1. Internet Access
[Warning: This is as really a long-winded wish list disguised as a question]
I've been reading (with some amount of envy) about the Rebol language
(http://www.rebol.com). Stuff like web access, ftp and email all appear
trivial in Rebol. Here's an example script that reads the raw contents of an
html file and emails it:
send luke at rebol.com read http://www.rebol.com/releases.html
Rebol supports at the following, across various platforms:
HTTP (web)
SMPT (sending email)
POP (reading email)
FTP (file transfer)
NNTP (news)
It seems to me that it would be a Good Thing if Euphoria also supported
these in a uniform, cross-platform manner.
Some of this no doubt exists already, and I'm simply to dense to understand
it. So I'm hoping that people can whack me with a clue stick and help me out
here.
I'm primarily interested in being able to read web pages. I was thinking of
something along the lines of:
read_http( <web address> )
that would return the contents a file from the web. The application I had in
mind would read through a bulletin board and gather conversation threads for
off-site viewing.
[Question: If the page required cookies, would the routine have to accept
cookies in order to work?]
Anyhoo, I took a look at Daniel Bernstein's EuFTP package, but that deals
with FTP, not HTTP.
[Taking a stroll into Wish List Land...]
For completeness, we also need:
read_ftp()
read_binary_ftp()
write_ftp()
write_binary_ftp()
And since Rebol provides an interface to email, of course I want Euphoria to
provide that as well. Stealing shamelessly, I was thinking of something
like:
open_mailbox() -- open a mailbox for reading
clear_mailbox() -- empty a mailbox
write_mailbox() -- send a message to a mailbox
read_mailbox() -- read the current message in the mailbox
remove_mailbox() -- remove item from mailbox
next_mailbox() -- read the next message in the mailbox
close_mailbox() -- close a mailbox
I took a brief look at the email demo that Robert includes with Euphoria. Is
there a special reason why the message goes to my Outbox, instead of being
send immediately? Where can I get more information on this protocol?
[End of Soap Box/Wish List]
If someone could point me in the right direction for writing a read_http()
function, I'd appreciate it.
Additionally, I'm wondering if there actually is any interest in putting
together a cross-platform library of Rebol-like web functions for Euphoria.
-- David Cuny
2. Re: Internet Access
On Thu, 15 Jun 2000 03:26:36 -0700, David Cuny <dcuny at LANSET.COM> wrote:
>[Warning: This is as really a long-winded wish list disguised as a
question]
>
>I've been reading (with some amount of envy) about the Rebol language
>(http://www.rebol.com). Stuff like web access, ftp and email all appear
>trivial in Rebol. Here's an example script that reads the raw contents of
an
>html file and emails it:
>
>send luke at rebol.com read http://www.rebol.com/releases.html
>
>Rebol supports at the following, across various platforms:
>
> HTTP (web)
> SMPT (sending email)
> POP (reading email)
> FTP (file transfer)
> NNTP (news)
>
>It seems to me that it would be a Good Thing if Euphoria also supported
>these in a uniform, cross-platform manner.
>
>Some of this no doubt exists already, and I'm simply to dense to understand
>it. So I'm hoping that people can whack me with a clue stick and help me
out
>here.
>
>I'm primarily interested in being able to read web pages. I was thinking of
>something along the lines of:
>
> read_http( <web address> )
>
>that would return the contents a file from the web. The application I had
in
>mind would read through a bulletin board and gather conversation threads
for
>off-site viewing.
>
>
>[Question: If the page required cookies, would the routine have to accept
>cookies in order to work?]
>
>
>Anyhoo, I took a look at Daniel Bernstein's EuFTP package, but that deals
>with FTP, not HTTP.
>
>[Taking a stroll into Wish List Land...]
>
>For completeness, we also need:
>
> read_ftp()
> read_binary_ftp()
> write_ftp()
> write_binary_ftp()
>
>And since Rebol provides an interface to email, of course I want Euphoria
to
>provide that as well. Stealing shamelessly, I was thinking of something
>like:
>
> open_mailbox() -- open a mailbox for reading
> clear_mailbox() -- empty a mailbox
> write_mailbox() -- send a message to a mailbox
> read_mailbox() -- read the current message in the mailbox
> remove_mailbox() -- remove item from mailbox
> next_mailbox() -- read the next message in the mailbox
> close_mailbox() -- close a mailbox
>
>I took a brief look at the email demo that Robert includes with Euphoria.
Is
>there a special reason why the message goes to my Outbox, instead of being
>send immediately? Where can I get more information on this protocol?
>
>
>[End of Soap Box/Wish List]
>
>If someone could point me in the right direction for writing a read_http()
>function, I'd appreciate it.
>
>Additionally, I'm wondering if there actually is any interest in putting
>together a cross-platform library of Rebol-like web functions for Euphoria.
>
>-- David Cuny
David,
Try searching MSDN for the winInet (Windows Internet transfer) control
documentation. It has wrappers for the HTTP, FTP, and GOPHER protocols.
Brian Jackson
3. Re: Internet Access
Brian Jackson wrote:
>> If someone could point me in the right direction for
>> writing a read_http() function, I'd appreciate it.
>
> Try searching MSDN for the winInet (Windows Internet
> transfer) control documentation. It has wrappers for
> the HTTP, FTP, and GOPHER protocols.
I found some tutorials, thanks.
I was wondering if WinSock was perhaps a more diplomatic choice, since (on
Win95 machines, anyway) the presence of WinInet seems to be dependant on IE
being loaded. But I didn't see any support for HTTP protocol in the WinSock
2 API, so I suppose it's pretty much the only game in town. 
-- David Cuny
4. Re: Internet Access
On 15 Jun 2000, at 3:26, David Cuny wrote:
End of Soap Box/Wish List]
>
> If someone could point me in the right direction for writing a
> read_http() function, I'd appreciate it.
Look at Webshepard in the archives. There are a couple bugs, but
easily fixed. It builds a dir tree on the hd corresponding to the url
you ask for. I wrote one in mirc too, and in REBOL, but the Eu one
is better to modify things, you can't get into the REBOL code at
that level, and the mirc doesn't do something right that i can't figure
out. Eu is better to coding for non-standard events. Thanks to you
and win32lib, Eu is great at web access.
> Additionally, I'm wondering if there actually is any interest in
> putting together a cross-platform library of Rebol-like web functions
> for Euphoria.
yes
Kat
5. Re: Internet Access
Kat wrote:
>> If someone could point me in the right
>> direction for writing a read_http() function,
>> I'd appreciate it.
>
> Look at [Web Shepherd] in the archives.
Great! Looks like Just like Mark Smith did a ton of work, and it's almost
exactly what I was looking for.
>> Additionally, I'm wondering if there actually
>> is any interest in putting together a cross-platform
>> library of Rebol-like web functions for Euphoria.
>
> yes
My wants are simple:
- Make it simple as Rebol
- Make it cross-platform
- Make it a standard Euphoria include
Actually being involved the coding isn't even on my priority list. Are you
interesting in running the project? You and Pete are far more knowlegeable
about this sort of thing than I am, although I'm quite willing to hack at
Mark's code.
-- David Cuny
6. Re: Internet Access
From: <color><param>0000,0000,8000</param>Kat <<gertie at
pell.net></color>
To: <color><param>0000,0000,8000</param>Cuny,
David@DSS</color>
<bold>Subject: <color><param>0000,0000,8000</param>Re: Internet
Access</bold></color>
Date sent: <color><param>0000,0000,8000</param>Fri, 16 Jun 2000
02:05:12 -0500</color>
On 15 Jun 2000, at 15:02, Cuny, David@DSS wrote:
<color><param>7F00,0000,0000</param>>
> My wants are simple:
>
> - Make it simple as Rebol
</color>Rebol has a *few* drawbacks that can be improved on, unless it's
been done already. Plus, i'd prefer to make it's smartness a sliding
variable factor, such as pass the parms you feel are relavant and
Eu figures out what to do with them. Not as it is (was?) in Rebol, if
you needed email on another port, you had to drop to writing
winsock code, or change the ini for it and restart Rebol. I
remember i had a problem with Rebol on some web pages too,
something in the header, and i haven't had that problem in Eu yet.
It was odd, cause if i had Rebol connect to mirc, then took the
same header from the Rebol http get and had mirc forward it to the
real web server, it was accepted.
<color><param>7F00,0000,0000</param>> - Make it cross-platform
</color>I did get two linux CDs, now if i could only find the drive.....
<color><param>7F00,0000,0000</param>> - Make it a standard Euphoria include
> Actually being involved the coding isn't even on my priority list.
Are
> you interesting in running the project?
</color><<speechless> errr,,, ok
<color><param>7F00,0000,0000</param>>You and Pete are far more
> knowlegeable about this sort of thing than I am, although I'm quite
> willing to hack at Mark's code.
</color>I have been working on it on/off for a bit, doing cleanup, adding
some options, but i hadn't considered ftp or email to it. I could tho,
it's only a matter of protocol really, and i have those plus telnet and
MUD programmed in mirc. The cookies, redirects, etc are just a
matter of storage and looping the code, not a problem.
Kat
<nofill>
7. Re: Internet Access
Kat wrote:
> Plus, i'd prefer to make it's smartness a sliding
> variable factor, such as pass the parms you feel
> are relavant and Eu figures out what to do with them.
Do you mean something like one of these:
net_read_ftp( { NET_USERNAME, "fredcheezebits", NET_TIMEOUT 2000,
... } )
net_read_ftp( "username=fredcheezebits timeout=2000" )
instead of using seperate calls, like this:
net_timeout( 2000 )
net_username( "fredcheezbits" )
My preference is to use seperate calls, since it's a bit simpler, and more
standard.
> Not as it is (was?) in Rebol, if you needed email
> on another port, you had to drop to writing winsock
> code, or change the ini for it and restart Rebol.
Ick. I like the idea of being able to read an INI file, though.
>> Make it cross-platform.
> I did get two linux CDs, now if i could only find the drive.....
Heh. You don't have to be the one to code it. I just want to make sure that
the interface is designed to make it non-platform specific. Eventually I'll
start begging Pete to code the Linux stuff, before work in the Real World
drags him away from Euphoria forever.
>> Are you interesting in running the project?
> <speechless> errr,,, ok
Great. I'm hacking out the read_http() routine this weekend. It looks like
it shouldn't be too tough to convert it into read_https, read_ftp and
read_gopher once it's up and running.
> I have been working on it on/off for a bit, doing cleanup,
> adding some options, but i hadn't [adding] considered ftp
> or email to it.
I figured that the Rebol list looked fairly complete, so it may as well be
used as a model for routines. To some extent, it's redundant, since the
destintation strings will specify the protocol:
net_read( "http://www.rapideuphoria.com/contrib.htm")
I'll try to read through the Rebol documentation and get a proposed list of
routines. If we can come to agreement on the list, people can start hacking
existing code. Maybe Mark Smith would like to take a stab at it, instead of
having me mangle his code?
You don't have to code it all yourself, just coordinate it and yell at
people to finish up their portions. I'll volunteer for the http: bits.
Eventually, you'll get tired of yelling, and code it all yourself... 
One thing that's I'd like to add to the function list is an 'idle' routine
to run between grabbing chunks of files, so the user could watch for an
escape key/update a progress bar/etc. Something like:
net_idle( routine_id( <callback name>) )
Comments?
-- David Cuny
8. Re: Internet Access
- Posted by Kat <gertie at PELL.NET>
Jun 16, 2000
-
Last edited Jun 17, 2000
On 16 Jun 2000, at 18:46, David Cuny wrote:
> Kat wrote:
>
> > Plus, i'd prefer to make it's smartness a sliding
> > variable factor, such as pass the parms you feel
> > are relavant and Eu figures out what to do with them.
>
> Do you mean something like one of these:
>
> net_read_ftp( { NET_USERNAME, "fredcheezebits", NET_TIMEOUT 2000,
> ... } )
> net_read_ftp( "username=fredcheezebits timeout=2000" )
>
> instead of using seperate calls, like this:
>
> net_timeout( 2000 )
> net_username( "fredcheezbits" )
>
> My preference is to use seperate calls, since it's a bit simpler, and
> more standard.
What if i wanted to get the main page thru the proxy at port 8082,
the frames thru the proxy at port 8080, and the gifs thru the port
80? I could fire off the mainpage get (after setting some defaults):
page = getweb("return some.domain.com myport=8082")
-- parse the page
junk = getweb("save=e:\\somedir\subdir
some.domain.com/number1.gif myport=80")
By doing it this way, you cannot change the vars out from under
the current webget, but i guess there are other way around that.
Besides, this approach can be a single function that then calls
whatever your idea is of the proper interface.
> > Not as it is (was?) in Rebol, if you needed email
> > on another port, you had to drop to writing winsock
> > code, or change the ini for it and restart Rebol.
>
> Ick. I like the idea of being able to read an INI file, though.
Well, before the webget, the app can store/read the ini file, doesn't
matter to me, just seems the read/write to the hd will take
unnecessary time, all it will be good for is saving defaults and/or
last known state.
> >> Make it cross-platform.
> > I did get two linux CDs, now if i could only find the drive.....
>
> Heh. You don't have to be the one to code it. I just want to make sure
> that the interface is designed to make it non-platform specific.
> Eventually I'll start begging Pete to code the Linux stuff, before
> work in the Real World drags him away from Euphoria forever.
Right, coordination so the function names are the same and do the
same.
> >> Are you interesting in running the project?
> > <speechless> errr,,, ok
>
> Great. I'm hacking out the read_http() routine this weekend. It looks
> like it shouldn't be too tough to convert it into read_https, read_ftp
> and read_gopher once it's up and running.
I already have read_http.
> > I have been working on it on/off for a bit, doing cleanup,
> > adding some options, but i hadn't [adding] considered ftp
> > or email to it.
>
> I figured that the Rebol list looked fairly complete, so it may as
> well be used as a model for routines. To some extent, it's redundant,
> since the destintation strings will specify the protocol:
>
> net_read( "http://www.rapideuphoria.com/contrib.htm")
But you didn't spec what to do with the page. What if the page is
on a puter on your intranet? Rewrite the ini file, make sure it is
written, then call the net_read()?
> I'll try to read through the Rebol documentation and get a proposed
> list of routines. If we can come to agreement on the list, people can
> start hacking existing code. Maybe Mark Smith would like to take a
> stab at it, instead of having me mangle his code?
>
> You don't have to code it all yourself, just coordinate it and yell at
> people to finish up their portions. I'll volunteer for the http: bits.
>
> Eventually, you'll get tired of yelling, and code it all yourself...
>
>
> One thing that's I'd like to add to the function list is an 'idle'
> routine to run between grabbing chunks of files, so the user could
> watch for an escape key/update a progress bar/etc. Something like:
>
> net_idle( routine_id( <callback name>) )
I haven't tried to code any routine_id().
Kat
9. Re: Internet Access
Kat wrote:
> Besides, this approach can be a single function
> that then calls whatever your idea is of the proper
> interface.
Yeah, I think the best plan would be to supply a set of bricks and let
people write their own specialized routines.
> Well, before the webget, the app can store/read the
> ini file, doesn't matter to me, just seems the read/write
> to the hd will take unnecessary time, all it will be good
> for is saving defaults and/or last known state.
You're right. It would be better to leave this sort of thing out, and let
people add it themselves if it was appropriate to the application. I sort of
liked the INI file that Rebol has, but it would mean writing a specialized
routine to read and write ini files, and would have dubious value for most
users.
> Right, coordination so the function names are the same and do the
> same.
Please bear with me if the proposed functions seem a bit naive; I bow to
your expertise.
> I already have read_http.
No point in coding it, then. I'll assume that you can pretty much code
everything on the Windows side, and will ask for others more qualified for
help on other bits.
> > net_read( "http://www.rapideuphoria.com/contrib.htm")
>
> But you didn't spec what to do with the page. What if the page is
> on a puter on your intranet? Rewrite the ini file, make sure it is
> written, then call the net_read()?
Shame on me. I was thinking of returning a sequence containing the result
(binary file), or a -1 if there was a failure. This matches the way gets()
works:
object result
result = net_read( http://www")
if integer( result ) then
-- there's been an error
end if
> > net_idle( routine_id( <callback name>) )
>
> I haven't tried to code any routine_id().
The idea was that Euphoria doesn't support threading. So the best that can
be offered is, between packets, the routine could trigger and check to see
if the user wants to abort, or it displays an update to the screen status,
or something like that.
Anyhoo, here's my proposal. Anyone, feel free to hack it to bits:
[general]
the prefix net_ is used on routines.
the prefix NET_ is used on constants.
[setup]
i have to defer to your expertise when it comes to anything more than
trivial settings.
net_domain( "user at domain.dom" )
net_smtp_server( "mail.server.dom" )
net_pop_server( "pop.server.dom" )
net_proxy_server( "proxy.server.dom", 1080 ) -- i'm already out of my
depth here!
[sending email]
the recipient and cc can either be a single item, or a sequence of items.
net_subject( "this is the header line" )
net_recipient( nowhere.man at dev.null" )
net_cc( robert.craig at euphoria.com" )
net_content( "this is the body of the message" )
result = net_email()
[reading email]
i'm assuming that you want to support more than one mailbox being open
at a time. so net_open_mailbox returns the id of the requested mailbox,
and net_close_mailbox is used to close an open mailbox.
net_read_mail is used to examine the contents of the mailbox, and
net_remove_mail is used to remove contents of a mailbox.
should blind cc be included as well?
-- these are used to index the sequence returned by net_read_mail
NET_DATE = 1,
NET_SUBJECT = 2,
NET_RECIPENT = 3,
NET_CC = 4,
NET_CONTENT = 5
-- these can be used instead of an index in net_read_mailbox
-- i.e.: message = read_mailbox( myMailbox, NET_FIRST_MESSAGE )
NET_FIRST_MESSAGE = -1 -- first message in mailbox
NET_NEXT_MESSAGE = -2, -- next message in mailbox
NET_PRIOR_MESSAGE = -3, -- previous message in mailbox
NET_LAST_MESSAGE = -4 -- final message in mailbox
mailbox = net_open_mailbox( "pop://orson:rosebud at mail.yoda.dom" )
index = net_count_mail( mailbox )
message = net_read_mail( mailbox, index )
net_ remove_mail( mailbox, index )
net_close_mailbox( mailbox )
[ network read/write ]
the type of operation (http/https/ftp) should be determined from the
text of the url. for example:
net_read( "ftp://ftp.site.com/something.txt" )
is obviously going to use ftp. results of the read are returned as
a sequence on success, or a -1 on failure.
i think the simplest way to specify flags is akin to how euphoria
does it with files:
-- read in binary mode
result = net_read( "http://www.mysite.com/file.zip", "b" )
-- write in binary mode, append
net_write( "myfile.zip", "ftp://www.mysite.com/file.zip, "ba" )
net_dir is used to get file information. it returns a sequence with
file information in it, similar to euphoria's dir.
NET_NAME = 1, -- file name
NET_SIZE = 2, -- file size
NET_DATE = 3, -- file create date
NET_MODIFIED = 4, -- file modified date
NET_DIR = 5 -- true if file is a directory
result = net_read( url, flags )
result = net_write( "filename", url, flags )
net_ftp_user( "whoami" )
net_ftp_password( "secret" )
net_dir( url )
net_make_dir( url, dir )
net_delete( url )
net_rename( url, new name )
[tcp]
i defer to your knowledge here.
Comments, anyone?
-- david cuny
