1. Internet Access
- Posted by David Cuny <dcuny at LANSET.COM> Jun 15, 2000
- 663 views
[Warning: This is as really a long-winded wish list disguised as a question] I've been reading (with some amount of envy) about the Rebol language (http://www.rebol.com). Stuff like web access, ftp and email all appear trivial in Rebol. Here's an example script that reads the raw contents of an html file and emails it: send luke at rebol.com read http://www.rebol.com/releases.html Rebol supports at the following, across various platforms: HTTP (web) SMPT (sending email) POP (reading email) FTP (file transfer) NNTP (news) It seems to me that it would be a Good Thing if Euphoria also supported these in a uniform, cross-platform manner. Some of this no doubt exists already, and I'm simply to dense to understand it. So I'm hoping that people can whack me with a clue stick and help me out here. I'm primarily interested in being able to read web pages. I was thinking of something along the lines of: read_http( <web address> ) that would return the contents a file from the web. The application I had in mind would read through a bulletin board and gather conversation threads for off-site viewing. [Question: If the page required cookies, would the routine have to accept cookies in order to work?] Anyhoo, I took a look at Daniel Bernstein's EuFTP package, but that deals with FTP, not HTTP. [Taking a stroll into Wish List Land...] For completeness, we also need: read_ftp() read_binary_ftp() write_ftp() write_binary_ftp() And since Rebol provides an interface to email, of course I want Euphoria to provide that as well. Stealing shamelessly, I was thinking of something like: open_mailbox() -- open a mailbox for reading clear_mailbox() -- empty a mailbox write_mailbox() -- send a message to a mailbox read_mailbox() -- read the current message in the mailbox remove_mailbox() -- remove item from mailbox next_mailbox() -- read the next message in the mailbox close_mailbox() -- close a mailbox I took a brief look at the email demo that Robert includes with Euphoria. Is there a special reason why the message goes to my Outbox, instead of being send immediately? Where can I get more information on this protocol? [End of Soap Box/Wish List] If someone could point me in the right direction for writing a read_http() function, I'd appreciate it. Additionally, I'm wondering if there actually is any interest in putting together a cross-platform library of Rebol-like web functions for Euphoria. -- David Cuny
2. Re: Internet Access
- Posted by Brian Jackson <bjackson at 2FARGON.COM> Jun 15, 2000
- 666 views
On Thu, 15 Jun 2000 03:26:36 -0700, David Cuny <dcuny at LANSET.COM> wrote: >[Warning: This is as really a long-winded wish list disguised as a question] > >I've been reading (with some amount of envy) about the Rebol language >(http://www.rebol.com). Stuff like web access, ftp and email all appear >trivial in Rebol. Here's an example script that reads the raw contents of an >html file and emails it: > >send luke at rebol.com read http://www.rebol.com/releases.html > >Rebol supports at the following, across various platforms: > > HTTP (web) > SMPT (sending email) > POP (reading email) > FTP (file transfer) > NNTP (news) > >It seems to me that it would be a Good Thing if Euphoria also supported >these in a uniform, cross-platform manner. > >Some of this no doubt exists already, and I'm simply to dense to understand >it. So I'm hoping that people can whack me with a clue stick and help me out >here. > >I'm primarily interested in being able to read web pages. I was thinking of >something along the lines of: > > read_http( <web address> ) > >that would return the contents a file from the web. The application I had in >mind would read through a bulletin board and gather conversation threads for >off-site viewing. > > >[Question: If the page required cookies, would the routine have to accept >cookies in order to work?] > > >Anyhoo, I took a look at Daniel Bernstein's EuFTP package, but that deals >with FTP, not HTTP. > >[Taking a stroll into Wish List Land...] > >For completeness, we also need: > > read_ftp() > read_binary_ftp() > write_ftp() > write_binary_ftp() > >And since Rebol provides an interface to email, of course I want Euphoria to >provide that as well. Stealing shamelessly, I was thinking of something >like: > > open_mailbox() -- open a mailbox for reading > clear_mailbox() -- empty a mailbox > write_mailbox() -- send a message to a mailbox > read_mailbox() -- read the current message in the mailbox > remove_mailbox() -- remove item from mailbox > next_mailbox() -- read the next message in the mailbox > close_mailbox() -- close a mailbox > >I took a brief look at the email demo that Robert includes with Euphoria. Is >there a special reason why the message goes to my Outbox, instead of being >send immediately? Where can I get more information on this protocol? > > >[End of Soap Box/Wish List] > >If someone could point me in the right direction for writing a read_http() >function, I'd appreciate it. > >Additionally, I'm wondering if there actually is any interest in putting >together a cross-platform library of Rebol-like web functions for Euphoria. > >-- David Cuny David, Try searching MSDN for the winInet (Windows Internet transfer) control documentation. It has wrappers for the HTTP, FTP, and GOPHER protocols. Brian Jackson
3. Re: Internet Access
- Posted by "Cuny, David at DSS" <David.Cuny at DSS.CA.GOV> Jun 15, 2000
- 594 views
Brian Jackson wrote: >> If someone could point me in the right direction for >> writing a read_http() function, I'd appreciate it. > > Try searching MSDN for the winInet (Windows Internet > transfer) control documentation. It has wrappers for > the HTTP, FTP, and GOPHER protocols. I found some tutorials, thanks. I was wondering if WinSock was perhaps a more diplomatic choice, since (on Win95 machines, anyway) the presence of WinInet seems to be dependant on IE being loaded. But I didn't see any support for HTTP protocol in the WinSock 2 API, so I suppose it's pretty much the only game in town. -- David Cuny
4. Re: Internet Access
- Posted by Kat <gertie at PELL.NET> Jun 15, 2000
- 581 views
On 15 Jun 2000, at 3:26, David Cuny wrote: End of Soap Box/Wish List] > > If someone could point me in the right direction for writing a > read_http() function, I'd appreciate it. Look at Webshepard in the archives. There are a couple bugs, but easily fixed. It builds a dir tree on the hd corresponding to the url you ask for. I wrote one in mirc too, and in REBOL, but the Eu one is better to modify things, you can't get into the REBOL code at that level, and the mirc doesn't do something right that i can't figure out. Eu is better to coding for non-standard events. Thanks to you and win32lib, Eu is great at web access. > Additionally, I'm wondering if there actually is any interest in > putting together a cross-platform library of Rebol-like web functions > for Euphoria. yes Kat
5. Re: Internet Access
- Posted by "Cuny, David at DSS" <David.Cuny at DSS.CA.GOV> Jun 15, 2000
- 577 views
Kat wrote: >> If someone could point me in the right >> direction for writing a read_http() function, >> I'd appreciate it. > > Look at [Web Shepherd] in the archives. Great! Looks like Just like Mark Smith did a ton of work, and it's almost exactly what I was looking for. >> Additionally, I'm wondering if there actually >> is any interest in putting together a cross-platform >> library of Rebol-like web functions for Euphoria. > > yes My wants are simple: - Make it simple as Rebol - Make it cross-platform - Make it a standard Euphoria include Actually being involved the coding isn't even on my priority list. Are you interesting in running the project? You and Pete are far more knowlegeable about this sort of thing than I am, although I'm quite willing to hack at Mark's code. -- David Cuny
6. Re: Internet Access
- Posted by Kat <gertie at PELL.NET> Jun 16, 2000
- 591 views
From: <color><param>0000,0000,8000</param>Kat <<gertie at pell.net></color> To: <color><param>0000,0000,8000</param>Cuny, David@DSS</color> <bold>Subject: <color><param>0000,0000,8000</param>Re: Internet Access</bold></color> Date sent: <color><param>0000,0000,8000</param>Fri, 16 Jun 2000 02:05:12 -0500</color> On 15 Jun 2000, at 15:02, Cuny, David@DSS wrote: <color><param>7F00,0000,0000</param>> > My wants are simple: > > - Make it simple as Rebol </color>Rebol has a *few* drawbacks that can be improved on, unless it's been done already. Plus, i'd prefer to make it's smartness a sliding variable factor, such as pass the parms you feel are relavant and Eu figures out what to do with them. Not as it is (was?) in Rebol, if you needed email on another port, you had to drop to writing winsock code, or change the ini for it and restart Rebol. I remember i had a problem with Rebol on some web pages too, something in the header, and i haven't had that problem in Eu yet. It was odd, cause if i had Rebol connect to mirc, then took the same header from the Rebol http get and had mirc forward it to the real web server, it was accepted. <color><param>7F00,0000,0000</param>> - Make it cross-platform </color>I did get two linux CDs, now if i could only find the drive..... <color><param>7F00,0000,0000</param>> - Make it a standard Euphoria include > Actually being involved the coding isn't even on my priority list. Are > you interesting in running the project? </color><<speechless> errr,,, ok <color><param>7F00,0000,0000</param>>You and Pete are far more > knowlegeable about this sort of thing than I am, although I'm quite > willing to hack at Mark's code. </color>I have been working on it on/off for a bit, doing cleanup, adding some options, but i hadn't considered ftp or email to it. I could tho, it's only a matter of protocol really, and i have those plus telnet and MUD programmed in mirc. The cookies, redirects, etc are just a matter of storage and looping the code, not a problem. Kat <nofill>
7. Re: Internet Access
- Posted by David Cuny <dcuny at LANSET.COM> Jun 16, 2000
- 579 views
Kat wrote: > Plus, i'd prefer to make it's smartness a sliding > variable factor, such as pass the parms you feel > are relavant and Eu figures out what to do with them. Do you mean something like one of these: net_read_ftp( { NET_USERNAME, "fredcheezebits", NET_TIMEOUT 2000, ... } ) net_read_ftp( "username=fredcheezebits timeout=2000" ) instead of using seperate calls, like this: net_timeout( 2000 ) net_username( "fredcheezbits" ) My preference is to use seperate calls, since it's a bit simpler, and more standard. > Not as it is (was?) in Rebol, if you needed email > on another port, you had to drop to writing winsock > code, or change the ini for it and restart Rebol. Ick. I like the idea of being able to read an INI file, though. >> Make it cross-platform. > I did get two linux CDs, now if i could only find the drive..... Heh. You don't have to be the one to code it. I just want to make sure that the interface is designed to make it non-platform specific. Eventually I'll start begging Pete to code the Linux stuff, before work in the Real World drags him away from Euphoria forever. >> Are you interesting in running the project? > <speechless> errr,,, ok Great. I'm hacking out the read_http() routine this weekend. It looks like it shouldn't be too tough to convert it into read_https, read_ftp and read_gopher once it's up and running. > I have been working on it on/off for a bit, doing cleanup, > adding some options, but i hadn't [adding] considered ftp > or email to it. I figured that the Rebol list looked fairly complete, so it may as well be used as a model for routines. To some extent, it's redundant, since the destintation strings will specify the protocol: net_read( "http://www.rapideuphoria.com/contrib.htm") I'll try to read through the Rebol documentation and get a proposed list of routines. If we can come to agreement on the list, people can start hacking existing code. Maybe Mark Smith would like to take a stab at it, instead of having me mangle his code? You don't have to code it all yourself, just coordinate it and yell at people to finish up their portions. I'll volunteer for the http: bits. Eventually, you'll get tired of yelling, and code it all yourself... One thing that's I'd like to add to the function list is an 'idle' routine to run between grabbing chunks of files, so the user could watch for an escape key/update a progress bar/etc. Something like: net_idle( routine_id( <callback name>) ) Comments? -- David Cuny
8. Re: Internet Access
- Posted by Kat <gertie at PELL.NET> Jun 16, 2000
- 598 views
- Last edited Jun 17, 2000
On 16 Jun 2000, at 18:46, David Cuny wrote: > Kat wrote: > > > Plus, i'd prefer to make it's smartness a sliding > > variable factor, such as pass the parms you feel > > are relavant and Eu figures out what to do with them. > > Do you mean something like one of these: > > net_read_ftp( { NET_USERNAME, "fredcheezebits", NET_TIMEOUT 2000, > ... } ) > net_read_ftp( "username=fredcheezebits timeout=2000" ) > > instead of using seperate calls, like this: > > net_timeout( 2000 ) > net_username( "fredcheezbits" ) > > My preference is to use seperate calls, since it's a bit simpler, and > more standard. What if i wanted to get the main page thru the proxy at port 8082, the frames thru the proxy at port 8080, and the gifs thru the port 80? I could fire off the mainpage get (after setting some defaults): page = getweb("return some.domain.com myport=8082") -- parse the page junk = getweb("save=e:\\somedir\subdir some.domain.com/number1.gif myport=80") By doing it this way, you cannot change the vars out from under the current webget, but i guess there are other way around that. Besides, this approach can be a single function that then calls whatever your idea is of the proper interface. > > Not as it is (was?) in Rebol, if you needed email > > on another port, you had to drop to writing winsock > > code, or change the ini for it and restart Rebol. > > Ick. I like the idea of being able to read an INI file, though. Well, before the webget, the app can store/read the ini file, doesn't matter to me, just seems the read/write to the hd will take unnecessary time, all it will be good for is saving defaults and/or last known state. > >> Make it cross-platform. > > I did get two linux CDs, now if i could only find the drive..... > > Heh. You don't have to be the one to code it. I just want to make sure > that the interface is designed to make it non-platform specific. > Eventually I'll start begging Pete to code the Linux stuff, before > work in the Real World drags him away from Euphoria forever. Right, coordination so the function names are the same and do the same. > >> Are you interesting in running the project? > > <speechless> errr,,, ok > > Great. I'm hacking out the read_http() routine this weekend. It looks > like it shouldn't be too tough to convert it into read_https, read_ftp > and read_gopher once it's up and running. I already have read_http. > > I have been working on it on/off for a bit, doing cleanup, > > adding some options, but i hadn't [adding] considered ftp > > or email to it. > > I figured that the Rebol list looked fairly complete, so it may as > well be used as a model for routines. To some extent, it's redundant, > since the destintation strings will specify the protocol: > > net_read( "http://www.rapideuphoria.com/contrib.htm") But you didn't spec what to do with the page. What if the page is on a puter on your intranet? Rewrite the ini file, make sure it is written, then call the net_read()? > I'll try to read through the Rebol documentation and get a proposed > list of routines. If we can come to agreement on the list, people can > start hacking existing code. Maybe Mark Smith would like to take a > stab at it, instead of having me mangle his code? > > You don't have to code it all yourself, just coordinate it and yell at > people to finish up their portions. I'll volunteer for the http: bits. > > Eventually, you'll get tired of yelling, and code it all yourself... > > > One thing that's I'd like to add to the function list is an 'idle' > routine to run between grabbing chunks of files, so the user could > watch for an escape key/update a progress bar/etc. Something like: > > net_idle( routine_id( <callback name>) ) I haven't tried to code any routine_id(). Kat
9. Re: Internet Access
- Posted by David Cuny <dcuny at LANSET.COM> Jun 17, 2000
- 606 views
Kat wrote: > Besides, this approach can be a single function > that then calls whatever your idea is of the proper > interface. Yeah, I think the best plan would be to supply a set of bricks and let people write their own specialized routines. > Well, before the webget, the app can store/read the > ini file, doesn't matter to me, just seems the read/write > to the hd will take unnecessary time, all it will be good > for is saving defaults and/or last known state. You're right. It would be better to leave this sort of thing out, and let people add it themselves if it was appropriate to the application. I sort of liked the INI file that Rebol has, but it would mean writing a specialized routine to read and write ini files, and would have dubious value for most users. > Right, coordination so the function names are the same and do the > same. Please bear with me if the proposed functions seem a bit naive; I bow to your expertise. > I already have read_http. No point in coding it, then. I'll assume that you can pretty much code everything on the Windows side, and will ask for others more qualified for help on other bits. > > net_read( "http://www.rapideuphoria.com/contrib.htm") > > But you didn't spec what to do with the page. What if the page is > on a puter on your intranet? Rewrite the ini file, make sure it is > written, then call the net_read()? Shame on me. I was thinking of returning a sequence containing the result (binary file), or a -1 if there was a failure. This matches the way gets() works: object result result = net_read( http://www") if integer( result ) then -- there's been an error end if > > net_idle( routine_id( <callback name>) ) > > I haven't tried to code any routine_id(). The idea was that Euphoria doesn't support threading. So the best that can be offered is, between packets, the routine could trigger and check to see if the user wants to abort, or it displays an update to the screen status, or something like that. Anyhoo, here's my proposal. Anyone, feel free to hack it to bits: [general] the prefix net_ is used on routines. the prefix NET_ is used on constants. [setup] i have to defer to your expertise when it comes to anything more than trivial settings. net_domain( "user at domain.dom" ) net_smtp_server( "mail.server.dom" ) net_pop_server( "pop.server.dom" ) net_proxy_server( "proxy.server.dom", 1080 ) -- i'm already out of my depth here! [sending email] the recipient and cc can either be a single item, or a sequence of items. net_subject( "this is the header line" ) net_recipient( nowhere.man at dev.null" ) net_cc( robert.craig at euphoria.com" ) net_content( "this is the body of the message" ) result = net_email() [reading email] i'm assuming that you want to support more than one mailbox being open at a time. so net_open_mailbox returns the id of the requested mailbox, and net_close_mailbox is used to close an open mailbox. net_read_mail is used to examine the contents of the mailbox, and net_remove_mail is used to remove contents of a mailbox. should blind cc be included as well? -- these are used to index the sequence returned by net_read_mail NET_DATE = 1, NET_SUBJECT = 2, NET_RECIPENT = 3, NET_CC = 4, NET_CONTENT = 5 -- these can be used instead of an index in net_read_mailbox -- i.e.: message = read_mailbox( myMailbox, NET_FIRST_MESSAGE ) NET_FIRST_MESSAGE = -1 -- first message in mailbox NET_NEXT_MESSAGE = -2, -- next message in mailbox NET_PRIOR_MESSAGE = -3, -- previous message in mailbox NET_LAST_MESSAGE = -4 -- final message in mailbox mailbox = net_open_mailbox( "pop://orson:rosebud at mail.yoda.dom" ) index = net_count_mail( mailbox ) message = net_read_mail( mailbox, index ) net_ remove_mail( mailbox, index ) net_close_mailbox( mailbox ) [ network read/write ] the type of operation (http/https/ftp) should be determined from the text of the url. for example: net_read( "ftp://ftp.site.com/something.txt" ) is obviously going to use ftp. results of the read are returned as a sequence on success, or a -1 on failure. i think the simplest way to specify flags is akin to how euphoria does it with files: -- read in binary mode result = net_read( "http://www.mysite.com/file.zip", "b" ) -- write in binary mode, append net_write( "myfile.zip", "ftp://www.mysite.com/file.zip, "ba" ) net_dir is used to get file information. it returns a sequence with file information in it, similar to euphoria's dir. NET_NAME = 1, -- file name NET_SIZE = 2, -- file size NET_DATE = 3, -- file create date NET_MODIFIED = 4, -- file modified date NET_DIR = 5 -- true if file is a directory result = net_read( url, flags ) result = net_write( "filename", url, flags ) net_ftp_user( "whoami" ) net_ftp_password( "secret" ) net_dir( url ) net_make_dir( url, dir ) net_delete( url ) net_rename( url, new name ) [tcp] i defer to your knowledge here. Comments, anyone? -- david cuny