OpenEuphoria

1. Sending a File Over HTTP

• Posted by euphoric (admin) Aug 07, 2013
• 2044 views

How do you send a large file over the Internet via HTTP with a Euphoria program?

I can program both the client and server (and both are programmed in Euphoria), I just need to know the best way. Is it necessary to use multi-part from the Euphoria program to push a file to a server?

Thanks!

3. Re: Sending a File Over HTTP

• Posted by useless_ Aug 07, 2013
• 2014 views

Use POST, but why use http?

useless

5. Re: Sending a File Over HTTP

• Posted by useless_ Aug 07, 2013
• 2012 views

Come to remember it, i was using Eu to http files on the internet in the v3.1 days, back when i was more useful and Tiggr was online. Peoples on irc could get urls from her to the domain she seemed to be on back then, but what would happen is a php script on the webhost sent the user req to an Eu app running on my home computer, which then sent the reply back upstream to the host and then to the user, and if necessary uploaded a 2nd file to the webhost and gave the url to it in the reply to the user. People on irc hated the idea because they had to launch a browser to get the file. The people were too lazy to click on an url and let a browser open. Tiggr could also download, read, and reply to email. Just some of the code i deleted later.

useless

7. Re: Sending a File Over HTTP

• Posted by jimcbrown (admin) Aug 07, 2013
• 2010 views

Aren't you using curl or libcurl for that? Posting a text file the standard way should work just fine with the stdlib, but your case is not standard.

Untested, but I just mocked up a version of http.e that should do what you need. See http://openeuphoria.org/pastey/229.wc

Here's how you'd call it (assuming that it works):

        -- custom data 
        sequence file_content = "COMMAND\tORDER\r\nAUTH\t\YES\r\n..." 
	-- if you have a text file in the right format with the right data, 
	-- you should be able to just read its contents into file_content as 
	-- a flat sequence 
 
        -- post file script gets size and file parameters, calls decode_base64, and sends 
        -- back SIZE\nDECODED_FILE_CONTENTS. The test script is written in Perl to test against 
        -- modules we did not code, i.e. CGI and Base64 in this case. 
        object content = http_post("http://confidential/program.cgi", { RAW_POST_DATA, "application/x-what-goes-here", file_content }) 

Your posts didn't specify if a particular content-type was required, and if so what that should be (e.g application/x-what-goes-here). Your data interface specs should have this information.

9. Re: Sending a File Over HTTP

• Posted by euphoric (admin) Aug 08, 2013
• 1943 views

Ah, yes! POST! Why didn't I think of that. Thanks, kat!

What else would I use, BTW?

11. Re: Sending a File Over HTTP

• Posted by useless_ Aug 08, 2013
• 1900 views

You could use PUT, it's like a hybrid FTP in the HTTP protocol. I imagine not everyone supports it.
https://www.google.com/search?q=PUT+http

useless

13. Re: Sending a File Over HTTP

• Posted by euphoric (admin) Aug 15, 2013
• 1761 views

OK, HTTP POST experts... What does a server-side program look like when receiving a MULTIPART HTTP POST request?

15. Re: Sending a File Over HTTP

• Posted by euphoric (admin) Aug 16, 2013
• 1702 views

I just mean a Euphoria CGI script running under Apache!

Let me add that I was using GET and parsing it just fine with the getenv("QUERY_STRING"). However, QUERY_STRING is empty and STDIN is -1 when trying to use POST.

Thanks!

17. Re: Sending a File Over HTTP

• Posted by euphoric (admin) Aug 19, 2013
• 1617 views

Do I have to send CONTENT_LENGTH? I thought http_post() would package that up for me (or the OS would know it), but can't find confirmation.

19. Re: Sending a File Over HTTP

• Posted by jimcbrown (admin) Aug 19, 2013
• 1603 views

http_post() does it for you. Take a look at the function itself, but search for Content-Length rather than CONTENT_LENGTH.

21. Re: Sending a File Over HTTP

• Posted by euphoric (admin) Aug 20, 2013
• 1534 views

Thanks for working with me, jim. My comments follow:

public function transfer(sequence fPath) 
sequence result = "FAIL" 
object filetxt 
 
	filetxt = read_file( fPath ) 
	if not atom(filetxt) then 
        sequence data, fname 
 
        fname = filename( fPath ) 
        data = { 
            { "name", fname }, 
            { "company", url:encode(COMPANY_NAME) }, 
            { "transfer", "true" }, 

All that "stuff" is data I need to send to the server side script so I can organize the file I'm sending.

For example, the cgi-script will take the company and create a directory for it on the server to store the data being sent.

         result = http_post( "http://my.ip.address/files/transfer.esp", { MULTIPART_FORM_DATA, data } ) 
	end if 
 
    if not equal(result[2],"SUCCESS!") then 
        ERR_MESSAGE = result[2] 
    else 
        ERR_MESSAGE = "" 
    end if 
	return equal(result[2],"SUCCESS!") 
end function 

My cgi-script returns "SUCCESS!" on a successful completion of the requested activity.

Even sending the size parameter, my cgi-script reports "Nothing received."

Here's the cgi-script I'm using (a modified version of yours that I'm using just for testing at the moment) called "transfer.esp":

#!c:/euphoria4.0.5/bin/eui.exe 
 
puts(1,"Content-type: text/html\n\n") 
 
without warning 
 
include std/filesys.e 
include std/dll.e 
include std/get.e 
include std/machine.e 
include std/sequence.e 
include std/io.e 
 
constant upload_dir = "licenses/" 
 
object nbytes, variables, notused, qstr 
 
constant alphanum = "abcdefghijklmnopqrstuvwxyz" & 
    "ABCDEFGHIJKLMNOPQRSTUVWXYZ" & "0123456789." 
& "\\/:" --a hack, not really alphanum, but it makes things easier -- not an issue as files can never have these anyways 
 
function cleaner(sequence s) 
    integer k 
    k = 1 
    while k <= length(s) do 
	if not find(s[k], alphanum) then 
	    s = s[1..k-1] & "%" & sprintf("%x", s[k]) & s[k+1..length(s)] 
	    k += 2 --we can skip the next 2 chars, they are part of 
	    --the curent char 
	end if 
	k += 1 
    end while 
    return s 
end function 
 
nbytes = getenv("CONTENT_LENGTH") 
qstr = getenv("QUERY_STRING") 
 
-- testing received values here... 
if atom(qstr) then 
	puts(1,"No query string.") 
else 
	if length(qstr) > 0 then 
		puts(1,"Query string: '" & qstr & "'") 
	else 
		puts(1,"Query string is empty.") 
	end if 
end if 
 
if atom(nbytes) then 
	nbytes = "" 
end if 
 
nbytes = value(nbytes) 
if nbytes[1] = GET_SUCCESS then 
  nbytes = nbytes[2] 
else 
  puts(1,"\n<html><body>Internal Server Error1<p>" & 
 "Please try again at a later time.</body></html>") 
  abort(1) 
end if 
 
variables = get_bytes(0,nbytes) 
 
object text, line, char, last 
text = {} 
line = {} 
last = -1 
for j = 1 to length(variables) do 
    char = variables[j] 
    line &= char 
    if char = '\n' and last = '\r' then 
	text &= {line} 
	line = {} 
    end if 
    last = char 
end for 
 
if length(text) > 0 then 
	if equal(text[$],"") then text = text[1..$-1] end if 
	variables = text 
 
	variables = variables[2..length(variables)-1] 
	notused = match("filename=", variables[1]) 
	nbytes = filename(variables[1][notused+length("filename=")+1..length(variables[1])-3]) 
	variables = variables[3..$] 
	notused = getenv("REMOTE_ADDR") --remote ip 
 
	if atom(notused) then 
		notused = "" 
	end if 
 
	nbytes = upload_dir&notused&"-"&nbytes --remote ip & filename 
	notused = open(nbytes, "w") 
	if notused = -1 then 
	      printf(1, "\nCouldn't open file %s.", {nbytes}) 
	      abort(1) 
	end if 
	variables = variables[2..$] -- bizarre but ok - have random leftover blank line 
	variables[$] = variables[$][1..length(variables[$])-2] 
	if equal(variables[$],"") then variables = variables[1..$-1] end if 
	for j = 1 to length(variables) do 
	    puts(notused, variables[j]) 
	end for 
	close(notused) 
	puts(1, "SUCCESS!") 
else 
	puts(1,"\nNothing received.") 
end if 

Here's the function to call transfer.esp from the client side:

public function transfer(sequence fPath) 
sequence result = "FAIL" 
object filetxt 
 
	filetxt = read_file( fPath ) 
	if not atom(filetxt) then 
        sequence data, fname 
 
        fname = filename( fPath ) 
        data = { 
            { "name", fname }, 
            { "company", url:encode(COMPANY_NAME) }, 
            { "transfer", "true" }, 
            { "data", filetxt, fname, "text/plain", ENCODE_NONE }, 
            { "size", sprintf("%d",{length(filetxt)}) } 
        } 
     
    puts(1,"\nsending this data:\n") 
    pretty_print(1,data,{3}) 
     
        result = http_post( "http://localhost/transfer.esp", { MULTIPART_FORM_DATA, data } ) 
	end if 
 
puts(1,"\nFrom server:\n") 
pretty_print(1,result,{3}) 
 
    if not equal(result[2],"SUCCESS!") then 
        ERR_MESSAGE = result[2] 
    else 
        ERR_MESSAGE = "" 
    end if 
	return equal(result[2],"SUCCESS!") 
end function 
 

I'm always getting "Query string is empty. Nothing received."

23. Re: Sending a File Over HTTP

• Posted by euphoric (admin) Aug 20, 2013
• 1567 views

I've met with great success!

Client-side code:

public function transfer(sequence fPath) 
sequence result = "FAIL" 
object filetxt 
 
	filetxt = read_file( fPath ) 
	if not atom(filetxt) then 
        sequence data, fname 
 
        fname = filename( fPath ) 
        data = { 
            { "data", filetxt, fname, "text/plain", ENCODE_NONE }, 
            { "size", sprintf("%d",{length(filetxt)}) } 
        } 
     
        result = http_post( "http://12.345.678.90/files/transfer.esp?transfer=true&name=" & fname & "&company=" & url:encode(COMPANY_NAME), { MULTIPART_FORM_DATA, data } ) 
 	else 
 		message_box( "Could not backup '" & licFile & "'\n\nPlease try again later or contact us for help.", "SAT Transfer Error", MB_OK+MB_ICONERROR) 
	end if 
 
    if not equal(result[2],"SUCCESS!") then 
        ERR_MESSAGE = result[2] 
    else 
        ERR_MESSAGE = "" 
    end if 
	return equal(result[2],"SUCCESS!") 
end function 

Server-side (cgi) code:

#!c:/euphoria4.0.5/bin/eui.exe 
 
puts(1,"Content-type: text/html\n\n") 
 
without warning 
 
include std/filesys.e 
include std/dll.e 
include std/get.e 
include std/machine.e 
include std/sequence.e 
include std/io.e 
 
include settings.e 
include shared.e 
 
object nbytes, qstr 
sequence params = "", text, LICENSES = get_setting("LICENSES") 
 
nbytes = getenv("CONTENT_LENGTH") 
qstr = getenv("QUERY_STRING") 
 
if atom(qstr) then 
	puts(1,"No query string.") 
	abort(1) 
else 
	if length(qstr) > 0 then 
		params = get_params( qstr ) 
	else 
		puts(1,"Query string is empty.") 
		abort(1) 
	end if 
end if 
 
if atom(nbytes) then 
	nbytes = "" 
end if 
 
nbytes = value(nbytes) 
if nbytes[1] = GET_SUCCESS then 
  nbytes = nbytes[2] 
else 
  puts(1,"Internal error. Please try again later.") 
  abort(1) 
end if 
 
text = get_bytes(0,nbytes) 
 
if length(text) > 0 and sequence(params) then 
 
	sequence cname, fname, buDir, buFile, bfile = "transfer" 
	integer i 
	 
    params[1] = lower(params[1]) 
     
    i = find("company",params[1]) 
    cname = params[2][i] 
	i = find("name",params[1]) 
	fname = params[2][i] 
	 
	if atom( dir( LICENSES & "\\" & cname & "\\" & bfile ) ) then 
		system( "mkdir \"" & LICENSES & "\\" & cname & "\\" & bfile & "\"", 2 ) 
	end if 
	 
	buDir = LICENSES & "\\" & cname & "\\" & bfile & "\\" 
   	buFile =  buDir & cname & "-" & fname 
   	 
    atom fn = open( buFile, "w" ) 
    if fn = -1 then 
		puts(1,"Could not open remote transfer file!") 
    else 
	    puts(fn,text) 
	    close(fn) 
	    puts(1,"SUCCESS!") 
	end if 
 
else 
	puts(1,"Bad params or nothing received.") 
end if 

The only problem (and it's not a problem for me, but might be for someone later down the line), is that variable 'text' needs to be parsed of its contents. Here's an example of what is received:

--rOQZtBFmBcvbVJhYJ13g 
Content-Disposition: form-data; name="data"; filename="foad.lic" 
Content-Type: text/plain 
 
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Duis condimentum lobortis tempus. Morbi eros augue, vestibulum nec ultrices at, aliquet et metus. Quisque et orci sit amet mauris volutpat sagittis. Sed pretium eget ante at consequat. Proin leo mi, accumsan quis vestibulum ac, sagittis varius urna. Morbi eget lorem vel purus ullamcorper laoreet non in lectus. Donec ultrices adipiscing sem nec elementum. Phasellus eu ullamcorper eros, ut tincidunt risus. In fringilla justo quis erat dignissim suscipit. Vivamus in ullamcorper posuere. 
--rOQZtBFmBcvbVJhYJ13g 
Content-Disposition: form-data; name="size" 
 
545 
--rOQZtBFmBcvbVJhYJ13g-- 

The content of the file is between the blank line after "Content-Type: text/plain" and second marker (--rOQZtBFmBcvbVJhYJ13g). Do we already have a function that will parse this content?