1. Hitting an API with Euphoria
- Posted by euphoric (admin) Dec 22, 2017
- 1895 views
Node.js has async functionality, so it's nice to use it to hit an API websocket and grab data while other processes continue.
I'm curious, could Euphoria be used to pull data from a web socket, keeping maybe a sequence populated with the latest data, while another part of the code utilizes that data?
Node.js has async/await, but is that really necessary for what I want to do?
I'm thinking about starting a pretty big project using Irv's GUI library and Eu4. Any input regarding reading web sockets would be much appreciated!
2. WellRe: Hitting an API with Euphoria
- Posted by Ekhnat0n Dec 22, 2017
- 1844 views
The possibility is there, but I don't know if Eu is capable executing really threaded code through 3 out of 4 cores,
in the mean time collecting your incoming data in a FIFO-pipe thru the remaining core.
That pipe ofc. can be approached at any moment you need to do so.
You even could use all 4 cores for processing the data, as long as the pipe it draws it data from isn't empty.
Just before that stage, set core* to collecting for a moment or 2.
Simple (binary) logic made me find the solution in abt 3 secs
3. Re: Hitting an API with Euphoria
- Posted by petelomax Dec 22, 2017
- 1856 views
Phix has proper multithreading
Pete
4. Re: Hitting an API with Euphoria
- Posted by Ekhnat0n Dec 22, 2017
- 1833 views
@Pete:
Hi Pete,
I know Phix has mutithreading, I really thoroughly studied the 'manual and help'.
Phix was running for a while on my Lenovo with Ubuntu 16.04.3 but it seems it is not fully compatible with OE 4.1 yet.
The speed of processing was really stunning.
Alas some OS-troubles threw me back and forced me to re-install, but with
the same result.Though I updated all necessary files I think, I probably have missed updating the only one that survives a reboot or restart.
I did put it in a directory of its own, making Euphoria a "daughter" of Phix, by putting that in /phix/euphoria, so $PATH
etc are targeting just one level too low for phix, but it will be found inside the path I did set.
Hopefully I will be able to solve this little problem soon, enabling me to have a real blazing start of 2018.
As said before, if in any way you could use my help (translations, o.s.a.), I can do that part in almost perfect German and
French as well,
Kind regards
Antoine
5. Re: Hitting an API with Euphoria
- Posted by jimcbrown (admin) Dec 22, 2017
- 1878 views
Node.js has async functionality, so it's nice to use it to hit an API websocket and grab data while other processes continue.
Cool.
Euphoria has HTTP support, but not WebSockets. Doesn't look too hard to add, though: https://en.wikipedia.org/wiki/WebSocket
I'm curious, could Euphoria be used to pull data from a web socket, keeping maybe a sequence populated with the latest data, while another part of the code utilizes that data?
Yes, the easiest way will be to use tasks.
Node.js has async/await, but is that really necessary for what I want to do?
If you need to do other things while waiting on the data, then you will need nonblocking/async sockets. It's not necessary if you are ok with blocking waits (but few are). Even with tasks, a blocking call will cause all tasks to wait.
True multithreading would resolve this, but that's still experimental for Eu.
6. Re: Hitting an API with Euphoria
- Posted by euphoric (admin) Dec 22, 2017
- 1859 views
Phix has proper multithreading
Pete, does Phix have websockets?
7. Re: Hitting an API with Euphoria
- Posted by euphoric (admin) Dec 22, 2017
- 1854 views
Euphoria has HTTP support, but not WebSockets. Doesn't look too hard to add, though: https://en.wikipedia.org/wiki/WebSocket
So sockets is different from websockets? Dang. I was hoping it meant the same thing.
8. Re: Hitting an API with Euphoria
- Posted by irv Dec 22, 2017
- 1850 views
libwebsockets has "only" 177 functions. It should be possible to create a EuGTK plugin for that, in the same manner as the libwebview and libsourceview are done. The documentation is good.
That said, I have no idea how any of this would be used, except for the basic http stuff, so I am not the person to be doing this.
9. Re: Hitting an API with Euphoria
- Posted by euphoric (admin) Dec 22, 2017
- 1839 views
libwebsockets has "only" 177 functions. It should be possible to create a EuGTK plugin for that, in the same manner as the libwebview and libsourceview are done. The documentation is good.
That said, I have no idea how any of this would be used, except for the basic http stuff, so I am not the person to be doing this.
You have to, Irv. You're my only hope!
Or Pete.
Or Greg.
Or Matt.
10. Re: Hitting an API with Euphoria
- Posted by ghaberek (admin) Dec 22, 2017
- 1818 views
You have to, Irv. You're my only hope!
Eh, hope shmope. Here's a wrapper for librws. It's a lot smaller than libwebsockets but it doesn't seem to support HTTPS.
It uses a background thread to transfer data so (IMPORTANT) I have no idea how stable it will be on Euphoria. Good luck!
librws-x86.zip (12 KB)
-Greg
11. Re: Hitting an API with Euphoria
- Posted by euphoric (admin) Dec 22, 2017
- 1839 views
Here's a wrapper for librws. It's a lot smaller than libwebsockets but it doesn't seem to support HTTPS.
It uses a background thread to transfer data so (IMPORTANT) I have no idea how stable it will be on Euphoria. Good luck!
librws-x86.zip (12 KB)
I think I need HTTPS.
12. Re: Hitting an API with Euphoria
- Posted by petelomax Dec 22, 2017
- 1823 views
- Last edited Dec 23, 2017
Phix has proper multithreading
Pete, does Phix have websockets?
No, sorry. If, however, you get something working in OE, that needs threads, I'll take a look.
Pete
13. Re: Hitting an API with Euphoria
- Posted by jmduro Dec 24, 2017
- 1774 views
In my last Standard library, I added a demo to show REST API management with libcurl. It is not yet ready for use. On published 1.0.5 version, it needs a little change in _curl_.e:
In curl_easy_setopt, the lines corresponding to slists have to be changed this way:
) then -- struct curl_slist * code = exec_func(xcurl_easy_setopt, {curl, option, param}, "+curl_easy_setopt")
I also added a curl_put function:
global function curl_put(atom curl, sequence url, sequence headers, object body) --<function> --<name>curl_put</name> --<digest>sends a PUT request to an URL and gets the page</digest> --<desc> -- intended to be used with REST APIs --</desc> --<param> --<type>atom</type> --<name>handle</name> --<desc>CURL session handle</desc> --</param> --<param> --<type>sequence</type> --<name>url</name> --<desc>URL to get the page from</desc> --</param> --<param> --<type>sequence</type> --<name>headers</name> --<desc>list of request headers</desc> --</param> --<param> --<type>object</type> --<name>body</name> --<desc>body of the request. Either a string or NULL.</desc> --</param> --<return> -- sequence -- * status : HTTP status -- * url : effective URL (useful if redirection is followed) -- * headers : sequence of headers -- * content : HTML Page content --</return> --<example> -- constant DEFAULT_HEADERS = { -- "Host: 192.168.1.10", -- "User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0 " & -- "Gecko/20100101 Firefox/45.0", -- "Accept: application/json" -- } -- -- object res = curl_put(curl, MY_REST_API & "/session/" & sessionId & -- "/timeouts/implicit_wait", DEFAULT_HEADERS, "{" & -- "\"sessionId\": \"" & sessionId & "\", " & -- sprintf("\"ms\": %d", ms) & -- "}") -- printf(1, "Status: %d\n", {res[1]}) -- printf(1, "Effective URL: %s\n", {res[2]}) -- analyze_object(res[3], "Headers", f_debug) -- analyze_object(res[4], "Content", f_debug) --</example> --<see_also>curl_easy_perform, curl_easy_perform_ex, curl_get, curl_post, curl_delete, curl_patch</see_also> --</function> sequence res atom pheaders log_puts("\nPUT " & object_dump(url) & " " & object_dump(headers) & " " & object_dump(body) & "\n") curl_easy_setopt(curl, CURLOPT_URL, url) pheaders = NULL for i = 1 to length(headers) do pheaders = curl_slist_append(pheaders, headers[i]) end for curl_easy_setopt(curl, CURLOPT_HTTPHEADER, pheaders) curl_easy_setopt(curl, CURLOPT_POST, 0) curl_easy_setopt(curl, CURLOPT_POSTFIELDS, body) curl_easy_setopt(curl, CURLOPT_CUSTOMREQUEST, "PUT") res = curl_send(curl) curl_slist_free_all(pheaders) return res end function
Here is the demo:
include get.e include lib/_curl_.e object res atom curl f_debug = 1 with_debug = 1 ------------------------------------------------------------------------------ -- common initialization ------------------------------------------------------------------------------ function init_curl_session() atom curl curl = curl_easy_init() log_printf("curl = %d\n", curl) if not curl then return 0 end if -- Proxy settings curl_easy_setopt(curl, CURLOPT_PROXYTYPE, CURLPROXY_HTTP) -- curl_easy_setopt(curl, CURLOPT_PROXY, "http://127.0.0.1:8888") -- ex: Fiddler2 proxy curl_easy_setopt(curl, CURLOPT_PROXY, "") -- no proxy -- Cookies settings -- beware of already registered cookies! -- if a connection fails, first remove or empty the cookie file -- before you verify your credentials if file_exists(InitialDir & SLASH & "cookies.txt") then write_file(InitialDir & SLASH & "cookies.txt", "") end if curl_easy_setopt(curl, CURLOPT_COOKIEFILE, InitialDir & SLASH & "cookies.txt") curl_easy_setopt(curl, CURLOPT_COOKIEJAR, InitialDir & SLASH & "cookies.txt") -- Common settings curl_easy_setopt(curl, CURLOPT_VERBOSE, 1) curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1) -- Skip Peer Verification curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 0) -- Skip Host Verification curl_easy_setopt(curl, CURLOPT_SSL_VERIFYHOST, 0) return curl end function -------------------------------------------------------------------------------- -- CURL POST - Beginner -------------------------------------------------------------------------------- res = curl_global_init(CURL_GLOBAL_DEFAULT) curl = init_curl_session() if curl then res = curl_post(curl, "https://jsonplaceholder.typicode.com/posts", {"Content-type: application/json; charset=UTF-8"}, "{ \"title\": \"foo\", \"body\": \"bar\", \"userId\": 1 }") if (res[HTTP_STATUS] < 200) or (res[HTTP_STATUS] > 226) then puts(1, "POST failed!\n" & res[HTTP_BODY] & "\n") end if puts(1, "Result:\n" & res[HTTP_BODY] & "\n") curl_easy_cleanup(curl) end if curl_global_cleanup() -------------------------------------------------------------------------------- -- CURL PUT - Beginner -------------------------------------------------------------------------------- res = curl_global_init(CURL_GLOBAL_DEFAULT) curl = init_curl_session() if curl then res = curl_put(curl, "https://jsonplaceholder.typicode.com/posts/1", {"Content-type: application/json; charset=UTF-8"}, "{ \"id\": 1, \"title\": \"foo\", \"body\": \"bar\", \"userId\": 1 }") if (res[HTTP_STATUS] < 200) or (res[HTTP_STATUS] > 226) then puts(1, "PUT failed!\n" & res[HTTP_BODY] & "\n") end if puts(1, "Result:\n" & res[HTTP_BODY] & "\n") curl_easy_cleanup(curl) end if curl_global_cleanup() -------------------------------------------------------------------------------- -- CURL PATCH - Beginner -------------------------------------------------------------------------------- res = curl_global_init(CURL_GLOBAL_DEFAULT) curl = init_curl_session() if curl then res = curl_patch(curl, "https://jsonplaceholder.typicode.com/posts/1", {"Content-type: application/json; charset=UTF-8"}, "{ \"title\": \"foo\" }") if (res[HTTP_STATUS] < 200) or (res[HTTP_STATUS] > 226) then puts(1, "PATCH failed!\n" & res[HTTP_BODY] & "\n") end if puts(1, "Result:\n" & res[HTTP_BODY] & "\n") curl_easy_cleanup(curl) end if curl_global_cleanup() -------------------------------------------------------------------------------- -- CURL GET - Beginner -------------------------------------------------------------------------------- void = curl_global_init(CURL_GLOBAL_DEFAULT) curl = init_curl_session() if curl then res = curl_get(curl, "https://jsonplaceholder.typicode.com/posts/1", {"Content-type: application/json; charset=UTF-8"}) if (res[HTTP_STATUS] < 200) or (res[HTTP_STATUS] > 226) then puts(1, "GET failed!\n" & res[HTTP_BODY] & "\n") end if puts(1, "Result:\n" & res[HTTP_BODY] & "\n") curl_easy_cleanup(curl) end if curl_global_cleanup() -------------------------------------------------------------------------------- -- CURL DELETE - Beginner -------------------------------------------------------------------------------- res = curl_global_init(CURL_GLOBAL_DEFAULT) curl = init_curl_session() if curl then res = curl_delete(curl, "https://jsonplaceholder.typicode.com/posts/1", {"Content-type: application/json; charset=UTF-8"}) if (res[HTTP_STATUS] < 200) or (res[HTTP_STATUS] > 226) then puts(1, "DELETE failed!\n" & res[HTTP_BODY] & "\n") end if puts(1, "Result:\n" & res[HTTP_BODY] & "\n") curl_easy_cleanup(curl) end if curl_global_cleanup() puts(1, "\nFinished\n") res = wait_key()
Jean-Marc
14. Re: Hitting an API with Euphoria
- Posted by jmduro Dec 24, 2017
- 1744 views
Sorry, it is not that easy. Between 1.0.5 and 1.0.9, I also added headers management in curl_get, curl_post, curl_patch and curl_delete:
Following lines have to be added in each function:
atom pheaders pheaders = NULL for i = 1 to length(headers) do pheaders = curl_slist_append(pheaders, headers[i]) end for curl_easy_setopt(curl, CURLOPT_HTTPHEADER, pheaders)
Functions declarations are changed as here:
global function curl_get(atom curl, sequence url, sequence headers) global function curl_post(atom curl, sequence url, sequence headers, object body) global function curl_put(atom curl, sequence url, sequence headers, object body) global function curl_delete(atom curl, sequence url, sequence headers) global function curl_patch(atom curl, sequence url, sequence headers, object body)
With Eu4, replace global by public.
15. Re: Hitting an API with Euphoria
- Posted by jimcbrown (admin) Dec 29, 2017
- 1682 views
No, sorry. If, however, you get something working in OE, that needs threads, I'll take a look.
It might still be an option to take a look (and maybe thread-ify it) if he gets something working in OE without threads, though.