1. Hitting an API with Euphoria

Node.js has async functionality, so it's nice to use it to hit an API websocket and grab data while other processes continue.

I'm curious, could Euphoria be used to pull data from a web socket, keeping maybe a sequence populated with the latest data, while another part of the code utilizes that data?

Node.js has async/await, but is that really necessary for what I want to do?

I'm thinking about starting a pretty big project using Irv's GUI library and Eu4. Any input regarding reading web sockets would be much appreciated!

new topic     » topic index » view message » categorize

2. WellRe: Hitting an API with Euphoria

The possibility is there, but I don't know if Eu is capable executing really threaded code through 3 out of 4 cores,
in the mean time collecting your incoming data in a FIFO-pipe thru the remaining core.
That pipe ofc. can be approached at any moment you need to do so.
You even could use all 4 cores for processing the data, as long as the pipe it draws it data from isn't empty.
Just before that stage, set core* to collecting for a moment or 2.
Simple (binary) logic made me find the solution in abt 3 secs

new topic     » goto parent     » topic index » view message » categorize

3. Re: Hitting an API with Euphoria

Phix has proper multithreading

Pete

new topic     » goto parent     » topic index » view message » categorize

4. Re: Hitting an API with Euphoria

@Pete:

Hi Pete,

I know Phix has mutithreading, I really thoroughly studied the 'manual and help'.
Phix was running for a while on my Lenovo with Ubuntu 16.04.3 but it seems it is not fully compatible with OE 4.1 yet.
The speed of processing was really stunning.
Alas some OS-troubles threw me back and forced me to re-install, but with
the same result.Though I updated all necessary files I think, I probably have missed updating the only one that survives a reboot or restart.
I did put it in a directory of its own, making Euphoria a "daughter" of Phix, by putting that in /phix/euphoria, so $PATH
etc are targeting just one level too low for phix, but it will be found inside the path I did set.
Hopefully I will be able to solve this little problem soon, enabling me to have a real blazing start of 2018.
As said before, if in any way you could use my help (translations, o.s.a.), I can do that part in almost perfect German and
French as well,
Kind regards

Antoine

new topic     » goto parent     » topic index » view message » categorize

5. Re: Hitting an API with Euphoria

euphoric said...

Node.js has async functionality, so it's nice to use it to hit an API websocket and grab data while other processes continue.

Cool.

Euphoria has HTTP support, but not WebSockets. Doesn't look too hard to add, though: https://en.wikipedia.org/wiki/WebSocket

euphoric said...

I'm curious, could Euphoria be used to pull data from a web socket, keeping maybe a sequence populated with the latest data, while another part of the code utilizes that data?

Yes, the easiest way will be to use tasks.

euphoric said...

Node.js has async/await, but is that really necessary for what I want to do?

If you need to do other things while waiting on the data, then you will need nonblocking/async sockets. It's not necessary if you are ok with blocking waits (but few are). Even with tasks, a blocking call will cause all tasks to wait.

True multithreading would resolve this, but that's still experimental for Eu.

new topic     » goto parent     » topic index » view message » categorize

6. Re: Hitting an API with Euphoria

petelomax said...

Phix has proper multithreading

Pete, does Phix have websockets?

new topic     » goto parent     » topic index » view message » categorize

7. Re: Hitting an API with Euphoria

jimcbrown said...

Euphoria has HTTP support, but not WebSockets. Doesn't look too hard to add, though: https://en.wikipedia.org/wiki/WebSocket

So sockets is different from websockets? Dang. I was hoping it meant the same thing.

new topic     » goto parent     » topic index » view message » categorize

8. Re: Hitting an API with Euphoria

libwebsockets has "only" 177 functions. It should be possible to create a EuGTK plugin for that, in the same manner as the libwebview and libsourceview are done. The documentation is good.

That said, I have no idea how any of this would be used, except for the basic http stuff, so I am not the person to be doing this.

new topic     » goto parent     » topic index » view message » categorize

9. Re: Hitting an API with Euphoria

irv said...

libwebsockets has "only" 177 functions. It should be possible to create a EuGTK plugin for that, in the same manner as the libwebview and libsourceview are done. The documentation is good.

That said, I have no idea how any of this would be used, except for the basic http stuff, so I am not the person to be doing this.

You have to, Irv. You're my only hope!

Or Pete.

Or Greg.

Or Matt.

grin

new topic     » goto parent     » topic index » view message » categorize

10. Re: Hitting an API with Euphoria

euphoric said...

You have to, Irv. You're my only hope!

Eh, hope shmope. Here's a wrapper for librws. It's a lot smaller than libwebsockets but it doesn't seem to support HTTPS.

It uses a background thread to transfer data so (IMPORTANT) I have no idea how stable it will be on Euphoria. Good luck!

librws-x86.zip (12 KB)

-Greg

new topic     » goto parent     » topic index » view message » categorize

11. Re: Hitting an API with Euphoria

ghaberek said...

Here's a wrapper for librws. It's a lot smaller than libwebsockets but it doesn't seem to support HTTPS.

It uses a background thread to transfer data so (IMPORTANT) I have no idea how stable it will be on Euphoria. Good luck!

librws-x86.zip (12 KB)

I think I need HTTPS.

new topic     » goto parent     » topic index » view message » categorize

12. Re: Hitting an API with Euphoria

euphoric said...
petelomax said...

Phix has proper multithreading

Pete, does Phix have websockets?

No, sorry. If, however, you get something working in OE, that needs threads, I'll take a look.

Pete

new topic     » goto parent     » topic index » view message » categorize

13. Re: Hitting an API with Euphoria

In my last Standard library, I added a demo to show REST API management with libcurl. It is not yet ready for use. On published 1.0.5 version, it needs a little change in _curl_.e:

In curl_easy_setopt, the lines corresponding to slists have to be changed this way:

  ) then  -- struct curl_slist * 
    code = exec_func(xcurl_easy_setopt, {curl, option, param}, "+curl_easy_setopt") 

I also added a curl_put function:

global function curl_put(atom curl, sequence url, sequence headers, 
                         object body) 
--<function> 
--<name>curl_put</name> 
--<digest>sends a PUT request to an URL and gets the page</digest> 
--<desc> 
-- intended to be used with REST APIs 
--</desc> 
--<param> 
--<type>atom</type> 
--<name>handle</name> 
--<desc>CURL session handle</desc> 
--</param> 
--<param> 
--<type>sequence</type> 
--<name>url</name> 
--<desc>URL to get the page from</desc> 
--</param> 
--<param> 
--<type>sequence</type> 
--<name>headers</name> 
--<desc>list of request headers</desc> 
--</param> 
--<param> 
--<type>object</type> 
--<name>body</name> 
--<desc>body of the request. Either a string or NULL.</desc> 
--</param> 
--<return> 
-- sequence 
-- * status  : HTTP status 
-- * url     : effective URL (useful if redirection is followed) 
-- * headers : sequence of headers 
-- * content : HTML Page content 
--</return> 
--<example> 
-- constant DEFAULT_HEADERS = { 
--   "Host: 192.168.1.10", 
--   "User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0 " & 
--     "Gecko/20100101 Firefox/45.0", 
--   "Accept: application/json" 
-- } 
-- 
-- object res = curl_put(curl, MY_REST_API & "/session/" & sessionId & 
--                        "/timeouts/implicit_wait", DEFAULT_HEADERS, "{" & 
--                   "\"sessionId\": \"" & sessionId & "\", " & 
--                   sprintf("\"ms\": %d", ms) & 
--                 "}") 
-- printf(1, "Status: %d\n", {res[1]}) 
-- printf(1, "Effective URL: %s\n", {res[2]}) 
-- analyze_object(res[3], "Headers", f_debug) 
-- analyze_object(res[4], "Content", f_debug) 
--</example> 
--<see_also>curl_easy_perform, curl_easy_perform_ex, curl_get, curl_post, curl_delete, curl_patch</see_also> 
--</function> 
  sequence res 
  atom pheaders 
 
  log_puts("\nPUT " & object_dump(url) & " " & object_dump(headers) & " "  & 
           object_dump(body) & "\n") 
  curl_easy_setopt(curl, CURLOPT_URL, url) 
  pheaders = NULL 
  for i = 1 to length(headers) do 
    pheaders = curl_slist_append(pheaders, headers[i]) 
  end for 
  curl_easy_setopt(curl, CURLOPT_HTTPHEADER, pheaders) 
  curl_easy_setopt(curl, CURLOPT_POST, 0) 
  curl_easy_setopt(curl, CURLOPT_POSTFIELDS, body) 
  curl_easy_setopt(curl, CURLOPT_CUSTOMREQUEST, "PUT") 
 
  res = curl_send(curl) 
  curl_slist_free_all(pheaders) 
  return res 
end function 

Here is the demo:

include get.e 
include lib/_curl_.e 
 
object res 
atom curl 
 
f_debug = 1 
with_debug = 1 
 
------------------------------------------------------------------------------ 
-- common initialization 
------------------------------------------------------------------------------ 
 
function init_curl_session() 
  atom curl 
 
  curl = curl_easy_init() 
  log_printf("curl = %d\n", curl) 
  if not curl then return 0 end if 
 
  -- Proxy settings 
 
  curl_easy_setopt(curl, CURLOPT_PROXYTYPE, CURLPROXY_HTTP) 
--  curl_easy_setopt(curl, CURLOPT_PROXY, "http://127.0.0.1:8888")  -- ex: Fiddler2 proxy 
  curl_easy_setopt(curl, CURLOPT_PROXY, "")                         -- no proxy 
 
  -- Cookies settings 
 
  -- beware of already registered cookies! 
  -- if a connection fails, first remove or empty the cookie file 
  -- before you verify your credentials 
  if file_exists(InitialDir & SLASH & "cookies.txt") then 
    write_file(InitialDir & SLASH & "cookies.txt", "") 
  end if 
  curl_easy_setopt(curl, CURLOPT_COOKIEFILE, InitialDir & SLASH & "cookies.txt") 
  curl_easy_setopt(curl, CURLOPT_COOKIEJAR, InitialDir & SLASH & "cookies.txt") 
 
  -- Common settings 
 
  curl_easy_setopt(curl, CURLOPT_VERBOSE, 1) 
  curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1) 
 
  -- Skip Peer Verification 
  curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 0) 
 
  -- Skip Host Verification 
  curl_easy_setopt(curl, CURLOPT_SSL_VERIFYHOST, 0) 
 
  return curl 
end function 
 
-------------------------------------------------------------------------------- 
-- CURL POST - Beginner 
-------------------------------------------------------------------------------- 
 
res = curl_global_init(CURL_GLOBAL_DEFAULT) 
curl = init_curl_session() 
if curl then 
 
  res = curl_post(curl, "https://jsonplaceholder.typicode.com/posts", 
                  {"Content-type: application/json; charset=UTF-8"}, 
                  "{ \"title\": \"foo\", \"body\": \"bar\", \"userId\": 1 }") 
  if (res[HTTP_STATUS] < 200) or (res[HTTP_STATUS] > 226) then 
    puts(1, "POST failed!\n" & res[HTTP_BODY] & "\n") 
  end if 
  puts(1, "Result:\n" & res[HTTP_BODY] & "\n") 
 
  curl_easy_cleanup(curl) 
end if 
curl_global_cleanup() 
 
-------------------------------------------------------------------------------- 
-- CURL PUT - Beginner 
-------------------------------------------------------------------------------- 
 
res = curl_global_init(CURL_GLOBAL_DEFAULT) 
curl = init_curl_session() 
if curl then 
 
  res = curl_put(curl, "https://jsonplaceholder.typicode.com/posts/1", 
                 {"Content-type: application/json; charset=UTF-8"}, 
                 "{ \"id\": 1, \"title\": \"foo\", \"body\": \"bar\", \"userId\": 1 }") 
  if (res[HTTP_STATUS] < 200) or (res[HTTP_STATUS] > 226) then 
    puts(1, "PUT failed!\n" & res[HTTP_BODY] & "\n") 
  end if 
  puts(1, "Result:\n" & res[HTTP_BODY] & "\n") 
 
  curl_easy_cleanup(curl) 
end if 
curl_global_cleanup() 
 
-------------------------------------------------------------------------------- 
-- CURL PATCH - Beginner 
-------------------------------------------------------------------------------- 
 
res = curl_global_init(CURL_GLOBAL_DEFAULT) 
curl = init_curl_session() 
if curl then 
 
  res = curl_patch(curl, "https://jsonplaceholder.typicode.com/posts/1", 
                   {"Content-type: application/json; charset=UTF-8"}, 
                   "{ \"title\": \"foo\" }") 
  if (res[HTTP_STATUS] < 200) or (res[HTTP_STATUS] > 226) then 
    puts(1, "PATCH failed!\n" & res[HTTP_BODY] & "\n") 
  end if 
  puts(1, "Result:\n" & res[HTTP_BODY] & "\n") 
 
  curl_easy_cleanup(curl) 
end if 
curl_global_cleanup() 
 
-------------------------------------------------------------------------------- 
-- CURL GET - Beginner 
-------------------------------------------------------------------------------- 
 
void = curl_global_init(CURL_GLOBAL_DEFAULT) 
curl = init_curl_session() 
if curl then 
 
  res = curl_get(curl, "https://jsonplaceholder.typicode.com/posts/1", 
                 {"Content-type: application/json; charset=UTF-8"}) 
  if (res[HTTP_STATUS] < 200) or (res[HTTP_STATUS] > 226) then 
    puts(1, "GET failed!\n" & res[HTTP_BODY] & "\n") 
  end if 
  puts(1, "Result:\n" & res[HTTP_BODY] & "\n") 
 
  curl_easy_cleanup(curl) 
end if 
curl_global_cleanup() 
 
-------------------------------------------------------------------------------- 
-- CURL DELETE - Beginner 
-------------------------------------------------------------------------------- 
 
res = curl_global_init(CURL_GLOBAL_DEFAULT) 
curl = init_curl_session() 
if curl then 
 
  res = curl_delete(curl, "https://jsonplaceholder.typicode.com/posts/1", 
                    {"Content-type: application/json; charset=UTF-8"}) 
  if (res[HTTP_STATUS] < 200) or (res[HTTP_STATUS] > 226) then 
    puts(1, "DELETE failed!\n" & res[HTTP_BODY] & "\n") 
  end if 
  puts(1, "Result:\n" & res[HTTP_BODY] & "\n") 
 
  curl_easy_cleanup(curl) 
end if 
curl_global_cleanup() 
 
puts(1, "\nFinished\n") 
res = wait_key() 

Jean-Marc

new topic     » goto parent     » topic index » view message » categorize

14. Re: Hitting an API with Euphoria

Sorry, it is not that easy. Between 1.0.5 and 1.0.9, I also added headers management in curl_get, curl_post, curl_patch and curl_delete:

Following lines have to be added in each function:

  atom pheaders 
 
  pheaders = NULL 
  for i = 1 to length(headers) do 
    pheaders = curl_slist_append(pheaders, headers[i]) 
  end for 
  curl_easy_setopt(curl, CURLOPT_HTTPHEADER, pheaders) 

Functions declarations are changed as here:

global function curl_get(atom curl, sequence url, sequence headers) 
global function curl_post(atom curl, sequence url, sequence headers, object body) 
global function curl_put(atom curl, sequence url, sequence headers, object body) 
global function curl_delete(atom curl, sequence url, sequence headers) 
global function curl_patch(atom curl, sequence url, sequence headers, object body) 

With Eu4, replace global by public.

new topic     » goto parent     » topic index » view message » categorize

15. Re: Hitting an API with Euphoria

petelomax said...

No, sorry. If, however, you get something working in OE, that needs threads, I'll take a look.

It might still be an option to take a look (and maybe thread-ify it) if he gets something working in OE without threads, though.

new topic     » goto parent     » topic index » view message » categorize

Search



Quick Links

User menu

Not signed in.

Misc Menu