1. Server-Sent Events
- Posted by CraigWelch May 08, 2017
- 3602 views
Server-Sent Events
I haven’t seen much discussion here about Server-Sent Events, which is not surprising given that the number of people doing CGI processing seems to be fairly small. There’s but one mention of this, by acEduardo in this thead .
I’ve been playing with this, successfully, to drive progress bars, and to populate log sections on web pages. I’m posting the results in case anyone else has a need for this function.
First, a few words on what it is. A short chronology, which many can skip.
- The first web pages were completely static. You entered a url, and back came a web page. Embedded in that page would be hyperlinks, images, video, sound … but a page was a page.
- With the development of CGI functionality, the ability to create the page on the fly allowed for the model in common use, including the page on which you’re reading this now, whereby the url included attributes describing how the page should be constructed. This typically involved looking up a database, and assembling the page with static elements and constructed elements based on processing the database results. As well as the base url, additional query options are sent as part of a GET or POST.
- The search bar on the right of this page is a good example, where a search for ‘CGI’ results in the construction of a url (using GET) such as: http://openeuphoria.org/search/results.wc?s=cgi&news=1&ticket=1&forum=1&wiki=1&manual=1 Everything after ‘?’ is used by the CGI program to determine the results. The individual queries are separated by ‘&’.
- One problem with this is that as page sizes got larger, the overhead in asking for and receiving a huge page to make a minor change became cumbersome. Javascript came to the rescue, with the AJAX function. Javascript is of course a programming language embedded in the HTML of the web page. An AJAX request is one that is made, to a url, on behalf of just one element of the page. This request will also involve query components.
- A good example of AJAX is when you start filling in a form field, and based on what you have typed the field is populated (or a drop-down is populated) with suggestions. After each keystroke, an AJAX request is sent which returns all appropriate words starting with that (or those) characters.
- In all of this the communications were initiated by the browser. The server could respond to any kind of request, but could never initiate a communication. To implement online chat, for example, in which case both parties send messages via a server, it’s necessary for the server to advise client B that there is a new message from client A. The traditional way has been by way of polling. Every interval the client page would, with an AJAX request, ask the server if there were any new data. This is certainly better than refreshing an entire page every few seconds, but the constant polling has its own undesirable overhead.
- That’s where Server-Set Events come in. A Javascript object is created that initiates a listener. When the server has something to say, it sends data to the listener, which has a function attached to it which will (perhaps after processing) put the new information somewhere on the web page. It might be the latest chat message. Or an updated stock ticker.
Implementation
Javascript
This is example code, updating a progress bar (“test”) as well as final status in a field (“Result3”). Changing the value in “test” will update the length of the progress bar.
{view:"button", id:"test_progress", value:"Test Progress", type:"form", autowidth:true, click: function(){ var source = new EventSource("./find_fix_tables.eui?action=test_progress"); source.onmessage = function(event) { .setValue(event.data * 1)}; // turn string into number source.onopen = function(event) { console.log("OPEN socket"); $ $("test").show(); $ $("Result3").setHTML("Processing")} source.onerror = function(event) { console.log("ERROR - CLOSING socket"); source.close(); $ $("test").hide(); $ $("Result3").setHTML("Completed")} } }, // I have had to write '$ $("test")' because if the two dollar signs are adjacent the // forum formatting code strips each such item out.
Euphoria
Here is the server code. For testing, it’s just a loop counting from 1 to 100, advising the client on each iteration.
Notes:
- The no-cache header, recommended in all the documentation for Server-Set Events, prevents this from working. It’s commented out.
- The usual Content-Type header, such as ‘text/html’, ’application/json’ etc. is now ‘text/event-stream’. This must be followed by two blank lines. With normal text headers, you need *at least* two, and can have more. In this case however you need *exactly* two.
- The data must be in the format ‘data:your_data’. In this case the loop counter is used, so the result is, form example ‘data: 5’. The Javascript receiver will parse only lines with ‘data: …’ and use the value of 5.
- Most CGI scripts written in Euphoria have the headers and final ‘puts()’ at the end of the program. The program then terminates. That serves to send the output to Apache or other web server. In this case, there is no termination, so it’s necessary to insert a flush() statement to ensure that the data are sent to Apache.
--******************************************** --* function test_progress() * --******************************************** function test_progress() --puts(1,"Cache-Control: no-cache") -- This header would prevent the function from working. puts(1,"Content-Type: text/event-stream\n\n") for i = 1 to 100 do sleep(0.1) puts(1, sprintf("data: %d \n\n", i)) -- must have the two newlines flush(1) end for puts(1, "Status: 415 OK") return {"Finished test"} end function
That’s it. It works. I’m happy to answer any questions or provide any help.
Craig
2. Re: Server-Sent Events
- Posted by ghaberek (admin) May 08, 2017
- 3606 views
--******************************************** --* function test_progress() * --******************************************** function test_progress() --puts(1,"Cache-Control: no-cache") -- This header would prevent the function from working. puts(1,"Content-Type: text/event-stream\n\n") for i = 1 to 100 do sleep(0.1) puts(1, sprintf("data: %d \n\n", i)) -- must have the two newlines flush(1) end for puts(1, "Status: 415 OK") return {"Finished test"} end function
I think the problem is that you're not ending your Cache-Control header with the expected "\r\n" terminator.
So this code:
puts(1,"Cache-Control: no-cache") puts(1,"Content-Type: text/event-stream\n\n")
Will output this text, which is incorrect:
Cache-Control: no-cacheContent-Type: text/event-stream\n\n
The correct code would look like this:
puts(1,"Cache-Control: no-cache\r\n") puts(1,"Content-Type: text/event-stream\r\n") puts(1,"\r\n")
And the output looks like this:
Cache-Control: no-cache\r\n Content-Type: text/event-stream\r\n \r\n
See: http://stackoverflow.com/a/5757349/2300395
-Greg
3. Re: Server-Sent Events
- Posted by CraigWelch May 08, 2017
- 3531 views
You're absolutely right.
I've been using "\n\n" with the "Content-Type" header for years. The earliest sample from Rob Craig had that. In the example above, the "Cache-Control" now works with "\r\n" but does not work with "\n\n" (as I had at first tried). However "\n\n" still works with the "Content-Type" header. However, for the sake of good practice (as per the StackOverflow article you pointed me to), it shall be "\r\n" for all my CGI from now on.
Many thanks,
Craig
4. Re: Server-Sent Events
- Posted by jmduro May 09, 2017
- 3433 views
That’s it. It works. I’m happy to answer any questions or provide any help.
I'm probably missing something cause Firefox allows me only one possibility: to save the source file. It does not execute as with other source code on the same server.
Here is my code:
include std/os.e include std/io.e --******************************************** --* function test_progress() * --******************************************** function test_progress() puts(1,"Cache-Control: no-cache\r\n") puts(1,"Content-Type: text/event-stream\r\n") puts(1,"\r\n") for i = 1 to 100 do sleep(0.1) puts(1, sprintf("data: %d \n\n", i)) -- must have the two newlines flush(1) end for puts(1, "Status: 415 OK") return {"Finished test"} end function test_progress()
Jean-Marc
5. Re: Server-Sent Events
- Posted by jmduro May 09, 2017
- 3421 views
Problem seems to be the content-type. Following code does not the job as expected but executes (EU4.1 on Linux Debian 64):
include std/os.e include std/io.e puts(1,"Cache-Control: no-cache\n") puts(1, "Content-Type: text/plain\n\n") -- puts(1,"Content-Type: text/event-stream\n\n") for i = 1 to 100 do sleep(0.1) puts(1, sprintf("data: %d \n\n", i)) -- must have the two newlines flush(1) end for puts(1, "Status: 415 OK")
Jean-Marc
6. Re: Server-Sent Events
- Posted by jmduro May 09, 2017
- 3413 views
It is not that simple: it works with Chrome 58 and not with Firefox ESR 45.9 (default on Debian Jessie).
7. Re: Server-Sent Events
- Posted by jmduro May 09, 2017
- 3409 views
Same behaviour with latest Firefox ESR 52.1 64 on Debian Jessie with EU4.1. Something must be missing to allow server-side events with Firefox that is installed by default in Chrome.
8. Re: Server-Sent Events
- Posted by CraigWelch May 09, 2017
- 3414 views
That's interesting - Firefox has apparently supported Server-Side Events since 6.0.
Does this thread or this one help?
Are you looking at the output with an empty browser, or do you have some Javascript code to establish a receiver? If so, can you post that code?
9. Re: Server-Sent Events
- Posted by jmduro May 10, 2017
- 3371 views
I'm just running an empty browser with code published above.
Both threads didn't help. EU code complies to the first one and as the script is not executed, I don't know how to send an event before it should run.
Jean-Marc
10. Re: Server-Sent Events
- Posted by acEduardo May 17, 2017
- 3248 views
I'd adapted a example I found in PHP, and it simply works.
sse.html:
<!DOCTYPE html> <html> <head> <title> SSE Example </title> <meta charset="utf-8" /> </head> <body> <script> var source = new EventSource('sse.ex'); source.onmessage = function(e) { document.body.innerHTML += e.data + '<br>'; }; </script> </body> </html>
sse.ex:
include std/datetime.e constant months = {"Jan","Feb","Mar","Apr","May","Jun","Jul","Aug","Sep","Oct","Nov","Dec"} constant days_week = {"Sun","Mon","Tue","Wed","Thu","Fri","Sat"} export enum year, month, day, hour, minute, second, weekday export function now() sequence _now = now_gmt() _now &= weeks_day(_now) return _now end function export function timestamp (sequence _date, sequence timezone = "UTC", integer offset = 0) integer i = 1 sequence weekday_name = days_week[_date[7]], month_name = months[_date[2]] return sprintf("%s, %02d %s %04d %02d:%02d:%02d %s", {weekday_name, _date[3], month_name, _date[1], _date[4], _date[5], _date[6], timezone} ) end function puts (1, "Content-Type: text/event-stream\r\n" & "Cache-Control: no-cache\r\n" & "\r\n") printf (1, "id: %d\n" & "data: %s\n", { time(), timestamp(now()) })
11. Re: Server-Sent Events
- Posted by jmduro May 18, 2017
- 3273 views
There must be something different between your configuration and mine: I get a blank page with only the right title on both Firefox and Google. I use Debian 8.6 64-bit with OpenEuphoria 4.1.0 beta 2 64-bit.
Jean-Marc
12. Re: Server-Sent Events
- Posted by jmduro May 18, 2017
- 3252 views
Maybe some Apache2 module?
Here are mine:
ls /etc/apache2/mods-enabled/ access_compat.load authn_core.load autoindex.load env.load negotiation.conf setenvif.conf actions.conf authn_file.load cgi.load filter.load negotiation.load setenvif.load actions.load authz_core.load deflate.conf mime.conf php5.conf status.conf alias.conf authz_host.load deflate.load mime.load php5.load status.load alias.load authz_user.load dir.conf mpm_prefork.conf reqtimeout.conf auth_basic.load autoindex.conf dir.load mpm_prefork.load reqtimeout.load
13. Re: Server-Sent Events
- Posted by jmduro May 18, 2017
- 3253 views
I get the same blank page on RaspberryPy 3.
14. Re: Server-Sent Events
- Posted by acEduardo May 22, 2017
- 3213 views
I'm using Lighttpd and running Euphoria scripts as CGI.
To put it to work: lighttpd -f ./config/lighttpd-something.conf -D
Example configuration (save it as lighttpd-something.conf):
server.modules = ( "mod_cgi", "mod_simple_vhost" ) server.document-root = "/home/aceduardo/apps/lighttpd/base/www/" server.port = 8080 server.bind = "127.0.0.1" server.pid-file = "/home/aceduardo/apps/lighttpd/running/server.pid" server.errorfile-prefix = "/home/aceduardo/apps/lighttpd/base/status/status-" server.upload-dirs = ( "/home/aceduardo/apps/lighttpd/base/uploads/" ) server.network-backend = "linux-sendfile" server.follow-symlink = "enable" dir-listing.activate = "enable" dir-listing.encoding = "utf-8" index-file.names = ( "index.xhtml", "index.html", "index.xml", "index.il", "index.ex", "index.pl", "index.cgi", "index.sh", ) simple-vhost.server-root = "/home/aceduardo/apps/lighttpd/base/" simple-vhost.default-host = "www" simple-vhost.document-root = "public" url.access-deny = ( "~", ".inc", ".cfg", ".err", ".log", ".e" ) cgi.assign = ( ".sh" => "/bin/sh", #-- Most relevant line: ".ex" => "/home/aceduardo/apps/euphoria41/bin/eui", #-- Shrouded Euphoria scripts as CGI, runs faster: ".il" => "/home/aceduardo/apps/euphoria41/bin/eub", ".pl" => "/usr/bin/perl", ".cgi" => "", )
15. Re: Server-Sent Events
- Posted by jmduro May 24, 2017
- 3507 views
No change, still blank page. It does not try to execute the script.
Here is my configuration:
root@raspberrypi:/home/pi# ls -lR apps/ apps/: total 4 drwxrwxrwx 4 pi pi 4096 mai 18 12:59 lighttpd apps/lighttpd: total 8 drwxrwxrwx 5 pi pi 4096 mai 18 13:00 base drwxrwxrwx 2 pi pi 4096 mai 18 13:16 running apps/lighttpd/base: total 12 drwxrwxrwx 2 pi pi 4096 mai 18 13:00 status drwxrwxrwx 2 pi pi 4096 mai 18 13:00 uploads drwxrwxrwx 2 pi pi 4096 mai 18 13:09 www apps/lighttpd/base/status: total 0 apps/lighttpd/base/uploads: total 0 apps/lighttpd/base/www: total 8 -rwxrwxrwx 1 root root 935 mai 12 16:02 sse.ex -rwxrwxrwx 1 root root 294 mai 12 16:01 sse.html apps/lighttpd/running: total 0 root@raspberrypi:/home/pi# cat ./config/lighttpd.conf server.modules = ( "mod_cgi", "mod_simple_vhost" ) server.document-root = "/home/pi/apps/lighttpd/base/www/" server.port = 80 server.bind = "127.0.0.1" server.pid-file = "/home/pi/apps/lighttpd/running/server.pid" server.errorfile-prefix = "/home/pi/apps/lighttpd/base/status/status-" server.upload-dirs = ( "/home/pi/apps/lighttpd/base/uploads/" ) server.network-backend = "linux-sendfile" server.follow-symlink = "enable" dir-listing.activate = "enable" dir-listing.encoding = "utf-8" index-file.names = ( "index.xhtml", "index.html", "index.xml", "index.il", "index.ex", "index.exu", "index.pl", "index.cgi", "index.sh", ) simple-vhost.server-root = "/home/pi/apps/lighttpd/base/" simple-vhost.default-host = "www" simple-vhost.document-root = "public" url.access-deny = ( "~", ".inc", ".cfg", ".err", ".log", ".e" ) cgi.assign = ( ".sh" => "/bin/sh", #-- Most relevant line: ".ex" => "/usr/local/euphoria-4.1.0-RaspberryPi/bin/eui", ".exu" => "/usr/local/euphoria-4.1.0-RaspberryPi/bin/eui", #-- Shrouded Euphoria scripts as CGI, runs faster: ".il" => "/usr/local/euphoria-4.1.0-RaspberryPi/bin/eub", ".pl" => "/usr/bin/perl", ".cgi" => "", ) root@raspberrypi:/home/pi# lighttpd -f ./config/lighttpd.conf -D 2017-05-18 13:19:23: (log.c.164) server started 2017-05-18 13:19:23: (server.c.1045) WARNING: unknown config-key: url.access-deny (ignored)
If I point directly to http://localhost/sse.ex, I can see an error in apps/lighttpd/base/www:
/home/pi/apps/lighttpd/base/www/sse.ex:1 <0052>:: can't find 'std/datetime.e' in any of ... /home/pi/apps/lighttpd/base/www /home/pi/apps/lighttpd/base/www/sse.ex include std/datetime.e ^
EUDIR is declared for root and for the user account (pi).
Jean-Marc
16. Re: Server-Sent Events
- Posted by jmduro May 24, 2017
- 3137 views
I guess the result of following command is decisive:
# grep EUDIR ~/.bashrc ~/.profile ~/.bash_profile ~/.bash_login /etc/profile /etc/environment /etc/bash.bashrc
Here is what I get:
/root/.bashrc:export EUDIR=/usr/local/euphoria-4.1.0-RaspberryPi grep: /root/.bash_profile: Aucun fichier ou dossier de ce type grep: /root/.bash_login: Aucun fichier ou dossier de ce type /etc/profile:EUDIR="/usr/local/euphoria-4.1.0-RaspberryPi" /etc/profile:export EUDIR /etc/environment:EUDIR=/usr/local/euphoria-4.1.0-RaspberryPi
Jean-Marc