1. News7.ex now runs at Ludicrous Speed.
- Posted by useless_ Sep 19, 2012
- 1737 views
- Last edited Sep 20, 2012
News7.ex now runs at Ludicrous Speed.
By running profile on news7 vs news3, the main loop in news7 runs 350 times vs 17 times in news3, so there's lot more time there to insert other tasks, if you like. In http.e, there's time to poll socks.e over 2000 times, in news3 it was under 400 times (while that doesn't help with only 15 urls, let your imagination roam).
I removed the global quit variable, and now use a task to set the quit time. I then didn't schedule main app as a task. With the freed time, i added a task counter to the screen, and you can see how it rather instantly keeps up with the search_url tasks as they complete. I also applied the local task_list() Matt came up with yesterday.
In the http://designerthinking.com/OpenEuphoria/ path you'll also find eui.bat to run instead of eui.exe, this will open the dosbox to full screen immeadiately for you.
You'll want to get http.e too, it has the task_yield patch.
I'd be real interested to hear if this version runs any faster than news.ex or news3.ex for those of you with internet download speeds over 3Megabits.
Ok, people trying to dl the directory.., here's the urls all spelled out:
http://designerthinking.com/OpenEuphoria/news7.ex
http://designerthinking.com/OpenEuphoria/news3.ex
http://designerthinking.com/OpenEuphoria/http.e
http://designerthinking.com/OpenEuphoria/eui.bat
I swear, if you are on dialup, you may not get 15 urls by the time either news3 or news7 times out. In which case, you won't notice any speedup anyhow. Of course, you can increase the value of the timeout in either one too.
If you are on an OC fiber, news7 should blow away news3.
useless
2. Re: News7.ex now runs at Ludicrous Speed.
- Posted by mattlewis (admin) Sep 20, 2012
- 1621 views
If you are on an OC fiber, news7 should blow away news3.
I'm on FiOS, and news3 takes 4sec, news7 3sec, so roughly a 25% improvement.
I also noticed that you took out the call to clear_screen(), which I suppose doesn't matter on Windows, but does when you run it as a console program (on Windows or Linux).
Matt
3. Re: News7.ex now runs at Ludicrous Speed.
- Posted by useless_ Sep 20, 2012
- 1594 views
If you are on an OC fiber, news7 should blow away news3.
I'm on FiOS, and news3 takes 4sec, news7 3sec, so roughly a 25% improvement.
I also noticed that you took out the call to clear_screen(), which I suppose doesn't matter on Windows, but does when you run it as a console program (on Windows or Linux).
Matt
FiOS is Verison, i wouldn't trust anything they advertise. What was your download speed during the test? With over 2000 calls to socks.e to see "is there data yet?", news7 should have been much faster than news3, news7 is idle so much more of the time.
Yes, i removed clear_screen() from news, it wasn't in the timing frame, and the dosbox starts cleared anyhow. I did add clear_line(), which is in the timing frame in search_url().
useless
4. Re: News7.ex now runs at Ludicrous Speed.
- Posted by mattlewis (admin) Sep 20, 2012
- 1587 views
FiOS is Verison, i wouldn't trust anything they advertise. What was your download speed during the test? With over 2000 calls to socks.e to see "is there data yet?", news7 should have been much faster than news3, news7 is idle so much more of the time.
At speedtest.net, I just clocked at 9-12Mbps down and 22-37Mbps up. I've also got a VPN with a remote NX session going, plus whatever my wife and the TV box is doing (most of all that is download, so that's probably slightly underestimating my actual connection). I think we're supposed to have 35/35.
I haven't done any serious profiling of the code. I would assume that the remaining bottlenecks are waiting for the remote servers to respond.
Hmm...translating, I get a segfault:
0x000000000046589a in update_screen_string (s=0x6f97c0 "task 15: http://news.yahoo.com") at be_w.c:820 820 screen_image[line][col].ascii = s[i];
Somehow screen_line (and therefore line) has become equal to 896808. 
Yes, i removed clear_screen() from news, it wasn't in the timing frame, and the dosbox starts cleared anyhow. I did add clear_line(), which is in the timing frame in search_url().
Yeah...not a big deal, so I added a clear before running the program on the command line. Otherwise it starts writing all over whatever's already there in my console.
Matt
5. Re: News7.ex now runs at Ludicrous Speed.
- Posted by useless_ Sep 20, 2012
- 1593 views
I haven't done any serious profiling of the code. I would assume that the remaining bottlenecks are waiting for the remote servers to respond.
I tried to detect that here in various ways, but my connection is slow enough, and this computer isn't a speed demon, that there wasn't a useful time-space big enough between sending the http req and getting back data. Of course, it could be that's because those OS calls are blocking, and cannot return to Eu until Microsoft says so. A call to task_delay(.01) after opening the socket didn't kill the program, and after recieving zero data, but did make it take much longer to run. I don't know if that's a windoze or tasks.e issue. My thinking was: there's no data yet, or there cannot possibly be data yet, so take a 10ms timeout before banging on winsock again. I tried various delay values.
Hmm...translating, I get a segfault:
0x000000000046589a in update_screen_string (s=0x6f97c0 "task 15: http://news.yahoo.com") at be_w.c:820 820 screen_image[line][col].ascii = s[i];
Somehow screen_line (and therefore line) has become equal to 896808.
I'll take a guess at atom-to-int conversion? I'm so glad i could help! 
useless
6. Re: News7.ex now runs at Ludicrous Speed.
- Posted by mattlewis (admin) Sep 20, 2012
- 1580 views
Hmm...translating, I get a segfault:
0x000000000046589a in update_screen_string (s=0x6f97c0 "task 15: http://news.yahoo.com") at be_w.c:820 820 screen_image[line][col].ascii = s[i];
Somehow screen_line (and therefore line) has become equal to 896808.
I'll take a guess at atom-to-int conversion? I'm so glad i could help! [/quote]
Close. The prototype for Position() in euphoria.h is (int, int). That's OK for 32-bits. But an object is bigger than an int on 64-bits.
Matt
7. Re: News7.ex now runs at Ludicrous Speed.
- Posted by CoJaBo2 Sep 22, 2012
- 1516 views
I'd be real interested to hear if this version runs any faster than news.ex or news3.ex for those of you with internet download speeds over 3Megabits.
Mostly just because I had it available...
$ wget 'http://cachefly.cachefly.net/100mb.test'
--2012-09-22 21:42:53-- http://cachefly.cachefly.net/100mb.test
Resolving cachefly.cachefly.net (cachefly.cachefly.net)... 205.234.175.175
Connecting to cachefly.cachefly.net (cachefly.cachefly.net)|205.234.175.175|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 104857600 (100M) [application/octet-stream]
Saving to: `100mb.test'
100%[==========================================================================================================================================>] 104,857,600 58.3M/s in 1.7s
2012-09-22 21:42:54 (58.3 MB/s) - `100mb.test' saved [104857600/104857600]
$ clear; yes | time ./bin/eui news3.ex
Looking for lines containing "Libya"
task 1: http://www.cbc.ca/news/
found 15 instances
task 2: http://www.juancole.com
found 90 instances
task 3: http://www.abc.net.au
found 0 instances
task 4: http://abcnews.go.com
found 2 instances
task 5: http://www.aljazeera.com
found 10 instances
task 6: http://www.bbc.co.uk/news/
found 5 instances
task 7: http://www.cbsnews.com
found 20 instances
task 8: http://www.cnn.com
found 7 instances
task 9: http://www.democracynow.org
found 2 instances
task 10: http://www.foxnews.com
found 10 instances
task 11: http://www.guardiannews.com
found 4 instances
task 12: http://www.nbcnews.com
found 2 instances
task 13: http://www.reuters.com
found 3 instances
task 14: http://whatreallyhappened.com
found 13 instances
task 15: http://news.yahoo.com
found 20 instances
all done in 2 seconds
All Done.
Press [enter] key to exit
1.64user 1.09system 0:03.80elapsed 71%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+8outputs (0major+16854minor)pagefaults 0swaps
$ clear; yes | time ./bin/eui news7.ex
Looking for lines containing "Libya"
Timeout is set to 24 seconds.
task 1: http://www.cbc.ca/news/
found 15 instances
task 2: http://www.juancole.com
found 90 instances
task 3: http://www.abc.net.au
found 0 instances
task 4: http://abcnews.go.com
found 2 instances
task 5: http://www.aljazeera.com
found 10 instances
task 6: http://www.bbc.co.uk/news/
found 5 instances
task 7: http://www.cbsnews.com
found 20 instances
task 8: http://www.cnn.com
found 7 instances
task 9: http://www.democracynow.org
found 2 instances
task 10: http://www.foxnews.com
found 10 instances
task 11: http://www.guardiannews.com
found 4 instances
task 12: http://www.nbcnews.com
found 2 instances
task 13: http://www.reuters.com
found 3 instances
task 14: http://whatreallyhappened.com
found 13 instances
task 15: http://news.yahoo.com
found 20 instances
SubTasks running right now: 0
All urls searched in: 1 seconds.
All Done.
Press Any Key to quit...
1.44user 0.72system 0:03.45elapsed 62%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+8outputs (0major+16828minor)pagefaults 0swaps
There is about a 2.5 second lag before anything is even printed; after that it takes about 1-2 sec to run. Is this normal?
8. Re: News7.ex now runs at Ludicrous Speed.
- Posted by useless_ Sep 23, 2012
- 1469 views
I'd be real interested to hear if this version runs any faster than news.ex or news3.ex for those of you with internet download speeds over 3Megabits.
Mostly just because I had it available...
$ wget 'http://cachefly.cachefly.net/100mb.test' <snip> 104,857,600 58.3M/s in 1.7s <snip> $ clear; yes | time ./bin/eui news3.ex Looking for lines containing "Libya" <snip> all done in 2 seconds All Done. Press [enter] key to exit 1.64user 1.09system 0:03.80elapsed 71%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+8outputs (0major+16854minor)pagefaults 0swaps ------------------------------------------------------ $ clear; yes | time ./bin/eui news7.ex Looking for lines containing "Libya" <snip> All urls searched in: 1 seconds. All Done. Press Any Key to quit... 1.44user 0.72system 0:03.45elapsed 62%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+8outputs (0major+16828minor)pagefaults 0swaps
So news7 used less cpu and still finished faster, cool.
There is about a 2.5 second lag before anything is even printed; after that it takes about 1-2 sec to run. Is this normal?
Seems to be that startup lag on all Eu 4 apps. I now put a flag of some sort in all my apps to let me know when the app i wrote is actually running, as opposed to starting up or hung. the flag beats sitting there staring at the screen saying "is it running, is it hung, is it running, it hung, did it really hang?..."
Thanks Cojabo.
useless
