1. Benchmarking Fun
- Posted by _tom (admin) Sep 28, 2012
- 2653 views
Interpreter Benchmarking
Some benchmark results comparing Euphoria against some common interpreters. Results are in seconds which includes the startup time for each interpreter; lowest time is best.
| Benchmark | Argument | Euphoria Interpreter |
Euphoria Compiler |
Python | Perl |
|---|---|---|---|---|---|
| ackermann | 10 | 24.9 | 0.97 | 73.9 | 92.4 |
| binary trees | 18 | 301 | 52.0 | 536 | 636 |
| fannkuch | 10 | 18.6 | 3.06 | 65.0 | 62.3 |
| n-body | 500000 | 37.3 | 15.4 | 57.5 | 68.7 |
| eui version 4.04 | euc version 4.04 | python 2.6.5 | perl 5.10.1 |
I tested on a netbook using Mint Linux.
The source-code for these benchmarks is now in the wiki.
2. Re: Benchmarking Fun
- Posted by jimcbrown (admin) Sep 28, 2012
- 2593 views
Interpreter Benchmarking
Some benchmark results comparing Euphoria against some common interpreters. Results are in seconds which includes the startup time for each interpreter; lowest time is best.
| Benchmark | Argument | Euphoria Interpreter |
Euphoria Compiler |
Python | Perl |
|---|---|---|---|---|---|
| ackermann | 10 | 24.9 | 0.97 | 73.9 | 92.4 |
| binary trees | 18 | 301 | 52.0 | 536 | 636 |
| fannkuch | 10 | 18.6 | 3.06 | 65.0 | 62.3 |
| n-body | 500000 | 37.3 | 15.4 | 57.5 | 68.7 |
| eui version 4.04 | euc version 4.04 | python 2.6.5 | perl 5.10.1 |
I tested on a netbook using Mint Linux.
The source-code for these benchmarks is now in the wiki.
This is pretty cool. Keep up the good work!
3. Re: Benchmarking Fun
- Posted by _tom (admin) Sep 28, 2012
- 2603 views
For more benchmarks see the Archive. Jason Gade, with help from Derek Parnell, produced a suite of Euphoria benchmark programs.
4. Re: Benchmarking Fun
- Posted by useless_ Sep 28, 2012
- 2572 views
Those look impressive. Can you include PHP in the test results?
useless
5. Re: Benchmarking Fun
- Posted by euphoric (admin) Sep 28, 2012
- 2633 views
Can you include PHP in the test results?
I second this request.
6. Re: Benchmarking Fun
- Posted by _tom (admin) Sep 28, 2012
- 2618 views
I don't have PHP working on my netbook (yet).
For now take a look at http://shootout.alioth.debian.org/ to get an idea of the relative speeds of various languages.
In the past year I had two AMD64 computers melt down; I am now tired of buying computers. It is difficult to say how the my results (Atom N450) will compare with current computers.
7. Re: Benchmarking Fun
- Posted by Insolor Sep 29, 2012
- 2611 views
It would be interesting for me to see results of Lua and especially LuaJIT in this table.
8. Re: Benchmarking Fun
- Posted by _tom (admin) Oct 02, 2012
- 2430 views
Benchmarking Update
| Benchmark | Argument | Euphoria Interpreter 4.0.0 |
Lua | Python | Php | Ruby | Euphoria Compiler 4.0.0 |
LuaJIT |
|---|---|---|---|---|---|---|---|---|
| ackermann | 10 | 14.5 | 19.3 | 68.3 | 81.8 | crash | 0.8 | 1.65 |
| 8 | 0.94 | 1.21 | 3.58 | 4.64 | 12.8 | 0.06 | 0.1 | |
| binary | 18 | 81.9 | 340 | 505 | 1451 | 1186 | 43.2 | 67.6 |
| fann | 10 | 12.2 | 41.6 | 58.1 | 64.8 | 209 | 2.42 | 3.42 |
| n_body | 500000 | 24.1 | 18.3 | 52.3 | 39.7 | 178 | 13.4 | 1.25 |
| 4.0.0 | 5.1.4 | 2.6.5 | 5.3.2 | 1,8.7 | 4.0.0 | beta 2.0.0 |
Tested: netbook N450 Mint 9 Linux -- from a USB pendrive.
As before, do not place too much faith in the absolute numbers presented. You have to do tests on your own computer.
The USB pendrive used for my previous tests failed--pendrives do have a limited lifespan. Using another USB pendrive I started testing again. This drive has Euphoria 4.0.0 (actualy a developmental version) and shows that Eu4.0.4 is slower for some reason.
9. Re: Benchmarking Fun
- Posted by euphoric (admin) Oct 02, 2012
- 2432 views
OK, so Euphoria kicks everybody's butt. That's obvious. Except for LuaJIT! (Except for compiled Euphoria.) Nice job, LuaJIT. I liked using Lua when I used it. Seemed very adaptable to some "free form" coding...heheh.
The one quirk, though, is Lua's running of n_body! What happened there? 
Oh, and LuaJIT beats compiled Euphoria in the n_body as well? WHAT?!
10. Re: Benchmarking Fun
- Posted by Insolor Oct 03, 2012
- 2412 views
OK, so Euphoria kicks everybody's butt. That's obvious. Except for LuaJIT! (Except for compiled Euphoria.) Nice job, LuaJIT. I liked using Lua when I used it. Seemed very adaptable to some "free form" coding...heheh.
The one quirk, though, is Lua's running of n_body! What happened there?
Oh, and LuaJIT beats compiled Euphoria in the n_body as well? WHAT?!
JIT means Just-in-time compilation, i.e. during execution of a program actually the native machine code is executed, so it's not surprising that in the results of the benchmarks of LuaJIT approaches very close to results of the translated euphoria ones (and even beats in n_body benchmark).
I hope, some day we will see Euphoria JIT version beating even some statically compilable languages :)
As for specific time 13.4 of euc vs. 1.25 of LuaJIT, I think there still are things to be optimized in the Euphoria translator.
11. Re: Benchmarking Fun
- Posted by mattlewis (admin) Oct 03, 2012
- 2329 views
As for specific time 13.4 of euc vs. 1.25 of LuaJIT, I think there still are things to be optimized in the Euphoria translator.
And some things have been optimized since 4.0.0 (the version used for that test).
Matt
12. Re: Benchmarking Fun
- Posted by EUWX Oct 03, 2012
- 2298 views
As for specific time 13.4 of euc vs. 1.25 of LuaJIT, I think there still are things to be optimized in the Euphoria translator.
And some things have been optimized since 4.0.0 (the version used for that test).
Matt
The figures are very respectable and grossly in favour of Euphoria.
There must be something specific in n-body which is causing LUA JIT to outperform Euphoria to such a great extent.
However, Euphoria does out perform others even in this test.
13. Re: Benchmarking Fun
- Posted by EUWX Oct 03, 2012
- 2265 views
Benchmarking Update
| Benchmark | Argument | Euphoria Interpreter 4.0.0 |
Lua 5.1.4 |
Python 2.6.5 |
Php 5.3.2 |
Ruby 1,8.7 |
Euphoria Compiler 4.0.0 |
LuaJIT 2.0.0 |
|---|---|---|---|---|---|---|---|---|
| ackermann | 10 | 14.5 | 19.3 | 68.3 | 81.8 | crash | 0.8 | 1.65 |
| 8 | 0.94 | 1.21 | 3.58 | 4.64 | 12.8 | 0.06 | 0.1 | |
| binary | 18 | 81.9 | 340 | 505 | 1451 | 1186 | 43.2 | 67.6 |
| fann | 10 | 12.2 | 41.6 | 58.1 | 64.8 | 209 | 2.42 | 3.42 |
| n_body | 500000 | 24.1 | 18.3 | 52.3 | 39.7 | 178 | 13.4 | 1.25 |
| Average | Performance | 26.73 | 84.08 | 137.46 | 328.39 | 396.45 | 11.98 | 14.804 |
| Compare to | EU 4.0 Compiled | 44.8% | 14.2% | 8.7% | 3.6% | 3.0% | 100% | 80.9% |
| Compare to | LuaJIT 2.0 | 55.4% | 17.6% | 10.4 | 4.5 | 3.7% | 124.6% | 100% |
14. Re: Benchmarking Fun
- Posted by mattlewis (admin) Oct 05, 2012
- 2062 views
I played around a little with the nbody code. I sped it up a fair amount. See here.
I'm not motivated enough to post numbers, but I found that 4.0.4 was significantly faster than 3.1. 4.1 32-bits was about the same interpreted and translated. 64-bits was a little slower interpreted, but faster translated.
Matt
15. Re: Benchmarking Fun
- Posted by _tom (admin) Oct 05, 2012
- 2045 views
I played around a little with the nbody code. I sped it up a fair amount. See here.
I'm not motivated enough to post numbers, but I found that 4.0.4 was significantly faster than 3.1. 4.1 32-bits was about the same interpreted and translated. 64-bits was a little slower interpreted, but faster translated.
Matt
Using the same conditions as the previous tests nbody.ex has a time of 17.3 seconds.
The test is using 4.0.0 'developmental' (a historical accident due to having a handful of USB sticks with various editions of Euphoria on them.)
Tom
16. Re: Benchmarking Fun
- Posted by jimcbrown (admin) Oct 05, 2012
- 2066 views
I played around a little with the nbody code. I sped it up a fair amount. See here.
I'm not motivated enough to post numbers, but I found that 4.0.4 was significantly faster than 3.1. 4.1 32-bits was about the same interpreted and translated. 64-bits was a little slower interpreted, but faster translated.
Matt
Using the same conditions as the previous tests nbody.ex has a time of 17.3 seconds.
The test is using 4.0.0 'developmental' (a historical accident due to having a handful of USB sticks with various editions of Euphoria on them.)
Tom
That's the interpreter right?
Do you have a time for the translator / compiler?
17. Re: Benchmarking Fun
- Posted by _tom (admin) Oct 05, 2012
- 2067 views
Benchmarking Update
| Benchmark | Argument | Euphoria Interpreter 4.0.0 |
Lua 5.1.4 |
Python 2.6.5 |
Php 5.3.2 |
Ruby 1,8.7 |
Euphoria Compiler 4.0.0 |
LuaJIT 2.0.0 |
|---|---|---|---|---|---|---|---|---|
| ackermann | 10 | 14.5 | 19.3 | 68.3 | 81.8 | crash | 0.8 | 1.65 |
| 8 | 0.94 | 1.21 | 3.58 | 4.64 | 12.8 | 0.06 | 0.1 | |
| binary | 18 | 81.9 | 340 | 505 | 1451 | 1186 | 43.2 | 67.6 |
| fann | 10 | 12.2 | 41.6 | 58.1 | 64.8 | 209 | 2.42 | 3.42 |
| n_body Jason Gade |
500000 | 24.1 | 18.3 | 52.3 | 39.7 | 178 | 13.4 | 1.25 |
| n_body Matt Lewis |
500000 | 17.3 | 10.0 |
18. Re: Benchmarking Fun
- Posted by jaygade Oct 05, 2012
- 2040 views
Awesome. I don't remember the code any more, but I DO remember that n-body was the hardest one to translate in that I really REALLY wanted to find a way to do it while effectively showing off Euphoria's sequences.
If I remember correctly.
19. Re: Benchmarking Fun
- Posted by petelomax Oct 06, 2012
- 1987 views
I played around a little with the nbody code. I sped it up a fair amount. See here.
I'm not motivated enough to post numbers, but I found that 4.0.4 was significantly faster than 3.1. 4.1 32-bits was about the same interpreted and translated. 64-bits was a little slower interpreted, but faster translated.
Matt
Just tried it on Phix: 2.54s 
20. Re: Benchmarking Fun
- Posted by jaygade Oct 23, 2012
- 1579 views
Okay, on my system, Python is faster than my version of nbody, but Matt's version of nbody takes half the time that mine does (and is faster than Python). I can't explain that.
Running Linux Mint 13 on a Lenovo R61i Intel 1.83GHz Core 2 Duo 2GB RAM
Linux 3.2.0-23-generic #36-Ubuntu SMP Tue Apr 10 20:39:51 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux
Euphoria Interpreter v4.0.5 Linux, Using System Memory Revision Date: 2012-10-15, Id: 62d94559f849
Vs.
Python 2.7.3
$ for i in 1 2 3; do time <nbody command> 50000; done
| Command | real | user | sys |
| python nbody.py 50000 | 0m1.469s | 0m1.444s | 0m0.020s |
| 0m1.482s | 0m1.476s | 0m0.008s | |
| 0m1.547s | 0m1.520s | 0m0.020s | |
| eui nbody.ex 50000 | 0m1.836s | 0m1.776s | 0m0.056s |
| 0m1.815s | 0m1.728s | 0m0.080s | |
| 0m1.810s | 0m1.728s | 0m0.076s | |
| eui nbody-2.ex 50000 | 0m0.878s | 0m0.852s | 0m0.024s |
| 0m0.877s | 0m0.860s | 0m0.012s | |
| 0m0.876s | 0m0.864s | 0m0.004s |
Matt's is by far the fastest.
21. Re: Benchmarking Fun
- Posted by jaygade Oct 23, 2012
- 1571 views
I upped n to 500,000. All three took approximately 10x as long (of course). However, my own implementation came closer and even surpassed Python's.
Matt's was still about 1/3rd or so faster.
22. Re: Benchmarking Fun
- Posted by jaygade Oct 23, 2012
- 1536 views
Okay, now that I've compared the two, I see what you did.
I wanted to use Euphoria's sequence operations when I first wrote it, but I drew a total blank on how to transform the sequences.
