1. Simple integer math benchmark

I know I'm preaching to the choir as it were here, but you might be interested in a little benchmarking I did recently.

I tested the following simple/naive/stupid prime number generator with a number of interpreters, and Euphoria came out looking very nice.

integer n, lim, k, p, pc 
 
pc  = 0 
n   = 1 
lim = 5000000 
 
while n < lim do 
    k = 3 
    p = 1 
    n = n + 2 
    while k * k <= n and p do 
        p = floor(n / k) * k != n 
        k = k + 2 
    end while 
    if p then 
        pc = pc + 1 
    end if 
end while 
? pc 

I adapted the above code to a number of interpreters, and here are the results:

prime.c         1.00 seconds - test program, compiled with gcc -O3 
Euphoria        8.27 seconds - openeuphoria.com 
gforth          8.56 seconds - complang.tuwien.ac.at/forth/gforth 
tinypas.c      15.82 seconds - 
tiny-c.c       17.10 seconds - iro.umontreal.ca/~felipe/IFT2030-Automne2002/Complements/tinyc.c 
toy2.c         17.64 seconds - QDepartment/Files/Ed-Davis-stuff/toy.c 
Pike           30.43 seconds - pike.lysator.liu.se 
jwillia basic  50.26 seconds - github.com/jwillia3/BASIC 
oc             66.44 seconds - exmortis.narod.ru/pcode_src/oc.zip 
atlast         77.08 seconds - fourmilab.ch/atlast 
NaaLaa         92.98 seconds - naalaa.com 
Lua           109.56 seconds - lua.org 
Perl          137.69 seconds - perl.org 
si            189.23 seconds - drdobbs.com/cpp/si-a-c-like-script-interpreter/184408141 
Yabasic       203.94 seconds - yabasic.de 
cint          217.31 seconds - root.cern.ch/drupal/content/cint 
Python        225.83 seconds - python.org 
Ruby          242.52 seconds - ruby-lang.org 
ch            294.08 seconds - softintegration.com 
LB Booster    533.65 seconds - bbcbasic.co.uk/lbb 
Little C      767.34 seconds - books.mcgraw-hill.com/downloads/products/0072121246/0072121246_code.zip 
PicoC         795.76 seconds - code.google.com/p/picoc 

Of course you can write a faster primer number generator, but that wasn't the goal. The goal was to get a (perhaps weak) sense of interpreter speed, compared to C, and a simple interpreter I'd written, regarding integer operations and looping.

  1. prime.c - Test program compiled with gcc -O3
  2. Euphoria - It rocks!
  3. gforth - Fast Forth interpreter
  4. tinypas.c - C translation of Jan van de Snepscheut's Pascal S. Compiles to a VM. The compiler does some simple peephole optimization.
  5. tiny-c - Marc Feeley's Tiny C. This is a tiny C subset VM compiler/interpreter. Modified to add the operators needed to run the benchmark.
  6. toy2.c - Slightly modified version of toy.c. The one listed at QDepartment takes 51.01 seconds.
  7. Pike - Neat C-like scripting language.
  8. jwillia basic- A really cool BASIC interpreter, all in one header file! Compiles to a VM.
  9. oc - A tiny C subset VM compiler/interpreter. Was as 1991 OCCC Winner. The unobfuscated version is actually quite readable.
  10. atlast - Forth interpreter
  11. NaaLaa - Cool language for making games.
  12. Lua - Popular scripting language.
  13. Perl - There is always more than one way to do it...
  14. si - A tiny C subset interpreter by Al Stevens, circa 1989. This is almost a pure interpreter, however it does tokenize the source before interpreting it.
  15. Yabasic - Nice basic interpreter - sadly, no longer developed as of 2013.
  16. cint - Full C interpreter.
  17. Python - Popular scripting language.
  18. Ruby - Popular scripting language.
  19. ch - Full C interpreter.
  20. LB Booster - Runs Liberty BASIC programs faster than Liberty BASIC.
  21. Little C - Herbert Schildt's - pure (infamous?) tiny C subset interpreter.
  22. PicoC - Large subset of C interpreter.

I actually did not realize that Euphoria was so fast and/or set out to show the same. I was testing the speed of a simple interpreter I'd written, and I wanted to see how it stacked up against a few other interpreters.

And finally, like all benchmarks, take this one with a grain of salt smile

new topic     » topic index » view message » categorize

2. Re: Simple integer math benchmark

One more detail. How does eui compare with euc?

_tom

new topic     » goto parent     » topic index » view message » categorize

3. Re: Simple integer math benchmark

_tom said...

One more detail. How does eui compare with euc?

_tom

4.77 seconds - for euc -wat -con and 1.95 seconds - for euc -gcc -con

new topic     » goto parent     » topic index » view message » categorize

4. Re: Simple integer math benchmark

ed_davis said...
_tom said...

One more detail. How does eui compare with euc?

_tom

4.77 seconds - for euc -wat -con and 1.95 seconds - for euc -gcc -con

And eui?

useless

new topic     » goto parent     » topic index » view message » categorize

5. Re: Simple integer math benchmark

What do they say... ?
There are lies, bloody lies and benchmarks! ;)
But thanks for the effort.
@admins - how about adding the benchmarks with the eui / watcom / gcc to the manual with a link on the homepage?
My son recently had an article published (he is a java programmer) where he says old school hand coding programming is dead,
I like to remind him that there is nothing like hand coding with a decent tool to get speed.
Pity there is no Java on that benchmark.

new topic     » goto parent     » topic index » view message » categorize

6. Re: Simple integer math benchmark

fizzpopsoft said...

What do they say... ?
There are lies, bloody lies and benchmarks! ;)
But thanks for the effort.

I did say take it with a grain of salt smile

fizzpopsoft said...

Pity there is no Java on that benchmark.

I can do something about that .....

class primes { 
    public static void main(String[] args) { 
        int n, lim; 
        int k; 
        int pc; 
        boolean is_prime; 
 
        pc = 0; 
        n = 1; 
        lim = 5000000; 
        while (n < lim) { 
            k=3; 
            is_prime=true; 
            n=n+2; 
            while ((k*k<=n) && (is_prime)) { 
                is_prime=n/k*k!=n; 
                k=k+2; 
            } 
            if (is_prime) { 
                pc = pc + 1; 
            } 
        } 
        System.out.print(pc); 
    } 
} 

1.48 seconds.

new topic     » goto parent     » topic index » view message » categorize

7. Re: Simple integer math benchmark

fizzpopsoft said...

What do they say... ?
There are lies, bloody lies and benchmarks! ;)
But thanks for the effort.

A benchmark is only meaningful if your write it yourself to test a particular idea. In this case testing a home-brew interpreter against a variety of others.

Otherwise its just fun to realize that O[ does very well.

fizzpopsoft said...

@admins - how about adding the benchmarks with the eui / watcom / gcc to the manual with a link on the homepage?

I added a link from the wiki page on benchmarking to this forum entry.

The updated user manual has a page on benchmarking results.

fizzpopsoft said...

My son recently had an article published (he is a java programmer) where he says old school hand coding programming is dead,
I like to remind him that there is nothing like hand coding with a decent tool to get speed.

I ran the benchmark on my netbook:

  • eui = 49.6 s
  • euc = 4.9 s

The obvious solution to getting a better benchmark score is to get a faster computer.

Or, I could use a less efficient interpreter and take much longer to execute the same program.

new topic     » goto parent     » topic index » view message » categorize

8. Re: Simple integer math benchmark

_tom said...

I ran the benchmark on my netbook:

  • eui = 49.6 s
  • euc = 4.9 s

I am very surprised Eu interpreted ran 10 times slower than euc. I don't recall that the spread was that wide in previous benchmarks.

useless

new topic     » goto parent     » topic index » view message » categorize

9. Re: Simple integer math benchmark

useless_ said...
ed_davis said...
_tom said...

One more detail. How does eui compare with euc?

_tom

4.77 seconds - for euc -wat -con and 1.95 seconds - for euc -gcc -con

And eui?

useless

eui was the interpreter used in the benchmark - 8.27 seconds.

new topic     » goto parent     » topic index » view message » categorize

10. Re: Simple integer math benchmark

Not wanting to short-change anyone, here are a couple of updates:

Lua and Ruby are _much_ faster if I make a couple of simple changes - no change in the actual algorithm (that would be cheating smile ).

My original Lua version was:

    pc = 0 
    n = 1 
    lim = 5000000 
    while n < lim do 
        k = 3 
        p = 1 
        n = n + 2 
        while k * k <= n and p do 
            p = math.floor(n / k) * k ~= n 
            k = k + 2 
        end 
        if p then 
            pc = pc + 1 
        end 
    end 
    print(pc) 
I changed it to:
    local pc = 0 
    local n = 1 
    local lim = 5000000 
    while n < lim do 
        local k = 3 
        local p = 1 
        ... 
    The rest is the same. 

And the Lua time goes from 105.56 seconds down to 38.16 seconds.

My original Ruby version was:

    include Math 
    pc = 0 
    n = 1 
    lim = 5000000 
    while n < lim do 
        k = 3 
        p = 1 
        n = n + 2 
        while k * k <= n and p do 
            p = (n / k).floor * k != n 
            k = k + 2 
        end 
        if p then 
            pc = pc + 1 
        end 
    end 
    puts pc 
I changed this line:
    p = (n / k).floor * k != n 
to:
    p = n.to_i / k.to_i * k != n 
And the Ruby time goes from 242.52 down to 43.38 seconds.

But Euphoria still beats them by a good margin. smile

new topic     » goto parent     » topic index » view message » categorize

11. Re: Simple integer math benchmark

fizzpopsoft said...

My son recently had an article published (he is a java programmer) where he says old school hand coding programming is dead,

Go on, give us a link then

new topic     » goto parent     » topic index » view message » categorize

12. Re: Simple integer math benchmark

Certainly.
http://www.it-online.co.za/2013/12/03/the-new-wave-of-software-development/

new topic     » goto parent     » topic index » view message » categorize

13. Re: Simple integer math benchmark

BTW,
I challenged him to Euphoria vs his favourite development engine, Talend.
We took about the same time to write the app, which was a simple database create/update etc,
and Euphoria was 10 percent quicker in the execution.
Talend generates Java via an impressive GUI. So yes I have to say if you are creating a significant app, maybe Euphoria isn't the fastest to develop, but if its something small you need to run fast, then I say Euphoria.

new topic     » goto parent     » topic index » view message » categorize

14. Re: Simple integer math benchmark

You cannot count on the bench mark having the same speed proportion among different machines. Generally speaking, you cannot even count on a set of x86 code being faster on all machines when faster on one.

new topic     » goto parent     » topic index » view message » categorize

15. Re: Simple integer math benchmark

Ed,

On my old (slow) i3 machine, eui (4.05) takes 17.879 seconds.

You might also want to benchmark against luajit, the optimizing, just-in-time compiler for lua. On my machine, it takes 3.395 seconds to run the improved lua code you provided.

I had earlier written some other code to compare eui, lua and luajit and found that luajit runs like a demon. And unlike euphoria, which can optimize for integers, lua (and luajit) only use floating point numbers internally (kind of like running with atoms exclusively under euphoria). Lua version 5.3, which is development right now, will offer the developer a choice of using integers or floating point numbers when appropriate.

The reason I mention all this is because I see lua and euphoria as being the closest languages "spiritually". They have many syntactic similarities, and euphoria's sequences and lua's tables act as the default native data structure. I have used both languages and am happy to use either of them when appropriate.

I also enjoyed your post as it mentions a few languages I had not heard of before. Being a polyglot and programming language collector, I now have some new ones to check out!

krg

new topic     » goto parent     » topic index » view message » categorize

Search



Quick Links

User menu

Not signed in.

Misc Menu