OpenEuphoria

2. Re: Well I did it! (variable_id, etc..)

• Posted by Greg Haberek <ghaberek at gmail.com> Nov 25, 2004
• 662 views

Sorry, scratch variable_type() for now, I've still got to work on that one...


On Thu, 25 Nov 2004 01:51:02 -0500, Greg Haberek <ghaberek at gmail.com> wrote:
> Here's the moment everyone's been waiting for:
> 
> I've alted eu.ex to include the following features:
> 
>      s = routine_name( i )   -- returns a routine name given an id
>      s = routine_list()      -- all routines registered with routine_id()
>      i = variable_id( s )    -- returns a unique variable id
>      s = variable_name( i )  -- returns a varable name given an id
>      s = variable_list()     -- all variables registerd with variable_id()
>      i = variable_type( i )  -- returns a variable's type given an id
>      o = read( i )           -- reads a variable's value
>      write( i, o )           -- writes a variable's value
> 
> I hope all of you are happy. I'm still looking into routine ids for
> built-in routines. Like Rob said, its a little difficult to implement.
> I'm not including source code at this point, just a translated
> executable and a demo program. I'll be updating these features with
> the beta and official releases of 2.5, or giving them to RDS.
> 
> Go here to download:
> http://www.merkur.000k2.com/euw.zip
> 
> 
> ~Greg
>

4. Re: Well I did it! (variable_id, etc..)

• Posted by "Kat" <gertie at visionsix.com> Nov 25, 2004
• 663 views

On 25 Nov 2004, at 1:51, Greg Haberek wrote:

> 
> Here's the moment everyone's been waiting for:
> 
> I've alted eu.ex to include the following features:
> 
>       s = routine_name( i )   -- returns a routine name given an id
>       s = routine_list()      -- all routines registered with routine_id()
>       i = variable_id( s )    -- returns a unique variable id
>       s = variable_name( i )  -- returns a varable name given an id
>       s = variable_list()     -- all variables registerd with variable_id() i
>       =
>       variable_type( i )  -- returns a variable's type given an id o = read( i
>       )
>                 -- reads a variable's value write( i, o )           -- writes
>                 a
>       variable's value
> 
> 
> I hope all of you are happy. I'm still looking into routine ids for
> built-in routines. Like Rob said, its a little difficult to implement.
> I'm not including source code at this point, just a translated
> executable and a demo program. I'll be updating these features with
> the beta and official releases of 2.5, or giving them to RDS.
> 
> Go here to download:
> http://www.merkur.000k2.com/euw.zip

That's nice, but we can't have them as an exe that runs at Eu's speed, and 
we can't have the source unless we bought what? How much slower is your 
interpreter now?

Kat

6. Re: Well I did it! (variable_id, etc..)

• Posted by Andy Serpa <ac at onehorseshy.com> Nov 25, 2004
• 675 views

Greg Haberek wrote:
> 
> > That's nice, but we can't have them as an exe that runs at Eu's speed, and
> > we can't have the source unless we bought what? How much slower is your
> > interpreter now?
> 
> I'll put out the source once I get all the features in that I'd like.
> As far as speed, well, it definitly needs to be run with the C
> backend.
> 
> sorting test - 10,000 random integers 0 - 999
> 
>     exw.exe -   0.020 seconds
>     euw.exe - 11.376 seconds
> 
> That's 568% slower!
> 
> It's my hope that since I've implemented these features, RDS may be
> more likely to use them, since I put in all the grunt work of thinking
> and all. Heck, the whole equals thing for sequences was a matter of
> changing one line. One line!!! Check this out:
> 
> execute.e: line 1139 in procedure opEQUALS_IFW()
>     if val[a] = val[b] then
> 
> change to:
>     if equal( val[a], val[b] ) then
> 
> That's it!!! I had to rack my brain figuring out the inner workings of
> the interpreter to put in the rest, but now its easy as pie. Rob's
> coding style isn't that bad once you get the hang of it.
> 

Someone needs to separate the PD front-end from the back-end so that the
front-end can just spit out an unshrouded .il byte-code file, and then the PD
back-end can read in this file and run it.  I will get to this eventually if no
one else does first.  Then we just have to create a new back-end that runs faster
than the PD back-end, or hope Rob "sees the light" and simply sells us the real
back-end (for a profit!).  I think I will try writing a new backend in Ocaml,
which shouldn't be too hard for the core of the language because Ocaml already
does everything Euphoria does (in a syntactically different way) and has garbage
collection etc.  (It also has some support for threads and other stuff people
have been looking for if you want to jump ship completely.)  It can run
interpreted, or as bytecode, or as native code, just a Eu can.  When compiled, it
is much faster than translated Eu (it approaches the speed of C for pure Ocaml,
of course it wouldn't run as fast as an Eu .il interpreter), so it might be
possible to make a halfway decent performing backend.  I'll probably just make a
mini-backend implementing the core at first to see how it performs...

8. Re: Well I did it! (variable_id, etc..)

• Posted by Robert Craig <rds at RapidEuphoria.com> Nov 26, 2004
• 652 views

Greg Haberek wrote:
> I'll put out the source once I get all the features in that I'd like.
> As far as speed, well, it definitly needs to be run with the C
> backend.
> 
> sorting test - 10,000 random integers 0 - 999
> 
>     exw.exe -   0.020 seconds
>     euw.exe - 11.376 seconds
> 
> That's 568% slower!

Try my new version of opASSIGN_SUBS...

procedure opASSIGN_SUBS() -- also ASSIGN_SUBS_CHECK, ASSIGN_SUBS_I
-- LHS single subscript and assignment
    object x, subs

    a = Code[pc+1]  -- the sequence
    b = Code[pc+2]  -- the subscript
    if sequence(val[b]) then
        RTFatal("subscript must be an atom\n(assigning to subscript of a sequenc
    end if
    c = Code[pc+3]  -- the RHS value
    x = val[a]
    lhs_check_subs(x, val[b])
    x = val[c]
    subs = val[b]
    val[a][subs] = x  -- single LHS subscript
    pc += 4
end procedure

I think you'll be pleasantly surprised.

Regards,
   Rob Craig
   Rapid Deployment Software
   http://www.RapidEuphoria.com

10. Re: Well I did it! (variable_id, etc..)

• Posted by Patrick Barnes <mrtrick at gmail.com> Nov 26, 2004
• 668 views

On Thu, 25 Nov 2004 21:21:25 -0500, Greg Haberek <ghaberek at gmail.com> wrote:
> P.S. I just added version() and now I'm working on a 'next' statement.
> I think, due to the internal working of for and while loops, I'll have
> to make 'next for' and 'next while' statements... or I need some way
> of determining what type of loop i'm in.

Well, what about if you are next'ing a for loop, you specify the
variable of the loop you mean?
I was thinking about this before, and it doesn't work too well for
while loops, because there is no loop variable...
That way, if there's no var, it's for a while loop, if there's a var,
it's for a for loop?

for a = 1 to 10 do
    for b = 1 to 200 do
         for c = 1 to 3 do
              if something(a,b,c) then
                    next c --skips to the next iteration in the 'c' loop
              end if
               dostuff(b,a*c)
              if something_else(a,b,c) then
                     next a  --skips to the next iteration in the 'a' loop
              end if
          end for
     end for
end for

Looks pretty intuitive to me!
-- 
MrTrick

12. Re: Well I did it! (variable_id, etc..)

• Posted by Patrick Barnes <mrtrick at gmail.com> Nov 26, 2004
• 653 views

On Thu, 25 Nov 2004 22:09:37 -0500, Greg Haberek <ghaberek at gmail.com> wrote:
> 
> > Looks pretty intuitive to me!
> 
> Um... kinda. In my opinion, the next statement should act like an exit
> statement, only instead of leaving the loop, it jumps back to the top
> and increments the counter when needed (for vs while). 

Well, that's what I meant. What did you think I meant?

>The problem is,
> I don't know if I'm in a for or a while loop. I wish I could explain
> it better, you'd understand if you've studied the source code as much
> as I have in the past few days... 

That's ok, my brain is twisted enough around the interprocess Windows
API right now.

>What you're offering would be nice to have, but it looks like it
could get a bit messy (internally, that is).

Well, lets see how it goes then! 
-- 
MrTrick

14. Re: Well I did it! (variable_id, etc..)

• Posted by Greg Haberek <ghaberek at gmail.com> Nov 26, 2004
• 665 views

I think I've got it figured out (I think...) To stick with current
keyword conventions, this is how next will operate:

    -- example 1:
    for i = 1 to 5 do
        if i = 4 then
            next    -- should skip 4
        end if
    end for

    -- example 2:
    while 1 do
        if some_condition then
            next -- skip the rest of the code
        end if
        -- do something else
    end while

This way next operates just like exit, except it goes to the top of
the loop, not the bottom. Give me a few days to get it worked out,
there's still a few parts of the interpreter I don't understand, which
is preventing me from getting everything working.

~Greg


On Fri, 26 Nov 2004 09:54:40 +0000, Pete Lomax
<petelomax at blueyonder.co.uk> wrote:
> 
> On Thu, 25 Nov 2004 22:09:37 -0500, Greg Haberek <ghaberek at gmail.com>
> 
> 
> wrote:
> 
> >
> >> Looks pretty intuitive to me!
> >
> >Um... kinda. In my opinion, the next statement should act like an exit
> >statement, only instead of leaving the loop, it jumps back to the top
> >and increments the counter when needed (for vs while). The problem is,
> >I don't know if I'm in a for or a while loop. I wish I could explain
> >it better, you'd understand if you've studied the source code as much
> >as I have in the past few days... What you're offering would be nice
> >to have, but it looks like it could get a bit messy (internally, that
> >is).
> >
> I was about to say that
>        next a  -- for loop
> might get confused with
>        next            -- while loop
>        a = ...
> 
> I don't know if this will help or not but you may have slightly more
> luck with next(a) since no valid euphoria statement begins with (
> 
> Pete
> 
> 
> 
> 
>

16. Re: Well I did it! (variable_id, etc..)

• Posted by Robert Craig <rds at RapidEuphoria.com> Nov 26, 2004
• 660 views

Andy Serpa wrote:
> 
> Greg Haberek wrote:
> > 
> > [sounds like Peter from Family Guy] "Holy friggin crap!!"
> > 
> > That's amazing. Here's a speed update:
> > 
> > sorting 10,000 random objects between 0 and 999
> > 
> > Old stats:
> >     exw: 0.020 seconds
> >     euw: 10.975 seconds (best run, some were 11+)
> > 
> > New stats:
> >     exw: 0.020 seconds
> >     euw: 0.281 seconds
> > 
> > I guess the interpreted interpreter isn't so slow after all!
> > 
> 
> Damn.
> 
> Explanation?  Sounds like a trick I ought to know.  I have often put in such
> "optimizations"
> and taken them back out because it actually made things slower.  And if you
> mess around
> with this much it goes back to being 10x slower.  What's the secret?

There's a very subtle, but in this case major, performance issue here.
I only found it by using profile_time. (It has nothing to do with
2.5. Any version of Euphoria would act the same.)

The new, fast version of the code is below.
I've added some comments with !!!, 
to show how I speeded it up:

procedure opASSIGN_SUBS() -- also ASSIGN_SUBS_CHECK, ASSIGN_SUBS_I
-- LHS single subscript and assignment
    object x, subs

    a = Code[pc+1]  -- the sequence
    b = Code[pc+2]  -- the subscript
    if sequence(val[b]) then
RTFatal("subscript must be an atom\n(assigning to subscript of a
        sequence)")
    end if
    c = Code[pc+3]  -- the RHS value
x = val[a]    -- !!! save val[a] in x, i.e. increments reference count on
    val[a]
    lhs_check_subs(x, val[b])  -- !!! pass x instead of val[a]
    x = val[c]  -- !!! x overwritten, removes a reference count on val[a]
    subs = val[b]
    val[a][subs] = x  -- val[a] now has one reference count, not two
    pc += 4
end procedure

In the original code I was passing val[a] as an argument, i.e.

   lhs_check_subs(val[a], val[b])

What it was actually doing internally is something like:

   temp1 = val[a]
   temp2 = val[b]
   lhs_check_subs(temp1, temp2)

later on when the statement

   val[a][subs] = x

was performed, there were always *two* reference counts
on val[a]. One from val itself, and a dangling reference 
from temp1, since temp1 was still pointing to val[a]. This means
that a full copy of val[a] had to be made in order to
safely change an element of it. i.e. without making a copy, we'd be
disturbing the value of temp1. val[a] is a 10000-element sequence. 
Ouch! 

Euphoria tries to re-use temps as soon as possible, and free their
references, but until a temp is overwritten with a new value, 
any reference count caused by that temp remains. In this case 
temp1 was not used again. Almost any kind of expression would 
have resulted in temp1 being overwritten with a new value, 
with the problematic reference count being removed.

I haven't seen a case as bad as this before, but
I've long considered doing an optimization where temps and vars
whose current value is not going to be used again could be
"erased" immediately, thus dropping the reference count on what they
are pointing to. To avoid problems, I'd set them to NOVALUE (or something), 
but if it was a var, you might sometimes see 
NOVALUE (or something) in ex.err or on the trace screen which 
could be very confusing, since they should really have a value.

Regards,
   Rob Craig
   Rapid Deployment Software
   http://www.RapidEuphoria.com

18. Re: Well I did it! (variable_id, etc..)

• Posted by Robert Craig <rds at RapidEuphoria.com> Nov 26, 2004
• 660 views

Andy Serpa wrote:
> 
> Robert Craig wrote:
> > 
> > What it was actually doing internally is something like:
> > 
> >    temp1 = val[a]
> >    temp2 = val[b]
> >    lhs_check_subs(temp1, temp2)
> > 
> 
> So I can understand, it uses these temps because the val is subscripted?  

Yes.

> And when
> you replace it with x it just uses x?  

Yes.

> Or does it then create a temp with a reference
> to x for the lhs_check_subs() call?
 
No temp will be used, but an extra ref will be put on the 
value of x during the call (in case the routine wants to modify the
value via it's parameter that receives the value of x).
It will be removed at the end of the call.
At the end of each routine call, all temps used by that routine
are "de-ref'd". The key thing here, is that I am able to
explicitly overwrite the value of x, and lose the ref that x
has to val[a]. When temp1 had a ref to val[a], I couldn't
explicitly kill that ref. I just had to hope that temp1
would be overwritten by some expression before I did the
LHS subscripting operation.

I wouldn't expect the average Euphoria user to be
concerned about internal reference counts. Usually
Euphoria does a reasonable job of handling these things.
This case is unusual. For instance, it wouldn't happen with a single
level of subscripting, because then temp1 would not be
required - val would be passed directly. Temps are typically needed
to hold expression results (val[a]), not copies of whole 
variables (val).

Regards,
   Rob Craig
   Rapid Deployment Software
   http://www.RapidEuphoria.com

20. Re: Well I did it! (variable_id, etc..)

• Posted by Greg Haberek <ghaberek at gmail.com> Nov 26, 2004
• 674 views
• Last edited Nov 27, 2004

> avoided it for so long. It's just plain messy, and that's now what he
> wants for the language.

sorry, i meant "..not what he wants..."