1. Why some ...+ FASTER PARSING than 2.4 !
- Posted by Al Getz <Xaxo at aol.com>
Jan 01, 2006
-
Last edited Jan 02, 2006
Juergen Luethje wrote:
>
> Robert Craig wrote:
>
> <snip>
>
> > However, Euphoria is supposed to be a simple language.
> > I would not want to add this complexity to the front end,
> > plus complexity to the manual, as well as having newbies
> > constantly say "what are those extra files for?".
> > In theory it would be "transparent", but people would still
> > see the extra files lying around, and if anything went wrong,
> > they would wonder if they were really up-to-date.
> >
> > At some point, I'm going to look into shrouding of include
> > files, to get a .il that can be included. I'm not sure how
> > it will go, but I'd rather do that than complicate the
> > whole process of running the interpreter. I do not want to
> > introduce another type of intermediate file.
>
> I think this is actually the way to go! I hope it will be possible
> to create .il files that can be included (or linked or whatever).
> Very much appreciated!
>
> <snip>
>
> Regards,
> Juergen
>
> --
> Have you read a good program lately?
Hi Juergen and Rob,
Some interesting ideas here. I can start to see something like:
without parse (perhaps in il code)
at the beginning of a file to tell the Eu interpreter not to parse
the file, that it's an il file so it doesnt need it.
If there was enough info stored in the il file the Eu inter. could just
add it to the collection instead of having to first parse it.
This would mean HUGE programs could parse actually FASTER than the
old method (pre 2.5) (!!!!!) if they were built using several includes.
The only include file we'd have to modify was the one we were
working on (debugging) at the time, and we could always work it
so that the untested code was in, yes, a different include file than
the main program. Once debugged, stick it in with the rest by
specifying that it's pre-parsed.
I can see load times actually being FASTER than 2.4 was!
This would be really GREAT!
Heck, i wouldnt even mind
include preparsed MyFile.ew (optional word: 'include')
This way we'd have direct control over which files we want to debug
and those already done and still have everything load super fast!
If it turned out that we did have an error in a file that was
included as 'preparsed' then we'd just have to go back and include
the original file, but that's life...no big deal really. I guess
the error printout info would have to be good enough to point us
back to the original file line numbers also (as now).
Just some ideas...
Perhaps a statement:
with preparsing
to tell the interp to convert all the files into il types, and
store them (somewhere where we wont get current_dir() errors).
Then, if we want to debug a given file perhaps
without preparsing MyFile.ew
or something like that (a little more thought here required).
Think about this: If we had a program that was 1,000,000 lines
split up into as little as only *two* files file1.e (large) and file2.e
(small) and we wanted to debug the code in file2.e once file1.e was
preparsed imagine how fast this would load on the next run when
only having to parse the small file2.e ? WOW!
The problem i can see coming up is how each file gets it's current
directory...if they were put in another folder they would have to have
some way to get the current directory of the project exw file, not the
il files themselves. This shouldnt be too hard?
Take care,
Al
And, good luck with your Euphoria programming!
My bumper sticker: "I brake for LED's"
2. Re: Why some ...+ FASTER PARSING than 2.4 !
- Posted by Juergen Luethje <j.lue at gmx.de>
Jan 01, 2006
-
Last edited Jan 02, 2006
Al Getz wrote:
> Juergen Luethje wrote:
>>
>> Robert Craig wrote:
>>
>> <snip>
>>
>>> However, Euphoria is supposed to be a simple language.
>>> I would not want to add this complexity to the front end,
>>> plus complexity to the manual, as well as having newbies
>>> constantly say "what are those extra files for?".
>>> In theory it would be "transparent", but people would still
>>> see the extra files lying around, and if anything went wrong,
>>> they would wonder if they were really up-to-date.
>>>
>>> At some point, I'm going to look into shrouding of include
>>> files, to get a .il that can be included. I'm not sure how
>>> it will go, but I'd rather do that than complicate the
>>> whole process of running the interpreter. I do not want to
>>> introduce another type of intermediate file.
>>
>> I think this is actually the way to go! I hope it will be possible
>> to create .il files that can be included (or linked or whatever).
>> Very much appreciated!
>>
>> <snip>
>>
>
> Hi Juergen and Rob,
>
> Some interesting ideas here.
The idea -- and the desire -- to include .il files came up shortly after
Eu 2.5 was released. 
> I can start to see something like:
>
> without parse (perhaps in il code)
>
> at the beginning of a file to tell the Eu interpreter not to parse
> the file, that it's an il file so it doesnt need it.
> If there was enough info stored in the il file the Eu inter. could just
> add it to the collection instead of having to first parse it.
Using Eu 2.4, it is possible to include pre-parsed (shrouded) files
without the need of a special "with" clause. This is simple and
straightforward. I don't see an advantage when we would have to use such
a "with" clause.
> This would mean HUGE programs could parse actually FASTER than the
> old method (pre 2.5) (!!!!!) if they were built using several includes.
Yes, faster than the pre 2.5 method when plain source code text files
are included. However, at least with Eu 2.4 we also could/can include
pre-parsed (shrouded) files. I think not many people have done so,
though.
BTW: You can shroud include files (e.g. Win32lib) with Eu 2.4, and then
include these shrouded (= pre-parsed) files in your Eu 2.5 programs!!!
> The only include file we'd have to modify was the one we were
> working on (debugging) at the time, and we could always work it
> so that the untested code was in, yes, a different include file than
> the main program. Once debugged, stick it in with the rest by
> specifying that it's pre-parsed.
> I can see load times actually being FASTER than 2.4 was!
> This would be really GREAT!
>
> Heck, i wouldnt even mind
>
> include preparsed MyFile.ew (optional word: 'include')
I hope this will not be necessary. Euphoria should be smart enough to
detect itself, which files contain pre-parsed code.
> This way we'd have direct control over which files we want to debug
> and those already done and still have everything load super fast!
>
> If it turned out that we did have an error in a file that was
> included as 'preparsed' then we'd just have to go back and include
> the original file, but that's life...no big deal really.
Yep. Even Rob will probably not be able to create a DWIM
(Do-What-I-Mean / not what I write) interpreter or translator.
<snip>
Regards,
Juergen
--
Have you read a good program lately?
3. Re: Why some ...+ FASTER PARSING than 2.4 !
Juergen Luethje wrote:
> Using Eu 2.4, it is possible to include pre-parsed (shrouded) files
> without the need of a special "with" clause. This is simple and
> straightforward. I don't see an advantage when we would have to use such
> a "with" clause.
Yea, but wouldn't it be faster just to bind or shroud the entire program? I mean
shrouding is an seperate step that must be peformed on numerous source files
consecutively.
> Yes, faster than the pre 2.5 method when plain source code text files
> are included. However, at least with Eu 2.4 we also could/can include
> pre-parsed (shrouded) files. I think not many people have done so,
> though.
> BTW: You can shroud include files (e.g. Win32lib) with Eu 2.4, and then
> include these shrouded (= pre-parsed) files in your Eu 2.5 programs!!!
It would be faster to bind or shroud the entire program once, than to shroud 24
library files that would be included into your main program.
The smallest modification to them would require a full re-shroud step.
I think obfuscated IL (shrouded) include files could be useful when one wishes
to release partial source code to their project, but needs other proprietary
libraries to function. But then again, they could just build DLLs or SOs instead
that execute up to five times faster.
Regards,
Vincent
4. Re: Why some ...+ FASTER PARSING than 2.4 !
I wrote:
> It would be faster to bind or shroud the entire program once, than to shroud
> 24 library files that would be included into your main program.
> The smallest modification to them would require a full re-shroud step.
I take that back...
You could obviously make a single file that contains a list of include
statements then just shroud that; one could also do that for a DLL/SO.
Regards,
Vincent
5. Re: Why some ...+ FASTER PARSING than 2.4 !
On Sun, 01 Jan 2006 17:42:36 -0800, "Juergen Luethje"
<guest at RapidEuphoria.com> said:
> The idea -- and the desire -- to include .il files came up shortly
> after Eu 2.5 was released.
Sadly, I've still not figured that out. Phil Russel showed me how to
dump
a complete .il, but not a partial (re-locating pre-defined globals,
etc).
Matt Lewis, if you are listening, any thoughts?
Regards,
Pete
--
petelomax at fastmail.fm
--
http://www.fastmail.fm - A fast, anti-spam email service.
6. Re: Why some ...+ FASTER PARSING than 2.4 !
petelomax wrote:
>
> On Sun, 01 Jan 2006 17:42:36 -0800, "Juergen Luethje"
> <guest at RapidEuphoria.com> said:
> > The idea -- and the desire -- to include .il files came up shortly
> > after Eu 2.5 was released.
> Sadly, I've still not figured that out. Phil Russel showed me how to
> dump a complete .il, but not a partial (re-locating pre-defined globals,
> etc).
> Matt Lewis, if you are listening, any thoughts?
>
I haven't put any serious effort into this. I suspect that the challenges
aren't terribly technical, but probably more drudgery than anything else.
I think the most difficult would be remapping all the symbol references
from their old values to their new values in the actual il code for each
subprogram.
You could possibly save these as sequences instead of integers when you emit
il code (knowing that you're doing a partial shroud). That would make later
stuff easier, though I doubt this would work when translated from eu.ex to
Rob's binary format for il files.
Alternatively, you'd need to 'reparse' the il code to remap the symtab
references. This would require knowing which parts were symtab indices.
You'd also need to add everything to the hash table, and update other and
sundry bookkeeping items along the way. So it wouldn't be a zero parse time,
but probably much reduced.
I still wish that we could supply il code to the binder and let it produce
an il file, though I doubt Rob's changed his mind on the matter.
Matt Lewis
7. Re: Why some ...+ FASTER PARSING than 2.4 !
Juergen Luethje wrote:
>
> Al Getz wrote:
>
> > Juergen Luethje wrote:
> >>
> >> Robert Craig wrote:
> >>
> >> <snip>
> >>
> >>> However, Euphoria is supposed to be a simple language.
> >>> I would not want to add this complexity to the front end,
> >>> plus complexity to the manual, as well as having newbies
> >>> constantly say "what are those extra files for?".
> >>> In theory it would be "transparent", but people would still
> >>> see the extra files lying around, and if anything went wrong,
> >>> they would wonder if they were really up-to-date.
> >>>
> >>> At some point, I'm going to look into shrouding of include
> >>> files, to get a .il that can be included. I'm not sure how
> >>> it will go, but I'd rather do that than complicate the
> >>> whole process of running the interpreter. I do not want to
> >>> introduce another type of intermediate file.
> >>
> >> I think this is actually the way to go! I hope it will be possible
> >> to create .il files that can be included (or linked or whatever).
> >> Very much appreciated!
> >>
> >> <snip>
> >>
> >
> > Hi Juergen and Rob,
> >
> > Some interesting ideas here.
>
> The idea -- and the desire -- to include .il files came up shortly after
> Eu 2.5 was released.
>
> > I can start to see something like:
> >
> > without parse (perhaps in il code)
> >
> > at the beginning of a file to tell the Eu interpreter not to parse
> > the file, that it's an il file so it doesnt need it.
> > If there was enough info stored in the il file the Eu inter. could just
> > add it to the collection instead of having to first parse it.
>
> Using Eu 2.4, it is possible to include pre-parsed (shrouded) files
> without the need of a special "with" clause. This is simple and
> straightforward. I don't see an advantage when we would have to use such
> a "with" clause.
>
> > This would mean HUGE programs could parse actually FASTER than the
> > old method (pre 2.5) (!!!!!) if they were built using several includes.
>
> Yes, faster than the pre 2.5 method when plain source code text files
> are included. However, at least with Eu 2.4 we also could/can include
> pre-parsed (shrouded) files. I think not many people have done so,
> though.
> BTW: You can shroud include files (e.g. Win32lib) with Eu 2.4, and then
> include these shrouded (= pre-parsed) files in your Eu 2.5 programs!!!
>
> > The only include file we'd have to modify was the one we were
> > working on (debugging) at the time, and we could always work it
> > so that the untested code was in, yes, a different include file than
> > the main program. Once debugged, stick it in with the rest by
> > specifying that it's pre-parsed.
> > I can see load times actually being FASTER than 2.4 was!
> > This would be really GREAT!
> >
> > Heck, i wouldnt even mind
> >
> > include preparsed MyFile.ew (optional word: 'include')
>
> I hope this will not be necessary. Euphoria should be smart enough to
> detect itself, which files contain pre-parsed code.
>
> > This way we'd have direct control over which files we want to debug
> > and those already done and still have everything load super fast!
> >
> > If it turned out that we did have an error in a file that was
> > included as 'preparsed' then we'd just have to go back and include
> > the original file, but that's life...no big deal really.
>
> Yep. Even Rob will probably not be able to create a DWIM
> (Do-What-I-Mean / not what I write) interpreter or translator.
>
> <snip>
>
> Regards,
> Juergen
>
> --
> Have you read a good program lately?
Hi there,
It's just fine with me if Rob wants to make it all automatic, but
there must be a way to turn it off too when something doesnt work right:
without preparsing
perhaps?
Take care,
Al
And, good luck with your Euphoria programming!
My bumper sticker: "I brake for LED's"
8. Re: Why some ...+ FASTER PARSING than 2.4 !
- Posted by Vincent <darkvincentdude at yahoo.com>
Jan 03, 2006
-
Last edited Jan 04, 2006
Al Getz wrote:
> It's just fine with me if Rob wants to make it all automatic, but
> there must be a way to turn it off too when something doesnt work right:
> without preparsing
> perhaps?
Before you get to excited, lets face it... RDS is *not* going to change the way
Euphoria parses. We might get an additional small speed improvement in the next
release, but in general, Euphoria won't ever parse quickly again on older
systems.
I see nothing wrong with shrouded includes, but remember that you could use DLL
and SOs as an alternative that not only doesn't require parsing, but could
improve execution upto five times and have the possibility to work with other
programmming languages.
Regards,
Vincent
9. Re: Why some ...+ FASTER PARSING than 2.4 !
Al Getz wrote:
<big snip>
> It's just fine with me if Rob wants to make it all automatic, but
> there must be a way to turn it off too when something doesnt work right:
>
> without preparsing
>
> perhaps?
I see it this way:
Either I want to include the original source code
include mylib.e
or the corresponding .il file:
include mylib.il
When including 'mylib.il' causes a problem, then I'll include
'mylib.e' and look for the source of the problem.
Regards,
Juergen
10. Re: Why some ...+ FASTER PARSING than 2.4 !
Vincent wrote:
> Juergen Luethje wrote:
>
>> Using Eu 2.4, it is possible to include pre-parsed (shrouded) files
>> without the need of a special "with" clause. This is simple and
>> straightforward. I don't see an advantage when we would have to use such
>> a "with" clause.
>
> Yea, but wouldn't it be faster just to bind or shroud the entire program? I
> mean shrouding is an seperate step that must be peformed on numerous source
> files consecutively.
It depends on the situation.
E.g. when I'm developing a program that uses Win32lib (which is not
changed by me, and which takes a rather long time to parse), then it
is a good idea to include Win32lib in a pre-parsed format.
But my own program is often changed during development (and is probably
not as huge as Win32lib). So I don't want to include my own files in a
pre-parsed format while they are under developement.
>> Yes, faster than the pre 2.5 method when plain source code text files
>> are included. However, at least with Eu 2.4 we also could/can include
>> pre-parsed (shrouded) files. I think not many people have done so,
>> though.
>> BTW: You can shroud include files (e.g. Win32lib) with Eu 2.4, and then
>> include these shrouded (= pre-parsed) files in your Eu 2.5 programs!!!
>
> It would be faster to bind or shroud the entire program once, than to shroud
> 24 library files that would be included into your main program.
> The smallest modification to them would require a full re-shroud step.
--------------------------------------------------------------------------
In a subsequent post Vincent wrote:
> I take that back...
>
> You could obviously make a single file that contains a list of include
> statements
> then just shroud that; one could also do that for a DLL/SO.
--------------------------------------------------------------------------
> I think obfuscated IL (shrouded) include files could be useful when one wishes
> to release partial source code to their project, but needs other proprietary
> libraries to function. But then again, they could just build DLLs or SOs
> instead
> that execute up to five times faster.
Someone who has the binder but not the translator can *not* build DLLs
or SOs. And even if s/he has the translator, s/he might prefer to
release a bound program rather than a translated program, e.g. because
the first one will give more meaningful error messages. The programmer
should have the choice.
Regards,
Juergen
--
Have you read a good program lately?
