1. [phix] what is a hybrid interpreter compiler?

searching "hybrid interpreter compiler"

  • leads me to Phix
  • leads me to Python being 'hybrid' because it uses byte-code
  • misc odd hits

wikipedia has no page on this

so, how does Phix actually work? Is there any byte code involved as in Euphoria?

Is Phix part of an endless loop, where no one understands what a hybrid is?

be well
_tom

new topic     » topic index » view message » categorize

2. Re: [phix] what is a hybrid interpreter compiler?

First and foremost Phix is (internally) a compiled language:

source -> tokens -> il -> hardware-specific [x86/64] machine code -> executable file -> run (from disk). 

However, by default (no -c flag on the command line) it behaves as an interpreter:

source -> tokens -> il -> hardware-specific [x86/64] machine code -> run (in memory). 

Four points regarding interpretation:
1: It executes just(/almost) as fast as compiled
2: Obviously omitting the disk write step means it is actually faster
3: Writing an exe usually triggers an anti-virus scan/stall before it can be run.
4: In practice interpretation omits some optimisation steps performed when compiling.

The "il" (intermediate language/representation) used internally by Phix is like an AST (abstract syntax tree) but is actually a flat linear list (with self-contained links) for improved performance reasons.

The main optimisation omitted during interpretation is "constant/type propagation", for instance in

function lower(object s) 
    if atom(s) then --- (a single character) 
        ... 
    elsif string(s) then 
        ... -- [2] 
    else            -- (a nested sequence) 
        ... 
    end if 
end function 

the compiler analyses the il and might find that in all calls to lower, s is a string. It can then only emit the middle [2] block and does not even have to emit the string test. Obviously you end up with a smaller and faster executable, but on average the cost of that analysis is more than what you save at runtime, acceptable and still a good move for compilation [any loss dwarfed by disk io & av checks anyway], however (usually) simply not worthwhile for interpretation.

The message we want to deliver is 1: no waiting for the compiler (eg Go is often touted as a fast compiler, but I almost always end up drumming my fingers waiting for it to start), and 2: interpretation does not mean slow.

In "hybrid interpreter/compiler" I suppose hybrid really just means "both". There does seem to be a common association of "hybrid" with "jit", which Phix is not.

Erm, yeah, I guess this probably deserves a paragraph/section all of it's own.

PS No argument that Go has a blindingly fast runtime, way better than Phix, it's the compile-time I find tardy.

new topic     » goto parent     » topic index » view message » categorize

3. Re: [phix] what is a hybrid interpreter compiler?

Not sure this is the right place or time to ask this...

Maybe i am just in a "ok, this can be hacked!" mood...

Can Phix say "hybrid" means a second stand-alone program can be "interpreted" and stashed in ram, and then a task (from tasks.e) be pointed at it? The "why" is so if the 2nd program crashes, it doesn't take the main program down with it.

Maybe i am using the wrong words again.

Kat

new topic     » goto parent     » topic index » view message » categorize

4. Re: [phix] what is a hybrid interpreter compiler?

katsmeow said...

if the 2nd program crashes, it doesn't take the main program down with it.

Grand sweeping statements about this sort of thing would be so littered with ifs and buts and dos and donts if accurate they would be unreadable, but I'll give it a go.

It is much better and far easier to design a specific solution for a specific problem. I'll focus on "recover from crash".

Forget multi-tasking, if an individual task crashes it's game over, and try/catch won't help with infinite loops.

Multi-threading is slightly better, but it is still tricky to kill a crashed or looping thread (though restarting it should be easy enough).

You could perhaps write some tricky inline assembly to determine the machine address and size of some bit of code.

Neither multi-tasking nor multi-threading would have gained anything from getting the machine address anyway.

You cannot pass that machine address to a separate process and have it execute it. (Not since DOS 0.0.1 anyway)

You could peek the code and pass the whole lot, either as PIC (position independent code) or manually relocate it, very messy either way.

Far easier just to put the code in a separate process in the first place.

Communicating sequential processes is usually best. If you open Windows Task Manager and look at your browser (well at least a modern chrome-based one) you should see it has dozens of processes, so that if one page crashes it doesn't bring down the whole thing and it can put a message along with a restart button on the affected page.

That is all done with ipc, and/or OS-specific message passing for performance. Separate processes, code where it belongs, no machine addresses, tasks, or threads involved.

new topic     » goto parent     » topic index » view message » categorize

5. Re: [phix] what is a hybrid interpreter compiler?

[edited away for posting features no one wants]

Kat

new topic     » goto parent     » topic index » view message » categorize

6. Re: [phix] what is a hybrid interpreter compiler?

Is this a valid understanding?

Given some source-code:

As an ''interpreter''

  • Phix reads, parses, makes IL code, ...
    • errors are detected and reported in a programmer friendly way
  • Phix then compiles the IL and saves to ram
  • program then executes

As a ''compiler''

  • instead of compiling to ram, it saves to a file
  • you can run the program at any time (no need for Phix anymore)

be well
_tom

new topic     » goto parent     » topic index » view message » categorize

7. Re: [phix] what is a hybrid interpreter compiler?

_tom said...

Is this a valid understanding?

Almost. Forget about "IL" - almost every compiler has some form of it, and certainly any that produce efficient machine code.
But it sends the wrong message: some (slow) interpreters run IL on a VM, whereas Phix is more like C & libc (only asm-based).

As an ''interpreter''

  • Phix reads, parses, makes executable code, then executes it (directly from memory)

As a ''compiler''

  • as above but saves the executable code to a file, which it then executes[*]
  • you can run the program at any time (and ship it, no need for Phix anymore)

[*] unless -norun is specified on the command line

Note: interpretation is often noticably faster than compilation because writing an executable file usually wakes up an anti-virus program which then stalls proceeedings (for ~1s).
However, the last stage of compilation also performs some additional optimisation steps, so sometimes but not always that ends up faster.

_tom said...

errors are detected and reported in a programmer friendly way

That is not a separate step, errors can occur at any stage, for instance illegal characters in the read stage, and
non_existent_routine() is in fact only reported after the machine code has been generated, when it finds it still
present on some "todo" list. Mention it before or after, sure, but not as part of the interpreter/compiler distinction.

new topic     » goto parent     » topic index » view message » categorize

8. Re: [phix] what is a hybrid interpreter compiler?

p hello.exw 

Runs as expected.

$ p -c hello.exw 
sh: 1: /media/mycomputer/Seagate: not found 

It compiles but does not also run.

I keep my work on an external USB drive. Looks like some Linux security is keeping the ''hello'' from running.

From the internal hard-drive, this all works as advertised.

be well
_tom
Forked into: sh: 1: /media/mycomputer/Seagate: not found

new topic     » goto parent     » topic index » view message » categorize

9. Re: [phix] what is a hybrid interpreter compiler?

katsmeow said...

Not sure this is the right place or time to ask this...

Maybe i am just in a "ok, this can be hacked!" mood...

Can Phix say "hybrid" means a second stand-alone program can be "interpreted" and stashed in ram, and then a task (from tasks.e) be pointed at it? The "why" is so if the 2nd program crashes, it doesn't take the main program down with it.

Maybe i am using the wrong words again.

Kat

Makes perfect sense to me! In fact, I can see a trival OE implementation of this as follows:

First program creates the stash in ram.

First program creates a pipe and dumps stash from ram into it.

First program launches stub as a new process, passing it the pipe somehow (on nix, just passing the fd on the command line most likely, and the stub knows to grab it from there and how to open the pipe from then on).

Stub reads from the pipe and stashes it into ram again.

Stub does whatever magickery is required (e.g. mmap() to make allocated memory executable) and then makes a machine level call into the ram stash.

If the ram stash returns, stub exits.

If exchange of variables is desired while both proccesses are running, pass them through the pipe and have the processes signal each other when data is ready (on nix, using signal(8) to do so).

Tweak as desired (for example, replacing the pipe with some implementation of shared memory, changing process to native thread, using windoze API instead of nix stuff, etc).

petelomax said...

Grand sweeping statements about this sort of thing would be so littered with ifs and buts and dos and donts if accurate they would be unreadable, but I'll give it a go.

Seems quite readable to me!

katsmeow said...

The hybrid connection : a Phix global program running on a cluster could be running compiled and interpreted processes on multiple machines, different OSs, as one program.

Reminds me of how Plan9 allows one to specify the cpu that a program runs on .. which may be on a different box over the network - and of a different type (so one could mix Intel with MIPS and have them appear to run on the same computer).

katsmeow said...

I agree with all you said, but you seem to have missed i was referring to completely separate processes, not tasks as OE does tasks now.

and being it's a separate task, calling it a task seems to fit.

No disagreements from me on that point. That said, in situations like this, I like to make up a new term (or sometimes multiple new terms) which helps other people avoid confusion. (I'm still using the old word in my head with my own meaning, and have to "translate" it when I write.) For example, https://openeuphoria.org/forum/m/126054.wc

YMMV

katsmeow said...

When declaring the type, i preceed the native OE types which i want to share with the word "global", and the preprocessor takes care of the machinery behind the scenes.

I'd probably have introduced a new keyword like 'shared' or 'universal' as 'global is already a keyword with a specific meaning in OE/Phix. NABD.

katsmeow said...

I use IPC now (but toying with TCP across the lan) to communicate back and forth between processes, and i can share selected functions, proceedures, and variables. The RDS trace/debug console functions with each separate process as normal. But it feels like this performance would be better served if it was part of the language,

Nice!

I think, in addition to support from the language/compiler+standard language libraries, the best performance would be gained if the OS also provided explicit support for this scenario. (Admittedly, this gets trickier as you try to use more and more OSes.) And how cool would it be if we could shared these things, not just between computers or between OSes, but between different languages? That said, one has to start somewhere, and your suggestion is a good one to start with.

new topic     » goto parent     » topic index » view message » categorize

10. Re: [phix] what is a hybrid interpreter compiler?

jimcbrown said...
katsmeow said...

When declaring the type, i preceed the native OE types which i want to share with the word "global", and the preprocessor takes care of the machinery behind the scenes.

I'd probably have introduced a new keyword like 'shared' or 'universal' as 'global is already a keyword with a specific meaning in OE/Phix. NABD.

"Shared" was my first choice too.

jimcbrown said...
katsmeow said...

I use IPC now (but toying with TCP across the lan) to communicate back and forth between processes, and i can share selected functions, proceedures, and variables. The RDS trace/debug console functions with each separate process as normal. But it feels like this performance would be better served if it was part of the language,

Nice!

I think, in addition to support from the language/compiler+standard language libraries, the best performance would be gained if the OS also provided explicit support for this scenario. (Admittedly, this gets trickier as you try to use more and more OSes.) And how cool would it be if we could shared these things, not just between computers or between OSes, but between different languages? That said, one has to start somewhere, and your suggestion is a good one to start with.

Seriously? Over the many years here, i was only shot down for linking EU/OE to Mirc. Mirc gave me easy gui, easy windows event-driven and threaded access. Mirc was a ready irc interface to #Euphoria on irc, even after my irc.e was rejected and i deleted all that code. OE gave me speed and unlimited variable size. But i wrote a threaded web server in mirc before OE had http.e, and the server wasn't blocking.

Anyhow, i deleted all the "shared" variables code from the computers. I've been hoping Eu would greatly improve, become a shining light in the dark, for over 20 years now, with attempts to copy a few features here and there. It's simply not going to happen.

Kat

new topic     » goto parent     » topic index » view message » categorize

11. Re: [phix] what is a hybrid interpreter compiler?

katsmeow said...

I've been hoping Eu would greatly improve, become a shining light in the dark, for over 20 years now, with attempts to copy a few features here and there. It's simply not going to happen.

Have you ever dabbled in any other languages over that time? I've programmed in PHP, Node.js, and some C#, all of which (well, except PHP) that let me do something that (at the time) I couldn't do with Euphoria.

new topic     » goto parent     » topic index » view message » categorize

12. Re: [phix] what is a hybrid interpreter compiler?

euphoric said...
katsmeow said...

I've been hoping Eu would greatly improve, become a shining light in the dark, for over 20 years now, with attempts to copy a few features here and there. It's simply not going to happen.

Have you ever dabbled in any other languages over that time? I've programmed in PHP, Node.js, and some C#, all of which (well, except PHP) that let me do something that (at the time) I couldn't do with Euphoria.

Pascal for many years, PHP on a remote server, mirc of course. Different Basics starting in ~1978. Z80, 8035, 6502, 8086 machine code. Too many incompatable versions of Python. Lua and Scheme and Haskel made me concentrate more on the programming language than on the task. And whatever that was HP used on their industrial contollers. Also projects that used 100's of ttl and ecl per project, and IEEE-488/HPIB "language" back in the day.

Also... jimcbrown , the shared vars could have been used to let OE execute strings, which was roundly shot down more than once.

And it would have been a patch for compiled programs that use http.e, which blocks, and won't run compiled if tasks are used. So if the incoming data in http.e was assigned to a shared variable in http.e, it would automagically appear in a non-blocking way in the main program as http.e recieved it. I was/am plenty happy running Eu as interpreted or bound.

Kat

I pinned enough hope on Eu that the cluster i built, with spares, totalled 14 P4 desktops, stuffed with as much ram and hdds as they'd take. And gigabit lan. I replaced it with five i5 machines and a i7 a few years ago. I admit i still have two P4 runnng. I have not disassembled all the P4 yet, but there's 48 330GB hdds in a box under the table. It's all such a waste.

I'm in my 60's. No one is going to improve OE enough in the time i have left.

new topic     » goto parent     » topic index » view message » categorize

13. Re: [phix] what is a hybrid interpreter compiler?

katsmeow said...

Pascal for many years, PHP on a remote server, mirc of course. Different Basics starting in ~1978. Z80, 8035, 6502, 8086 machine code. Too many incompatable versions of Python. Lua and Scheme and Haskel made me concentrate more on the programming language than on the task. And whatever that was HP used on their industrial contollers. Also projects that used 100's of ttl and ecl per project, and IEEE-488/HPIB "language" back in the day.

I really liked Lua when I was playing around with it. I've not done much with Python because I always heard it was so slow and (when I was looking) it had no good GUI implementations or integrations. I'm sure much has changed since then.

katsmeow said...

Also... jimcbrown , the shared vars could have been used to let OE execute strings, which was roundly shot down more than once... I'm in my 60's. No one is going to improve OE enough in the time i have left.

Sounds like you need to bend Pete's ear and make some suggestions for Phix. With its in-built threads and possibility of integrating assembly, you might could approach your lofty dreams and whims. It would be interesting to see how far you could go.

Or... maybe... fork Euphoria and make it do what you want. That might be your penultimate pathway!

new topic     » goto parent     » topic index » view message » categorize

14. Re: [phix] what is a hybrid interpreter compiler?

katsmeow said...

"Shared" was my first choice too.

Well, they do say that great minds think alike...;)

katsmeow said...

Over the many years here, i was only shot down for linking EU/OE to Mirc.

I vaguely recall this, but my recollection was that this occured in the early aughts - before RDS EU became open source. (Back then, RobCraig wasn't accepting any code contributions or library changes from anyone - aside from minor bug fixes. The best you could hope for was to get something into the archives.) If that's right, then sadly, the stuff was already deleted by the time that the OE dev team first came together.

It is quite sad, as if you still had the code and were willing to share it, I could guarantee that your code would get a very different reception today.

katsmeow said...

Seriously?

Yes, seriously.

katsmeow said...

Mirc gave me easy gui, easy windows event-driven and threaded access. Mirc was a ready irc interface to #Euphoria on irc,

OE gave me speed and unlimited variable size

Sounds like a nice pairing.

katsmeow said...

But i wrote a threaded web server in mirc before OE had http.e, and the server wasn't blocking.

Sounds like OE was a little late to the party. We can already have non-blocking http in OE today, many years later. Still not threaded (unless you count Phix or the experimental branches).

katsmeow said...

Anyhow, i deleted all the "shared" variables code from the computers. even after my irc.e was rejected and i deleted all that code.

Someone reading this might wonder how mIRC is relevant to the earlier posts - basically, with all the things you claim to have been able to do with RDS Eu with the extra boost from mIRC, adding the shared variable stuff across processes, networks, and so on would have been so so much easier.

At this point in time I do not recollect that anyone aside from jeremy_c ever submitting an irc.e to the Eu dev team. Jeremy wasn't happy with his own irc.e from his EuChat project (though now I don't recollect why) and there weren't really any other contenders. If someone were to submit something now, I'm sure it'd be accepted, albeit with perhaps some modifications.

It's still an open ticket for OE to add a net/irc.e in, https://openeuphoria.org/ticket/357.wc

katsmeow said...

I've been hoping Eu would greatly improve, become a shining light in the dark, for over 20 years now, with attempts to copy a few features here and there. It's simply not going to happen.

I'm in my 60's. No one is going to improve OE enough in the time i have left.

Kat

I've come around to agreeing with you. Nowadays I have not enough free time to help out myself, and Greg can't go at it alone. (Not that I've given up on Greg, just that I think it's unreasonable to expect too much from him too fast unless he's able to get some help.)

I've got more hopes for Phix now (and I wouldn't mind if Phix took over as the official Euphorian dialect).

katsmeow said...

Also... jimcbrown , the shared vars could have been used to let OE execute strings,

Yes. I can see how one possible implementation might have worked - copy the strings into a temp file, execute it in a new process, and pass the results back. Instant REPL.

katsmeow said...

which was roundly shot down more than once.

Not by me. Remember, I set up an experimental branch to add support for IL execution and self-modifying code.

katsmeow said...

And it would have been a patch for compiled programs that use http.e, which blocks,

Actually, it's pretty trivial to make it non-blocking on nix.

katsmeow said...

and won't run compiled if tasks are used.

The issue IIRC was that compiling a shared library meant that tasks couldn't be used. And someone was trying to compile http.e into a shared library, so that was modified so it still used tasks (compiled or interpreted or bound) UNLESS it was compiled as a shared library - then it'd still work, just without tasks (since in that mode tasks were not available). So it's not an issue you'd hit unless you started using compiled Eu.

katsmeow said...

I was/am plenty happy running Eu as interpreted or bound.

It's not an issue you'd hit unless you started using compiled Eu.

katsmeow said...

So if the incoming data in http.e was assigned to a shared variable in http.e, it would automagically appear in a non-blocking way in the main program as http.e recieved it.

That's pretty neat!

katsmeow said...

I pinned enough hope on Eu that the cluster i built, with spares, totalled 14 P4 desktops, stuffed with as much ram and hdds as they'd take. And gigabit lan. I replaced it with five i5 machines and a i7 a few years ago. I admit i still have two P4 runnng. I have not disassembled all the P4 yet, but there's 48 330GB hdds in a box under the table. It's all such a waste.

Hmm, I don't get it. It still sounds like you could be using that cluster if you did one of:

1. Using Eu + mIRC to provide threads/non-blocking/shared vars

2. Phix, with it's native thread support

3. The experimental OE branches that I set up for threads and self-modifying code.

Perhaps it's just timing - just guessing here, but maybe the cluster was already taken apart by the time Phix supported threads and the experimental branches came out.

euphoric said...

Or... maybe... fork Euphoria and make it do what you want. That might be your penultimate pathway!

OE would be a poor choice due to the fact that so much of it is still written in C.

Phix, which is done in just assembly and .. Phix, would be a much better choice. It already has a lot of what was wanted, so the gap to close would also be much smaller.

new topic     » goto parent     » topic index » view message » categorize

Search



Quick Links

User menu

Not signed in.

Misc Menu