OpenEuphoria

2. Re: [phix] what is a hybrid interpreter compiler?

• Posted by petelomax Jan 28, 2021
• 1208 views

First and foremost Phix is (internally) a compiled language:

source -> tokens -> il -> hardware-specific [x86/64] machine code -> executable file -> run (from disk). 

However, by default (no -c flag on the command line) it behaves as an interpreter:

source -> tokens -> il -> hardware-specific [x86/64] machine code -> run (in memory). 

Four points regarding interpretation:
1: It executes just(/almost) as fast as compiled
2: Obviously omitting the disk write step means it is actually faster
3: Writing an exe usually triggers an anti-virus scan/stall before it can be run.
4: In practice interpretation omits some optimisation steps performed when compiling.

The "il" (intermediate language/representation) used internally by Phix is like an AST (abstract syntax tree) but is actually a flat linear list (with self-contained links) for improved performance reasons.

The main optimisation omitted during interpretation is "constant/type propagation", for instance in

function lower(object s) 
    if atom(s) then --- (a single character) 
        ... 
    elsif string(s) then 
        ... -- [2] 
    else            -- (a nested sequence) 
        ... 
    end if 
end function 

the compiler analyses the il and might find that in all calls to lower, s is a string. It can then only emit the middle [2] block and does not even have to emit the string test. Obviously you end up with a smaller and faster executable, but on average the cost of that analysis is more than what you save at runtime, acceptable and still a good move for compilation [any loss dwarfed by disk io & av checks anyway], however (usually) simply not worthwhile for interpretation.

The message we want to deliver is 1: no waiting for the compiler (eg Go is often touted as a fast compiler, but I almost always end up drumming my fingers waiting for it to start), and 2: interpretation does not mean slow.

In "hybrid interpreter/compiler" I suppose hybrid really just means "both". There does seem to be a common association of "hybrid" with "jit", which Phix is not.

Erm, yeah, I guess this probably deserves a paragraph/section all of it's own.

PS No argument that Go has a blindingly fast runtime, way better than Phix, it's the compile-time I find tardy.

4. Re: [phix] what is a hybrid interpreter compiler?

• Posted by petelomax Jan 28, 2021
• 1175 views

Grand sweeping statements about this sort of thing would be so littered with ifs and buts and dos and donts if accurate they would be unreadable, but I'll give it a go.

It is much better and far easier to design a specific solution for a specific problem. I'll focus on "recover from crash".

Forget multi-tasking, if an individual task crashes it's game over, and try/catch won't help with infinite loops.

Multi-threading is slightly better, but it is still tricky to kill a crashed or looping thread (though restarting it should be easy enough).

You could perhaps write some tricky inline assembly to determine the machine address and size of some bit of code.

Neither multi-tasking nor multi-threading would have gained anything from getting the machine address anyway.

You cannot pass that machine address to a separate process and have it execute it. (Not since DOS 0.0.1 anyway)

You could peek the code and pass the whole lot, either as PIC (position independent code) or manually relocate it, very messy either way.

Far easier just to put the code in a separate process in the first place.

Communicating sequential processes is usually best. If you open Windows Task Manager and look at your browser (well at least a modern chrome-based one) you should see it has dozens of processes, so that if one page crashes it doesn't bring down the whole thing and it can put a message along with a restart button on the affected page.

That is all done with ipc, and/or OS-specific message passing for performance. Separate processes, code where it belongs, no machine addresses, tasks, or threads involved.

6. Re: [phix] what is a hybrid interpreter compiler?

• Posted by _tom (admin) Jan 29, 2021
• 1133 views

Is this a valid understanding?

Given some source-code:

As an ''interpreter''

• Phix reads, parses, makes IL code, ...
  • errors are detected and reported in a programmer friendly way
• Phix then compiles the IL and saves to ram
• program then executes

As a ''compiler''

• instead of compiling to ram, it saves to a file
• you can run the program at any time (no need for Phix anymore)

be well
_tom

8. Re: [phix] what is a hybrid interpreter compiler?

• Posted by _tom (admin) Jan 30, 2021
• 1096 views

p hello.exw 

Runs as expected.

$ p -c hello.exw 
sh: 1: /media/mycomputer/Seagate: not found 

It compiles but does not also run.

I keep my work on an external USB drive. Looks like some Linux security is keeping the ''hello'' from running.

From the internal hard-drive, this all works as advertised.

be well
_tom
Forked into: sh: 1: /media/mycomputer/Seagate: not found

10. Re: [phix] what is a hybrid interpreter compiler?

• Posted by katsmeow Feb 01, 2021
• 1046 views

"Shared" was my first choice too.

Seriously? Over the many years here, i was only shot down for linking EU/OE to Mirc. Mirc gave me easy gui, easy windows event-driven and threaded access. Mirc was a ready irc interface to #Euphoria on irc, even after my irc.e was rejected and i deleted all that code. OE gave me speed and unlimited variable size. But i wrote a threaded web server in mirc before OE had http.e, and the server wasn't blocking.

Anyhow, i deleted all the "shared" variables code from the computers. I've been hoping Eu would greatly improve, become a shining light in the dark, for over 20 years now, with attempts to copy a few features here and there. It's simply not going to happen.

Kat

12. Re: [phix] what is a hybrid interpreter compiler?

• Posted by katsmeow Feb 01, 2021
• 1054 views

Pascal for many years, PHP on a remote server, mirc of course. Different Basics starting in ~1978. Z80, 8035, 6502, 8086 machine code. Too many incompatable versions of Python. Lua and Scheme and Haskel made me concentrate more on the programming language than on the task. And whatever that was HP used on their industrial contollers. Also projects that used 100's of ttl and ecl per project, and IEEE-488/HPIB "language" back in the day.

Also... jimcbrown , the shared vars could have been used to let OE execute strings, which was roundly shot down more than once.

And it would have been a patch for compiled programs that use http.e, which blocks, and won't run compiled if tasks are used. So if the incoming data in http.e was assigned to a shared variable in http.e, it would automagically appear in a non-blocking way in the main program as http.e recieved it. I was/am plenty happy running Eu as interpreted or bound.

Kat

I pinned enough hope on Eu that the cluster i built, with spares, totalled 14 P4 desktops, stuffed with as much ram and hdds as they'd take. And gigabit lan. I replaced it with five i5 machines and a i7 a few years ago. I admit i still have two P4 runnng. I have not disassembled all the P4 yet, but there's 48 330GB hdds in a box under the table. It's all such a waste.

I'm in my 60's. No one is going to improve OE enough in the time i have left.

14. Re: [phix] what is a hybrid interpreter compiler?

• Posted by jimcbrown (admin) Feb 07, 2021
• 1007 views

Well, they do say that great minds think alike...;)

I vaguely recall this, but my recollection was that this occured in the early aughts - before RDS EU became open source. (Back then, RobCraig wasn't accepting any code contributions or library changes from anyone - aside from minor bug fixes. The best you could hope for was to get something into the archives.) If that's right, then sadly, the stuff was already deleted by the time that the OE dev team first came together.

It is quite sad, as if you still had the code and were willing to share it, I could guarantee that your code would get a very different reception today.

Yes, seriously.

Sounds like a nice pairing.

Sounds like OE was a little late to the party. We can already have non-blocking http in OE today, many years later. Still not threaded (unless you count Phix or the experimental branches).

Someone reading this might wonder how mIRC is relevant to the earlier posts - basically, with all the things you claim to have been able to do with RDS Eu with the extra boost from mIRC, adding the shared variable stuff across processes, networks, and so on would have been so so much easier.

At this point in time I do not recollect that anyone aside from jeremy_c ever submitting an irc.e to the Eu dev team. Jeremy wasn't happy with his own irc.e from his EuChat project (though now I don't recollect why) and there weren't really any other contenders. If someone were to submit something now, I'm sure it'd be accepted, albeit with perhaps some modifications.

It's still an open ticket for OE to add a net/irc.e in, https://openeuphoria.org/ticket/357.wc

I've come around to agreeing with you. Nowadays I have not enough free time to help out myself, and Greg can't go at it alone. (Not that I've given up on Greg, just that I think it's unreasonable to expect too much from him too fast unless he's able to get some help.)

I've got more hopes for Phix now (and I wouldn't mind if Phix took over as the official Euphorian dialect).

Yes. I can see how one possible implementation might have worked - copy the strings into a temp file, execute it in a new process, and pass the results back. Instant REPL.

Not by me. Remember, I set up an experimental branch to add support for IL execution and self-modifying code.

Actually, it's pretty trivial to make it non-blocking on nix.

The issue IIRC was that compiling a shared library meant that tasks couldn't be used. And someone was trying to compile http.e into a shared library, so that was modified so it still used tasks (compiled or interpreted or bound) UNLESS it was compiled as a shared library - then it'd still work, just without tasks (since in that mode tasks were not available). So it's not an issue you'd hit unless you started using compiled Eu.

It's not an issue you'd hit unless you started using compiled Eu.

That's pretty neat!

Hmm, I don't get it. It still sounds like you could be using that cluster if you did one of:

1. Using Eu + mIRC to provide threads/non-blocking/shared vars

2. Phix, with it's native thread support

3. The experimental OE branches that I set up for threads and self-modifying code.

Perhaps it's just timing - just guessing here, but maybe the cluster was already taken apart by the time Phix supported threads and the experimental branches came out.

OE would be a poor choice due to the fact that so much of it is still written in C.

Phix, which is done in just assembly and .. Phix, would be a much better choice. It already has a lot of what was wanted, so the gap to close would also be much smaller.