1. Multiple interpreter instances
- Posted by ryanj Mar 01, 2015
- 3455 views
Forked from Re: string_exec()
Unless we're talking about this as something sandboxed away from the current program.
That's the easiest version to implement though. Just do it eueval-style (aka turning eu.ex into a library). (And even here, eueval5.0 could easily be modified to recreate and throw actual exceptions to the caller, if a fatal or uncaught exception was detected in the simulated code.)
From my point of view this is the only way to implement!
Eval in the current program is something i do not want to even think about.
(eval needs to live in a Sandbox and nowhere else!)
Thank you
Andreas
I have had an idea several years ago. What if the euphoria "backend" was able to run in multiple instances, each instance in a different thread? I need to study the internal working of OE more, but i believe it would require converting all global state data to be put inside arrays/sequences and add a manager that can create and destroy instances, each with it's own index/id number, and a sharing interface which all instances can access, with some sort of semaphore mechanism. This would open up some interesting possibilities:
- Multi-threading and blocking I/O would be possible, because a new thread would actually be a different interpreter, completely independent.
- Running scripts or generating code on-the-fly would be possible, by simply starting a new interpreter instance.
- Processing data with multiple cores would be possible, by starting multiple instances of the same code (library or routine) on multiple cores.
- By adding a few built-in routines, a "interpreter instance" or "thread" manager could be accessed to start, stop, or monitor threads, as well as share data between them in a thread-safe way.
- A eui.dll or eui.so could be linked to a main program to execute euphoria code, and share data with that program using the same system that shares data between interpreter instances.
What do you think?
2. Re: Multiple interpreter instances
- Posted by katsmeow Mar 01, 2015
- 3426 views
What do you think?
Generally, great ideas, but i suspect the GreatProcess that syncs entire duplicate cloned environments will be massive and eat lots of cpu cycles. I don't think any of it will happen.
Tomorrow, i am disassembling my cluster, stashing the whole thing away in storage. I'll be down to one laptop, one desktop puter for myself, and one desktop for occasional hardware experiments and tests. I don't plan on any more Euphoria projects.
Kat
3. Re: Multiple interpreter instances
- Posted by ghaberek (admin) Mar 01, 2015
- 3481 views
I have had an idea several years ago. What if the euphoria "backend" was able to run in multiple instances, each instance in a different thread? I need to study the internal working of OE more, but i believe it would require converting all global state data to be put inside arrays/sequences and add a manager that can create and destroy instances, each with it's own index/id number, and a sharing interface which all instances can access, with some sort of semaphore mechanism. This would open up some interesting possibilities:
- Multi-threading and blocking I/O would be possible, because a new thread would actually be a different interpreter, completely independent.
- Running scripts or generating code on-the-fly would be possible, by simply starting a new interpreter instance.
- Processing data with multiple cores would be possible, by starting multiple instances of the same code (library or routine) on multiple cores.
- By adding a few built-in routines, a "interpreter instance" or "thread" manager could be accessed to start, stop, or monitor threads, as well as share data between them in a thread-safe way.
- A eui.dll or eui.so could be linked to a main program to execute euphoria code, and share data with that program using the same system that shares data between interpreter instances.
What do you think?
I've considered this project a few times and I think it is entirely possible. What you need at the basic level is something like memcached for shared data storage and then some sort of message-passing interface like ZeroMQ to coordinate all of the nodes. These pieces could be written in Euphoria or you could use those libraries directly. The best part is that such a system wouldn't really care what a "node" looks like, be it a thread, CPU, or another machine entirely. You could probabyl scale out to hundreds of nodes quite easily. Another option would be to use Gearman to do a lot of the heavy lifting. It is already designed for such things.
-Greg
4. Re: Multiple interpreter instances
- Posted by jimcbrown (admin) Mar 02, 2015
- 3366 views
I have had an idea several years ago. What if the euphoria "backend" was able to run in multiple instances, each instance in a different thread?
Looks like I beat you to it. http://scm.openeuphoria.org/hg/euphoria/shortlog/335f3b5cd19b
I need to study the internal working of OE more, but i believe it would require converting all global state data to be put inside arrays/sequences and add a manager that can create and destroy instances, each with it's own index/id number, and a sharing interface which all instances can access, with some sort of semaphore mechanism.
In my version, each thread has its own copy of everything, and only accesses that. So by default all variables are unique to a thread. (Not yet implemented.) For the future, there was to be a special shared data instance for variables that were declared to be shared between threads, that each thread would access in common. (Not yet implemented.)
This would open up some interesting possibilities:
- Multi-threading and blocking I/O would be possible, because a new thread would actually be a different interpreter, completely independent.
Yep. This already works in the experimental branch.
- Processing data with multiple cores would be possible, by starting multiple instances of the same code (library or routine) on multiple cores.
Yep. This already works.
- A eui.dll or eui.so could be linked to a main program to execute euphoria code, and share data with that program using the same system that shares data between interpreter instances.
This is more-or-less how the experimental branch is implemented.
- Running scripts or generating code on-the-fly would be possible, by simply starting a new interpreter instance.
Not in this implementation. All Euphoria code is compiled into IL first, and a single static, unmodifiable instance of IL is shared between all threads. (Since it's effectively read-only, this is thread-safe.)
- By adding a few built-in routines, a "interpreter instance" or "thread" manager could be accessed to start, stop, or monitor threads, as well as share data between them in a thread-safe way.
I hadn't gotten this far yet. Only starting threads is supported. Though one can still wrap the lower level OS/libc routines to manage native threads.
