1. The Big Plan - was Euphoria 2.5 Features..... ??
Mario Steele wrote:
> First off, The wearing out Idea/Problem. A String Type Variable.
> Yeah Yeah, I realize we can do custom string type variables in Euphoria,
> but there's still 1 major problem with it all. Even if it's a string
> type variable, Euphoria still allocates 4 bytes of memory to store a
> simple 0 to 255 (Or 0 to 255*2) character into memory. Now for little
> baby strings, okay, that's kewl, no big loss. But when we get into big
> files, like for example, Win32lib _BEFORE_ the breakup into seperate
> libraries, and still somewhat now, we get a very big memory waste for
> all of it. Now we all know, the interpreter doesn't allocate 4 bytes of
> memory for each character, when reading the Plain Text euphoria code
> into memory, for parsing, and execution. So my quesiton is, why should
> we have to waste memory to read a file into memory, especially big
> files, which could double, even triple in size, when loaded into a
> simple Euphoria Sequence.
It would be great to store strings as one byte per element.
The problem is the interpreter would be significantly
bigger, buggier and slower. Every operation on sequences
(and there are lots of them!) would have to check if the
sequence consisted of all chars, and then use different code for
processing the chars. There would be lots of checking to
see if a non-char was being assigned to a "string" sequence.
There would be lots of conversions from "byte" format to
normal Euphoria object format. etc. If I did something like this,
I would not introduce a new type of sequence in the
language manual, I would do it purely as an
internal optimization.
> Next, a actual new Idea:
> Now, I noticed, when binding, it uses the Public Edition of Euphoria as
> the base for it, which is all fine and dandy, cause you can remove the
> source, and have the original Public Edition interpreter with no
> problems. But, I see two major problems with this idea. One being the
> file size. yes, considering what other interpreters have so far,
> Euphoria is alot smaller, but there is allways room for improvement.
> When Euphoria source code is bound to the interpreter, one specifically
> isn't going to be removing the source code, to get the interpreter,
> unless they know exactly how to do that, without destroying the
> interpreter. Now, what I propose is, that you can proabbly loose about
> 100k or more on the Interpreter for binding, if you divide the
> interpreter into two parts in the C code. Consider them if you would,
> modules.
>
> The two modules would be, 1 the reading of clear, un-objectized, plain
> text euphoria code. This is really only needed for being a Interpreter,
> not as a Bound Executable. The second module, would be the objectized,
> shrouded code, which only Euphoria knows how to decipher for, weither it
> be a external file, or bound to the file. Since the objectized code
> would be more orientated to sort of a "Binary Script" file, which is
> native only to Euphoria, it would grately enhance the load time, and
> parsing time for bound Euphoria programs, in which strictly relies on
> the source of the code, bound or not, to be purely objectized. Since
> there is no need for the plain text module, this would proabbly reduce
> the size of bound programs by another 100kb or more in size. This alone
> would make Euphoria even more desirable, for the yet even smaller
> executables then what other interpreters come out with.
It's funny you should mention this because I am planning
to do just what you say (and more).
Here's the plan:
- Make a clear distinction between the Euphoria
"front-end" and the Euphoria "back-end".
- The front-end handles scanning, parsing, and emission of
low-level Euphoria intermediate language code (IL).
The front-end would be written in Euphoria.
- A back-end processes the IL.
- The interpreter back-end, written in C, executes the IL.
- The Translator back-end, written in Euphoria,
converts IL to C source, to be linked with the
run-time library portion of the back-end.
- What we call today "shrouded code" will in the future be IL.
(Some extra encryption may be applied as well).
- The binder will tack this IL onto the Interpreter back-end.
The Interpreter back-end will obviously be smaller than
the current Interpreter.
- When you bind you'll be able to choose whether debugging info
will be included or not in the bound file. It will be possible
to get a full ex.err dump even when your program's source code
is not included in the bound file (just symbol table,
line table info).
I would still release a single interpreter .exe file formed
by fusing the Euphoria-written front-end with the C-written
back-end, and keeping the IL in memory.
Advantages:
- Translator will be 100% Euphoria code
- Interpreter will be 30% Euphoria code
- binder (has always been written 100% in Euphoria)
becomes radically simpler, smaller (reduced to 1/10 the size?,
plus shared copy of front-end to produce the IL),
easier to maintain in the future
- bound files are bound with a smaller "back-end" only, though
the IL (even without debug info) might turn out to be a bit
larger, so this might not be an advantage overall
Disadvantages:
- The Interpreter front-end will be slower. Translator-generated
C won't be as fast as the current hand-coded C.
I tested "include win32lib.ew"
on my P4, with time() calls before and after the include.
It took less than 1/10 of a second to parse. So if it rises to
a few tenths of a second, who cares? On my old P2, purchased
4 1/2 years ago, it took several tenths of a second.
So I guess that might rise to a couple of seconds (on
a Win32Lib-based program, with windows popping up etc.
it will barely be noticed.)
When you bind or shroud a program, this extra parse time
will go away completely, and will be less than now,
since you are running the IL directly without any parsing.
Currently, bound programs are parsed.
Since we will no longer be binding with the full interpreter,
there will be no need to compress it. Not decompressing will
improve the start-up time slightly.
The size of programs that we write has not kept up with
the increases in CPU speed. At best, programs can only
grow linearly in size over time, whereas CPU speeds
keep increasing geometrically.
- The Translator will be slower. I don't care. It's currently
surprisingly fast, and the subsequent C compiles take
most of the time anyway. (The Translator could be run
by the interpreter, or it could translate itself into
a faster .exe for a given platform.)
I've already started converting front-end C code to Euphoria,
and I'm happy with the result so far. The code is
often simpler, cleaner, easier to understand, fewer lines, etc.
The whole mess involving malloc/free and growing
and shrinking countless data structures has been eliminated.
Strings are nicer to work with.
And of course I have subscript checking, etc. etc.
Also, there are places in the C code where I used a
complicated, ugly algorithm to gain a few cycles
or save a few bytes (on a 8 Mb RAM, 50 Mhz 486),
and it's no longer needed.
Regards,
Rob Craig
Rapid Deployment Software
http://www.RapidEuphoria.com
2. Re: The Big Plan - was Euphoria 2.5 Features..... ??
Hi Rob, Hi EuMario!
----------
> From: Robert Craig <rds at RapidEuphoria.com>
> Subject: The Big Plan - was Euphoria 2.5 Features..... ??
[[snipped]]
> It's funny you should mention this because I am planning
> to do just what you say (and more).
WOW! EuMario! Thank you very much for your question!
Good question! Good suggestion! Good news!
> Here's the plan:
>
> - Make a clear distinction between the Euphoria
> "front-end" and the Euphoria "back-end".
>
> - The front-end handles scanning, parsing, and emission of
> low-level Euphoria intermediate language code (IL).
> The front-end would be written in Euphoria.
I know just now what I can make
with that "front-end" first of all!
> - A back-end processes the IL.
OK, I like The Big Plan !
[[snipped]]
Good Luck Rob!
Best Regards,
Igor Kachan
kinz at peterlink.ru
3. Re: The Big Plan - was Euphoria 2.5 Features..... ??
----- Original Message -----
From: "Robert Craig" <rds at RapidEuphoria.com>
To: <EUforum at topica.com>
Subject: The Big Plan - was Euphoria 2.5 Features..... ??
[snip]
>
> It's funny you should mention this because I am planning
> to do just what you say (and more).
>
> Here's the plan:
>
Excellent!
And if you can please publish the specs for the IL too 
--
Derek
4. Re: The Big Plan - was Euphoria 2.5 Features..... ??
This is really wonderful news. Another good effect I see is that the front
end of the interpreter will be mostly platform independent (error messages
instead of IL generation for a few built-ins). This should substantially
reduce the effort in porting to a new platform.
As for Euphoria.Net, let's use Rob's own nearly (entirely?)
platform-independent IL. I'm sure I'd pay good money for a Eu IL to .Net IL
translator. Since I would be using .Net professionally, this would be
considerable value added for me, while imposing no costs on our Windows
hobbyists, Linux enthusiasts, or DOS dinosaurs. (I also count myself among
the latter.)
Even without major language changes, this is radical enough that if I were
in charge of versioning, I'd call this Euphoria 3.0!
-- Mike Nelson
5. Re: The Big Plan - was Euphoria 2.5 Features..... ??
Below
----- Original Message -----
From: "Derek Parnell" <ddparnell at bigpond.com>
To: <EUforum at topica.com>
Subject: Re: The Big Plan - was Euphoria 2.5 Features..... ??
>
>
> ----- Original Message -----
> From: "Robert Craig" <rds at RapidEuphoria.com>
> To: <EUforum at topica.com>
> Sent: Saturday, December 06, 2003 6:00 AM
> Subject: The Big Plan - was Euphoria 2.5 Features..... ??
>
>
> [snip]
> >
> > It's funny you should mention this because I am planning
> > to do just what you say (and more).
> >
> > Here's the plan:
> >
>
> Excellent!
>
> And if you can please publish the specs for the IL too
Somehow I'm guessing that will be one of those registered things, if
published at all.
But I would be nice to to have it published.
Daniel Kluss
>
> --
> Derek
>
>
>
> TOPICA - Start your own email discussion group. FREE!
>
>
6. Re: The Big Plan - was Euphoria 2.5 Features..... ??
- Posted by akusaya at gmx.net
Dec 09, 2003
R> - binder (has always been written 100% in Euphoria)
R> becomes radically simpler, smaller (reduced to 1/10 the size?,
R> plus shared copy of front-end to produce the IL),
R> easier to maintain in the future
In addition, I have a tip to reduce the back-end interpreter size,
that is removing the machine_proc/func code that can be replaced by
Windows/Linux API.
The code of the API implementation is only automatically included when
it is used in the program.
So, if the program uses (for example) free_console(), the bound
program will consist of back-end interpreter, free_console.lib and
the program itself. But if the program doesn't use free_console(), the
bound program will consist of back-end interpreter and the program
itself. So it's smaller.
Some machine_func/proc code that can be replaced that I found are:
(win32 only, i don't know linux APIs)
M_FREE_CONSOLE = 54 change to kernel32:FreeConsole
M_SEEK = 19,
M_WHERE = 20,
M_DIR = 22,
M_CURRENT_DIR = 23, change to kernel32: file functions
M_FLUSH = 60,
M_LOCK_FILE = 61,
M_UNLOCK_FILE = 62,
M_CHDIR = 63
M_SOUND = 1,
M_LINE = 2,
M_PALETTE = 3,
M_GRAPHICS_MODE = 5,
M_CURSOR = 6, are these still needed?
M_WRAP = 7, maybe these can be changed to
M_SCROLL = 8, respective machine-code (for the dos32)
M_SET_T_COLOR = 9,
M_SET_B_COLOR = 10, or to ncurses library/win32 console API
M_POLYGON = 11, (in separate file)
M_TEXTROWS = 12,
M_VIDEO_CONFIG = 13,
M_ELLIPSE = 18,
M_GET_POSITION = 25,
M_ALL_PALETTE = 27
M_SLEEP = 64 change to kernel32:Sleep
M_GET_MOUSE = 14,
M_MOUSE_EVENTS = 15, don't know
M_MOUSE_POINTER = 24
7. Re: The Big Plan - was Euphoria 2.5 Features..... ??
On Tue, 9 Dec 2003 14:05:05 +0700, aku saya <akusaya at gmx.net> wrote:
>
>
>R> - binder (has always been written 100% in Euphoria)
>R> becomes radically simpler, smaller (reduced to 1/10 the size?,
>R> plus shared copy of front-end to produce the IL),
>R> easier to maintain in the future
>
>In addition, I have a tip to reduce the back-end interpreter size,
>that is removing the machine_proc/func code that can be replaced by
>Windows/Linux API.
>
>The code of the API implementation is only automatically included when
>it is used in the program.
>
>So, if the program uses (for example) free_console(), the bound
>program will consist of back-end interpreter, free_console.lib and
>the program itself. But if the program doesn't use free_console(), the
>bound program will consist of back-end interpreter and the program
>itself. So it's smaller.
>
>Some machine_func/proc code that can be replaced that I found are:
>(win32 only, i don't know linux APIs)
>
>
>M_FREE_CONSOLE = 54 change to kernel32:FreeConsole
>
>
>M_SEEK = 19,
>M_WHERE = 20,
>M_DIR = 22,
>M_CURRENT_DIR = 23, change to kernel32: file functions
>M_FLUSH = 60,
>M_LOCK_FILE = 61,
>M_UNLOCK_FILE = 62,
>M_CHDIR = 63
>
With the exception of M_SLEEP, I think the remainder are all DOS32
routines. I think it would make sense, and be alot simpler, to put all
of these in a single [optional] DOS32 link pack. It wouldn't surprise
me if it adds up to less than 1K, though.
>
>M_SOUND = 1,
>M_LINE = 2,
>M_PALETTE = 3,
>M_GRAPHICS_MODE = 5,
>M_CURSOR = 6, are these still needed?
>M_WRAP = 7, maybe these can be changed to
>M_SCROLL = 8, respective machine-code (for the dos32)
>M_SET_T_COLOR = 9,
>M_SET_B_COLOR = 10, or to ncurses library/win32 console API
>M_POLYGON = 11, (in separate file)
>M_TEXTROWS = 12,
>M_VIDEO_CONFIG = 13,
>M_ELLIPSE = 18,
>M_GET_POSITION = 25,
>M_ALL_PALETTE = 27
>
>
>M_SLEEP = 64 change to kernel32:Sleep
>
>
>M_GET_MOUSE = 14,
>M_MOUSE_EVENTS = 15, don't know
>M_MOUSE_POINTER = 24
>
>
>
>TOPICA - Start your own email discussion group. FREE!
>
>
