1. Size of translated programs
Hi Rob, hi all,
I happened to make an interesting discovery.
When I translate one of my programs, using the command
ecw.exe -bor <prog>.exw
with the Eu2C translator 2.4, and then running
emake.bat
the resulting exe file has a size of 157184 Bytes.
When I run
shroud -clear <prog>.exw
before translating the program, the resulting exe file has a size of
only 144896 Bytes (7.8% smaller). Maybe sometimes even more than 7.8%
can be gained? I didn't do any statistical research so far. 
I'm somewhat surprised, because in e2c.htm#size it reads:
<quote>
The Translator deletes routines that are not used, including
those from the standard Euphoria include files. After deleting unused
routines, it checks again for more routines that have now become unused,
and so on.
<unquote>
According to the docs, shrouding also deletes unused constants, not
"only" unused routines. Does that cause the difference?
In order to automatically translate to smaller executables, I wrote the
following batch file:
@echo off
shroud -clear -out %2 %1
ecw.exe -bor %2
but it does not work. The last command is not executed. Does anyone know
the reason why?
I'd like to suggest for Eu 2.5, that the translator would be as
effective as the shrouder in deleting unused stuff.
Regards,
Juergen
--
/"\ ASCII ribbon campain |
\ / against HTML in | This message has been ROT-13 encrypted
X e-mail and news, | twice for higher security.
/ \ and unneeded MIME |
2. Re: Size of translated programs
Juergen Luethje wrote:
>
> In order to automatically translate to smaller executables, I wrote the
> following batch file:
> @echo off
> shroud -clear -out %2 %1
> ecw.exe -bor %2
>
> but it does not work. The last command is not executed. Does anyone know
> the reason why?
'shroud' is actually a batch file that calls the real shroud.ex* and DOS
has a nasty habit of not transferring control back to a calling batch
file after another has been run. i.e. in your case it means that control
isn't returned to your script when shroud quits.
You have to tell it to return control by using the DOS 'call' command:
@echo off
call shroud -clear -out %2 %1
ecw.exe -bor %2
--
[ Carl R White == aka () = The Domain of Cyrek = ]
[ Cyrek the Illogical /\ www.cyreksoft.yorks.com ]
3. Re: Size of translated programs
Kenneth wrote:
> Juergen Luethje wrote:
>> In order to automatically translate to smaller executables, I wrote the
>> following batch file:
>> @echo off
>> shroud -clear -out %2 %1
>> ecw.exe -bor %2
>>
>> but it does not work. The last command is not executed. Does anyone know
>> the reason why?
>
> Shroud is a batch file. You need to use "call"
> call shroud ...
> Otherwise control won't return to your bat file.
Works fine.
Thanks, Kenneth and Carl!
Regards,
Juergen
--
The difference between men and boys
is the price of their toys.
4. Re: Size of translated programs
Juergen Luethje wrote:
> I happened to make an interesting discovery.
>
> When I translate one of my programs, using the command
> ecw.exe -bor <prog>.exw
>
> with the Eu2C translator 2.4, and then running
> emake.bat
>
> the resulting exe file has a size of 157184 Bytes.
>
> When I run
> shroud -clear <prog>.exw
>
> before translating the program, the resulting exe file has a size of
> only 144896 Bytes (7.8% smaller). Maybe sometimes even more than 7.8%
> can be gained? I didn't do any statistical research so far.
>
> I'm somewhat surprised, because in e2c.htm#size it reads:
> <quote>
> The Translator deletes routines that are not used, including
> those from the standard Euphoria include files. After deleting unused
> routines, it checks again for more routines that have now become unused,
> and so on.
> <unquote>
>
> According to the docs, shrouding also deletes unused constants, not
> "only" unused routines. Does that cause the difference?
I think most constants are simple integer ones like:
constant MAX = 999
The Translator deletes these constants,
even if they are used. It just plugs in the value of the
constant everywhere it is needed.
I think where the difference lies is that the shrouder is smart
enough to delete unused atom constants and unused constants
that are set to a side-effect-free expression,
whereas the translator will keep those ones.
> I'd like to suggest for Eu 2.5, that the translator would be as
> effective as the shrouder in deleting unused stuff.
Although they both delete unused stuff,
the algorithms used by the Translator and the
Binder/Shrouder are quite different. Some things
aren't as easy to optimize in one versus the other,
but I'll keep your suggestion in mind. Thanks.
Regards,
Rob Craig
Rapid Deployment Software
http://www.RapidEuphoria.com
5. Re: Size of translated programs
On Thu, Aug 21, 2003 at 07:28:47PM -0400, Robert Craig wrote:
> I think most constants are simple integer ones like:
> constant MAX = 999
> The Translator deletes these constants,
> even if they are used. It just plugs in the value of the
> constant everywhere it is needed.
>
> I think where the difference lies is that the shrouder is smart
> enough to delete unused atom constants and unused constants
> that are set to a side-effect-free expression,
> whereas the translator will keep those ones.
>
> >I'd like to suggest for Eu 2.5, that the translator would be as
> >effective as the shrouder in deleting unused stuff.
>
> Although they both delete unused stuff,
> the algorithms used by the Translator and the
> Binder/Shrouder are quite different. Some things
> aren't as easy to optimize in one versus the other,
> but I'll keep your suggestion in mind. Thanks.
>
> Regards,
> Rob Craig
> Rapid Deployment Software
> http://www.RapidEuphoria.com
>
Why are the algorithms so different? Does it have to do with the difference
between how C code and Eu code is parsed?
jbrown
>
>
> TOPICA - Start your own email discussion group. FREE!
>
>
--
Outlook Users, please don't put my email address in your address book. That way,
my email address won't appear in forged emails sent by email viruses. (Which are
technically worms btw :P)
--
Linux User:190064
Linux Machine:84163
6. Re: Size of translated programs
jbrown105 at speedymail.org wrote:
> Why are the algorithms so different?
They both remove unused stuff, and then
remove other stuff that is now seen to be unused,
and so on, and so on, but the structure of the two programs
is different, and it made more sense to do it one
way in the binder/shrouder and a very different
way in the Translator.
> Does it have to do with the difference
> between how C code and Eu code is parsed?
No.
Regards,
Rob Craig
Rapid Deployment Software
http://www.RapidEuphoria.com
7. Re: Size of translated programs
Rob wrote:
> Juergen Luethje wrote:
<snip>
>> When I run
>> shroud -clear <prog>.exw
>>
>> before translating the program, the resulting exe file has a size of
>> only 144896 Bytes (7.8% smaller). Maybe sometimes even more than 7.8%
>> can be gained? I didn't do any statistical research so far.
see below
<snip>
> I think where the difference lies is that the shrouder is smart
> enough to delete unused atom constants and unused constants
> that are set to a side-effect-free expression,
> whereas the translator will keep those ones.
>
>> I'd like to suggest for Eu 2.5, that the translator would be as
>> effective as the shrouder in deleting unused stuff.
>
> Although they both delete unused stuff,
> the algorithms used by the Translator and the
> Binder/Shrouder are quite different. Some things
> aren't as easy to optimize in one versus the other,
> but I'll keep your suggestion in mind. Thanks.
I see. Thanks for the explanation.
I made some tests and found out, that the .exe size sometimes is
reduced _considerably_ by shrouding before translating. This applies
e.g. to several Win32Lib programs. I got the following results with
'boxes.exw', that ships with Win32Lib (version 0.58.08).
Translated for, and compiled with Borland C++ 5.5.1 Command-line
Compiler (size of .exe files in bytes):
normal reduced
------ -------
811008 594432 ==> 26.7% smaller
Translated for, and compiled with Open Watcom C/C++ 1.0
(size of .exe files in bytes):
normal reduced
------ -------
807424 590336 ==> 26.9% smaller
Regards,
Juergen
--
/"\ ASCII ribbon campain | |\ _,,,---,,_
\ / against HTML in | /,`.-'`' -. ;-;;,_
X e-mail and news, | |,4- ) )-,_..;\ ( `'-'
/ \ and unneeded MIME | '---''(_/--' `-'\_)
8. Re: Size of translated programs
Carl wrote:
> Juergen Luethje wrote:
>>
>> In order to automatically translate to smaller executables, I wrote the
>> following batch file:
>> @echo off
>> shroud -clear -out %2 %1
>> ecw.exe -bor %2
>>
>> but it does not work. The last command is not executed. Does anyone know
>> the reason why?
>
> 'shroud' is actually a batch file that calls the real shroud.ex* and DOS
> has a nasty habit of not transferring control back to a calling batch
> file after another has been run. i.e. in your case it means that control
> isn't returned to your script when shroud quits.
>
> You have to tell it to return control by using the DOS 'call' command:
> @echo off
> call shroud -clear -out %2 %1
> ecw.exe -bor %2
Thanks again.
As I wrote, this worked fine -- but only for shrouding/translating small
programs, as I see now. There is an issue with big files, which I wasn't
aware of. 'call shroud' runs asynchronously, and this may lead to
strange results. I detected that by shrouding/translating 'boxes.exw'
(ships with Win32Lib). Now I use
@echo off
start /w exw.exe bind.ex -SHROUD_ONLY -clear -out %2 %1
ecw.exe -bor %2
This seems to work as expected.
Regards,
Juergen
--
/"\ ASCII ribbon campain | |\ _,,,---,,_
\ / against HTML in | /,`.-'`' -. ;-;;,_
X e-mail and news, | |,4- ) )-,_..;\ ( `'-'
/ \ and unneeded MIME | '---''(_/--' `-'\_)
9. Re: Size of translated programs
Juergen Luethje wrote:
> I made some tests and found out, that the .exe size sometimes is
> reduced _considerably_ by shrouding before translating. This applies
> e.g. to several Win32Lib programs. I got the following results with
> 'boxes.exw', that ships with Win32Lib (version 0.58.08).
>
>
> Translated for, and compiled with Borland C++ 5.5.1 Command-line
> Compiler (size of .exe files in bytes):
>
> normal reduced
> ------ -------
> 811008 594432 ==> 26.7% smaller
>
>
> Translated for, and compiled with Open Watcom C/C++ 1.0
> (size of .exe files in bytes):
>
> normal reduced
> ------ -------
> 807424 590336 ==> 26.9% smaller
Thanks. That's surprising.
When I tried to duplicate your results using
boxes.exw in Win32Lib 58.8 and 59.1 (I don't have 58.08)
I got:
normal reduced
------ -------
620,544 607,744
shroud -clear boxes.exw
gave me:
deleted 2249 of 3945 constants
deleted 413 of 677 routines
You might want to double check your results.
I guess it's possible that in your case
there was maybe one critical constant declaration
deleted by the shrouder but not the translator.
The removal of one key declaration could trigger the
removal of scores of other routines and constants.
Regards,
Rob Craig
Rapid Deployment Software
http://www.RapidEuphoria.com
10. Re: Size of translated programs
Hello again Rob,
you wrote:
> Juergen Luethje wrote:
>> I made some tests and found out, that the .exe size sometimes is
>> reduced _considerably_ by shrouding before translating. This applies
>> e.g. to several Win32Lib programs. I got the following results with
>> 'boxes.exw', that ships with Win32Lib (version 0.58.08).
>>
>>
>> Translated for, and compiled with Borland C++ 5.5.1 Command-line
>> Compiler (size of .exe files in bytes):
>>
>> normal reduced
>> ------ -------
>> 811008 594432 ==> 26.7% smaller
>>
>>
>> Translated for, and compiled with Open Watcom C/C++ 1.0
>> (size of .exe files in bytes):
>>
>> normal reduced
>> ------ -------
>> 807424 590336 ==> 26.9% smaller
>
> Thanks. That's surprising.
> When I tried to duplicate your results using
> boxes.exw in Win32Lib 58.8 and 59.1 (I don't have 58.08)
Hmm, regarding the Win32Lib versions, I'm a little confused at the
moment. In the 'index.htm' file in the 'docs' folder of my copy of
Win32Lib, it reads: "Version: 0.58.08 15/May/2003".
At Derek's main Euphoria page
<http://www.users.bigpond.com/ddparnell/euphoria/euphoria.htm>
there is no version 0.58.08 mentioned at all, but a version 0.58.8,
with the same date as my copy of version 0.58.08.
At Derek's page
<http://www.users.bigpond.com/ddparnell/euphoria/changes.htm>
there is no version 0.58.8 mentioned at all, but a version 0.58.08,
dated 15/May/2003.
I just uploaded the original ZIP file, that I downloaded from Derek
at that time, to my website. I hope and believe Derek won't mind
(the file will be there only temporarily, of course).
http://luethje.de.vu/temp/w32005808.zip (789 KB)
So if you like, you can try to duplicate my results, using exactly the
same version of Win32Lib that I used.
> I got:
>
> normal reduced
> ------ -------
> 620,544 607,744
>
> shroud -clear boxes.exw
> gave me:
> deleted 2249 of 3945 constants
> deleted 413 of 677 routines
>
> You might want to double check your results.
I also regard this as important. It's easy to make a mistake in a
calculation. But my current results are consistent with the results
I got before. I'm also sure, that I've only the official Euphoria
version 2.4 installed -- no files left from earlier releases.
> I guess it's possible that in your case
> there was maybe one critical constant declaration
> deleted by the shrouder but not the translator.
> The removal of one key declaration could trigger the
> removal of scores of other routines and constants.
The following table shows the results of translating the first five .exw
files (in alphabetical order) in the 'demo folder of Win32Lib 58.08 with
the Eu2C translator 2.4 (ecw.exe -bor <prog>.exw) and compiling with
Borland C++ 5.5.1 Command-line Compiler:
| size of .exe file | |
| (in bytes) | | output of
name | | shroud -clear | reduction | shroud -clear
| normal | before | of size | ("deleted ...")
| translation | translate | |
==============+=============+===============+===========+===================
advanc.exw | 802,816 | 591,360 | 26.3% | 2207 of 3844 const
| | | | 420 of 676 rout
--------------+-------------+---------------+-----------+-------------------
appselect.exw | 813,056 | 596,480 | 26.6% | 2210 of 3842 const
| | | | 414 of 673 rout
--------------+-------------+---------------+-----------+-------------------
bitmap14.exw | 793,600 | 582,144 | 26.6% | 2210 of 3846 const
| | | | 421 of 672 rout
--------------+-------------+---------------+-----------+-------------------
bkgnd16.exw | 819,712 | 600,064 | 26.8% | 2206 of 3842 const
| | | | 417 of 674 rout
--------------+-------------+---------------+-----------+-------------------
boxes.exw | 811,008 | 594,432 | 26.7% | 2203 of 3842 const
| | | | 415 of 676 rout
Regards,
Juergen
--
/"\ ASCII ribbon campain |
\ / against HTML in | This message has been ROT-13 encrypted
X e-mail and news, | twice for higher security.
/ \ and unneeded MIME |
11. Re: Size of translated programs
How to make a shrouded executable:
Step 1: Strip code of unused routines, constants, etc.
Step 2: Shroud code into unreadable file
Step 3: Bind shrouded code to the end of PDE (Public Domain Edition)
Interpreter
Yields: (size of interpreter) + (size of shrouded code file)
How to make a translated executable:
Step 1: Strip code of unused routines, constants, etc..
Step 2: Translate code from Euphoia to C.
Add extra routines for compatibility when necessary.
Step 3: Compile C code using appropriate C compiler. (Suggest Watcom
or Borland.)
Yields: (final size of compiled C code)
Clearly the difference in size can never be compared. A shrouded text file
full of code will always be smaller than the same program in compiled C. I
suggest using UPX to compress your translated executables. I usually get
somewhere between 25% and 45% compression on my .exe files.
12. Re: Size of translated programs
Juergen Luethje wrote:
> I just uploaded the original ZIP file, that I downloaded from Derek
> at that time, to my website. I hope and believe Derek won't mind
> (the file will be there only temporarily, of course).
> http://luethje.de.vu/temp/w32005808.zip (789 KB)
>
> So if you like, you can try to duplicate my results, using exactly the
> same version of Win32Lib that I used.
>
> snipped: table showing savings from running shroud -clear
> before running translator
I downloaded your file and duplicated your results.
The .exe files are much bigger when using 58.8 because there's
a "with trace" statement in effect for most of win32lib.ew.
It was removed for 59.1.
In the Complete edition translator, "with trace" causes
ctrace("...") statements to be emitted into the C code,
provided there is also a call to trace() somewhere in the
program. These ctrace() statements increase the .exe size
considerably. You would have been warned about this,
but win32lib.ew has "without warning" in effect.
By running shroud -clear before running the translator,
you eliminate the lone trace() statement, since it is
located in a routine that gets deleted. This stops
the ctrace() statements from being emitted.
By itself, the translator notes the existence of the
trace() call before it decides to delete the routine.
The remaining small difference in size is due to
a few complex constant declarations that the shrouder eliminates
but the Translator doesn't. There seems to be no difference
in the routines deleted.
Regards,
Rob Craig
Rapid Deployment Software
http://www.RapidEuphoria.com
13. Re: Size of translated programs
On Sun, 24 Aug 2003 23:02:13 -0400, Robert Craig
<rds at RapidEuphoria.com> wrote:
>By itself, the translator notes the existence of the
>trace() call before it decides to delete the routine.
Can we assume that is something you plan to fix, one day?
Pete
14. Re: Size of translated programs
Pete Lomax wrote:
> On Sun, 24 Aug 2003 23:02:13 -0400, Robert Craig
> <rds at RapidEuphoria.com> wrote:
>>By itself, the translator notes the existence of the
>>trace() call before it decides to delete the routine.
>
> Can we assume that is something you plan to fix, one day?
Yes. That optimization will be improved, but
people should really remove "with trace" from finished code.
It slows you down 10% to 20% when running the
interpreter too, even without any call to trace().
I should also have the Translator output some more
statistics, such as the number of unused routines deleted,
and the number of lines of debug (ctrace) code inserted.
That way you'd know that debug code was being inserted
even if you have "without warning".
Regards,
Rob Craig
Rapid Deployment Software
http://www.RapidEuphoria.com
15. Re: Size of translated programs
Rob wrote:
> Juergen Luethje wrote:
>> I just uploaded the original ZIP file, that I downloaded from Derek
>> at that time, to my website. I hope and believe Derek won't mind
>> (the file will be there only temporarily, of course).
>> http://luethje.de.vu/temp/w32005808.zip (789 KB)
>>
>> So if you like, you can try to duplicate my results, using exactly the
>> same version of Win32Lib that I used.
>>
>> snipped: table showing savings from running shroud -clear
>> before running translator
>
> I downloaded your file and duplicated your results.
>
> The .exe files are much bigger when using 58.8 because there's
> a "with trace" statement in effect for most of win32lib.ew.
Aaaah..
> It was removed for 59.1.
Good.
> In the Complete edition translator, "with trace" causes
> ctrace("...") statements to be emitted into the C code,
> provided there is also a call to trace() somewhere in the
> program. These ctrace() statements increase the .exe size
> considerably. You would have been warned about this,
> but win32lib.ew has "without warning" in effect.
>
> By running shroud -clear before running the translator,
> you eliminate the lone trace() statement, since it is
> located in a routine that gets deleted. This stops
> the ctrace() statements from being emitted.
> By itself, the translator notes the existence of the
> trace() call before it decides to delete the routine.
>
> The remaining small difference in size is due to
> a few complex constant declarations that the shrouder eliminates
> but the Translator doesn't. There seems to be no difference
> in the routines deleted.
Thanks for the comprehensive explanation!
It looks, as if you had some extra work, because I made strange
experiments with an outdated version of Win32Lib. I'm sorry for that.
Regards,
Juergen
--
/"\ ASCII ribbon campain | |\ _,,,---,,_
\ / against HTML in | /,`.-'`' -. ;-;;,_
X e-mail and news, | |,4- ) )-,_..;\ ( `'-'
/ \ and unneeded MIME | '---''(_/--' `-'\_)
16. Re: Size of translated programs
Rob wrote:
> Pete Lomax wrote:
>> On Sun, 24 Aug 2003 23:02:13 -0400, Robert Craig
>> <rds at RapidEuphoria.com> wrote:
>>> By itself, the translator notes the existence of the
>>> trace() call before it decides to delete the routine.
>>
>> Can we assume that is something you plan to fix, one day?
>
> Yes. That optimization will be improved, but
> people should really remove "with trace" from finished code.
> It slows you down 10% to 20% when running the
> interpreter too, even without any call to trace().
Oh, that's remarkable.
> I should also have the Translator output some more
> statistics, such as the number of unused routines deleted,
> and the number of lines of debug (ctrace) code inserted.
> That way you'd know that debug code was being inserted
> even if you have "without warning".
Thanks!
Regards,
Juergen
--
/"\ ASCII ribbon campain | "Everything should be made as simple
\ / against HTML in | as possible, but not simpler."
X e-mail and news, |
/ \ and unneeded MIME | [Albert Einstein]
