1. Modifying the Euphoria interpreter
Are there any restritions on the statements thay may appear in the front end
source files?
Specifically, I plan to add an open_dll() and c_func() call in scanner.e to
fix the problem of paths with accented characters in EUINC or EUDIR not being
searched properly by path_open() under Windows. In order to build succesfully,
can they remain as-is or should they be replaced by their native
machine_func() equivalents?
TIA
CChris
2. Re: Modifying the Euphoria interpreter
CChris wrote:
> Are there any restritions on the statements thay may appear in the front end
> source files?
I don't think there are any restrictions.
I think you can even have multitasking if you want.
Obviously, I haven't tested everything though.
> Specifically, I plan to add an open_dll() and c_func() call in scanner.e to
> fix the problem of paths with accented characters in EUINC or EUDIR not being
> searched properly by path_open() under Windows. In order to build succesfully,
> can they remain as-is or should they be replaced by their native
> machine_func() equivalents?
If you try to *execute* an open_dll under DOS,
you'll get an error. So you'll need to check platform()
and avoid that. You should make sure your modified front-end
runs with ex.exe, not just exw.exe.
I'm not sure we ever agreed on EUforum that
what you are doing is fine with everyone else,
but you can certainly try it and let us know how it works.
Thanks,
Rob Craig
Rapid Deployment Software
http://www.RapidEuphoria.com
3. Re: Modifying the Euphoria interpreter
Robert Craig wrote:
>
> CChris wrote:
> > Are there any restritions on the statements thay may appear in the front end
> > source files?
>
> I don't think there are any restrictions.
> I think you can even have multitasking if you want.
> Obviously, I haven't tested everything though.
>
> > Specifically, I plan to add an open_dll() and c_func() call in scanner.e to
> > fix the problem of paths with accented characters in EUINC or EUDIR not
> > being
> > searched properly by path_open() under Windows. In order to build
> > succesfully,
> > can they remain as-is or should they be replaced by their native
> > machine_func() equivalents?
>
> If you try to *execute* an open_dll under DOS,
> you'll get an error. So you'll need to check platform()
> and avoid that. You should make sure your modified front-end
> runs with ex.exe, not just exw.exe.
>
> I'm not sure we ever agreed on EUforum that
> what you are doing is fine with everyone else,
> but you can certainly try it and let us know how it works.
>
> Thanks,
> Rob Craig
> Rapid Deployment Software
> <a href="http://www.RapidEuphoria.com">http://www.RapidEuphoria.com</a>
Definitely, this is a Windows only patch, so I check platform() before
calling open_dll() and before using c_func().
I tested my mod standalone, and it works. This does not replace full
testing, but means I can go ahead with it.
The proposed modification is actually a _bug fix_:
* Problem: Under Windows NT/2K/XP and later, paths may legitimately contain
non-ASCII characters, as long as they belong to the system codepage. If
the path in EUDIR, or any path inside EUINC, contains such a character,
said path is not processed by path_open(), and include files therein are not
found.
* Cause of the problem: when getenv() returns a string, that string is
encoded using the current Windows codepage. But the strings open() passes
are interpreted using the OEM current codepage. Since they have different
layouts, some characters are misinterpreted, and path_open() searches a path
that very likely doesn't exist.
* Current workaround: In the Advanced tab of system properties, change
offending characters in paths to the ones with the right codes.
Example: if a path ontains a e acute accent, coded #E9 in the Windows code
page and #82 in the OEM codepage, replace that e acute by the character with
code #82 in the Windows codepage. This happens to be a U acute.
This workaround implies tweaking environment variables and rebooting the
computer for changes to take effect; it is not satisfactory, but at least it
works.
* Suggested mod to front end, specifically to path_open():
If trying to open a file using a path from EUINC/EUDIR fails, if platform()
is Windows and if the path has accented characters, then try again after
converting this path using the CharToOemA() WinAPI. It has been in
kernel32.dll since Windows 3.0 and probably before.
Conversion applies to the path only, not the file name, since it is not
stored in the environment variable.
At least this is the right API to use, and the modified path_open() now finds
files it was missing before.
When I have tested with a real life interpreter (built under OW1.5), I'll
(try to) submit that mod. Unless someone has a brighter idea.
CChris
PS: if indeed mentioning open_dll() will cause ex.exe to error out because
it is not supported under DOS32, even though it will never execute the
function, then I'll have to use the machine_func() equivalent, that's all
it will take I bet. Thanks for pointing this potential problem out.
4. Re: Modifying the Euphoria interpreter
CChris wrote:
>
> Robert Craig wrote:
> >
> > If you try to *execute* an open_dll under DOS,
>
> PS: if indeed mentioning open_dll() will cause ex.exe to error out because
> it is not supported under DOS32, even though it will never execute the
> function, then I'll have to use the machine_func() equivalent, that's all
> it will take I bet. Thanks for pointing this potential problem out.
I felt Rob was pretty clear on that. An "if platform()=WIN32" wrap around the
open_dll() call (and etc) should be fine.
Regards,
Pete
5. Re: Modifying the Euphoria interpreter
Pete Lomax wrote:
>
> CChris wrote:
> >
> > Robert Craig wrote:
> > >
> > > If you try to *execute* an open_dll under DOS,
> >
> > PS: if indeed mentioning open_dll() will cause ex.exe to error out because
> > it is not supported under DOS32, even though it will never execute the
> > function, then I'll have to use the machine_func() equivalent, that's all
> > it will take I bet. Thanks for pointing this potential problem out.
>
> I felt Rob was pretty clear on that. An "if platform()=WIN32" wrap around the
> open_dll() call (and etc) should be fine.
>
> Regards,
> Pete
No, it's even worse than that.
Remember that, if ex.exe tries to compile a program which mentions
open_dll(), it errors out even though the the function won't ever be
executed because the call is protected by a "if platform()=??? then".
The error is generated before the platform dependent code is removed.
But replacing the call with "machine_func(50,"kernel32.dll") won't hurt.
CChris
6. Re: Modifying the Euphoria interpreter
CChris wrote:
>
> Pete Lomax wrote:
> >
> > CChris wrote:
> > >
> > > Robert Craig wrote:
> > > >
> > > > If you try to *execute* an open_dll under DOS,
> > >
> > > PS: if indeed mentioning open_dll() will cause ex.exe to error out because
> > > it is not supported under DOS32, even though it will never execute the
> > > function, then I'll have to use the machine_func() equivalent, that's all
> > > it will take I bet. Thanks for pointing this potential problem out.
> >
> > I felt Rob was pretty clear on that. An "if platform()=WIN32" wrap around
> > the
> > open_dll() call (and etc) should be fine.
> >
> > Regards,
> > Pete
>
> No, it's even worse than that.
> Remember that, if ex.exe tries to compile a program which mentions
> open_dll(), it errors out even though the the function won't ever be
> executed because the call is protected by a "if platform()=??? then".
> The error is generated before the platform dependent code is removed.
> But replacing the call with "machine_func(50,"kernel32.dll") won't hurt.
This works fine for me with ex.exe:
include dll.e
procedure foo()
atom x
x = open_dll("xxxxx.dll")
end procedure
puts(1, "No problem")
Regards,
Rob Craig
Rapid Deployment Software
http://www.RapidEuphoria.com
7. Re: Modifying the Euphoria interpreter
CChris wrote:
> Remember that, if ex.exe tries to compile a program which mentions
> open_dll(), it errors out even though the the function won't ever be
> executed because the call is protected by a "if platform()=??? then".
> The error is generated before the platform dependent code is removed.
I can't confirm that. Code like the following always worked fine for me:
include misc.e
include dll.e
atom Kernel32
if platform() = WIN32 then
Kernel32 = open_dll("kernel32.dll")
else
Kernel32 = -1
end if
? Kernel32
if getc(0) then end if
Interpreted with exw.exe the code prints e.g. 2088763392, interpreted with
ex.exe it prints -1 as expected. The code never "errors out".
Regards,
Juergen
8. Re: Modifying the Euphoria interpreter
Juergen Luethje wrote:
>
> CChris wrote:
>
> > Remember that, if ex.exe tries to compile a program which mentions
> > open_dll(), it errors out even though the the function won't ever be
> > executed because the call is protected by a "if platform()=??? then".
> > The error is generated before the platform dependent code is removed.
>
> I can't confirm that. Code like the following always worked fine for me:
> }}}
<eucode>
> include misc.e
> include dll.e
>
> atom Kernel32
>
> if platform() = WIN32 then
> Kernel32 = open_dll("kernel32.dll")
> else
> Kernel32 = -1
> end if
> ? Kernel32
> if getc(0) then end if
> </eucode>
{{{
> Interpreted with exw.exe the code prints e.g. 2088763392, interpreted with
> ex.exe it prints -1 as expected. The code never "errors out".
>
> Regards,
> Juergen
Good. I'll have to check if this result holds even when the open_dll() call
is part of the interpreter. Will hopefully check this tonight (5pm here).
CChris
9. Re: Modifying the Euphoria interpreter
CChris wrote:
> Definitely, this is a Windows only patch, so I check platform() before
> calling open_dll() and before using c_func().
>
> I tested my mod standalone, and it works. This does not replace full
> testing, but means I can go ahead with it.
>
> The proposed modification is actually a _bug fix_:
> * Problem: Under Windows NT/2K/XP and later, paths may legitimately contain
> non-ASCII characters, as long as they belong to the system codepage. If
> the path in EUDIR, or any path inside EUINC, contains such a character,
> said path is not processed by path_open(), and include files therein are not
> found.
> * Cause of the problem: when getenv() returns a string, that string is
> encoded using the current Windows codepage. But the strings open() passes
> are interpreted using the OEM current codepage. Since they have different
> layouts, some characters are misinterpreted, and path_open() searches a path
> that very likely doesn't exist.
> * Current workaround: In the Advanced tab of system properties, change
> offending characters in paths to the ones with the right codes.
> Example: if a path ontains a e acute accent, coded #E9 in the Windows code
> page and #82 in the OEM codepage, replace that e acute by the character with
> code #82 in the Windows codepage. This happens to be a U acute.
> This workaround implies tweaking environment variables and rebooting the
> computer for changes to take effect; it is not satisfactory, but at least it
> works.
> * Suggested mod to front end, specifically to path_open():
> If trying to open a file using a path from EUINC/EUDIR fails, if platform()
> is Windows and if the path has accented characters, then try again after
> converting this path using the CharToOemA() WinAPI. It has been in
> kernel32.dll since Windows 3.0 and probably before.
> Conversion applies to the path only, not the file name, since it is not
> stored in the environment variable.
>
> At least this is the right API to use, and the modified path_open() now finds
> files it was missing before.
> When I have tested with a real life interpreter (built under OW1.5), I'll
> (try to) submit that mod. Unless someone has a brighter idea.
This sounds reasonable to me.
Maybe others have an opinion?
I can give you full developer permissions on SourceForge
so you can check out official source code using SVN
(e.g. TortoiseSVN) and check it back in again after testing it
on your system. You just need to sign up for a free SourceForge id
and let me know what it is.
I realize that 90% of the job will be learning how
to use SVN, but I want to wean the community off using
me (or Matt) to make all changes. It will be better if
we have several people who are able to make changes and
check them in (and then handle the almost-inevitable bug
reports and enhancement requests that will flow from that).
I hope to get back to the 3.1 FreeBSD Release Candidate
later today. I had several distractions over the past few days.
It might be better to hold off on checking in your changes
until I do the full 3.1 official release for all platforms,
this week I hope.
Thanks,
Rob Craig
Rapid Deployment Software
http://www.RapidEuphoria.com
10. Re: Modifying the Euphoria interpreter
CChris wrote:
> No, it's even worse than that.
> Remember that, if ex.exe tries to compile a program which mentions
> open_dll(), it errors out even though the the function won't ever be
> executed because the call is protected by a "if platform()=??? then".
> The error is generated before the platform dependent code is removed.
> But replacing the call with "machine_func(50,"kernel32.dll") won't hurt.
As well as the same things others have tried, I just tested this here:
constant k32=open_dll("kernel32")
constant k32=machine_func(50,"kernel32")
Despite beginning to mistrust myself, I was right. Both the above fail equally
on ex.exe, since you cannot put a platform() check around either.
It is true, btw, to say that you cannot use constant definitions like the above
in platform independent code. Instead you must use atom vars.
The point I am trying to make here is that you seem to be locked into using a
direct call to machine_func when it will make no significant difference - see the
definition of open_dll() in dll.e. It is the machine_func call itself which fails
on ex.exe, NOT the trivial open_dll() wrapper... you can even prove this by
putting a display inside open_dll just before the call to machine_func: it will
show fine, just before ex.exe crashes.
Regards,
Pete
11. Re: Modifying the Euphoria interpreter
- Posted by CChris <christian.cuvier at agriculture.gouv.fr>
Jun 12, 2007
-
Last edited Jun 13, 2007
Robert Craig wrote:
>
> CChris wrote:
> > Definitely, this is a Windows only patch, so I check platform() before
> > calling open_dll() and before using c_func().
> >
> > I tested my mod standalone, and it works. This does not replace full
> > testing, but means I can go ahead with it.
> >
> > The proposed modification is actually a _bug fix_:
> > * Problem: Under Windows NT/2K/XP and later, paths may legitimately contain
> > non-ASCII characters, as long as they belong to the system codepage. If
> > the path in EUDIR, or any path inside EUINC, contains such a character,
> > said path is not processed by path_open(), and include files therein are not
> > found.
> > * Cause of the problem: when getenv() returns a string, that string is
> > encoded using the current Windows codepage. But the strings open() passes
> > are interpreted using the OEM current codepage. Since they have different
> > layouts, some characters are misinterpreted, and path_open() searches a path
> > that very likely doesn't exist.
> > * Current workaround: In the Advanced tab of system properties, change
> > offending characters in paths to the ones with the right codes.
> > Example: if a path ontains a e acute accent, coded #E9 in the Windows code
> > page and #82 in the OEM codepage, replace that e acute by the character with
> > code #82 in the Windows codepage. This happens to be a U acute.
> > This workaround implies tweaking environment variables and rebooting the
> > computer for changes to take effect; it is not satisfactory, but at least it
> > works.
> > * Suggested mod to front end, specifically to path_open():
> > If trying to open a file using a path from EUINC/EUDIR fails, if platform()
> > is Windows and if the path has accented characters, then try again after
> > converting this path using the CharToOemA() WinAPI. It has been in
> > kernel32.dll since Windows 3.0 and probably before.
> > Conversion applies to the path only, not the file name, since it is not
> > stored in the environment variable.
> >
> > At least this is the right API to use, and the modified path_open() now
> > finds
> > files it was missing before.
> > When I have tested with a real life interpreter (built under OW1.5), I'll
> > (try to) submit that mod. Unless someone has a brighter idea.
>
> This sounds reasonable to me.
> Maybe others have an opinion?
>
> I can give you full developer permissions on SourceForge
> so you can check out official source code using SVN
> (e.g. TortoiseSVN) and check it back in again after testing it
> on your system. You just need to sign up for a free SourceForge id
> and let me know what it is.
>
> I realize that 90% of the job will be learning how
> to use SVN, but I want to wean the community off using
> me (or Matt) to make all changes. It will be better if
> we have several people who are able to make changes and
> check them in (and then handle the almost-inevitable bug
> reports and enhancement requests that will flow from that).
>
> I hope to get back to the 3.1 FreeBSD Release Candidate
> later today. I had several distractions over the past few days.
> It might be better to hold off on checking in your changes
> until I do the full 3.1 official release for all platforms,
> this week I hope.
>
> Thanks,
> Rob Craig
> Rapid Deployment Software
> <a href="http://www.RapidEuphoria.com">http://www.RapidEuphoria.com</a>
It's Ok; my SF id is CChris.
I have other things waiting (most urgently fixing text printing using
nonstandard options in win32lib), so that I certainly can wait.
And I was aware too that there would be some learning investment on SVN...
Feel free to email privately oedoc at free dot fr for any relevant
request/info.
CChris
12. Re: Modifying the Euphoria interpreter
- Posted by CChris <christian.cuvier at agriculture.gouv.fr>
Jun 12, 2007
-
Last edited Jun 13, 2007
Pete Lomax wrote:
>
> CChris wrote:
> > No, it's even worse than that.
> > Remember that, if ex.exe tries to compile a program which mentions
> > open_dll(), it errors out even though the the function won't ever be
> > executed because the call is protected by a "if platform()=??? then".
> > The error is generated before the platform dependent code is removed.
> > But replacing the call with "machine_func(50,"kernel32.dll") won't hurt.
>
> As well as the same things others have tried, I just tested this here:
> }}}
<eucode>
> constant k32=open_dll("kernel32")
> constant k32=machine_func(50,"kernel32")
> </eucode>
{{{
> Despite beginning to mistrust myself, I was right. Both the above fail equally
> on ex.exe, since you cannot put a platform() check around either.
> It is true, btw, to say that you cannot use constant definitions like the
> above
> in platform independent code. Instead you must use atom vars.
>
> The point I am trying to make here is that you seem to be locked into using
> a direct call to machine_func when it will make no significant difference -
> see the definition of open_dll() in dll.e. It is the machine_func call itself
> which fails on ex.exe, NOT the trivial open_dll() wrapper... you can even
> prove
> this by putting a display inside open_dll just before the call to
> machine_func:
> it will show fine, just before ex.exe crashes.
>
> Regards,
> Pete
Of course I know the wrapper is trivial, and makes no difference.
I thought I remembered that, if running a program using ex.exe, and if
that program had lines like
atom k32
if platform()=WINDOWS then
k32=open_dll("kernel32.dll")
end if
I was getting an error for open_dll() not being supported under DOS. And then
of course the wrapper would make a difference, since no one could object to
a machine_func() call at compile time.
But again, I'll have to check what actually happens.
And as you said, the entry point can't be a constant, as it must appear
inside an if block.
CChris
13. Re: Modifying the Euphoria interpreter
- Posted by Bernie Ryan <xotron at bluefrog.com>
Jun 12, 2007
-
Last edited Jun 13, 2007
CChris wrote:
> }}}
<eucode>
> atom k32
> if platform()=WINDOWS then
> k32=open_dll("kernel32.dll")
> end if
> </eucode>
{{{
It should be :
if platform()=WIN32 then
Bernie
My files in archive:
WMOTOR, XMOTOR, W32ENGIN, MIXEDLIB, EU_ENGIN, WIN32ERU, WIN32API
Can be downloaded here:
http://www.rapideuphoria.com/cgi-bin/asearch.exu?dos=on&win=on&lnx=on&gen=on&keywords=bernie+ryan
14. Re: Modifying the Euphoria interpreter
CChris wrote:
> I was getting an error for open_dll() not being supported under DOS.
Ah, of course: that error message *IS* caused by a machine_func(50,...)
statement being run, *NOT* from an open_dll() statement being defined.
I think I can imagine the thought process you had, what you have to think of is
what message should Eu best display on a bad machine_func(50,...) call?
It is not going to be "machine_func opcode 50 not supported", is it?
I trust that has got through now, but if not please voice any remaining items of
confusion, as I think that it is vital anyone making such mods understands
*exactly* what is going on.
Regards,
Pete
15. Re: Modifying the Euphoria interpreter
Pete Lomax wrote:
>
> CChris wrote:
>
> > I was getting an error for open_dll() not being supported under DOS.
>
> Ah, of course: that error message *IS* caused by a machine_func(50,...)
> statement
> being run, *NOT* from an open_dll() statement being defined.
>
>
> I think I can imagine the thought process you had, what you have to think of
> is what message should Eu best display on a bad machine_func(50,...) call?
>
> It is not going to be "machine_func opcode 50 not supported", is it?
>
>
> I trust that has got through now, but if not please voice any remaining items
> of confusion, as I think that it is vital anyone making such mods understands
> *exactly* what is going on.
>
> Regards,
> Pete
Ok, I found some time to check it out.
Protecting the open_dll() statement by an "if platform()=" test enables a
program containing it to run both under exw and ex, both under Windows and
in a DOS box. So I assume this protection will be enough, although I'll
perform a test with the newly built interpreters just to be sure.
CChris
16. Re: Modifying the Euphoria interpreter
CChris wrote:
<snip>
> Ok, I found some time to check it out.
> Protecting the open_dll() statement by an "if platform()=" test enables a
> program containing it to run both under exw and ex, both under Windows and
> in a DOS box. So I assume this protection will be enough, although I'll
> perform a test with the newly built interpreters just to be sure.
What is the reason for integrating open_dll() into the interpreter?
Regards,
Juergen
17. Re: Modifying the Euphoria interpreter
Juergen Luethje wrote:
>
> CChris wrote:
>
> <snip>
>
> > Ok, I found some time to check it out.
> > Protecting the open_dll() statement by an "if platform()=" test enables a
> > program containing it to run both under exw and ex, both under Windows and
> > in a DOS box. So I assume this protection will be enough, although I'll
> > perform a test with the newly built interpreters just to be sure.
>
> What is the reason for integrating open_dll() into the interpreter?
>
> Regards,
> Juergen
You may wish to check http://www.listfilter.com/EUforum/m14385.html for a
detailed report.
I initially considered using direct translation, not involving any API call.
After all, it is only a matter of character recoding, which can be done using
a simple lookup table.
I quickly backed off considering that there are a few OEM and Windows code
pages to cater for, each with its own 128-255 range mapping. It looked
far more straightforward and reliable to ask the system for a proper
translation.
The bottom line is: since there is a Windows only issue which some Windows
dll handles, the most sensible thing to do is to use it, hence an open_dll()
call in the interpreter, as well as a c_func(). The call won't occur on other
platforms.
If similar issues arise under Linux/BSD, can anyone tell me about it, so that
the fix could be extended to these cases?
CChris
18. Re: Modifying the Euphoria interpreter
CChris wrote:
> Juergen Luethje wrote:
> >
> > CChris wrote:
> >
> > <snip>
> >
> > > Ok, I found some time to check it out.
> > > Protecting the open_dll() statement by an "if platform()=" test enables a
> > > program containing it to run both under exw and ex, both under Windows and
> > > in a DOS box. So I assume this protection will be enough, although I'll
> > > perform a test with the newly built interpreters just to be sure.
> >
> > What is the reason for integrating open_dll() into the interpreter?
> >
> > Regards,
> > Juergen
>
> You may wish to check <a
> href="http://www.listfilter.com/EUforum/m14385.html">http://www.listfilter.com/EUforum/m14385.html</a>
> for a
> detailed report.
Ah yes, I see.
> I initially considered using direct translation, not involving any API call.
> After all, it is only a matter of character recoding, which can be done using
> a simple lookup table.
>
> I quickly backed off considering that there are a few OEM and Windows code
> pages to cater for, each with its own 128-255 range mapping. It looked
> far more straightforward and reliable to ask the system for a proper
> translation.
I agree absolutely. The developers of Windows already have done that work,
and there is no need for you to do it again.
<snip>
Regards,
Juergen
