1. Modifying the Euphoria interpreter
- Posted by CChris <christian.cuvier at agriculture.gouv.fr> Jun 11, 2007
- 593 views
Are there any restritions on the statements thay may appear in the front end source files? Specifically, I plan to add an open_dll() and c_func() call in scanner.e to fix the problem of paths with accented characters in EUINC or EUDIR not being searched properly by path_open() under Windows. In order to build succesfully, can they remain as-is or should they be replaced by their native machine_func() equivalents? TIA CChris
2. Re: Modifying the Euphoria interpreter
- Posted by Robert Craig <rds at RapidEuphoria.com> Jun 12, 2007
- 593 views
CChris wrote: > Are there any restritions on the statements thay may appear in the front end > source files? I don't think there are any restrictions. I think you can even have multitasking if you want. Obviously, I haven't tested everything though. > Specifically, I plan to add an open_dll() and c_func() call in scanner.e to > fix the problem of paths with accented characters in EUINC or EUDIR not being > searched properly by path_open() under Windows. In order to build succesfully, > can they remain as-is or should they be replaced by their native > machine_func() equivalents? If you try to *execute* an open_dll under DOS, you'll get an error. So you'll need to check platform() and avoid that. You should make sure your modified front-end runs with ex.exe, not just exw.exe. I'm not sure we ever agreed on EUforum that what you are doing is fine with everyone else, but you can certainly try it and let us know how it works. Thanks, Rob Craig Rapid Deployment Software http://www.RapidEuphoria.com
3. Re: Modifying the Euphoria interpreter
- Posted by CChris <christian.cuvier at agriculture.gouv.fr> Jun 12, 2007
- 610 views
Robert Craig wrote: > > CChris wrote: > > Are there any restritions on the statements thay may appear in the front end > > source files? > > I don't think there are any restrictions. > I think you can even have multitasking if you want. > Obviously, I haven't tested everything though. > > > Specifically, I plan to add an open_dll() and c_func() call in scanner.e to > > fix the problem of paths with accented characters in EUINC or EUDIR not > > being > > searched properly by path_open() under Windows. In order to build > > succesfully, > > can they remain as-is or should they be replaced by their native > > machine_func() equivalents? > > If you try to *execute* an open_dll under DOS, > you'll get an error. So you'll need to check platform() > and avoid that. You should make sure your modified front-end > runs with ex.exe, not just exw.exe. > > I'm not sure we ever agreed on EUforum that > what you are doing is fine with everyone else, > but you can certainly try it and let us know how it works. > > Thanks, > Rob Craig > Rapid Deployment Software > <a href="http://www.RapidEuphoria.com">http://www.RapidEuphoria.com</a> Definitely, this is a Windows only patch, so I check platform() before calling open_dll() and before using c_func(). I tested my mod standalone, and it works. This does not replace full testing, but means I can go ahead with it. The proposed modification is actually a _bug fix_: * Problem: Under Windows NT/2K/XP and later, paths may legitimately contain non-ASCII characters, as long as they belong to the system codepage. If the path in EUDIR, or any path inside EUINC, contains such a character, said path is not processed by path_open(), and include files therein are not found. * Cause of the problem: when getenv() returns a string, that string is encoded using the current Windows codepage. But the strings open() passes are interpreted using the OEM current codepage. Since they have different layouts, some characters are misinterpreted, and path_open() searches a path that very likely doesn't exist. * Current workaround: In the Advanced tab of system properties, change offending characters in paths to the ones with the right codes. Example: if a path ontains a e acute accent, coded #E9 in the Windows code page and #82 in the OEM codepage, replace that e acute by the character with code #82 in the Windows codepage. This happens to be a U acute. This workaround implies tweaking environment variables and rebooting the computer for changes to take effect; it is not satisfactory, but at least it works. * Suggested mod to front end, specifically to path_open(): If trying to open a file using a path from EUINC/EUDIR fails, if platform() is Windows and if the path has accented characters, then try again after converting this path using the CharToOemA() WinAPI. It has been in kernel32.dll since Windows 3.0 and probably before. Conversion applies to the path only, not the file name, since it is not stored in the environment variable. At least this is the right API to use, and the modified path_open() now finds files it was missing before. When I have tested with a real life interpreter (built under OW1.5), I'll (try to) submit that mod. Unless someone has a brighter idea. CChris PS: if indeed mentioning open_dll() will cause ex.exe to error out because it is not supported under DOS32, even though it will never execute the function, then I'll have to use the machine_func() equivalent, that's all it will take I bet. Thanks for pointing this potential problem out.
4. Re: Modifying the Euphoria interpreter
- Posted by Pete Lomax <petelomax at blueyonder.co.uk> Jun 12, 2007
- 616 views
CChris wrote: > > Robert Craig wrote: > > > > If you try to *execute* an open_dll under DOS, > > PS: if indeed mentioning open_dll() will cause ex.exe to error out because > it is not supported under DOS32, even though it will never execute the > function, then I'll have to use the machine_func() equivalent, that's all > it will take I bet. Thanks for pointing this potential problem out. I felt Rob was pretty clear on that. An "if platform()=WIN32" wrap around the open_dll() call (and etc) should be fine. Regards, Pete
5. Re: Modifying the Euphoria interpreter
- Posted by CChris <christian.cuvier at agriculture.gouv.fr> Jun 12, 2007
- 585 views
Pete Lomax wrote: > > CChris wrote: > > > > Robert Craig wrote: > > > > > > If you try to *execute* an open_dll under DOS, > > > > PS: if indeed mentioning open_dll() will cause ex.exe to error out because > > it is not supported under DOS32, even though it will never execute the > > function, then I'll have to use the machine_func() equivalent, that's all > > it will take I bet. Thanks for pointing this potential problem out. > > I felt Rob was pretty clear on that. An "if platform()=WIN32" wrap around the > open_dll() call (and etc) should be fine. > > Regards, > Pete No, it's even worse than that. Remember that, if ex.exe tries to compile a program which mentions open_dll(), it errors out even though the the function won't ever be executed because the call is protected by a "if platform()=??? then". The error is generated before the platform dependent code is removed. But replacing the call with "machine_func(50,"kernel32.dll") won't hurt. CChris
6. Re: Modifying the Euphoria interpreter
- Posted by Robert Craig <rds at RapidEuphoria.com> Jun 12, 2007
- 586 views
CChris wrote: > > Pete Lomax wrote: > > > > CChris wrote: > > > > > > Robert Craig wrote: > > > > > > > > If you try to *execute* an open_dll under DOS, > > > > > > PS: if indeed mentioning open_dll() will cause ex.exe to error out because > > > it is not supported under DOS32, even though it will never execute the > > > function, then I'll have to use the machine_func() equivalent, that's all > > > it will take I bet. Thanks for pointing this potential problem out. > > > > I felt Rob was pretty clear on that. An "if platform()=WIN32" wrap around > > the > > open_dll() call (and etc) should be fine. > > > > Regards, > > Pete > > No, it's even worse than that. > Remember that, if ex.exe tries to compile a program which mentions > open_dll(), it errors out even though the the function won't ever be > executed because the call is protected by a "if platform()=??? then". > The error is generated before the platform dependent code is removed. > But replacing the call with "machine_func(50,"kernel32.dll") won't hurt. This works fine for me with ex.exe:
include dll.e procedure foo() atom x x = open_dll("xxxxx.dll") end procedure puts(1, "No problem")
Regards, Rob Craig Rapid Deployment Software http://www.RapidEuphoria.com
7. Re: Modifying the Euphoria interpreter
- Posted by Juergen Luethje <j.lue at gmx.de> Jun 12, 2007
- 587 views
CChris wrote: > Remember that, if ex.exe tries to compile a program which mentions > open_dll(), it errors out even though the the function won't ever be > executed because the call is protected by a "if platform()=??? then". > The error is generated before the platform dependent code is removed. I can't confirm that. Code like the following always worked fine for me:
include misc.e include dll.e atom Kernel32 if platform() = WIN32 then Kernel32 = open_dll("kernel32.dll") else Kernel32 = -1 end if ? Kernel32 if getc(0) then end if
Interpreted with exw.exe the code prints e.g. 2088763392, interpreted with ex.exe it prints -1 as expected. The code never "errors out". Regards, Juergen
8. Re: Modifying the Euphoria interpreter
- Posted by CChris <christian.cuvier at agriculture.gouv.fr> Jun 12, 2007
- 572 views
Juergen Luethje wrote: > > CChris wrote: > > > Remember that, if ex.exe tries to compile a program which mentions > > open_dll(), it errors out even though the the function won't ever be > > executed because the call is protected by a "if platform()=??? then". > > The error is generated before the platform dependent code is removed. > > I can't confirm that. Code like the following always worked fine for me: > }}} <eucode> > include misc.e > include dll.e > > atom Kernel32 > > if platform() = WIN32 then > Kernel32 = open_dll("kernel32.dll") > else > Kernel32 = -1 > end if > ? Kernel32 > if getc(0) then end if > </eucode> {{{ > Interpreted with exw.exe the code prints e.g. 2088763392, interpreted with > ex.exe it prints -1 as expected. The code never "errors out". > > Regards, > Juergen Good. I'll have to check if this result holds even when the open_dll() call is part of the interpreter. Will hopefully check this tonight (5pm here). CChris
9. Re: Modifying the Euphoria interpreter
- Posted by Robert Craig <rds at RapidEuphoria.com> Jun 12, 2007
- 603 views
CChris wrote: > Definitely, this is a Windows only patch, so I check platform() before > calling open_dll() and before using c_func(). > > I tested my mod standalone, and it works. This does not replace full > testing, but means I can go ahead with it. > > The proposed modification is actually a _bug fix_: > * Problem: Under Windows NT/2K/XP and later, paths may legitimately contain > non-ASCII characters, as long as they belong to the system codepage. If > the path in EUDIR, or any path inside EUINC, contains such a character, > said path is not processed by path_open(), and include files therein are not > found. > * Cause of the problem: when getenv() returns a string, that string is > encoded using the current Windows codepage. But the strings open() passes > are interpreted using the OEM current codepage. Since they have different > layouts, some characters are misinterpreted, and path_open() searches a path > that very likely doesn't exist. > * Current workaround: In the Advanced tab of system properties, change > offending characters in paths to the ones with the right codes. > Example: if a path ontains a e acute accent, coded #E9 in the Windows code > page and #82 in the OEM codepage, replace that e acute by the character with > code #82 in the Windows codepage. This happens to be a U acute. > This workaround implies tweaking environment variables and rebooting the > computer for changes to take effect; it is not satisfactory, but at least it > works. > * Suggested mod to front end, specifically to path_open(): > If trying to open a file using a path from EUINC/EUDIR fails, if platform() > is Windows and if the path has accented characters, then try again after > converting this path using the CharToOemA() WinAPI. It has been in > kernel32.dll since Windows 3.0 and probably before. > Conversion applies to the path only, not the file name, since it is not > stored in the environment variable. > > At least this is the right API to use, and the modified path_open() now finds > files it was missing before. > When I have tested with a real life interpreter (built under OW1.5), I'll > (try to) submit that mod. Unless someone has a brighter idea. This sounds reasonable to me. Maybe others have an opinion? I can give you full developer permissions on SourceForge so you can check out official source code using SVN (e.g. TortoiseSVN) and check it back in again after testing it on your system. You just need to sign up for a free SourceForge id and let me know what it is. I realize that 90% of the job will be learning how to use SVN, but I want to wean the community off using me (or Matt) to make all changes. It will be better if we have several people who are able to make changes and check them in (and then handle the almost-inevitable bug reports and enhancement requests that will flow from that). I hope to get back to the 3.1 FreeBSD Release Candidate later today. I had several distractions over the past few days. It might be better to hold off on checking in your changes until I do the full 3.1 official release for all platforms, this week I hope. Thanks, Rob Craig Rapid Deployment Software http://www.RapidEuphoria.com
10. Re: Modifying the Euphoria interpreter
- Posted by Pete Lomax <petelomax at blueyonder.co.uk> Jun 12, 2007
- 607 views
CChris wrote: > No, it's even worse than that. > Remember that, if ex.exe tries to compile a program which mentions > open_dll(), it errors out even though the the function won't ever be > executed because the call is protected by a "if platform()=??? then". > The error is generated before the platform dependent code is removed. > But replacing the call with "machine_func(50,"kernel32.dll") won't hurt. As well as the same things others have tried, I just tested this here:
constant k32=open_dll("kernel32") constant k32=machine_func(50,"kernel32")
Despite beginning to mistrust myself, I was right. Both the above fail equally on ex.exe, since you cannot put a platform() check around either. It is true, btw, to say that you cannot use constant definitions like the above in platform independent code. Instead you must use atom vars. The point I am trying to make here is that you seem to be locked into using a direct call to machine_func when it will make no significant difference - see the definition of open_dll() in dll.e. It is the machine_func call itself which fails on ex.exe, NOT the trivial open_dll() wrapper... you can even prove this by putting a display inside open_dll just before the call to machine_func: it will show fine, just before ex.exe crashes. Regards, Pete
11. Re: Modifying the Euphoria interpreter
- Posted by CChris <christian.cuvier at agriculture.gouv.fr> Jun 12, 2007
- 569 views
- Last edited Jun 13, 2007
Robert Craig wrote: > > CChris wrote: > > Definitely, this is a Windows only patch, so I check platform() before > > calling open_dll() and before using c_func(). > > > > I tested my mod standalone, and it works. This does not replace full > > testing, but means I can go ahead with it. > > > > The proposed modification is actually a _bug fix_: > > * Problem: Under Windows NT/2K/XP and later, paths may legitimately contain > > non-ASCII characters, as long as they belong to the system codepage. If > > the path in EUDIR, or any path inside EUINC, contains such a character, > > said path is not processed by path_open(), and include files therein are not > > found. > > * Cause of the problem: when getenv() returns a string, that string is > > encoded using the current Windows codepage. But the strings open() passes > > are interpreted using the OEM current codepage. Since they have different > > layouts, some characters are misinterpreted, and path_open() searches a path > > that very likely doesn't exist. > > * Current workaround: In the Advanced tab of system properties, change > > offending characters in paths to the ones with the right codes. > > Example: if a path ontains a e acute accent, coded #E9 in the Windows code > > page and #82 in the OEM codepage, replace that e acute by the character with > > code #82 in the Windows codepage. This happens to be a U acute. > > This workaround implies tweaking environment variables and rebooting the > > computer for changes to take effect; it is not satisfactory, but at least it > > works. > > * Suggested mod to front end, specifically to path_open(): > > If trying to open a file using a path from EUINC/EUDIR fails, if platform() > > is Windows and if the path has accented characters, then try again after > > converting this path using the CharToOemA() WinAPI. It has been in > > kernel32.dll since Windows 3.0 and probably before. > > Conversion applies to the path only, not the file name, since it is not > > stored in the environment variable. > > > > At least this is the right API to use, and the modified path_open() now > > finds > > files it was missing before. > > When I have tested with a real life interpreter (built under OW1.5), I'll > > (try to) submit that mod. Unless someone has a brighter idea. > > This sounds reasonable to me. > Maybe others have an opinion? > > I can give you full developer permissions on SourceForge > so you can check out official source code using SVN > (e.g. TortoiseSVN) and check it back in again after testing it > on your system. You just need to sign up for a free SourceForge id > and let me know what it is. > > I realize that 90% of the job will be learning how > to use SVN, but I want to wean the community off using > me (or Matt) to make all changes. It will be better if > we have several people who are able to make changes and > check them in (and then handle the almost-inevitable bug > reports and enhancement requests that will flow from that). > > I hope to get back to the 3.1 FreeBSD Release Candidate > later today. I had several distractions over the past few days. > It might be better to hold off on checking in your changes > until I do the full 3.1 official release for all platforms, > this week I hope. > > Thanks, > Rob Craig > Rapid Deployment Software > <a href="http://www.RapidEuphoria.com">http://www.RapidEuphoria.com</a> It's Ok; my SF id is CChris. I have other things waiting (most urgently fixing text printing using nonstandard options in win32lib), so that I certainly can wait. And I was aware too that there would be some learning investment on SVN... Feel free to email privately oedoc at free dot fr for any relevant request/info. CChris
12. Re: Modifying the Euphoria interpreter
- Posted by CChris <christian.cuvier at agriculture.gouv.fr> Jun 12, 2007
- 568 views
- Last edited Jun 13, 2007
Pete Lomax wrote: > > CChris wrote: > > No, it's even worse than that. > > Remember that, if ex.exe tries to compile a program which mentions > > open_dll(), it errors out even though the the function won't ever be > > executed because the call is protected by a "if platform()=??? then". > > The error is generated before the platform dependent code is removed. > > But replacing the call with "machine_func(50,"kernel32.dll") won't hurt. > > As well as the same things others have tried, I just tested this here: > }}} <eucode> > constant k32=open_dll("kernel32") > constant k32=machine_func(50,"kernel32") > </eucode> {{{ > Despite beginning to mistrust myself, I was right. Both the above fail equally > on ex.exe, since you cannot put a platform() check around either. > It is true, btw, to say that you cannot use constant definitions like the > above > in platform independent code. Instead you must use atom vars. > > The point I am trying to make here is that you seem to be locked into using > a direct call to machine_func when it will make no significant difference - > see the definition of open_dll() in dll.e. It is the machine_func call itself > which fails on ex.exe, NOT the trivial open_dll() wrapper... you can even > prove > this by putting a display inside open_dll just before the call to > machine_func: > it will show fine, just before ex.exe crashes. > > Regards, > Pete Of course I know the wrapper is trivial, and makes no difference. I thought I remembered that, if running a program using ex.exe, and if that program had lines like
atom k32 if platform()=WINDOWS then k32=open_dll("kernel32.dll") end if
I was getting an error for open_dll() not being supported under DOS. And then of course the wrapper would make a difference, since no one could object to a machine_func() call at compile time. But again, I'll have to check what actually happens. And as you said, the entry point can't be a constant, as it must appear inside an if block. CChris
13. Re: Modifying the Euphoria interpreter
- Posted by Bernie Ryan <xotron at bluefrog.com> Jun 12, 2007
- 581 views
- Last edited Jun 13, 2007
CChris wrote: > }}} <eucode> > atom k32 > if platform()=WINDOWS then > k32=open_dll("kernel32.dll") > end if > </eucode> {{{ It should be :
if platform()=WIN32 then
Bernie My files in archive: WMOTOR, XMOTOR, W32ENGIN, MIXEDLIB, EU_ENGIN, WIN32ERU, WIN32API Can be downloaded here: http://www.rapideuphoria.com/cgi-bin/asearch.exu?dos=on&win=on&lnx=on&gen=on&keywords=bernie+ryan
14. Re: Modifying the Euphoria interpreter
- Posted by Pete Lomax <petelomax at blueyonder.co.uk> Jun 13, 2007
- 586 views
CChris wrote: > I was getting an error for open_dll() not being supported under DOS. Ah, of course: that error message *IS* caused by a machine_func(50,...) statement being run, *NOT* from an open_dll() statement being defined. I think I can imagine the thought process you had, what you have to think of is what message should Eu best display on a bad machine_func(50,...) call? It is not going to be "machine_func opcode 50 not supported", is it? I trust that has got through now, but if not please voice any remaining items of confusion, as I think that it is vital anyone making such mods understands *exactly* what is going on. Regards, Pete
15. Re: Modifying the Euphoria interpreter
- Posted by CChris <christian.cuvier at agriculture.gouv.fr> Jun 13, 2007
- 609 views
Pete Lomax wrote: > > CChris wrote: > > > I was getting an error for open_dll() not being supported under DOS. > > Ah, of course: that error message *IS* caused by a machine_func(50,...) > statement > being run, *NOT* from an open_dll() statement being defined. > > > I think I can imagine the thought process you had, what you have to think of > is what message should Eu best display on a bad machine_func(50,...) call? > > It is not going to be "machine_func opcode 50 not supported", is it? > > > I trust that has got through now, but if not please voice any remaining items > of confusion, as I think that it is vital anyone making such mods understands > *exactly* what is going on. > > Regards, > Pete Ok, I found some time to check it out. Protecting the open_dll() statement by an "if platform()=" test enables a program containing it to run both under exw and ex, both under Windows and in a DOS box. So I assume this protection will be enough, although I'll perform a test with the newly built interpreters just to be sure. CChris
16. Re: Modifying the Euphoria interpreter
- Posted by Juergen Luethje <j.lue at gmx.de> Jun 13, 2007
- 580 views
CChris wrote: <snip> > Ok, I found some time to check it out. > Protecting the open_dll() statement by an "if platform()=" test enables a > program containing it to run both under exw and ex, both under Windows and > in a DOS box. So I assume this protection will be enough, although I'll > perform a test with the newly built interpreters just to be sure. What is the reason for integrating open_dll() into the interpreter? Regards, Juergen
17. Re: Modifying the Euphoria interpreter
- Posted by CChris <christian.cuvier at agriculture.gouv.fr> Jun 13, 2007
- 578 views
Juergen Luethje wrote: > > CChris wrote: > > <snip> > > > Ok, I found some time to check it out. > > Protecting the open_dll() statement by an "if platform()=" test enables a > > program containing it to run both under exw and ex, both under Windows and > > in a DOS box. So I assume this protection will be enough, although I'll > > perform a test with the newly built interpreters just to be sure. > > What is the reason for integrating open_dll() into the interpreter? > > Regards, > Juergen You may wish to check http://www.listfilter.com/EUforum/m14385.html for a detailed report. I initially considered using direct translation, not involving any API call. After all, it is only a matter of character recoding, which can be done using a simple lookup table. I quickly backed off considering that there are a few OEM and Windows code pages to cater for, each with its own 128-255 range mapping. It looked far more straightforward and reliable to ask the system for a proper translation. The bottom line is: since there is a Windows only issue which some Windows dll handles, the most sensible thing to do is to use it, hence an open_dll() call in the interpreter, as well as a c_func(). The call won't occur on other platforms. If similar issues arise under Linux/BSD, can anyone tell me about it, so that the fix could be extended to these cases? CChris
18. Re: Modifying the Euphoria interpreter
- Posted by Juergen Luethje <j.lue at gmx.de> Jun 14, 2007
- 599 views
CChris wrote: > Juergen Luethje wrote: > > > > CChris wrote: > > > > <snip> > > > > > Ok, I found some time to check it out. > > > Protecting the open_dll() statement by an "if platform()=" test enables a > > > program containing it to run both under exw and ex, both under Windows and > > > in a DOS box. So I assume this protection will be enough, although I'll > > > perform a test with the newly built interpreters just to be sure. > > > > What is the reason for integrating open_dll() into the interpreter? > > > > Regards, > > Juergen > > You may wish to check <a > href="http://www.listfilter.com/EUforum/m14385.html">http://www.listfilter.com/EUforum/m14385.html</a> > for a > detailed report. Ah yes, I see. > I initially considered using direct translation, not involving any API call. > After all, it is only a matter of character recoding, which can be done using > a simple lookup table. > > I quickly backed off considering that there are a few OEM and Windows code > pages to cater for, each with its own 128-255 range mapping. It looked > far more straightforward and reliable to ask the system for a proper > translation. I agree absolutely. The developers of Windows already have done that work, and there is no need for you to do it again. <snip> Regards, Juergen