OpenEuphoria

1. Starting project EuphoriaVista Interpreter

• Posted by jcmarsh1 Jan 02, 2009
• 1351 views

Euphoria on Vista

• Today, I am starting a project aimed at producing a stable and safe version of the Interpreter targeting the MS Windows Vista platform. I will be calling it tentatively "EuphoriaVista". This comes after the Euphoria Interpreter v3.1.1 for Windows, which seems to be targeting WinXP, crashed my system twice. I am not a full-time programmer, so I would be interested in whatever community support I could get with it.

1. One of the problems is that GUI Windows (API calls from the Interpreter) wont work with DEP on. DEP protects Vista from malicious code and leaving it off leaves my system vulnerable to attack and file system corruption from other programs, as well as the ones I'm debugging. File system corruption could lead to "Genuine Windows" Anti-Piracy Limited Access Auto-Protect Mode (which happened to me once). In that mode, an annoying dialog pops up and you are limited to Internet access only, while you try to verify that your Windows copy is genuine. Once it's verified as genuine, you get your computer back.
2. The next problem is that Microsoft is dropping support for older code and older APIs in favor of newer and "safer" APIs and system library functions (such as UNICODE instead of "ascii", TCHAR instead of CHAR for character data, StringCchPrintf instead of "sprintf" to avoid buffer overflow the length and the character count is passed, etc.)
3. Since Windows 2000 (perhaps since NT), Windows is a UNICODE system internally. Any "ascii" strings passed to API functions using "functionA" format have to be converted to a UNICODE string before they are processed. This is provided for back-wards compatibility, however, all new programs targeting NT systems should be built using the "functionW" version of the API function passing a full UNICODE UTF-16 "like" string to the function. [The exact UNICODE encoding is based on the OS version, but they are all based on a 16-bit WCHAR]
4. Most new programs targeting the MS Vista and later are meant to be written in "dot net" (.NET) a compiled executable format with a program compress inside it similar to Euphoria's *.il method. [Sun's Java uses a similar technique to produce Java class files.]
5. Microsoft provides a free [for a home user at least, I don't know about commercial, someone would have to help me understand the legal jargon] programming tool "Microsoft Visual C++ 2008 Express Edition" (and other Express editions in VB, C#, Web) that compiles for the Vista platform [I think all Vista versions and supported processors, there are many output formats it supports] and can produce "dot net" (.NET) executables. (http://www.microsoft.com/Express/)
6. 64-bit Operating Systems. A newer data type "long long" or "__int64" is used extensively, and the standard method for calling functions is different, but uniform across all function calls.
7. Different privilege levels, and security descriptors that a programmer now has to account for.
8. Vista and later systems use a method for protecting memory so that traditional Euphoria machine code embedded in source files does not work. [It has to be called from the program's own memory, I think, or it has to be hard-coded in the executable or in a project *.dll that the Interpreter can link to.]
9. It may be easier to map the "dot net" (.NET) framework to the Interpreter its self. Each call to a "dot net" (.NET) function that the Interpreter uses could be in a C style switch statement.

• Euphoria v3.1.1 still uses the "functionA" version of the Windows API. Many older libraries use outdated machine code that wont work on Vista.
• EuphoriaVista will only use "safe" functions as specified by Microsoft and use recommended Windows API functions for the target Windows version whenever possible.
• The goal is to write source code in Euphoria, and have it run on Vista in a safe and predictable manner.

These are only some of the problems when porting to Vista. I will be working on a "Hello World" Windowed App demo of my "EuphoriaVista" porting project to post to the archive.

I will also be reading up on results on the forum matching keyword Vista in the next few days, to see where the community stands on supporting the Vista OS.

jcmarsh

3. Re: Starting project EuphoriaVista Interpreter

• Posted by irv Jan 02, 2009
• 1349 views

Good luck with that. I have enough trouble trying to get Vista to run in a safe and predictable manner.

5. Re: Starting project EuphoriaVista Interpreter

• Posted by jimcbrown (admin) Jan 02, 2009
• 1319 views

Other than stating that 3.1.1 caused Vista to BSOD and the issue with DEP that iis being fixed in 4.0 alpha 3 (the DEP issue also applies to XP), you have not given a clear statement on what specifically is broken on Euphoria that makes it Vista incompatible. A few basic tests were done on Vista and eu 4.0 seems to work fine there. (More testing and bug reports are welcome of course.)

uname() is already Windows 7 compatible. :P

Integration with .NET and Java isn't necessarily bad. It is very complex (and would probably need to be a separate development branch at a minimum), but it could be worthwhile. Also, JNI and Mono work on non Windows platforms, so this could benefit other OSes besides Vista. (Mono even has some API compatibility with .NET I believe.)

You have some great ideas, jcmarsh. I look forward to seeing how you progress your project with them.

7. Re: Starting project EuphoriaVista Interpreter

• Posted by jimcbrown (admin) Jan 02, 2009
• 1321 views

That's up the the user who owns the box. That person might want to have DEP off for valid reasons.

XP supports DEP iirc. The code to detect different versions of Windows has already been written, so DEP support will exist when the executable runs on XP or Vista but that executable will still run on 98 without any problems.

The only examples you've stated of this are DEP and new Vista-only APIs. The former (aside from working on XP) has a backwards compatible approach and the latter can be done without the need for a separate Vista-only executable. (The APIs just need to be wrapped.)

DOS32 is totally different (almost as far apart from Vista as it is from OS X).

However, XP and 98 are mostly compatible. XP just has a larger API.

I don't know enough about Vista to comment, but I've heard it said that it was a smaller change from XP than XP was from NT.

ex.exe works fine on Vista IF you perform a few changes to the OS. You've correctly noted that this is supported through a backwards compatibility layer. (The majority of the issue is with ex.exe's usage of full screen mode.)

Not sure if I should mention the EUVISTA=1 option here.

This is not a bad idea. However, it is possible to have a single win32lib that handles all cases - attempts to actually use a function that isn't supported on 98 would be detected at run time and fail but the same code from the same library should work fine on Vista.

On the other hand, XP and NT are not going to lose support for the old 3.1 API so segregation here may not make sense. It makes more sense for Vista if Vista and WIndows 7 have lost or will lose older APIs (but this can be tested at run time using the same library and the same executable as that which runs on 98).

I guess there aren't enough copies of XP 64-bit to make sense to support it.

9. Re: Starting project EuphoriaVista Interpreter

• Posted by jcmarsh1 Jan 02, 2009
• 1327 views

Specifically, I would not run ex.exe on Vista even with DEP "off".

With DEP "on" window.exw doesn't work (but I heard that is being fixed). It returns with an error, the ex.err dump begins with:

C:\EU4a\demo\win32\window.exw:230 in procedure WinMain()  
A machine-level exception occurred during execution of this statement  
    szAppName = 2249440 
    hwnd = <no value> 
    msg = 21404904 
    wndclass = 21077648 
    WndProcAddress = 21050440 
    class = 13812233 
    id = 5 
    icon_handle = 60686645 
 
... called from C:\EU4a\demo\win32\window.exw:256  
 
 
Global & Local Variables 
 
[...] 
 
 C:\EU4a\demo\win32\window.exw: 
    LoadIcon = 22 
    LoadCursor = 23 
    GetStockObject = 24 
    RegisterClassEx = 25 
    CreateWindow = 26 
    ShowWindow = 27 
    UpdateWindow = 28 
    GetMessage = 29 
    TranslateMessage = 30 
    DispatchMessage = 31 
    PlaySound = 32' ' 
    BeginPaint = 33'!' 
    GetClientRect = 34'"' 
    DrawText = 35'#' 
    EndPaint = 36'$' 
    PostQuitMessage = 37'%' 
    DefWindowProc = 38'&' 
    wav_file = 2446312 
    Euphoria = 21404856 
    ps = 2427656 
    rect = 2249416 
    my_title = 2446344 

window.exw:230

    hwnd = c_func(CreateWindow, { 
                    0,                       -- extended style 
                    szAppName,               -- window class name 
                    my_title,                -- window caption 
                    WS_OVERLAPPEDWINDOW,     -- window style 
                    CW_USEDEFAULT,           -- initial x position 
                    CW_USEDEFAULT,           -- initial y position 
                    CW_USEDEFAULT,           -- initial x size 
                    CW_USEDEFAULT,           -- initial y size 
                    NULL,                    -- parent window handle 
                    NULL,                    -- window menu handle 
                    0 ,                 --hInstance // program instance handle 
                    NULL})              -- creation parameters 

It fails at the Window creation. I don't know if many of the programmers have tested it with DEP "on" so that's why I'm sharing this output. With DEP "off" it will run even with sound, however, it makes the system more unstable (and vulnerable). In my opinion, it would be best to leave DEP "on" if you get Vista. I don't know how Genuine Windows protection works on WinXP, as far as I know it just disables updates. On Vista it can lock you out with only limited Internet connection, until it can be restored under Safe Mode. Changing the DEP option while working with exw.exe activated the Genuine windows protection mode. I don't know what exactly set it off, but I would be glad when exw.exe fully supports DEP "on" with Windowed Apps.

Deciding which ideas I have time for and the know-how to do is a tough question for me. I can design things that will work, but they might not be the best way to do them. That's one reason I'll need help on creating something other people can make use of as well.

11. Re: Starting project EuphoriaVista Interpreter

• Posted by jcmarsh1 Jan 02, 2009
• 1308 views

Sorry, this doesn't work like instant messenger or phone, then I'd be able to correct the things I didn't know. I haven't set up the chat client yet.

You are right, I am only voicing my opinions for my new project with Vista and Euphoria.
I had forgot that DEP was on XP as well as Vista. It is more apparent to me on Vista.

lol. You guys are farther ahead than I thought. I knew that was possible, but I didn't think that you had written it already. However, Bernnie has been mentioning problems on the forum in the past about getting a stable version working for Win98. That is why I thought it might be better to split the installation package to have different executables for those platforms.

I see you don't understand my enthusiasm. A year ago, I wouldn't think it would be possible for me to customize the Euphoria Interpreter for my own needs, to add to it the stuff that I wanted. Today, many of those features have been added into version 4.0 including regular expressions and a decent math library. I want to add a graphics library for windows that is as easy to use as the one for DOS. Vista does not support DOS, especially Dos graphics in full-screen mode. With the Windows API working, even under DEP, then the programmer can make any sized window and draw to it, even making full screened (fast action or slow action) games with the Windows API. It would return the functionality that Euphoria had under DOS and under Win98 for newer versions of Windows under differing modes of operation, particularly Vista under DEP mode of operation.

ex.exe usage of memory poking has caused me a few problems, which is why I don't run it on Vista. It has caused a few crashes when I was first testing it on Vista. It doesn't like when ex.exe pokes to memory and then tries to call that as an Assembly function. The same program worked fine on Win98, but crashed on Vista (32-bit) x86. I image the problem has already been noted, that Vista doesn't like a program calling a function in a certain part of memory. I don't think many of the old DOS techniques work. I've only tested a few samples, and figured that Vista dropped a lot of support for DOS.

I did read something about a desire at least from Microsoft to "phase out" older technology in favor of new technology. It goes something like this, with Windows 3.1 there was one way to do something, with Windows 98 they added two more ways to do it, with Windows NT they added another way, with .NET there is yet another way. Which way should we do it now? Let's recommend the newest API syntax.

We'll have to wait and see if they really do "phase out" the older APIs.

13. Re: Starting project EuphoriaVista Interpreter

• Posted by mattlewis (admin) Jan 03, 2009
• 1282 views

I think that MS would *like* for everyone to use .NET, but I don't think that there's any danger of native code going away. Having said that, I did have a mostly working (mostly, as in, not all features of euphoria were supported) MSIL version of ooeu working. It outputted MSIL code which used an assembly written in C# that served as the 'back-end.' It was horribly slow, due to the way I emulated euphoria's datatypes. There's probably a better way.

At some point, euphoria will likely make the jump to 64 bits, but I think there's a lot of work to be done before that. And 64-bit operating systems aren't ready for prime time, other than in the server room.

I think you must be thinking of win32lib. It will take a fair amount of work to make it work with unicode. [shameless plusg]I'd recommend using wxEuphoria, since it already does unicode.[/plug]

Matt

15. Re: Starting project EuphoriaVista Interpreter

• Posted by jcmarsh1 Jan 03, 2009
• 1275 views

That's cool.

Yes, I was thinking more of win32lib when I wrote about that. I have not tested wxEuphoria much (it was in early stages when I first looked at it), I will have to look at it again.

I think what I am talking about is there should be standard include files for the Windows functions that are exposed on a more primitive level so that other libraries can be more easily based on them. This might make win32lib and IDE maintenance much easier.

It would define all of the constants, a few management functions, and a bunch of definitions.

17. Re: Starting project EuphoriaVista Interpreter

• Posted by jcmarsh1 Jan 03, 2009
• 1263 views

Would I have to recompile EX.EXE to get EUVISTA=1 to work???

[Windowed Dialog Box] 
16 bit MS-DOS Subsystem 
 
 
C:\Windows\system32\cmd.exe - ex wire 
This system does not support fullscreen mode. Choose 'Close' to terminate the  
application. 
 
	[Close]	[Ignore] 
[End Windowed Dialog Box] 

19. Re: Starting project EuphoriaVista Interpreter

• Posted by jimcbrown (admin) Jan 03, 2009
• 1282 views
• Last edited Jan 04, 2009

Take a look at dos_rescue.ew from http://www.rapideuphoria.com/langwar.zip

21. Re: Starting project EuphoriaVista Interpreter

• Posted by jcmarsh1 Jan 04, 2009
• 1311 views

If you have time, or specific interest, take a look at: "List of Microsoft Windows application programming interfaces and frameworks" (Beware before you look at it, it is a long dizzing list.)

http://en.wikipedia.org/wiki/List_of_Microsoft_Windows_application_programming_interfaces_and_frameworks

I'd like to be able to link to some of those frameworks now that call_back() works using xcall_back() from dep.zip (the DEP enabled version of the function)

After that I could write some code to link from Euphoria to Sun's Java using the generic C linkage method. For now, I am simply dynamically linking from the Interpreter, later once I know that works, I can start thinking about a Euphoria to Java (bit-code) translator.

23. Re: Starting project EuphoriaVista Interpreter

• Posted by jcmarsh1 Jan 07, 2009
• 1261 views

lol - jcmarsh is just my screen name for forums, James Cook is my real name.

Those are some good ideas.

25. Re: Starting project EuphoriaVista Interpreter

• Posted by mattlewis (admin) Jan 07, 2009
• 1241 views

I think the right approach is to make changes to euphoria, so that call_back works as expected with or without DEP turned on by the user.

Submitting code for euphoria is pretty informal. Either post here or [better] email the developer list telling us about your patch. Then, if it's reasonable, etc, a developer can commit it.

Matt