1. FreeLibrary()


FreeLibrary() seems to be a windose call, and it's equated to FreeModule in winbase.h. Seems to be equated to a few other terms also. But it's not made availale for us lowly mortals to call? How can we commoners, we dregs, access FreeLibrary in Eu?

useless

new topic     » topic index » view message » categorize

2. Re: FreeLibrary()

useless_ said...


FreeLibrary() seems to be a windose call, and it's equated to FreeModule in winbase.h. Seems to be equated to a few other terms also. But it's not made availale for us lowly mortals to call? How can we commoners, we dregs, access FreeLibrary in Eu?

Here's a simple wrapper for FreeLibrary:

include std/dll.e 
 
constant 
    KERNEL32 = open_dll( "kernel32.dll" ), 
    FREELIBRARY = define_c_func( KERNEL32, "FreeLibrary", { C_POINTER }, C_INT ) 
 
public function free_library( atom dll ) 
    return c_func( FREELIBRARY, { dll } ) 
end function 

This should work with dlls opened with open_dll(), as we simply return the value returned by LoadLibrary.

Matt

new topic     » goto parent     » topic index » view message » categorize

3. Re: FreeLibrary()

mattlewis said...
include std/dll.e 
 
constant 
    KERNEL32 = open_dll( "kernel32.dll" ), 
    FREELIBRARY = define_c_func( KERNEL32, "FreeLibrary", { C_POINTER }, C_INT ) 
 
public function free_library( atom dll ) 
    return c_func( FREELIBRARY, { dll } ) 
end function 

This should work with dlls opened with open_dll(), as we simply return the value returned by LoadLibrary.

I'm sure it's a rare use case, but I can see this being quote useful for a larger program that had to dynamically load modules in and out of memory.

And so...

  • can we add this to the standard library?
  • would it just be a call to dlclose() on UNIX side?
  • should it be called close_dll() instead?

-Greg

new topic     » goto parent     » topic index » view message » categorize

4. Re: FreeLibrary()

ghaberek said...
mattlewis said...
include std/dll.e 
 
constant 
    KERNEL32 = open_dll( "kernel32.dll" ), 
    FREELIBRARY = define_c_func( KERNEL32, "FreeLibrary", { C_POINTER }, C_INT ) 
 
public function free_library( atom dll ) 
    return c_func( FREELIBRARY, { dll } ) 
end function 

This should work with dlls opened with open_dll(), as we simply return the value returned by LoadLibrary.

I'm sure it's a rare use case, but I can see this being quote useful for a larger program that had to dynamically load modules in and out of memory.

And so...

I agree - it's a rare case.

ghaberek said...
  • would it just be a call to dlclose() on UNIX side?

Yes.

ghaberek said...
  • should it be called close_dll() instead?

-Greg

Yes.

ghaberek said...
  • can we add this to the standard library?

Yes!

new topic     » goto parent     » topic index » view message » categorize

5. Re: FreeLibrary()


I ran a tiny app for loading and unloading a dll 10k times, to time it, and found a memory leak. The app is at http://openeuphoria.org/pastey/153.wc , there's a short pause at the end of the first loop to catch the memory use total, and then the any_key() at the end to see the final memory use. First run uses ~10 megabytes, but use grows to ~100 megabytes after 10k runs. It's keeping ~9k of ram for each loop thru the load-unload loop. The dll source is at http://openeuphoria.org/pastey/154.wc .



useless

new topic     » goto parent     » topic index » view message » categorize

6. Re: FreeLibrary()

eukat said...


I ran a tiny app for loading and unloading a dll 10k times, to time it, and found a memory leak. The app is at http://openeuphoria.org/pastey/153.wc , there's a short pause at the end of the first loop to catch the memory use total, and then the any_key() at the end to see the final memory use. First run uses ~10 megabytes, but use grows to ~100 megabytes after 10k runs. It's keeping ~9k of ram for each loop thru the load-unload loop. The dll source is at http://openeuphoria.org/pastey/154.wc .



eukat

Part of this is due to the internals of define_c_func/proc. Everytime define_c_func/proc is called, a wrapper is made of the raw function pointer along with extra info (like the parameters and return type), and then a "C routine id" is returned.

Since you use the w32api to unload the shared object, the backend/runtime doesn't know that the shared object has been unloaded and keeps those wrappers around, wasting memory. AFAIK there's no way to free that memory short of ending the entire program.

9 kilobytes for a single define_c_func() call seems excessive, though. I was expecting more like 10 bytes or something (which times a million would only take up under 10 megabytes). Hopefully Matt can shed some light on this.

new topic     » goto parent     » topic index » view message » categorize

7. Re: FreeLibrary()

jimcbrown said...

Part of this is due to the internals of define_c_func/proc. Everytime define_c_func/proc is called, a wrapper is made of the raw function pointer along with extra info (like the parameters and return type), and then a "C routine id" is returned.

Since you use the w32api to unload the shared object, the backend/runtime doesn't know that the shared object has been unloaded and keeps those wrappers around, wasting memory. AFAIK there's no way to free that memory short of ending the entire program.

9 kilobytes for a single define_c_func() call seems excessive, though. I was expecting more like 10 bytes or something (which times a million would only take up under 10 megabytes). Hopefully Matt can shed some light on this.

So, the backend could probably track those resources and then release them with a call to close_dll(). That would probably be good...

-Greg

new topic     » goto parent     » topic index » view message » categorize

8. Re: FreeLibrary()

jimcbrown said...
eukat said...


I ran a tiny app for loading and unloading a dll 10k times, to time it, and found a memory leak. The app is at http://openeuphoria.org/pastey/153.wc , there's a short pause at the end of the first loop to catch the memory use total, and then the any_key() at the end to see the final memory use. First run uses ~10 megabytes, but use grows to ~100 megabytes after 10k runs. It's keeping ~9k of ram for each loop thru the load-unload loop. The dll source is at http://openeuphoria.org/pastey/154.wc .



eukat

Part of this is due to the internals of define_c_func/proc. Everytime define_c_func/proc is called, a wrapper is made of the raw function pointer along with extra info (like the parameters and return type), and then a "C routine id" is returned.

Since you use the w32api to unload the shared object,


I am open to using other methods, if you know of one, show me it working, please.

jimcbrown said...

the backend/runtime doesn't know that the shared object has been unloaded and keeps those wrappers around, wasting memory. AFAIK there's no way to free that memory short of ending the entire program.


If the back end creates those wrappers on demand, then the back end should have a named pointer to them, so an unload operation by that name should make that memory available for the next load(open_dll) operation. In this case, i am performing the loop on a DLL of the same name, it's not even like i am changing name, or DLL size, or number of functions in the DLL, or etc etc..

jimcbrown said...

9 kilobytes for a single define_c_func() call seems excessive, though. I was expecting more like 10 bytes or something (which times a million would only take up under 10 megabytes). Hopefully Matt can shed some light on this.


I agree. Granted the OS prolly cached the whole operation, since i didn't change up the DLL i loaded, but swapping in-out 500 DLLs per second is impressive, i think.

useless

new topic     » goto parent     » topic index » view message » categorize

9. Re: FreeLibrary()

eukat said...


I am open to using other methods, if you know of one, show me it working, please.

The only other method I know of is to modify the backend to do what you and Greg describe, below. Obviously, I can't show this working without actually implementing it.

eukat said...
jimcbrown said...

the backend/runtime doesn't know that the shared object has been unloaded and keeps those wrappers around, wasting memory. AFAIK there's no way to free that memory short of ending the entire program.


If the back end creates those wrappers on demand, then the back end should have a named pointer to them, so an unload operation by that name should make that memory available for the next load(open_dll) operation. In this case, i am performing the loop on a DLL of the same name, it's not even like i am changing name, or DLL size, or number of functions in the DLL, or etc etc..

new topic     » goto parent     » topic index » view message » categorize

10. Re: FreeLibrary()

ghaberek said...

So, the backend could probably track those resources and then release them with a call to close_dll(). That would probably be good...

-Greg

I agree. We definitely need a proper close_dll(). Right now, the only way to workaround this would be to wrap dlopen()/dlsym()/dlclose() and directly use the operating system calls to manage everything, bypassing the stdlib layer.

new topic     » goto parent     » topic index » view message » categorize

11. Re: FreeLibrary()

jimcbrown said...

Since you use the w32api to unload the shared object, the backend/runtime doesn't know that the shared object has been unloaded and keeps those wrappers around, wasting memory. AFAIK there's no way to free that memory short of ending the entire program.

9 kilobytes for a single define_c_func() call seems excessive, though. I was expecting more like 10 bytes or something (which times a million would only take up under 10 megabytes). Hopefully Matt can shed some light on this.

I'm not so sure that this is the full story now.

I tried cutting out std/dll.e entirely here, using only the operating system library (plus a small helper c lib to deal with the E_SEQUENCE transition), and on startup it uses 14MB but then only uses 2MB more (16MB total) for 10k runs.

However, for 90k runs (which takes 27 seconds), it uses 75MB total.

http://openeuphoria.org/pastey/155.wc

I even tried cutting out the calls to the helper lib, in case repeated c_func() calls or the creation of new sequences was the culprit, but I still saw 75MB after 90k runs.

I guess even using dlclose() doesn't do the job.

new topic     » goto parent     » topic index » view message » categorize

Search



Quick Links

User menu

Not signed in.

Misc Menu