OpenEuphoria

1. mem leaks win32 call_back

• Posted by raseu Jan 22, 2010
• 1519 views

first, apologies if i'm posting to the wrong place

version : Euphoria Interpreter 4.0.0 beta 2 (r2670) for Windows

using the following code under win32
causes memory leaks when viewed under Process tab
of Windows Task Manager.

include dll.e 
 
function callback_routine() 
   return 0 
end function 
 
procedure test_callback(sequence rname, integer loopcount) 
   atom headfunc, writefunc 
   headfunc  = routine_id(rname) 
   writefunc = routine_id(rname) 
   headfunc  = call_back({ '+', headfunc }) 
   writefunc = call_back({ '+', writefunc }) 
   printf (1, "loop = %4d\n", { loopcount }) 
end procedure 
 
--// main 
   integer i = 1 
   while (1) do 
      test_callback("callback_routine", i) 
      i += 1 
   end while 

the memory leaks can be traced to call_back( ...) function call.

after taking a look at the sources (.c and dll.e) it does appear
that call_back is allocating memory for function pointer handles
under windows.

is there any way to reclaim the memory allocated?

i am almost complete with writing an LGP (linear genetic programming)
library, that uses call_back() extensively (loads eui dlls as plugins,
then fixes up function pointers)


TIA

3. Re: mem leaks win32 call_back

• Posted by SDPringle Jan 24, 2010
• 1353 views

free() wont work. One is expected to call call_back() once for each function you want to get called by a C or assembler routine once. I never expected anyone to want to release the memory from this. You need to call free_code().

5. Re: mem leaks win32 call_back

• Posted by raseu Jan 24, 2010
• 1399 views

for the test program i posted earlier,
call_back calls allocate_protect with a
sequence length of 80 (size + BORDER_SPACE * 2)

i retested with the following additions
to the test code

include std/machine.e 

and

free_code(headfunc, 80) 
free_code(writefunc, 80) 

after creating call_back(s)


with an additional free(addr) inside std/dll.e
before return page_offset + page_addr

same issue with memory leaks

any pointers (no pun intended) in the right direction
would be of great help

7. Re: mem leaks win32 call_back

• Posted by raseu Feb 06, 2010
• 1187 views

1) ok, the call_back function is meant
to provide external routines the ability to call into euphoria,
but i would like the ability to dynamically setup/teardwon
call back interfaces on the fly as with other languages (namely 'C').
yes, i could resdesign my apps to use a more static/less flexible/fixed
function lookup table, but that's not what i want.

2) more importantly, if at some time in the future euphoria will
be available as an embedded language then the ability to provide
setup\teardown of embedded scripts that provides call backs surely
would need some method of allocating/deallocating function pointers

9. Re: mem leaks win32 call_back

• Posted by DerekParnell (admin) Feb 06, 2010
• 1222 views

Is this because you don't know which of your Euphoria routines is going to be called until the time you invoke the external function?

I think that you will find that this would be automated along with Garbage Collection for the released embedded script.

11. Re: mem leaks win32 call_back

• Posted by mattlewis (admin) Feb 06, 2010
• 1180 views

Maybe. It's pretty rare, though it's basically what I did with the callbacks for eu.ex. I've also used fptr.e to use somewhat exotic calling conventions, like MS's thiscall. There were also some standard callbacks required in COM programming that required more than the then-allowed number of parameters.

Matt

13. Re: mem leaks win32 call_back

• Posted by mattlewis (admin) Feb 06, 2010
• 1176 views

I don't follow you about eu.ex. It uses the same technique (just transplanted from fptr.e into execute.e) as fptr.e. Take a look at cb_std and cb_cdecl to see what I was talking about as far as adding an additional parameter for dispatching.

And I don't see what use freeing callbacks would be in eu.ex. When would you free them? Why?

Matt

15. Re: mem leaks win32 call_back

• Posted by mattlewis (admin) Feb 06, 2010
• 1238 views

OK, so for callbacks in general: When would you free them? Why?

Sorry, maybe I missed it, or just not getting it. This is an honest question about how you would decide to free a callback. In what sort of situation would this be useful? If there's a good reason for it, it's definitely something we could do.

I haven't looked closely at the changes Shawn recently made, but I suppose there's no reason why we couldn't recycle the memory used for callbacks. But it would complicate the code, add places for us to write bugs, etc, and I just can't justify it, because it seems like something that's likely unsafe, and doesn't really do all that much for you. I'd be happy to be shown to be wrong here, though, because maybe there is something cool that can be done with it.

Matt

17. Re: mem leaks win32 call_back

• Posted by mattlewis (admin) Feb 07, 2010
• 1138 views

I guess something that's guaranteed to only be called once, like for initialization is a good example of something that could be freed safely. I was thinking of what I believe is the main use of a callback, which is used in conjunction with GUI libraries as ways for the OS/window manager/etc to send messages to the user's program. About the only time it's safe to free that sort of callback is at the end of your program. Maybe some timer callbacks (assuming it even gets its own callback) are throw aways.

But maybe raseu is doing something completely different. That's what I'm interested in finding out. I guess I view this leaked memory in a similar manner to having messages sitting around in RAM. In fact, strings stored in constants probably take up way more memory in a typical program than the callbacks. Especially than callbacks for things like initialization.

Maybe. An example would be much more persuasive.

Yes, that's probably the obvious next step. But I still think we need a better case made for doing something like this.

It depends on the platform. On windows, we're allocating chunks of memory (page size, I think) due to the way we have to protect the memory. I suspect we should probably be doing that on other platforms, too. So in that case, we've got a custom malloc process where we could, in theory, add an ability to recycle old callbacks that are no longer required.

This all just seems like so much premature optimization though.

Matt

19. Re: mem leaks win32 call_back

• Posted by raseu Feb 08, 2010
• 1065 views

one of the projects i have is a wrapper for
the CURL libraries where i have been converting
some of my older 'C' code that creates/downloads/destroys connections
for url/ftp downloads as and when needed.

as part of the connection process i create callbacks
to buffer data.

porting the 'C' code to the euphoria equivalent yields
the following

... 
self[headfunc]  = routine_id("strip_cookies") 
self[writefunc] = routine_id("strip_data") 
self[headfunc]  = call_back({ '+', self[headfunc] }) 
self[writefunc] = call_back({ '+', self[writefunc] }) 
curl:setopt(self[handle], CURLOPT_HEADERFUNCTION, self[headfunc]) 
curl:setopt(self[handle], CURLOPT_WRITEFUNCTION,  self[writefunc]) 
etc ... 

which, if created/used/destroyed 'on the fly', will run into the mem leak
issues as outlined at the top of this thread

i was hoping to maintain 'like for like' code and flow control
between the two language versions as an aid for testing/debugging

anyways, not a show stopper.

for now, i'll just have to redesign some aspects of the code

would be ideal to have the have as much user control as possible
over allocated memory at some stage though.

thanks for the updates

btw, me is a he