Euphoria
Ticket #149:
memory leak
-
Reported by
SDPringle
Apr 10, 2010
I have a EUPHORIA program that leaks memory. It's runs events based on a timer system every 800 milliseconds. The program isn't continuously collecting data into a sequence.
I suspect this can be demonstrated by any program that calls subroutines as frequently. This could be caused by unbalanced referencing objects. The program usually uses around 21MB at the beginning and then overtime leaks. I will be running a test with 3040 vs 3145 (with system memory) and post the results here.
Details
1. Comment by SDPringle
Apr 12, 2010
This is the memory usage according to Windows task manager of running a Windows clock program written in EUPHORIA interpreted by versions 3040 and 3145 respectively.
time version memory starting... 09:48 3040 15 MB 09:48 3145 14 MB
okay Window is up and going through winMain... 09:49 3040 23356kB 09:49 3145 21184kB 09:54 3040 23372kB 09:54 3145 21436kB 10:03 3040 23572kB 10:03 3145 21544kB 10:11 3040 23572kB 10:11 3145 21728kB 10:52 3040 43820kB 10:52 3145 39296kB
2. Comment by mattlewis
Apr 12, 2010
Can you tell us what the program is? There's nothing here that anyone other than you can do about this.
3. Comment by mattlewis
Apr 12, 2010
Converted Shawn's data to table:
| Time |
r3040 |
r3145 |
| 09:49 |
23,356kB |
21,184kB |
| 09:54 |
23,372kB |
21,436kB |
| 10:03 |
23,572kB |
21,544kB |
| 10:11 |
23,572kB |
21,728kB |
| 10:52 |
43,820kB |
39,296kB |
4. Comment by mattlewis
Apr 12, 2010
Ouch...this page needs some table stylin' love. Added extra columns for better legibility:
| Time |
... |
r3040 |
... |
r3145 |
| 09:49 |
|
23,356kB |
|
21,184kB |
| 09:54 |
|
23,372kB |
|
21,436kB |
| 10:03 |
|
23,572kB |
|
21,544kB |
| 10:11 |
|
23,572kB |
|
21,728kB |
| 10:52 |
|
43,820kB |
|
39,296kB |
Well, looks like we've plugged something since r3040.
5. Comment by SDPringle
Apr 13, 2010
The program is closed source. I'll try to reproduce it with some other program....
memusetest.exw
starts at 18136k and goes up include
Win32Lib.ew include std/machine.e include std/win32/msgbox.e constant
TheWindow = createEx( Window, "The Window", 0, 0, 0, 100, 100, 0, 0 ) constant
CleanFace = create( Pixmap, "",
TheWindow, 0, 0, 100, 100, WS_VISIBLE ) constant
ClockFace = create( Pixmap, "",
TheWindow, 0, 0, 100, 100, WS_VISIBLE )
integer counter counter = 0 sequence mem procedure TheWindow_onTimer (integer self, integer event, sequence params)params is ( int timerId) counter = counter + 1 printf(1,"Timer rang %d!\n", {counter} ) copyBlt(ClockFace, 0,0, CleanFace ) copyBlt(TheWindow, 0,0,ClockFace) ifdef SAFE then if compare( mem, { machine:memory_used(), machine:allocations() } ) != 0 then puts(1, "There has been a change in memory.") abort(0) end if end ifdef end procedure setHandler( TheWindow, w32HTimer, routine_id("TheWindow_onTimer")) mem = { machine:memory_used(), machine:allocations() } setTimer( TheWindow, 0, 10 ) WinMain(TheWindow, Normal) ifdef SAFE then message_box( machine:info(), "Memory Information", MB_OK ) end ifdef
Now add the following to safe.e:
public function info() integer tm = 0 for i = 1 to length( safe_address_list ) do tm += safe_address_list[i][BLOCK_LENGTH] end for return sprintf(""" Total memory allocations %10d Total memory allocated %10dB""", { length(safe_address_list), tm } ) end function
public function memory_used() integer tm = 0 for i = 1 to length( safe_address_list ) do tm += safe_address_list[i][BLOCK_LENGTH] end for return tm end function
public function allocations() return length( safe_address_list ) end function
now run with the -D SAFE option. Run this program for ten minutes and record the memory used in the task manager. Then do it again an hour after that and you should see some difference.
6. Comment by SDPringle
Apr 13, 2010
With the last test, I compiled EUPHORIA with all of the defaults configure gives you: Managed memory, no debug and no checking options. My ticket addition didn't come out well. I should preview first...
Let's try that again:
Add this to safe.e:
public function info()
integer tm = 0
for i = 1 to length( safe_address_list ) do
tm += safe_address_list[i][BLOCK_LENGTH]
end for
return sprintf("""
Total memory allocations %10d
Total memory allocated %10dB""",
{ length(safe_address_list), tm } )
end function
public function memory_used()
integer tm = 0
for i = 1 to length( safe_address_list ) do
tm += safe_address_list[i][BLOCK_LENGTH]
end for
return tm
end function
public function allocations()
return length( safe_address_list )
end function
Run memusagetest.exw with -D SAFE. Here is the text of memusagetest.exw:
--starts at 18136k and goes up
include Win32Lib.ew
include std/machine.e
include std/win32/msgbox.e
constant TheWindow = createEx( Window, "The Window", 0, 0, 0, 100, 100, 0, 0 )
constant CleanFace = create( Pixmap, "", TheWindow, 0, 0, 100, 100, WS_VISIBLE )
constant ClockFace = create( Pixmap, "", TheWindow, 0, 0, 100, 100, WS_VISIBLE )
integer counter
counter = 0
sequence mem
procedure TheWindow_onTimer (integer self, integer event, sequence params)--params is ( int timerId)
counter = counter + 1
printf(1,"Timer rang %d!\n", {counter} )
copyBlt(ClockFace, 0,0, CleanFace )
copyBlt(TheWindow, 0,0,ClockFace)
ifdef SAFE then
if compare( mem, { machine:memory_used(), machine:allocations() } ) != 0 then
puts(1, "There has been a change in memory.")
abort(0)
end if
end ifdef
end procedure
setHandler( TheWindow, w32HTimer, routine_id("TheWindow_onTimer"))
mem = { machine:memory_used(), machine:allocations() }
setTimer( TheWindow, 0, 10 )
WinMain(TheWindow, Normal)
ifdef SAFE then
message_box( machine:info(), "Memory Information", MB_OK )
end ifdef
7. Comment by LarryMiller
Apr 29, 2010
This program also leaks memory:
include std/text.e
sequence void
while 1 do
void=upper("aaaaaaaaaaaaaa")
for i=1 to 10000 do
end for
end while
I don't see anything wrong in the upper() function so I suspect the problem is in the interpreter.
8. Comment by mattlewis
Apr 29, 2010
Larry, which platform?
9. Comment by jimcbrown
Apr 29, 2010
I've never seen Larry use Linux or any other kind of Unix.
10. Comment by mattlewis
Apr 29, 2010
I was kinda hoping for that, since the windows version of upper uses C calls to windows. I'm thinking that the leak has to do with c calls and/or callbacks, though I haven't nailed it down yet.
11. Comment by LarryMiller
Apr 29, 2010
I should have mentioned, I am using XP. I have an application in development that has a serious memory leak that I have not been able to locate. I eventually found that a call to upper() was at least a major contributor to the problem.
12. Comment by mattlewis
May 02, 2010
I found and fixed a memory leak in svn:3198. I'm not sure that it's what was causing the problems here, however. The problem was not cleaning up temps created by sequences:
c_proc( foo, { bar, baz } )
The sequence passed to c_proc wouldn't be cleaned up properly. I'm not convinced that this would cause a large memory leak, since the next time the same code was executed, the temp should be cleaned up during its assignment.
13. Comment by LarryMiller
May 03, 2010
Using r3198 I get the same results as before.
14. Comment by RStowasser
May 06, 2010
If I do not use the for loop, the memory leak is still more impressive. In Windows Vista I used this code with rev 3200:
include std/text.e
--include wildcard.e
sequence txt
while 1 do
txt = upper("a")
-- txt = lower("a")
-- txt = "a" -- works
end while
I have to hurry to kill euiw.exe with the task manager. Using euc and openWatcom there seems to be no memory leak. If I include wildcard.e this seems to work ok too. So maybe the problem is in std/text.e?
If I leave the include statements away using txt = "a" with euiw.exe this will use 1588 kb of memory. Using wildcard.e will use 1748 kb of memory and applying std/text.e will use 6000 kb. Is this additional ram justified?
15. Comment by jimcbrown
May 06, 2010
The fact that std/text.e leaks but wildcard.e works, I think, makes it pretty clear that this is at least a bug in how c funcs are handled.
I strongly suspect that the leak is DEP related.
16. Comment by mattlewis
May 07, 2010
Yeah, I've been suspecting either callbacks or c calls, but memory leaks are a real pain in windows, and I haven't been able to reproduce a leak with similar code under linux.
17. Comment by mattlewis
May 07, 2010
Try svn:3201. It seems to have fixed the upper() leak, and I suspect it may fix the initial, win32lib leak as well.
The problem was a spurious reference when an inlined function was assigned to a variable instead of a temp.
18. Comment by LarryMiller
May 07, 2010
When testing with r3201 I no longer see the leak in the sample I submitted. Early tests with my own application also do not show the memory leak.
Testing on XP.
19. Comment by SPringle
May 13, 2010
I am using 3202 and if you hide its window and then show its Window repeatedly in the example left here you will see the memory use going up.
20. Comment by LarryMiller
May 15, 2010
After further testing with my app I find that the leak is till present. I cannot be 100% certain that there is not a problem with my code. After translation with euc the leak is much reduced but still present.
I have been using the "VM Size" column in Task Manager with XP. This is the private bytes of the process. This is a more reliable indicator of a leak than "Mem Usage" which refers to physical memory. Physical memory usage is always under the control of the OS and the levels will not be entirely predictable.
I am sure the developers know this but others may not.
21. Comment by DerekParnell
Aug 02, 2010
Just asking for an update on this issue.
It seems to me that there is no leak in Euphoria anymore but there might be one in win32lib. Can anyone show me some code that demonstrates a leak in Euphoria? If not, I'll close this issue.
22. Comment by DerekParnell
Aug 04, 2010
No longer seems to be a problem.
GitHub