1. RE: 2.4 weirdness -- first report
Robert Craig wrote:
> Andy Serpa writes:
> > I sometimes get a machine-level crash with both 2.3 & 2.4. I'm not
> > doing any peeking or poking. I cannot reliably reproduce this crash
>
> Please send me the program. I'll try to reproduce it on
> one of my machines here.
>
Sorry, can't. It's huge, and is tied up with a big database. I'll try
to come up with something leaner that is doing more or less the same
thing and see if I can get it to crash.
> > Now then, I have a free RAM indicator on my toolbar,
> > and I can watch the amount of free memory go down as
> > my programs run. With 2.3, the memory
> > goes down to a certain level and stays there. With 2.4, during those 30
> > seconds, the memory creeps back up (about 30 MB). So I assume you've
> > done something different with the garbage collection, and it is taking
> > forever to release those temporary sequences for some reason?
>
> Yes, I've changed the storage allocator significantly.
> I'm using a WIN32 API routine directly.
> It might be giving the memory back to the system in this situation,
> rather than holding on to many megabytes of it. That takes more time
> though.
> Perhaps you are now getting some disk paging activity too.
>
Hmmm...
No disk activity that I can tell. And I've got plenty of RAM.
This could be trouble -- please consider going back to the old allocator
or doing something about that. In theory, giving the memory back to the
system is good, but not if is going to take forever to do it. That
particular program is now basically unusable with 2.4. With 2.3 it is 0
seconds, with 2.4 it is 30 seconds -- for a function that will be called
over & over. Am I going to be stuck with 2.3 forever now?
(See my other post about "speed" -- surely you don't want a general
slowdown for any program which happens to use lots of sequences in a
function. Euphoria is all about sequences!)
> > return NIL
>
> When you execute a "return" statement, it will free up all the
> private variables and temps used in the subroutine.
>
Yeah, make it go faster. Please. I'm worried now.
2. RE: 2.4 weirdness -- first report
>
> No disk activity that I can tell. And I've got plenty of RAM.
>
> This could be trouble -- please consider going back to the old allocator
>
> or doing something about that. In theory, giving the memory back to the
>
> system is good, but not if is going to take forever to do it. That
> particular program is now basically unusable with 2.4. With 2.3 it is 0
>
> seconds, with 2.4 it is 30 seconds -- for a function that will be called
>
> over & over. Am I going to be stuck with 2.3 forever now?
>
> (See my other post about "speed" -- surely you don't want a general
> slowdown for any program which happens to use lots of sequences in a
> function. Euphoria is all about sequences!)
>
I made some small programs with functions to eat up some memory with
giant sequences, and in every case 2.4 is faster than 2.3 in both
filling up the memory (*many* times faster when building up a sequence
with '&' or append) and returning from the function. But the memory
does not appear to be released (returned to system) as it does it my
other program. So benchmark programs show 2.4 faster than 2.3, but my
"real" programs run slower, in the one case massively so (I just
verified that again to make sure I wasn't crazy -- still takes 30
seconds to exit function.) Now I'm really confused...
-- Andy
3. RE: 2.4 weirdness -- first report
- Posted by gertie at visionsix.com
Feb 24, 2003
On 23 Feb 2003, at 23:59, Andy Serpa wrote:
>
>
> Robert Craig wrote:
> > Andy Serpa writes:
> > > I sometimes get a machine-level crash with both 2.3 & 2.4. I'm not
> > > doing any peeking or poking. I cannot reliably reproduce this crash
> >
> > Please send me the program. I'll try to reproduce it on
> > one of my machines here.
> >
>
> Sorry, can't. It's huge, and is tied up with a big database. I'll try
> to come up with something leaner that is doing more or less the same
> thing and see if I can get it to crash.
>
> > > Now then, I have a free RAM indicator on my toolbar,
> > > and I can watch the amount of free memory go down as
> > > my programs run. With 2.3, the memory
> > > goes down to a certain level and stays there. With 2.4, during those 30
> > > seconds, the memory creeps back up (about 30 MB). So I assume you've done
> > > something different with the garbage collection, and it is taking forever
> > > to
> > > release those temporary sequences for some reason?
> >
> > Yes, I've changed the storage allocator significantly.
> > I'm using a WIN32 API routine directly.
> > It might be giving the memory back to the system in this situation,
> > rather than holding on to many megabytes of it. That takes more time
> > though.
> > Perhaps you are now getting some disk paging activity too.
> >
>
> Hmmm...
>
> No disk activity that I can tell. And I've got plenty of RAM.
>
> This could be trouble -- please consider going back to the old allocator
> or doing something about that. In theory, giving the memory back to the
> system is good, but not if is going to take forever to do it. That
> particular program is now basically unusable with 2.4. With 2.3 it is 0
> seconds, with 2.4 it is 30 seconds -- for a function that will be called
> over & over. Am I going to be stuck with 2.3 forever now?
Possibly. I am stuck with old versions of win32lib.
> (See my other post about "speed" -- surely you don't want a general
> slowdown for any program which happens to use lots of sequences in a
> function. Euphoria is all about sequences!)
>
>
> > > return NIL
> >
> > When you execute a "return" statement, it will free up all the
> > private variables and temps used in the subroutine.
> >
>
> Yeah, make it go faster. Please. I'm worried now.
That can't be. Any return from a function or procedure will free it's vars, in
any
language i have used. The way around it is to use global vars and null them.
What would help is a separate way to enable garbage collection, at times
you know the program can be idle. So run the 2.3 gc all the time, and the 2.4
version gc as called by the programmer?
Kat
4. RE: 2.4 weirdness -- first report
Robert Craig wrote:
> Andy Serpa writes:
> > I just verified that again to make sure I
> > wasn't crazy -- still takes 30
> > seconds to exit function. Now I'm really confused...
>
> There's not much I can do without seeing your code.
> I can imagine that if you have millions of floating-point numbers,
> you might get into a situation where it would take a long time
> to free them. It's also remotely possible that the crash you
> occasionally see might be related to the 30 seconds problem.
> i.e. the bug is manifesting itself in two different ways.
> There are things I can do to address this problem, for instance
> I keep a cache of recently-free blocks to avoid calling HeapAlloc
> (or malloc) so often. That could be tuned if I had some statistics
> on what your program is doing.
>
I've semi-isolated the situation that causes this -- it only occurs (the
huge delay) when the function in question calls a certain 2nd function.
Funny thing is that 2nd function also just returns NIL, so it seems that
the cleanup would occur at that point instead of when the 1st function
exits, but I will try to come with with a short program that reproduces
the behavior.
