Re: De-allocating memory - time issue
On 19 Sep 2002, at 1:17, petelomax at blueyonder.co.uk wrote:
>
> I have a *HUGE* object of some 10 million entries. (I now know the
> exact size 'cos I coded a status window to let me know)
> [please read this all the way thru before responding]
>
> Given that I have 48MB of ram, that 10 million bears resemblance to me
> running out of physical memory (4 bytes per integer, yep), and the ole
> PC begins to disk thrash.
>
> So, I code a "Cancel" button, press it and the program waddles on
> longer than expected.
>
> So I trace it to the line
>
> stack={}
>
> That line alone takes 3 minutes to execute.
>
> Now, I kind of understand there are difficulties abound in the general
> case deallocating ten million entries, most of which are copies of
> copies of copies (however many it takes to get to ten million).
>
> stack is an array of approx repeat({},55000) of which only the middle
> 3200 have been updated, however each such that stack[i] is set to
> stack[i+/-1]&{x,y}.
>
> So I guess reference counts are off the scale.
>
> The annoying point here, then, assuming it *is* the problem, is that
> the ref count handling is, in this unusual case, wholly irrelevant.
>
> I definately doubt there is any way to improve this, it really is not
> any kind of major problem in any sense, but on the off-chance, I am
> just asking if anyone has any suggestions.
Once you run out of ram, you are at the mercy of the OS and disk speeds.
I'd point you to the Bladed Beowulf site, but the DOE took it down about 2
days before 9-11. Not that i can afford that either, assuming it ever makes it
into production, but i ran across it while looking into a similar problem as
your's: the problem doesn't fit into the computer, and no single cpu/ram
combination is going to be fast enough.
Just last night, one program ate 304megabytes, and stopped itself. I still
haven't figured out how it used so much ram.
Kat
|
Not Categorized, Please Help
|
|
