Re: What's holding Euphoria back?
Robert Craig wrote:
> If you want to set up a large database, where the size
> of the disk files matters, and the speed matters
I have written such a database, which so far consists
of some 15,000 files, each around 50K, growing at
the rate of some 200 a week. Associated with those
files, are two dictionaries, also growing, one containing
some 24,000 horses, the other 1,800 jockeys, each with
a list of all the past races they were in (at least,
those my 15,000 data files know about). There is more:
cross-correlation matrices, reconstructed results of
past races (one per month per track), etc. When you
have such a mess to manage, speed does matter, and
disk space I suppose. For speed, no way out. Most of
the data and statistics maintainance is slowed down
by disk access. The only way out is to stick files onto
a RAM disk and leave them there for as long as they
are needed. At any rate, I once ran a benchmark
comparing Euphoria 1.5a and Borland Pascal. At best,
Euphoria was as fast as Pascal, at worst 7.5 times
as slow. So what? I started the job in Pascal on a
486DX running at 50MHz, I continued on an AMD-K6 at
200MHz. It ran *faster*. Space? Since once upon
a time I wrote on a Kaypro II (Z80, 2MHz), with two
floppy disks, each holding 180K, I remember what
it was to save disk space. And that hangover stayed
with me for a long time. Writing correlation matrices
in binary representation rather than EBCDIC. Writing
*everything* like that. Having to write utilities to
be able to *read* that human-hostile stuff. Then I
realized that I was wrecking my nerves for just a few bytes
less. It was still tenable when I had only 85M of hard
disk. But with 3.2G, then another 6.4G? No longer.
Everything is now in "long-hand", even the correlation
matrices are formatted so that *I* can read them.
At any rate, it's all very silly. Most of the
commercial stuff I have seen (and hacked into)
was awfully wasteful of disk space. They never seemed
to have heard of variable-length fields, for instance.
And the disk-thrashing was deafening. And poor sods
paid good money for that.
and R. Craig continues:
> I think
> you can easily write your own set of very simple low-level routines
> to take specific sequences and store them on disk compactly
> and efficiently
It is easy, I have done it. But now I know better.
|
Not Categorized, Please Help
|
|
