1. high file count in application
Hello to Euphoria programming fans...
I've been playing with Euphoria 1.5 (more recently, the
complete edition of 2.3), attempting to build an Earth
science application. It relies on editing of EU sequences,
assembly of libraries of small files consisting of text docs
and EU sequences (scripts) - the scripts are then assembled
into hierarchy - into a "model", which can be saved and
then run, using a "viewer" application (choosing the
small file approach in order to conserve memory
for the application, flexibility and compatibility
between the DOS and Windows version.) The problem with
this is that the directories will surely have hundreds
of these files,to slow up people's scandisk and
virus scanners. Does anyone know the best, fast EU
include for archiving such files so that they can be
broken out for each session? (Or is a call to an
external zip/unzip application be more efficient?)
Peter
2. Re: high file count in application
Euphoria has a pre-written database library called EDS (Euphoria Database
System) and it's quite easy to use. I'm sure you could create a table called
"Files" then for each record, have the key be the filename and the data be
the file data. So your database would look like this:
table "Files"
record 1:
key: "my_file01.dat"
data: {sequence of data here...}
record 2:
key: "another file.dat"
data: {more file data...}
You could then list the files in your application, thereby having a
'virtual' directory in one file. When the user requests a file, grab the
data from the database. Simple. Quick. Moderately pain-free.
HTH,
~Greg
----- Original Message -----
From: "Peter Spaulding" <oregeopeter at aol.com>
To: <EUforum at topica.com>
Cc: <orepeter at yahoo.com>
Sent: Monday, December 01, 2003 10:07 AM
Subject: high file count in application
>
>
> Hello to Euphoria programming fans...
> I've been playing with Euphoria 1.5 (more recently, the
> complete edition of 2.3), attempting to build an Earth
> science application. It relies on editing of EU sequences,
> assembly of libraries of small files consisting of text docs
> and EU sequences (scripts) - the scripts are then assembled
> into hierarchy - into a "model", which can be saved and
> then run, using a "viewer" application (choosing the
> small file approach in order to conserve memory
> for the application, flexibility and compatibility
> between the DOS and Windows version.) The problem with
> this is that the directories will surely have hundreds
> of these files,to slow up people's scandisk and
> virus scanners. Does anyone know the best, fast EU
> include for archiving such files so that they can be
> broken out for each session? (Or is a call to an
> external zip/unzip application be more efficient?)
> Peter
>
>
>
> TOPICA - Start your own email discussion group. FREE!
>
>
3. Re: high file count in application
On 1 Dec 2003, at 15:07, Peter Spaulding wrote:
>
>
> Hello to Euphoria programming fans...
> I've been playing with Euphoria 1.5 (more recently, the
> complete edition of 2.3), attempting to build an Earth
> science application. It relies on editing of EU sequences,
> assembly of libraries of small files consisting of text docs
> and EU sequences (scripts) - the scripts are then assembled
> into hierarchy - into a "model", which can be saved and
> then run, using a "viewer" application (choosing the
> small file approach in order to conserve memory
> for the application, flexibility and compatibility
> between the DOS and Windows version.) The problem with
> this is that the directories will surely have hundreds
> of these files,to slow up people's scandisk and
> virus scanners. Does anyone know the best, fast EU
> include for archiving such files so that they can be
> broken out for each session? (Or is a call to an
> external zip/unzip application be more efficient?)
Tiggr does data mining, and as a result also has 100's of files per directory,
adding at least one directory per day and 250 to 600 files per day. That's at
least 500K bytes/day, just *one* subdirectory built last year has 414
megabytes in 118,306 files,, a bit more than EDS can deal with in ram.
Loading that as one EDS file isn't possible. But scandisk needs only(??) 45
minutes to scan that one harddrive, a few gigabytes. Good arguement for a
separate fileserver box, yes? And an 8bit string type, Rob? <wink@Karl>
Kat
4. Re: high file count in application
Have a look at the generic library "Save Compressed Sequences" by Ralf
Nieuwenhuijsen. It has routines for loading and saving sequences..
Also take a look at generic library "Save Euphoria Objects In Memory" by
Jordah Ferguson.
The "LZ" or "BZip2" compression libraries offer some pretty good
compression and are supported by many mainstream archivers. And could be
viewd mod'd easily.
Personaly I think incorperating compression, loading and saving metheds
in your proggie itself would be better than relying on a third party
archiver. In theory this would save alot of access time reading files.
----- Original Message -----
From: "Peter Spaulding" <oregeopeter at aol.com>
To: <EUforum at topica.com>
Cc: <orepeter at yahoo.com>
Sent: Tuesday, December 02, 2003 2:07 AM
Subject: high file count in application
>
>
> Hello to Euphoria programming fans...
> I've been playing with Euphoria 1.5 (more recently, the
> complete edition of 2.3), attempting to build an Earth
> science application. It relies on editing of EU sequences,
> assembly of libraries of small files consisting of text docs
> and EU sequences (scripts) - the scripts are then assembled
> into hierarchy - into a "model", which can be saved and
> then run, using a "viewer" application (choosing the
> small file approach in order to conserve memory
> for the application, flexibility and compatibility
> between the DOS and Windows version.) The problem with
> this is that the directories will surely have hundreds
> of these files,to slow up people's scandisk and
> virus scanners. Does anyone know the best, fast EU
> include for archiving such files so that they can be
> broken out for each session? (Or is a call to an
> external zip/unzip application be more efficient?)
> Peter
>
>
>
> TOPICA - Start your own email discussion group. FREE!
>
>
> --
> Incoming mail is certified Virus Free.
> Checked by AVG anti-virus system (http://www.grisoft.com).
> Version: 6.0.545 / Virus Database: 339 - Release Date: 27/11/03
>
---
--
