1. Question regarding EDS

Yes, it's me, back again. I have an EDS file "taa.edb" The file size a few days ago was 65,412 KB. It slowly grows. I now have the problem that if I add something (about 100 bytes or so) the file size jumps to 2,473,092 KB. It seems that the number of tables has jumped to 32654. It seems that all these numbers are approximately an integer power of 2. Has anyone seen this before? What can I do?

Regards,
jd

new topic     » topic index » view message » categorize

2. Re: Question regarding EDS

I've had a few edita.edb that were corrupt beyond recovery.

I assume db_compress fails, but essentially what that does internally is pretty much the best you can do: read what you can and create a new file.
If, as you say, there are apparently 32,654 tables, you may fare better with a bespoke program that reads the dozen-or-so tables that should exist and does not try to copy the 32,6nn that are gibberish.
Ultimately you have no choice but to delete the corrupt taa.edb - and be thankful this sort of thing does not happen very often.

Regards,
Pete

new topic     » goto parent     » topic index » view message » categorize

3. Re: Question regarding EDS

Have you tried the check_free_list() routine?

Also, you can use the db_dump() routine to get a better picture of what your DB ontains.

new topic     » goto parent     » topic index » view message » categorize

4. Re: Question regarding EDS

petelomax said...

I've had a few edita.edb that were corrupt beyond recovery.

I assume db_compress fails, but essentially what that does internally is pretty much the best you can do: read what you can and create a new file.
If, as you say, there are apparently 32,654 tables, you may fare better with a bespoke program that reads the dozen-or-so tables that should exist and does not try to copy the 32,6nn that are gibberish.
Ultimately you have no choice but to delete the corrupt taa.edb - and be thankful this sort of thing does not happen very often.

Regards,
Pete

Yes, I had written a program to open the database and copy the relevant tables into a new database. Not difficult as long as the database will open. I have used the check routine but all it says is that blocks are too big. I have for some time now made automatic backups (saves MANY historical copies) each time any program opens the data base. I do a file_copy BEFORE I open the database. This saves a lot of pain and agony. I guess the thrust of my comment is why do I have these problems. A database is useless if it is not reliable. I like EDS because it is integral. I don't like to carry along extra software with a program. It is also useful for all kinds of temporary storage. I should look at the code for EDS I suppose; but, I'm no programmer. Maybe there is a way to implement checking when reading or after writing the blocks. Anyway, thanks for your help.

Do you have any ideas for alternate ways to implement a database in euphoria?

Regards,
jd

new topic     » goto parent     » topic index » view message » categorize

5. Re: Question regarding EDS

DerekParnell said...

Have you tried the check_free_list() routine?

Also, you can use the db_dump() routine to get a better picture of what your DB ontains.

Thanks...

check_free_list just says block too big. db_get_errors reports nothing amiss. db_dump looks normal.

Thanks for your help.

Regards,
jd

new topic     » goto parent     » topic index » view message » categorize

6. Re: Question regarding EDS

jessedavis said...

A database is useless if it is not reliable.

IMHO, to call EDS a "database" is really a misnomer. I mean, it satisfies the conditions for storing and retrieving records, but it has no protections for data corruption and integrity.

Here's an interesting thread about the state of EDS and a (hopefully) more stable simple data storage system that Derek wrote: Reliability of small database system. I haven't tested it personally so I cannot vouch for its stability.

jessedavis said...

Do you have any ideas for alternate ways to implement a database in euphoria?

I know it's often hard to include additional dependencies with your applications, but something like SQLite is probably better suited to what you need here.

If reliability is of any concern whatsoever, I always recommend using something that's been thoroughly tested and developed with that in mind (which EDS apparently was not).

-Greg

new topic     » goto parent     » topic index » view message » categorize

7. Re: Question regarding EDS

jessedavis said...

I guess the thrust of my comment is why do I have these problems. A database is useless if it is not reliable.

I believe the core problem is that database.e is not transactional (there is no commit/rollback handling) and it is not a separate process. A crash anywhere else in your application may perform some writes but leave some other updates in memory that simply never make it to disk, and that way the database becomes inconsistent. If anything the problem is more likely to do with caching rather than database.e per se, though maybe things would be improved (potentially at a significant performance cost) by adding a few flush(current_db) to database.e (just don't ask me where).

I solved many problems in edita by adopting a close-often approach. All database handling is done by eaedb.e and almost all the routines in that call db_close() before returning. The overhead is certainly measurable, but acceptable in that particular application, and nowadays database corruptions seem to be pretty rare - not that I can categorically rule out other factors, in particular moving from rds eu to phix, or even win98 to win10, or whether my old box had flakey ram, etc.

jessedavis said...

Do you have any ideas for alternate ways to implement a database in euphoria?

No. I do however rather fancy some kind of (cross platform|properly documented) builtins/odbc.e for Phix, but, y'know, too many other things on me plate.

Pete

new topic     » goto parent     » topic index » view message » categorize

8. Re: Question regarding EDS

Hi

I agree, if you require reliability on medium to large databases then use sqlite. It's dll is a tiny add on to the program footprint, its as simple (if not simpler), than EDS, it has rollback and it has a huge userbase. I used it for years on my veterinary database, there was never a size related slowdown, and never had a software / database related data loss issue. Its not inherently multiuser, but I used it with multiple users. If you want client / server very large dataset models, then use one of the other databases.

Cheers

Chris

new topic     » goto parent     » topic index » view message » categorize

Search



Quick Links

User menu

Not signed in.

Misc Menu