Re: Reliability of small database system
- Posted by Spock Aug 27, 2013
- 6218 views
If you need any kind of error detection or reliability, you need to go with something like SQLite.
The only advantage EDB has is that it "comes with" Eu; however, it is very basic, and does not have any of the typical ACID features of modern databases. Notably, with EDB, a crash or power outage can easily destroy data.
I wrote an experimental version of EDS that improved the reliability a great deal, but at the cost of a minor slowdown. Not that you'd notice on a small database with modern disk drives. I'll see if I can dig it up. It uses a 'Before Image' file that records raw transactions as well as flushing transactions to the D/B. So that in the event of an unexpected application failure (power outage, crash, etc...) it can perform an automatic roll-forward of transactions to bring the database back up a stable state. The only thing you'll lose is the one transaction that was in the process of being written to the BI file.
Derek,
if you describe your process in a little more detail I might be able to just adopt the concept into my program.
regards, Spock
Derek,
For the small dataset I have to manipulate I would much rather prefer being able to load the whole thing in memory (into a Euphoria sequence) and work on it directly rather than via some database interface. Periodically (each 10 seconds or so) the whole file, if it had been changed, would be saved to disk using a unique filename based on time/date that is set once at startup. Duplicate files would accumulate but these can be purged from time to time, even automatically so that we keep, say, the last 20 day's worth.
Transactions are simple, only: ADD a new entry, DEL(ete) an existing entry, SET (change) an existing entry. [Entries correspond to single rows]. The app will GET the data directly from the sequence.
Inspired by your idea I am thinking of having a log file containing ALL transactions from the beginning. The process for any operation would then be:
1 Append the ADD/DEL/SET transaction to the log file. 2 Update the internal sequence
At program close, save the sequence to file. Also save a timestamped hash value to the ini file. At program startup the hash value will be checked to ensure the program closed successfully the previous time. If it didn't the user will be alerted and given the option of advising IT or letting the system recreate the database using the log file.
At any time the user can run a verify action on the log file in order to prove the current data is good.
I'm eager to hear any comments about this approach.
regards, Spock
