Re: Discarding EDS
- Posted by ghaberek (admin) Nov 10, 2021
- 837 views
I think it's a bit unfortunate that Rob Craig called EDS a "database" because it's really just key-value format.
If maps are writable to disk, then what's the need of EDS? (Just curious.)
Or, what's the different use-case for maps and EDS?
Mostly it comes down to memory usage. Maps are an "all or nothing" approach with all keys and values stored in memory at once, which makes it very fast. EDS only loads the keys of the selected table into memory and then only retrieves the values from disk when you ask for them, which makes it quite a bit slower. But these differences aren't strictly about memory consumption: you could probably fit most EDS databases entirely in memory without issue, but when keeping everything in memory, if your app crashes all of our maps go with it. EDS at least keeps most things on disk most of the time, so you're never really in an unsafe state unless you crash during a write, which is what Derek's improvements had tried to correct. And behind the scenes they're quite different as well. They may be the same "here is a key please give me the value" kinds of systems, but EDS ensures table keys are always sorted and uses binary search to locate its records, while maps don't care what order things are in and relies on hashing and buckets to its locate records. The differences in performance might start showing at massive scales, but I'm not sure anyone is pushing either system that hard. As far as use case, that's mostly down to occasional access and long-term storage (EDS) versus frequent access and short-to-zero-term storage (maps). Think of EDS like a library and maps like a phone book.
-Greg
