1. DBMS library in Euphoria
Hello All,
Because of the recent discussion, I am considdering
writing a DBMS lib and contributing it to the archive.
However, I need to know what features would be required
to be useful to others before I decide whether or not I'm
up to the challenge.
First of all this lib would not have a user interface
(at least at first), but just consist of a bunch of routines
to do the following:
Create tables
create a table file *.TBL
identify the fields, their data type, and size
Create relationships
relationships between tables (1:n)
identifying primary and foreign keys.
inforce referential integrity
Modify tables
Add or delete or change fields
Data entry and deletion
Add, delete, or change records
Data retreival
search for records
retrieve contents of fields in a record
possibly query generation features
I only have a vague idea of what I want to do:
The table files (and any others needed) would be
in my own format, basically random access files.
I am unsure what format the routines' input should be in:
a script language or separate arguments for routines.
Please comment on whether or not a lib like this is needed
and the features that would be necesary and/or preferred.
I'll then decide if I can and want to do it.
I need to know if the gratification of providing a needed
tool to the community outways the trouble of creating it.
:)
later,
Lewis Townsend
______________________________________________________
Get Your Private, Free Email at http://www.hotmail.com
2. Re: DBMS library in Euphoria
This kind of project has been on my list of things to do, but it's not =
anywhere near enough the top. I could most certainly use a DBMS that =
handles key files. But not too soon, 'cause that'll mean I'll have to =
program faster to get to that part of my list ;-P. =20
Please do it!
Michael J. Sabal
>>> keroltarr at HOTMAIL.COM 02/25/00 01:59PM >>>
Hello All,
Because of the recent discussion, I am considdering
writing a DBMS lib and contributing it to the archive.
However, I need to know what features would be required
to be useful to others before I decide whether or not I'm
up to the challenge.
First of all this lib would not have a user interface
(at least at first), but just consist of a bunch of routines
to do the following:
Create tables
create a table file *.TBL
identify the fields, their data type, and size
Create relationships
relationships between tables (1:n)
identifying primary and foreign keys.
inforce referential integrity
Modify tables
Add or delete or change fields
Data entry and deletion
Add, delete, or change records
Data retreival
search for records
retrieve contents of fields in a record
possibly query generation features
I only have a vague idea of what I want to do:
The table files (and any others needed) would be
in my own format, basically random access files.
I am unsure what format the routines' input should be in:
a script language or separate arguments for routines.
Please comment on whether or not a lib like this is needed
and the features that would be necesary and/or preferred.
I'll then decide if I can and want to do it.
I need to know if the gratification of providing a needed
tool to the community outways the trouble of creating it.
:)
later,
Lewis Townsend
______________________________________________________
Get Your Private, Free Email at http://www.hotmail.com
3. Re: DBMS library in Euphoria
Personally, I would write a lib for an existing RDBMS server.
PostGreSQL would be my choice, but it isn't the most popular. If I were going
to write the routines that actually manipulate the data, I'd write an XBase
lib. The reason being that these things are already in wide use, adding support
for popular things will make your lib more widely useful, and in genereal
better.
There are a few ways to do this. There's many (free) libs like the
already written in C and Perl. I'd do the 'call C routine' thing and you won't
have to do much work =), just write a Euphoria wrapper.
That's my (condensed) thoughts. I'd write more, but it's all theory and
babble and irrelevant and has lots of run-on sentances =P.
On Fri, 25 Feb 2000, you wrote:
> Hello All,
>
> Because of the recent discussion, I am considdering
> writing a DBMS lib and contributing it to the archive.
> However, I need to know what features would be required
> to be useful to others before I decide whether or not I'm
> up to the challenge.
> First of all this lib would not have a user interface
> (at least at first), but just consist of a bunch of routines
> to do the following:
>
> Create tables
> create a table file *.TBL
> identify the fields, their data type, and size
> Create relationships
> relationships between tables (1:n)
> identifying primary and foreign keys.
> inforce referential integrity
> Modify tables
> Add or delete or change fields
> Data entry and deletion
> Add, delete, or change records
> Data retreival
> search for records
> retrieve contents of fields in a record
> possibly query generation features
>
> I only have a vague idea of what I want to do:
> The table files (and any others needed) would be
> in my own format, basically random access files.
> I am unsure what format the routines' input should be in:
> a script language or separate arguments for routines.
>
> Please comment on whether or not a lib like this is needed
> and the features that would be necesary and/or preferred.
> I'll then decide if I can and want to do it.
> I need to know if the gratification of providing a needed
> tool to the community outways the trouble of creating it.
> :)
>
> later,
> Lewis Townsend
> ______________________________________________________
> Get Your Private, Free Email at http://www.hotmail.com
4. Re: DBMS library in Euphoria
Lewis Townsend wrote:
> Please comment on whether or not a [DBMS} lib like
> this is needed and the features that would be
> necesary and/or preferred.
Why write your own file format? I'd think that xBase file compatibility
would be the way to go. The format is well understood, and used by lots of
programs - especially DOS programs, which is where the library would be
targeted. You can find xBase file formats at:
http://www.wotsit.org/cgi-bin/search.cgi?database
http://www.e-bachmann.dk/docs/xbase.htm
-- David Cuny
5. Re: DBMS library in Euphoria
On Fri, 25 Feb 2000 11:43:25 -0800, Cuny, David at DSS <David.Cuny at
DSS.CA.GOV>
wrote:
>Lewis Townsend wrote:
>
>> Please comment on whether or not a [DBMS} lib like
>> this is needed and the features that would be
>> necesary and/or preferred.
>
>Why write your own file format? I'd think that xBase file compatibility
>would be the way to go. The format is well understood, and used by lots of
>programs - especially DOS programs, which is where the library would be
>targeted. You can find xBase file formats at:
>
>http://www.e-bachmann.dk/docs/xbase.htm
>
>-- David Cuny
I agree that xBase is the way to go. Daniel Bernstein has an xBase library
for Eu in the archive already, so step one could be done quite quickly. I
think that the most important feature would be some kind of indexed lookup,
probably a BTree, or something similar. I started down that path a long
time ago, but abandoned the project due to my loathing converting the
Btrieve libraries from C to Eu, and a general lack of support at the time.
I was using a proprietary file format that was based on xBase, so the code
might give you a few ideas. It's on my downloads page if you want to have
a look.
Brian
6. Re: DBMS library in Euphoria
On Fri, 25 Feb 2000 15:02:50 -0400, Steve Mosher <goat at DEVIL.ISN.NET> wrote:
> There are a few ways to do this. There's many (free) libs like the
>already written in C and Perl. I'd do the 'call C routine' thing and you
won't
>have to do much work =), just write a Euphoria wrapper.
What is the name of a FREE "C" library for xbase that doesn't
have to have it's object code linked into your program ??
How can you use wrappers in DOS ??
Where do you find this FREE xbase "C" source code that can use create
the data base do indexs, WRITE and read data ??
7. Re: DBMS library in Euphoria
----- Original Message -----
From: "Cuny, David at DSS" <David.Cuny at DSS.CA.GOV>
To: <EUPHORIA at LISTSERV.MUOHIO.EDU>
Sent: Friday, February 25, 2000 1:43 PM
Subject: Re: DBMS library in Euphoria
> Lewis Townsend wrote:
>
> > Please comment on whether or not a [DBMS} lib like
> > this is needed and the features that would be
> > necesary and/or preferred.
>
> Why write your own file format? I'd think that xBase file compatibility
> would be the way to go. The format is well understood, and used by lots of
> programs - especially DOS programs, which is where the library would be
> targeted. You can find xBase file formats at:
>
> http://www.wotsit.org/cgi-bin/search.cgi?database
> http://www.e-bachmann.dk/docs/xbase.htm
Ewwwwwww:
"Xbase is a complex of data files (.DBF), indexes (NDX, MDX, CDX etc.) and
eventually note files (DBT) for storing large amounts of formatted data in a
structured form. "
Ewwwww,, i *know* Eu can be put to better use than fixed length fields,
fixed field count per record, and fixed types in it! I need real random
access, variable length everything, and fields which may or may not be
there, which is why i have been using an XML-type of DB for years. If i
don't put anything in a particular record's field, that field doesn't exist
in that record; and if i put in a megabyte in the same field in the next
record, that's ok too. Nesting anything into anything else is not a problem.
Using the right tags, the DB looks nice in an HTML browser, and it's
editable in any text editor. Naturally, the more the data is optimised for
speed or size, the less human readable it becomes, but that is just one more
choice you have in your program as to how you save the data and spec the
tags and do the type checking.
Kat
8. Re: DBMS library in Euphoria
> What is the name of a FREE "C" library for xbase that doesn't
>
> have to have it's object code linked into your program ??
>
Erm, when I was thinking XBase, I was also thinking Perl (DBD::XBase). I
don't know but it might use a C lib. At any rate, have a look around, I'm
certain that free C code can be found for XBase. As for dynamic linking, any
programmer competant with a C compiler (not necessarily the language) can make
a shared object =). To become that competant with a C compiler (say gcc) it
takes all of five minutes of reading.
> How can you use wrappers in DOS ??
>
The same way you do in any other OS. Wrapper in this context means a
function that simply calls another function. It sets up some variables or uses
some defaults or whatever. Sometimes they're implemented with macros. The whole
point is to save sanity for the programmer.
> Where do you find this FREE xbase "C" source code that can use create
>
> the data base do indexs, WRITE and read data ??
See above.
9. Re: DBMS library in Euphoria
Kat wrote:
> Ewwwwwww
True, but there's a lot of legacy stuff sitting in xBase format. Being able
to at least read and write this format is important.
Remember your comments about limited hardware? I suspect that the end user
will:
1. Be using on a legacy DOS machine
2. Have limited memory on their machine
3. Want to use the database file with existing applicaitons
I could be wrong on all the above. I'm assuming DOS because there seems
little point in writing a DBMS for Win32 instead of wrapping ODBC.
I'll go a bit further out on a limb and assume that you implement your XML
database by loading it into memory. If this is the case, all the
transactions are lost if there is a power loss or code glitch while the
application is running. If you choose to update the file with each
transaction, the program becomes a slug when working with large databases.
OK, that should be enough assumptions to hang myself. 
BTW, you might want to take a look at WDDX (http://www.wddx.org/), one of
the zillion competing standards for XML data.
-- David Cuny
10. Re: DBMS library in Euphoria
- Posted by Everett Williams <rett at GVTC.COM>
Feb 25, 2000
-
Last edited Feb 26, 2000
On Fri, 25 Feb 2000 14:56:47 -0600, Kat <gertie at ZEBRA.NET> wrote:
>Ewwwww,, i *know* Eu can be put to better use than fixed length fields,
>fixed field count per record, and fixed types in it! I need real random
>access, variable length everything, and fields which may or may not be
>there, which is why i have been using an XML-type of DB for years. If i
>don't put anything in a particular record's field, that field doesn't exist
>in that record; and if i put in a megabyte in the same field in the next
>record, that's ok too. Nesting anything into anything else is not a problem.
>Using the right tags, the DB looks nice in an HTML browser, and it's
>editable in any text editor. Naturally, the more the data is optimised for
>speed or size, the less human readable it becomes, but that is just one more
>choice you have in your program as to how you save the data and spec the
>tags and do the type checking.
>
>Kat
And how were we, that is me and the frog in my pocket, going to dependably
index this lovely structure that you have built. Display it, yes...linearly
search
it, probably...but index and access linked records...that can be done with
this type of record, but for many reasons technical and some not so
technical indexed access is a bear. If you have solved it, I am interested. I
believe that a company called Progressive or some such had out a fully
programmable database that fit the description of what you have laid out, but
it was fairly slow, questionably scalable and quite expensive. Lead on.
Everett L.(Rett) Williams
rett at gvtc.com
11. Re: DBMS library in Euphoria
- Posted by "Lucius L. Hilley III" <lhilley at CDC.NET>
Feb 25, 2000
-
Last edited Feb 26, 2000
> ---------------------- Information from the mail
header -----------------------
> Sender: Euphoria Programming for MS-DOS
<EUPHORIA at LISTSERV.MUOHIO.EDU>
> Poster: Lewis Townsend <keroltarr at HOTMAIL.COM>
> Subject: DBMS library in Euphoria
> --------------------------------------------------------------------------
-----
>
> Hello All,
>
> Because of the recent discussion, I am considdering
> writing a DBMS lib and contributing it to the archive.
> However, I need to know what features would be required
> to be useful to others before I decide whether or not I'm
> up to the challenge.
> First of all this lib would not have a user interface
> (at least at first), but just consist of a bunch of routines
> to do the following:
>
> Create tables
> create a table file *.TBL
> identify the fields, their data type, and size
> Create relationships
> relationships between tables (1:n)
> identifying primary and foreign keys.
> inforce referential integrity
> Modify tables
> Add or delete or change fields
> Data entry and deletion
> Add, delete, or change records
> Data retreival
> search for records
> retrieve contents of fields in a record
> possibly query generation features
>
> I only have a vague idea of what I want to do:
> The table files (and any others needed) would be
> in my own format, basically random access files.
> I am unsure what format the routines' input should be in:
> a script language or separate arguments for routines.
>
> Please comment on whether or not a lib like this is needed
> and the features that would be necesary and/or preferred.
> I'll then decide if I can and want to do it.
> I need to know if the gratification of providing a needed
> tool to the community outways the trouble of creating it.
> :)
>
> later,
> Lewis Townsend
I have been wanting to do some databasing in Euphoria for quite
some time now but just haven't taken the time to build the tools
required to achieve what I want to do.
Good Luck on your path to fullfilment. :)
Lucius L. Hilley III
lhilley at cdc.net
+----------+--------------+--------------+
| Hollow | ICQ: 9638898 | AIM: LLHIII |
| Horse +--------------+--------------+
| Software | http://www.cdc.net/~lhilley |
+----------+-----------------------------+
12. Re: DBMS library in Euphoria
----- Original Message -----
From: "Cuny, David at DSS" <David.Cuny at DSS.CA.GOV>
To: <EUPHORIA at LISTSERV.MUOHIO.EDU>
Sent: Friday, February 25, 2000 3:32 PM
Subject: Re: DBMS library in Euphoria
> Kat wrote:
>
> > Ewwwwwww
>
> True, but there's a lot of legacy stuff sitting in xBase format. Being
able
> to at least read and write this format is important.
>
> Remember your comments about limited hardware? I suspect that the end user
> will:
>
> 1. Be using on a legacy DOS machine
Errrr,, so how will you pick the correct legacy DB format they will be
using?
> 2. Have limited memory on their machine
A reason to use file-based DBs, not a reason to strictly format those DBs.
> 3. Want to use the database file with existing applicaitons
See 1., how will you know which one? Will you write a new DB interface or
wrapper each time someone needs a DB? Standards are great, but i saw no
mention this time of a preferred format, i saw a call for opinions on which
format to go with. It's been a mite rough here the last week, i'll clean my
eyes and reboot tonite. I think i remember maybe 10 requests for various
forms of DBs over the last couple months, maybe we aren't talking about the
same one(s).
> I could be wrong on all the above. I'm assuming DOS because there seems
> little point in writing a DBMS for Win32 instead of wrapping ODBC.
Well, if writing a new DB in Eu, i'd go with a dos compatable, just for
backwards compatability and ease of translation to linux. But then, i don't
know linux.
> I'll go a bit further out on a limb and assume that you implement your XML
> database by loading it into memory. If this is the case, all the
> transactions are lost if there is a power loss or code glitch while the
> application is running. If you choose to update the file with each
> transaction, the program becomes a slug when working with large databases.
Well, in this case i use a UPS now, being very familiar with power outages,
having spent over a month once recovering from one brownout. But, yeas, i
use the disk based write-before-erase, and in some cases,
write-cacheflush-verify-erase. If the DB is 500Meg, and i have 8Meg of ram,
how could i do otherwise? With an index file (which i haven't done in Eu
yet), keeping the records located, even if the DB file itself is fragmented,
is easy. Databases are all about tradeoffs, speed vs flexibility vs size vs
ease of coding it.
Re: my mention of not done in Eu yet, most of the code is still in pascal, i
translated enough to get started *reading* the DB in Eu, but not adding to
it or sorting it or indexing yet. I need to, i just haven't gotten that far
yet, due to real life stuff. I ported enough from pascal to Eu to verify Eu
is the way to go, eventually it will all be ported over.
Kat
13. Re: DBMS library in Euphoria
- Posted by Lewis Townsend <keroltarr at HOTMAIL.COM>
Feb 28, 2000
-
Last edited Feb 29, 2000
Hello all,
I have come to the understanding that at least 2 other people
are involved in writing a DBMS libraries: (Kat and Michael Nelson?)
Also I realize that most useful databases require network
connectivity and multi-userability which frankly I haven't a clue
as to how to empliment. With all the conversation about it
I realize that what most people need is more than I can provide. Really all
I had in mind was a random-access file database with
fixed length records using normal Euphoria routines. Since
others are working on this sort of thing, I'm more than happy
to get back to work on my other projects, however I will be glad
to help others on their (DB) projects if I can.
later,
Lewis Townsend
______________________________________________________
Get Your Private, Free Email at http://www.hotmail.com
14. Re: DBMS library in Euphoria
----- Original Message -----
From: "Lewis Townsend" <keroltarr at HOTMAIL.COM>
To: <EUPHORIA at LISTSERV.MUOHIO.EDU>
Sent: Monday, February 28, 2000 3:24 PM
Subject: Re: DBMS library in Euphoria
> Hello all,
>
> I have come to the understanding that at least 2 other people
> are involved in writing a DBMS libraries: (Kat and Michael Nelson
Let me try again to make this clear, i am porting a working DB i wrote in
pascal. I am porting it as i need the functions in current Eu apps. And
since i am needing then entire DB access code in Eu asap, i am working on it
as i can. But this DB app ( and/or the speed i am working on it ) prolly
will not suit most people, especially anyone who has speed as the top
priority. It's entirely free-form, with records and fields being nestable
inside each other at runtime, and it's disk file based. I am exploring how
to create *and use* new fields during runtime.
I have other apps i am working on too, and am a week behind my own schedule.
As much as i try to, i can't seem to do any form of multitasking but
time-slice.
Kat,
in slowmo atm.
