1. eDBI is born, feedback wanted
- Posted by jeremy (admin) Aug 27, 2009
- 2373 views
- Last edited Aug 28, 2009
eDBI is born. Please see eDBI for more information. A quick run down here, though:
- Allows a common API to many different database servers/clients.
- Database drivers are loaded dynamically, allowing your application to work with database drivers you've never even seen before.
- Database drivers are easily written and do not require any changes to eDBI or your application to make them work.
- eDBI works with Euphoria types, for instance, when you perform a query that contains a varchar, integer, decimal and a date/time value, you will not get back all "string" sequences. You will get back a sequence, an integer, an atom and a datetime (from std/datetime.e). Further, when doing queries, you can use these types directly as well.
An example:
-- -- Example edbi use -- include std/datetime.e include edbi/edbi.e sequence data = { { "Ronald Mc'Donald", 29382, datetime:subtract(datetime:new(), 32, YEARS) }, { "Super Man", 55555, datetime:new(1944, 5, 18) }, { "Wonder Woman", 21232, datetime:new(1972, 9, 29) } } edbi:set_driver_path("drivers") edbi:db_handle dbh = edbi:open("sqlite3://example.db") edbi:execute("DROP TABLE people") edbi:execute("CREATE TABLE people (name VARCHAR(30), zip INTEGER, dob datetime)") for i = 1 to length(data) do edbi:execute("INSERT INTO people VALUES (%s, %d, %D)", data[i]) end for edbi:dbr_handle dbr = edbi:query("SELECT * FROM people") object person while sequence(person) with entry do printf(1, "Name=%s, Zip=%d, Dob=%s\n", { person[1], person[2], datetime:format(person[3], "%m/%d/%Y") }) entry person = edbi:next(dbr) end while edbi:close()
Now, to make that work, on say, MySQL, just change the opening line to:
edbi:db_handle dbh = edbi:open("mysql://localhost?dbname=people")
The same is true for any database driver that may exist.
What database drivers exist right now? Just the reference implementation, SQLite3. I am looking for help on other ones. The next highest priority is MySQL then PgSQL.
If you are interested in joining in on this project, please let me know.
Jeremy
2. Re: eDBI is born, feedback wanted
- Posted by sergelli Aug 29, 2009
- 2186 views
The "edbi.e," his new work, will greatly facilitate the use of DBs.
You use "SNV" for download, however this is a Linux tool. But I also see that you use a directory "c: \ projects>", signaling that is using the "CVN" in the Windows environment.
How is this possible?
There is a version "SVN" for Windows? Where found it? eDBI is only for use in 4.o ?
Sergio Gelli
3. Re: eDBI is born, feedback wanted
- Posted by jeremy (admin) Aug 29, 2009
- 2238 views
- Last edited Aug 30, 2009
The "edbi.e," his new work, will greatly facilitate the use of DBs.
You use "SNV" for download, however this is a Linux tool. But I also see that you use a directory "c: \ projects>", signaling that is using the "CVN" in the Windows environment.
How is this possible?
There is a version "SVN" for Windows? Where found it? eDBI is only for use in 4.o ?
Yes, SVN exists for Windows and most platforms. If you are a command line user, Setup-Subversion-1.6.5.msi will do the trick. If you like GUI clients, then you may want TortoiseSVN.
Right now, while eDBI is in heavy development, SVN will be the only real way to download it. Once it get's to a stable state, then I will start producing binaries.
Now, right now, if you wish to play with it, all you need is really 2 files: edbi.e and edbi_sqlite3.dll. That will enable you to start playing. You may also want to look at the example: example.ex to get some simple ideas. You can run eudoc -o edbi.txt edbi.ex to get full docs on it.
Oh, about 4.0. Yes, eDBI is for 4.0 only. 4.0 adds too many nice features to try and program for 4.0 and 3.1. Nothing I produce will be backward compatible with 3.1, it's just not worth it.
Jeremy
4. Re: eDBI is born, feedback wanted
- Posted by useless Aug 29, 2009
- 2186 views
- Last edited Aug 30, 2009
Now, right now, if you wish to play with it, all you need is really 2 files: edbi.e and edbi_sqlite3.dll. That will enable you to start playing. You may also want to look at the example: example.ex to get some simple ideas. You can run eudoc -o edbi.txt edbi.ex to get full docs on it.
I get a 404 on example.ex
useless
5. Re: eDBI is born, feedback wanted
- Posted by mattlewis (admin) Aug 29, 2009
- 2167 views
- Last edited Aug 30, 2009
Now, right now, if you wish to play with it, all you need is really 2 files: edbi.e and edbi_sqlite3.dll. That will enable you to start playing. You may also want to look at the example: example.ex to get some simple ideas. You can run eudoc -o edbi.txt edbi.ex to get full docs on it.
I get a 404 on example.ex
Those are the URIs to the files in svn. They won't resolve using http. You need to use an svn client to either check them out or export.
Matt
6. Re: eDBI is born, feedback wanted
- Posted by jeremy (admin) Aug 30, 2009
- 2158 views
I get a 404 on example.ex
Opps! I gave the wrong path. I corrected my original post citing this file (the other ones are correct). For completeness sake, the corrected path is example.ex.
Jeremy
7. Re: eDBI is born, feedback wanted
- Posted by jeremy (admin) Aug 30, 2009
- 2123 views
Those are the URIs to the files in svn. They won't resolve using http. You need to use an svn client to either check them out or export.
Matt, SVN will resolve just fine from a web browser. It's just a standard http/https request with GET/PUT actions.
Jeremy
8. Re: eDBI is born, feedback wanted
- Posted by jeremy (admin) Aug 30, 2009
- 2080 views
eDBI now has a MySQL driver committed and working. For an examples sake, the MySQL driver is written in Euphoria. So, the SQLite driver is pure C and the MySQL driver is pure Euphoria.
Now it should be easy for anyone to write a driver.
Jeremy
9. Re: eDBI is born, feedback wanted
- Posted by sergelli Aug 30, 2009
- 2155 views
Matt, SVN will resolve just fine from a web browser. It's just a standard http/https request with GET/PUT actions.
Jeremy
The address in the Open Wiki Euphoria
"http://jeremy.cowgar.com/svn/edbi/trunk edbi"
intended to be used with the "SVN"
Nor ok, This is the answer in "SVN"
---------------------------------------
URL
'http://jeremy.cowgar.com/svn/edbi/trunk%20edbi'
non-existent in that revision
Sergio
10. Re: eDBI is born, feedback wanted
- Posted by jeremy (admin) Aug 30, 2009
- 2142 views
The address in the Open Wiki Euphoria
"http://jeremy.cowgar.com/svn/edbi/trunk edbi"
intended to be used with the "SVN"
Nor ok, This is the answer in "SVN"
---------------------------------------
URL
'http://jeremy.cowgar.com/svn/edbi/trunk%20edbi'
non-existent in that revision
The entire URL does not include the trailing edbi. The trailing edbi was to tell SVN what directory to check the project out into. If you are using the command line client, you should use:
C:\projects> svn co http://jeremy.cowgar.com/svn/edbi/trunk edbi
That says checkout http://jeremy.cowgar.com/svn/edbi/trunk into the edbi directory.
Jeremy
11. Re: eDBI is born, feedback wanted
- Posted by consultjohan Oct 14, 2009
- 1991 views
Jeremy,
This is good stuff man! I'm trying it with mysql, and it works great.
Regards,
CJ
12. Re: eDBI is born, feedback wanted
- Posted by jeremy (admin) Oct 14, 2009
- 1957 views
This is good stuff man! I'm trying it with mysql, and it works great.
Great! If you find any bugs or see needed additions let me know. It's still pretty early on in it's development but as you've found out, it is functional
Jeremy
13. Re: eDBI is born, feedback wanted
- Posted by raseu Oct 14, 2009
- 1931 views
- Last edited Oct 15, 2009
great work,
poss mem leak in edbi_sqlite3.c ?
object __cdecl edbi_execute(object dbh, object sql) { sqlite3 *db = (sqlite3 *) dbh; char *c_sql; int result; c_sql = EMalloc(SEQ_PTR(sql)->length + 1); MakeCString(c_sql, sql, SEQ_PTR(sql)->length + 1); result = sqlite3_exec(db, c_sql, 0, 0, 0); // MISSING EFree(c_sql); return result; }
14. Re: eDBI is born, feedback wanted
- Posted by jeremy (admin) Oct 14, 2009
- 1916 views
- Last edited Oct 15, 2009
great work,
Thanks!
poss mem leak in edbi_sqlite3.c ?
object __cdecl edbi_execute(object dbh, object sql) { sqlite3 *db = (sqlite3 *) dbh; char *c_sql; int result; c_sql = EMalloc(SEQ_PTR(sql)->length + 1); MakeCString(c_sql, sql, SEQ_PTR(sql)->length + 1); result = sqlite3_exec(db, c_sql, 0, 0, 0); // MISSING EFree(c_sql); return result; }
Oh! Thanks for the catch. This is fixed in the latest SVN.
Jeremy
15. Re: eDBI is born, feedback wanted
- Posted by euphoric (admin) Oct 14, 2009
- 1929 views
- Last edited Oct 15, 2009
Oh! Thanks for the catch. This is fixed in the latest SVN.
How's that EuSQL plugin coming along?
16. Re: eDBI is born, feedback wanted
- Posted by jeremy (admin) Oct 15, 2009
- 1892 views
Oh! Thanks for the catch. This is fixed in the latest SVN.
How's that EuSQL plugin coming along?
That's your project! I have not done any work on it since we identified a problem internally to Euphoria. I made the port, things should work, you should give it a try and see if it works now and pick up from there.
Jeremy
17. Re: eDBI is born, feedback wanted
- Posted by euphoric (admin) Oct 15, 2009
- 1908 views
How's that EuSQL plugin coming along?
That's your project! I have not done any work on it since we identified a problem internally to Euphoria. I made the port, things should work, you should give it a try and see if it works now and pick up from there.
If you can get example_2.exw working, I'll try to take over from there.
18. Re: eDBI is born, feedback wanted
- Posted by jeremy (admin) Oct 15, 2009
- 1882 views
How's that EuSQL plugin coming along?
That's your project! I have not done any work on it since we identified a problem internally to Euphoria. I made the port, things should work, you should give it a try and see if it works now and pick up from there.
If you can get example_2.exw working, I'll try to take over from there.
I'm not going to be able to get to it for a while yet, so... The code should be there to make it work. I'm sure you can figure it out from here. I forget the state it was in but it was nearly done, IIRC.
Jeremy
19. Re: eDBI is born, feedback wanted
- Posted by euphoric (admin) Oct 15, 2009
- 1865 views
If you can get example_2.exw working, I'll try to take over from there.
The code should be there to make it work. I'm sure you can figure it out from here.
Well, there's a "bad routine number" problem I don't have time to diagnose. Maybe in the near future I can get at it.
20. Re: eDBI is born, feedback wanted
- Posted by jimcbrown (admin) Oct 15, 2009
- 1861 views
If you can get example_2.exw working, I'll try to take over from there.
The code should be there to make it work. I'm sure you can figure it out from here.
Well, there's a "bad routine number" problem I don't have time to diagnose. Maybe in the near future I can get at it.
Where? Where can I get the code to test this?
21. Re: eDBI is born, feedback wanted
- Posted by jimcbrown (admin) Oct 15, 2009
- 1879 views
If you can get example_2.exw working, I'll try to take over from there.
The code should be there to make it work. I'm sure you can figure it out from here.
Well, there's a "bad routine number" problem I don't have time to diagnose. Maybe in the near future I can get at it.
Where? Where can I get the code to test this?
Hmm, ok, I downloaded http://downloads.sourceforge.net/project/eusql/eusql/v0.78.1/eusql-0-78-1-full.zip?use_mirror=voxel but that requires symeval.e which is missing???
22. Re: eDBI is born, feedback wanted
- Posted by jeremy (admin) Oct 15, 2009
- 1927 views
Hmm, ok, I downloaded http://downloads.sourceforge.net/project/eusql/eusql/v0.78.1/eusql-0-78-1-full.zip?use_mirror=voxel but that requires symeval.e which is missing???
I don't remember all the things necessary but there are quite a few things to download to make it work. I know matheval is needed as well. Matt or CK will need to step in and give you all the dependencies.
Jeremy
23. Re: eDBI is born, feedback wanted
- Posted by jimcbrown (admin) Oct 15, 2009
- 1920 views
Hmm, ok, I downloaded http://downloads.sourceforge.net/project/eusql/eusql/v0.78.1/eusql-0-78-1-full.zip?use_mirror=voxel but that requires symeval.e which is missing???
I don't remember all the things necessary but there are quite a few things to download to make it work. I know matheval is needed as well. Matt or CK will need to step in and give you all the dependencies.
Jeremy
Ah, ok. I got matheval from http://downloads.sourceforge.net/project/matheval/matheval/v1.65/matheval-1-65.zip?use_mirror=voxel but then got the following ex.err:
eusql.e:7651 <0074>:: Errors resolving the following references: eusql.e (7651): db_rename_table db_rename_table( table_name, new_name ) ^ Warning ( not_used ): <0320>:: /opt/euphoria/include/machine.e - module variable 'check_calls' is assigned but never used --- Defined Words --- EU4 EU400 EU40000 EUC EUC UNIX LINUX -------------------
So I changed that to rename_table( db_name, table_name, new_name ) ... but at this point I'm wondering if eusql ever worked.
24. Re: eDBI is born, feedback wanted
- Posted by euphoric (admin) Oct 15, 2009
- 1887 views
Ah, ok. I got matheval from [an extremely old repository]... I'm wondering if eusql ever worked.
EuSQL is the database for BBCMF, which is working just fine!
Get matheval here (SVN repo). I think you'll also need EusLibs.
Let me know if you run into anything else.
25. Re: eDBI is born, feedback wanted
- Posted by mattlewis (admin) Oct 15, 2009
- 1970 views
Ah, ok. I got matheval from http://downloads.sourceforge.net/project/matheval/matheval/v1.65/matheval-1-65.zip?use_mirror=voxel but then got the following ex.err:
eusql.e:7651 <0074>:: Errors resolving the following references: eusql.e (7651): db_rename_table db_rename_table( table_name, new_name ) ^
So I changed that to rename_table( db_name, table_name, new_name ) ... but at this point I'm wondering if eusql ever worked.
You've got an old version. Yes, it's the last released version, but it's pretty old. Even so, I can't see why db_rename_table() wouldn't work. It's in both database.e (which you'd be using) and std/eds.e (which the head uses).
I'd recommend using the latest from svn for each:
- https://eusql.svn.sourceforge.net/svnroot/eusql/trunk
- https://matheval.svn.sourceforge.net/svnroot/matheval/trunk
That code has been updated for version 4 of euphoria. Plus lots of other fixes. The eusql release is now about 50 revisions behind the head of the trunk. Since the main users of eusql have been me and CK, it hasn't been released as often as it probably should have.
Matt
26. Re: eDBI is born, feedback wanted
- Posted by jimcbrown (admin) Oct 15, 2009
- 1938 views
Ah, ok. I got matheval from http://downloads.sourceforge.net/project/matheval/matheval/v1.65/matheval-1-65.zip?use_mirror=voxel but then got the following ex.err:
eusql.e:7651 <0074>:: Errors resolving the following references: eusql.e (7651): db_rename_table db_rename_table( table_name, new_name ) ^
So I changed that to rename_table( db_name, table_name, new_name ) ... but at this point I'm wondering if eusql ever worked.
You've got an old version. Yes, it's the last released version, but it's pretty old. Even so, I can't see why db_rename_table() wouldn't work. It's in both database.e (which you'd be using) and std/eds.e (which the head uses).
I'd recommend using the latest from svn for each:
- https://eusql.svn.sourceforge.net/svnroot/eusql/trunk
- https://matheval.svn.sourceforge.net/svnroot/matheval/trunk
That code has been updated for version 4 of euphoria. Plus lots of other fixes. The eusql release is now about 50 revisions behind the head of the trunk. Since the main users of eusql have been me and CK, it hasn't been released as often as it probably should have.
Matt
Ah, ok. So I got those two, plus euslibs from https://euslibs.svn.sourceforge.net/svnroot/euslibs/trunk, all checked out into the edbi/drivers/eusql directory.
Then in the edbi/drivers/eusql directory, I run euc -i ../.. -i ./euslibs -i ./eusql -i ./matheval -dll -lib ../../eu.a edbi_eusql.e
That works, and from there I'm able to run example_2.ex and see the machine exception when calling edbi_open()! :) According to gdb, this happens in de_reference() (of be_runtime.c, line 1591) in eui r2932.
27. Re: eDBI is born, feedback wanted
- Posted by jimcbrown (admin) Oct 15, 2009
- 1917 views
The real issue is that I seem to be missing example.db - so the edbi_open() call fails.
28. Re: eDBI is born, feedback wanted
- Posted by mattlewis (admin) Oct 15, 2009
- 1877 views
The real issue is that I seem to be missing example.db - so the edbi_open() call fails.
No, looking at it, the problem is that at the end of main-.c:eu_init(), everything is dereferenced. The translator shouldn't be doing that, because you end up with a bunch of freed data. It's treating the function (which is really all the top level euphoria code) like a normal euphoria function, and cleaning up after itself.
For a short while, the translator did this with normal executables, to clean up after itself at the end. This caused other problems, but for a dll, it obviously shouldn't be doing this, since the data is meant to stay around and be used when the main program calls the dll.
Matt
29. Re: eDBI is born, feedback wanted
- Posted by jimcbrown (admin) Oct 15, 2009
- 1855 views
The real issue is that I seem to be missing example.db - so the edbi_open() call fails.
No, looking at it, the problem is that at the end of main-.c:eu_init(), everything is dereferenced. The translator shouldn't be doing that, because you end up with a bunch of freed data. It's treating the function (which is really all the top level euphoria code) like a normal euphoria function, and cleaning up after itself.
For a short while, the translator did this with normal executables, to clean up after itself at the end. This caused other problems, but for a dll, it obviously shouldn't be doing this, since the data is meant to stay around and be used when the main program calls the dll.
Matt
Ok, so for the time being I've modified edbi.e to allow one to specify either a dll or a set of routine ids. So now I can just include edbi_eusql.e directly in example_2.ex, and create a list of routine ids, and it just works.
I discovered a bug in edbi_eusql.e, edbi_next() should take two paramenters (dbr and row) but only took one. After fixing this, I can uncomment the stuff in example_2.ex and it prints out a list of names.
$ eui -i drivers/eusql/matheval -i drivers/eusql/euslibs -i drivers/eusql/eusql example_2.ex edbi_open eumem:malloc() open_db returning from edbi_open: 2 Name=Ronald Mc''Donald, Zip=29382, Dob=10/15/1977 Name=Super Man, Zip=55555, Dob=05/18/1944 Name=Wonder Woman, Zip=21232, Dob=09/29/1972 $
Forked into: Using native euphoria in edbi
30. Re: eDBI is born, feedback wanted
- Posted by jimcbrown (admin) Oct 15, 2009
- 1878 views
The real issue is that I seem to be missing example.db - so the edbi_open() call fails.
I forgot to add that I solved this by downloading http://www.usingeuphoria.com/files/words.edb and renaming it to example.db and putting it into the same directory as example_2.ex
31. Re: eDBI is born, feedback wanted
- Posted by mattlewis (admin) Oct 15, 2009
- 1848 views
No, looking at it, the problem is that at the end of main-.c:eu_init(), everything is dereferenced.
This is fixed now. See ticket:99.
Matt
32. Re: eDBI is born, feedback wanted
- Posted by jimcbrown (admin) Oct 15, 2009
- 1846 views
No, looking at it, the problem is that at the end of main-.c:eu_init(), everything is dereferenced.
This is fixed now. See ticket:99.
Matt
Using r2939 to build the dll and run example_2.ex works. I get the same output from using the translated edbi_eusql.so library.
It looks like (other than edbi_next() ) edbi's eusql() plugin works fine.
33. Re: eDBI is born, feedback wanted
- Posted by euphoric (admin) Oct 15, 2009
- 1841 views
Using r2939 to build the dll and run example_2.ex works. I get the same output from using the translated edbi_eusql.so library.
It looks like (other than edbi_next() ) edbi's eusql() plugin works fine.
Are you going to commit any changes?
34. Re: eDBI is born, feedback wanted
- Posted by jimcbrown (admin) Oct 15, 2009
- 1844 views
Using r2939 to build the dll and run example_2.ex works. I get the same output from using the translated edbi_eusql.so library.
It looks like (other than edbi_next() ) edbi's eusql() plugin works fine.
Are you going to commit any changes?
If I ever had credentials to jeremy's svn server, I've since forgotten them.
So, no, I can't commit anything.
Mind you, the only change required is the oneliner for edbi_next().
35. Re: eDBI is born, feedback wanted
- Posted by consultjohan Oct 16, 2009
- 1861 views
For what it's worth, I'm trying to figure out how to
- get mysql_num_rows working. (I need to know how big to make my wxgrid)
- catch connection errors - I get a machine-level exception in edbi.e when I supply the wrong database password
- catch SQL errors. It brings down my whole application
- best work with fields of type date/datetime. (I'm not hardcoding field names in my application, so to get a date formated, I'll have to go and get metadata to see if there are date fields, and when populating the grid, I'll have to format the column as date/datetime in the row loop, which is inefficient and too much work)
What do you think?
36. Re: eDBI is born, feedback wanted
- Posted by jeremy (admin) Oct 16, 2009
- 1888 views
- get mysql_num_rows working. (I need to know how big to make my wxgrid)
Hm. I can see where this would be useful but it has to be done in a cross-database manner. Some database servers will not report the number of rows returned by a result until you consume all the available data (MySQL for example, see: mysql_num_rows and mysql_store_result). The downside to this is that the entire result set is then in memory at one point in time, which many will not want.
I'll have to ponder how to best handle this. It would be easy to add a new element to the query to store the number of rows but the idea of reading all available query data first just to get the number of rows isn't too exciting. I wonder if it wouldn't be best (as this is the rare case not the norm) to read all the rows into a sequence in Euphoria then just doing a length() on the resulting sequence? You would wind up with the data in memory twice for a short period of time but you do that anyway when you use mysql_store_result so that mysql_num_rows will work.
- catch connection errors - I get a machine-level exception in edbi.e when I supply the wrong database password
- catch SQL errors. It brings down my whole application
Yes... The early stage of eDBI (most products?) means that error handling is not the best. Right now a lot of things are assumed to just work . This is obviously not good but at first I wanted to get the API working then go back and do the important stuff
- best work with fields of type date/datetime. (I'm not hardcoding field names in my application, so to get a date formated, I'll have to go and get metadata to see if there are date fields, and when populating the grid, I'll have to format the column as date/datetime in the row loop, which is inefficient and too much work)
I would probably just do something like:
edbi:query("SELECT abc, def, DATE_FORMAT(mydate, '%m/%d/%Y') AS mydate FROM xyz")
That is, if you do not want to do any date processing on it in Euphoria. That will return a string value.
Jeremy
37. Re: eDBI is born, feedback wanted
- Posted by consultjohan Oct 28, 2009
- 1740 views
Jeremy,
Thanks for the response. I managed to work around some issues with a small patch here and there. Thanks also for the advice. For example:
public type dbr_handle(object o) if not atom(o) then return 0 end if if o = 0 then return 1 end if if error_code() then return 1 end if integer l = length(ram_space) object d = 0 if l >= o then d = ram_space[o] end if if not sequence(d) then return 0 end if if not length(d) = Q_END then return 0 end if return 1 end type
Regards,
CJ