1. EuSQLite
- Posted by Juergen Luethje <j.lue at gmx.de> Jun 08, 2007
- 635 views
- Last edited Jun 09, 2007
Hi Chris, out of curiosity, I just downloaded your file "eusqlite.zip". At the first glance, I stumbled across the function peek_sequence(). Below there are some other functions which might be faster. I don't know whether that really matter, though. Regards, Juergen ------------------------------------------------------------------------ global function peek_sequence (atom addr, integer len) integer i, tmp sequence seq seq="" i=0 tmp=peek(addr) while (tmp != 0 and i<len) do seq = append(seq, tmp) i = i + 1 tmp = peek(addr+i) end while return seq end function --====================================================================== global function peek_sequence1 (atom addr, integer len) integer i, tmp sequence seq seq = repeat(0, len) i = 0 tmp = peek(addr) while tmp != 0 and i < len do i += 1 seq[i] = tmp tmp = peek(addr+i) end while return seq[1..i] end function ------------------------------------------------------------------------ global function peek_sequence2 (atom addr, integer len) integer i sequence seq seq = peek({addr, len}) i = find(0, seq) - 1 if i = -1 then i = len end if return seq[1..i] end function ------------------------------------------------------------------------ atom Kernel32 integer Lstrlen if platform() = WIN32 then Kernel32 = open_dll("kernel32.dll") Lstrlen = define_c_func(Kernel32, "lstrlenA", {C_POINTER}, C_ULONG) end if global function peek_sequence3 (atom addr, integer len) integer i i = c_func(Lstrlen, {addr}) if i > len then i = len end if return peek({addr, len}) end function ------------------------------------------------------------------------
2. Re: EuSQLite
- Posted by Ray Smith <ray at RaymondSmith.com> Jun 08, 2007
- 590 views
- Last edited Jun 09, 2007
Juergen Luethje wrote: > Below there are some other functions which might be faster. > I don't know whether that really matter, though. > > Regards, > Juergen Hi Juergen, The "len" variable in the original peek_sequence really was meant to be "max_len". It didn't need to return a sequence of len, only not > len. So peek_sequence1 is not a valid substitute. Your peek_sequence2 and 3 look much better though. I'd "assume" (I know we should never do that) but peek_sequence3 looks the faster ... the only problem being it isn't portable to Linux or FreeBSD. So, my vote would be to compromise and go with peek_sequence2. So Chris ... would you consider putting euSQLite onto sourceforge to easily allow others access to improve it .. did you know they just added wikispaces to sourceforge ;)?? Regards, Ray Smith http://RaymondSmith.com
3. Re: EuSQLite
- Posted by Jesus Consuegra <jconsuegra at yahoo.com> Jun 09, 2007
- 616 views
Hi Juergen! I'm using heavily EuSqlite3, so I'm interested in any optimization that can be done... I've been taking a glimpse to your changes, and what I see is that in "peek_sequence2" you replaced
seq = "" by seq = repeat(0, len)
and
return seq by return seq[1..i]
Would this mean a big difference? Because if so, I should adapt it yet. I'm writing a web page generator for a friend that does historical investigation. He puts all the information inside big text files, where we've agreed a set of markers to identify titles, actors, references and document text. Document text can be quite big and is stored inside a sequence.... What I do is to split the big text in pieces and store it on a Sqlite3 database. From there, some other programs to extract the data and build the site. (I set a trial output site for my friend, just to see the data, formatting is awful and texts are in spanish... http://watersmurf.homelinux.com/julio.html, select the left link over the picture) I would appreciate very much any suggestion to improve the access to Sqlite3 databases. Thanks for your suggestions!. Jes.
4. Re: EuSQLite
- Posted by Jesus Consuegra <jconsuegra at yahoo.com> Jun 09, 2007
- 599 views
Ray, Thanks for pointing it out. I've replaced peek_sequence by peek_sequence2 and it seems that something went wrong in:
seq = peek({addr, len})
"C:\EUPHORIA\INCLUDE\wrapper.ew:99 in function peek_sequence() A machine-level exception occurred during execution of this statement addr = 3694432 len = 32768 i = <no value> seq = <no value> ... called from C:\EUPHORIA\INCLUDE\eusqlite3.ew:450 in function sqlite_get_table() "
5. Re: EuSQLite
- Posted by Juergen Luethje <j.lue at gmx.de> Jun 09, 2007
- 604 views
Ray Smith wrote: > Juergen Luethje wrote: > > Below there are some other functions which might be faster. > > I don't know whether that really matter, though. > > > > Regards, > > Juergen > > Hi Juergen, > > The "len" variable in the original peek_sequence really was meant to be > "max_len". > It didn't need to return a sequence of len, only not > len. > > So peek_sequence1 is not a valid substitute. > > Your peek_sequence2 and 3 look much better though. > > I'd "assume" (I know we should never do that) but peek_sequence3 looks the > faster > ... > the only problem being it isn't portable to Linux or FreeBSD. <snip> Hi Ray and Jesus, I solved that problem by writing a small cross-platform length() function for null-terminated strings using ASM. Like you Ray, currently I only assume that this function is the fastest (but I think it's an educated guess). :)
-- * Euphoria 2.4 Official Release or later required * -- (because parameters are passed to machine code) include machine.e include dll.e ------------------------------------------------------------------------ constant SL_CODE = allocate(26) poke(SL_CODE, { -- After the routine has pushed 4 bytes on the stack, first int -- argument is at stack offset +8. #57, -- push edi #8B, #7C, #24, #08, -- mov edi, [esp+8] #B9, #FF, #FF, #FF, #FF, -- mov ecx, #FFFFFFFF #B0, #00, -- mov al, 0 #FC, -- cld #F2, #AE, -- repne scasb #B8, #FE, #FF, #FF, #FF, -- mov eax, #FFFFFFFE #29, #C8, -- sub eax, ecx #5F, -- pop edi #C2, #04, #00 -- ret 4 -- pop 4 bytes off the stack }) global constant STRLEN = define_c_func("", SL_CODE, {C_POINTER}, C_ULONG) -- Usage: -- len = c_func(STRLEN, {addr}) -- -- 'addr' is a pointer to the first character of a null-terminated -- string in memory; -- returns the length of the string in bytes (not including the -- terminating null character) ------------------------------------------------------------------------ global function peek_string (atom addr) -- read a null-terminated string, starting at address 'addr' in memory return peek({addr, c_func(STRLEN,{addr})}) end function ------------------------------------------------------------------------ -- Demo sequence s atom addr s = "Hello" -- s = repeat('x', 90000) addr = allocate_string(s) printf(1, "len = %d", {c_func(STRLEN,{addr})}) free(addr)
Regards, Juergen
6. Re: EuSQLite
- Posted by ChrisBurch2 <crylex at freeuk.co.uk> Jun 09, 2007
- 586 views
Hi Jeurgen, thanks for the input, I am really grateful for any input to eusqlite. A little Background. Eusqlite3 as it is now, is derived from eusqlite2 from Ray Smiths library, updated to take account of the different return types. Eusqlite2 came with wrapper.ew, derived from win32lib, originally by Dave Cuny. As such, it worked, and I have made no changes to it. I am a lazy programmer too. Other libraries also use wrapper.ew, eg euallegro, and any changes shouldn't break these. I have added one or two other functions to it, or have used more updated versions as they came out (honestly can't remember which). I've looked at the functions, and agree with Ray, that reducing any platform() dependencies to a minimum is desirable. I've tested the functions for speed, and as yet have found no human noticeable difference between the original, and peek_sequence2 (which intuitively should be slightly faster). Of course, this view is subjective, not objective. I also didn't get any errors as Jesus did. Remember, most of the functions are carried out in the external dll or shared library, and peek_sequence is called to read every row of the dataset returned after sqlite has done its job, so unless you are expecting a _very_ large dataset, then in most real world usages there will be little noticeable differences. It will certainly be faster than a human being able to read a web page. The actual size of the database has no bearing on the speed of this function. I would also be a little careful about using the machine coded length function (as impressed as I am at the speed with which this was thrown together) for this reason - if euphoria is to be cross platform, will this still work on ARM and other processors, will it still work on 64 bit processors, etc. Note, I haven't tested this - yet! Finally eusqlite3 on sourceforge? It's an idea - I'll look into it. But anyone is more than welcom to add functions to the library, and forward them to me for inclusion. Full credits are given naturally Chris
7. Re: EuSQLite
- Posted by Juergen Luethje <j.lue at gmx.de> Jun 09, 2007
- 596 views
Hi Chris, thanks for the comprehensive explanation of the background -- which I didn't know previously. What you wrote makes a lot of sense. Regards, Juergen