1. Why doesn't the db grow?
- Posted by bruce_axtens Nov 16, 2019
- 1413 views
So i pulled this out of mothballs. I'm not even sure why I wrote it but as I'm in the process of putting almost everything up on github ...
When I run this the edb file never grows beyond 442 bytes. What was I doing wrong?
include std/net/dns.e include std/rand.e include std/math.e include std/eds.e include std/filesys.e --with trace --trace(2) object eds eds = dir("bofls.edb") if atom( eds ) then if db_create("bofls.edb", DB_LOCK_NO) != DB_OK then puts(2, "Couldn't create the database!\n") abort(1) end if else if db_open("bofls.edb") != DB_OK then puts(2, "Couldn't create the database!\n") abort(1) end if end if if db_create_table("ip") != DB_OK then puts(2, "ip table already exists!\n") end if check_free_list() sequence errs = db_get_errors() if length(errs) != 0 then --display_errors(errs) abort(1) end if if db_compress() != DB_OK then puts(2, "compress failed!\n") end if set_rand("") integer c = 0 while 1 label "one" do integer i = rand_range(1,254) integer j = rand_range(1,254) integer k = rand_range(1,254) integer l = rand_range(1,254) if i = 1 or i = 10 or i = 192 then continue "one" end if sequence addr = sprintf("%d.%d.%d.%d", {i, j, k, l}) if db_find_key( addr ) < 0 then object hba = host_by_addr(addr) puts( 1, addr ) if not atom(hba) then puts(1, " " & hba[1] & "\n") if db_insert( addr, hba[1] ) != DB_OK then puts( 1, "Cannot insert!\n") end if else puts(1, "\n") end if else -- puts(1, "Found " & addr & "\n") end if c = c + 1 printf(1, "%d\r", c ) integer key = get_key() if key != -1 then --puts(1, sprintf("%d\r", {key})) if key = 27 then exit "one" end if end if end while db_close()
2. Re: Why doesn't the db grow?
- Posted by irv Nov 16, 2019
- 1369 views
include std/console.e, then add this just after the includes:
procedure ShowDBErrors(object x) display(db_get_errors()) abort(0) end procedure db_fatal_id = routine_id("ShowDBErrors")
You'll see something like this, which should help with debugging:
{ { 903, "no table selected", "db_find_key", { "37.83.183.188", "" } } }
Why no table selected? Your compress routine causes the problem. Move it up to just after the db_open()
3. Re: Why doesn't the db grow?
- Posted by irv Nov 16, 2019
- 1387 views
This seems to work:
include std/net/dns.e include std/rand.e include std/math.e include std/eds.e include std/os.e include std/filesys.e include std/console.e procedure ShowDBErrors(object x) display(db_get_errors()) abort(0) end procedure db_fatal_id = routine_id("ShowDBErrors") object eds eds = dir("bofls.edb") if atom( eds ) then if db_create("bofls.edb", DB_LOCK_NO) != DB_OK then puts(2, "Couldn't create the database!\n") abort(1) end if else if db_open("bofls.edb") != DB_OK then puts(2, "Couldn't create the database!\n") abort(1) end if end if if db_compress() != DB_OK then puts(2, "compress failed!\n") end if if db_create_table("ip") != DB_OK then puts(2, "ip table already exists!\n") sleep(1) if db_select_table("ip") != DB_OK then puts(2,"error selecting table!") abort(1) end if end if check_free_list() set_rand("") integer c = 0 while 1 label "one" do integer i = rand_range(1,254) integer j = rand_range(1,254) integer k = rand_range(1,254) integer l = rand_range(1,254) if i = 1 or i = 10 or i = 192 then continue "one" end if sequence addr = sprintf("%d.%d.%d.%d", {i, j, k, l}) if db_find_key(addr) = 0 then display("Error reading database []",{}) end if if db_find_key( addr ) < 0 then object hba = host_by_addr(addr) if not atom(hba) then display("+ Addr: [] hba: []",{addr,hba[1]}) if db_insert( addr, hba[1] ) != DB_OK then display("Cannot insert!") end if end if end if c = c + 1 end while db_close()
+ Addr: 87.234.64.20 hba: port-87-234-64-20.dynamic.qsc.de + Addr: 213.161.251.37 hba: static251-37.mimer.net + Addr: 171.230.26.252 hba: dynamic-ip-adsl.viettel.vn + Addr: 64.216.23.72 hba: adsl-64-216-23-72.dsl.snantx.swbell.net
4. Re: Why doesn't the db grow?
- Posted by bruce_axtens Nov 17, 2019
- 1361 views
Thank you. Daft error. Arguably daft programmer. Nevertheless, much appreciated.
Bruce.
5. Re: Why doesn't the db grow?
- Posted by bruce_axtens Nov 21, 2019
- 1222 views
Okay so here we are a few days later. The program has been running 24 hours a day. I have a little program to dump the contents of the database. When I run that program I get
C:\SCRATCH>eui foo.ex bofls.edb ip C:\Euphoria\include\std\eds.e:1535 in function db_select_table() subscript value 171210 is out of bounds, assigning to a sequence of length 171209 - in subscript #1 of 'key_pointers' ... called from C:\SCRATCH\foo.ex:31 --> See ex.err Press Enter...And the code for that is
include std/eds.e with trace trace(1) integer tries = 0 object cmd = command_line() if (length(cmd) < 2) then abort(1) end if integer err = db_open(cmd[3], DB_LOCK_SHARED) if err != DB_OK then abort(1) end if --if db_select("iptable") != DB_OK then -- puts(2, "Could not select iptable database\n") --end if sequence names = db_table_list() for i = 1 to length(names) do puts(1, names[i] & '\n') end for --if db_select("ip") != DB_OK then -- puts(2, "Could not select ip\n") --end if if db_select_table("ip") != DB_OK then puts(2, "Couldn't find ip table!\n") abort(1) end if for i = 1 to db_table_size() do puts(1, db_record_data(i) & '\t' & db_record_key(i) & '\n') --if db_record_key(i) = DB_OK then -- puts(1, "0 key found\n") -- exit --end if end for
Am I hitting a hard limit in EDB files? Meanwhile, the other program is still running and adding to the edb.
6. Re: Why doesn't the db grow?
- Posted by bruce_axtens Nov 22, 2019
- 1182 views
Belay that! Must have been a bad copy (running on Azure server, copied to local drive). Working fine now.