1. Why doesn't the db grow?

So i pulled this out of mothballs. I'm not even sure why I wrote it but as I'm in the process of putting almost everything up on github ...

When I run this the edb file never grows beyond 442 bytes. What was I doing wrong?

include std/net/dns.e 
include std/rand.e 
include std/math.e 
include std/eds.e 
include std/filesys.e 
 
--with trace 
--trace(2) 
 
object eds 
eds = dir("bofls.edb") 
if atom( eds ) then 
	if db_create("bofls.edb", DB_LOCK_NO) != DB_OK then 
		puts(2, "Couldn't create the database!\n") 
		abort(1) 
	end if 
else 
	if db_open("bofls.edb") != DB_OK then 
		puts(2, "Couldn't create the database!\n") 
		abort(1) 
	end if 
end if 
 
if db_create_table("ip") != DB_OK then 
    puts(2, "ip table already exists!\n") 
end if 
 
check_free_list() 
 
sequence errs = db_get_errors() 
if length(errs) != 0 then 
    --display_errors(errs) 
    abort(1) 
end if 
 
if db_compress() != DB_OK then 
    puts(2, "compress failed!\n") 
end if 
 
set_rand("") 
 
integer c = 0 
while 1 label "one" do 
	integer i = rand_range(1,254) 
	integer j = rand_range(1,254) 
	integer k = rand_range(1,254) 
	integer l = rand_range(1,254) 
	if i = 1 or i = 10 or i = 192 then 
		continue "one" 
	end if 
	sequence addr = sprintf("%d.%d.%d.%d", {i, j, k, l}) 
	if db_find_key( addr ) < 0 then 
		object hba = host_by_addr(addr) 
		puts( 1, addr ) 
		if not atom(hba) then 
			puts(1, " " & hba[1] & "\n") 
			if db_insert( addr, hba[1] ) != DB_OK then 
				puts( 1, "Cannot insert!\n") 
			end if 
		else 
			puts(1, "\n") 
		end if 
	else  
	    -- puts(1, "Found " & addr & "\n") 
	end if 
	c = c + 1 
	printf(1, "%d\r", c ) 
	integer key = get_key() 
	if key != -1 then 
		--puts(1, sprintf("%d\r", {key})) 
		if key = 27 then 
			exit "one"  
		end if 
	end if 
end while 
 
db_close() 
new topic     » topic index » view message » categorize

2. Re: Why doesn't the db grow?

include std/console.e, then add this just after the includes:

procedure ShowDBErrors(object x) 
display(db_get_errors()) 
abort(0) 
end procedure 
  
db_fatal_id = routine_id("ShowDBErrors") 

You'll see something like this, which should help with debugging:

{ 
  { 
    903, 
    "no table selected", 
    "db_find_key", 
    { 
      "37.83.183.188", 
      "" 
    } 
  } 
} 

Why no table selected? Your compress routine causes the problem. Move it up to just after the db_open()

new topic     » goto parent     » topic index » view message » categorize

3. Re: Why doesn't the db grow?

This seems to work:

include std/net/dns.e  
include std/rand.e  
include std/math.e  
include std/eds.e  
include std/os.e 
include std/filesys.e 
include std/console.e  
 
procedure ShowDBErrors(object x) 
display(db_get_errors()) 
abort(0) 
end procedure 
  
db_fatal_id = routine_id("ShowDBErrors") 
  
object eds  
eds = dir("bofls.edb")  
if atom( eds ) then  
	if db_create("bofls.edb", DB_LOCK_NO) != DB_OK then  
		puts(2, "Couldn't create the database!\n")  
		abort(1)  
	end if  
else  
	if db_open("bofls.edb") != DB_OK then  
		puts(2, "Couldn't create the database!\n")  
		abort(1)  
	end if  
end if  
   
if db_compress() != DB_OK then  
   puts(2, "compress failed!\n")  
end if  
  
if db_create_table("ip") != DB_OK then  
    puts(2, "ip table already exists!\n") sleep(1) 
    if db_select_table("ip") != DB_OK then 
       puts(2,"error selecting table!") abort(1) 
    end if 
end if  
  
check_free_list()  
 
set_rand("")  
  
integer c = 0  
while 1 label "one" do  
	integer i = rand_range(1,254)  
	integer j = rand_range(1,254)  
	integer k = rand_range(1,254)  
	integer l = rand_range(1,254)  
	if i = 1 or i = 10 or i = 192 then  
		continue "one"  
	end if  
	sequence addr = sprintf("%d.%d.%d.%d", {i, j, k, l})  
	 
	if db_find_key(addr) = 0 then  
		display("Error reading database []",{})  
	end if 
	 
	if db_find_key( addr ) < 0 then  
		object hba = host_by_addr(addr)  
		if not atom(hba) then  
			display("+ Addr: [] hba: []",{addr,hba[1]}) 
			if db_insert( addr, hba[1] ) != DB_OK then  
				display("Cannot insert!")  
			end if 
		end if 
	end if  
	c = c + 1  
end while  
  
db_close()  

+ Addr: 87.234.64.20 hba: port-87-234-64-20.dynamic.qsc.de 
+ Addr: 213.161.251.37 hba: static251-37.mimer.net 
+ Addr: 171.230.26.252 hba: dynamic-ip-adsl.viettel.vn 
+ Addr: 64.216.23.72 hba: adsl-64-216-23-72.dsl.snantx.swbell.net 

new topic     » goto parent     » topic index » view message » categorize

4. Re: Why doesn't the db grow?

Thank you. Daft error. Arguably daft programmer. Nevertheless, much appreciated.

Bruce.

new topic     » goto parent     » topic index » view message » categorize

5. Re: Why doesn't the db grow?

Okay so here we are a few days later. The program has been running 24 hours a day. I have a little program to dump the contents of the database. When I run that program I get

C:\SCRATCH>eui foo.ex bofls.edb 
ip 
 
C:\Euphoria\include\std\eds.e:1535 in function db_select_table() 
subscript value 171210 is out of bounds, assigning to a sequence of length 171209 - in subscript #1 of 'key_pointers' 
 
... called from C:\SCRATCH\foo.ex:31 
 
--> See ex.err 
 
 
Press Enter... 
And the code for that is

include std/eds.e 
 
with trace  
trace(1) 
 
integer tries = 0 
object cmd = command_line() 
 
	if (length(cmd) < 2) then 
		abort(1) 
	end if 
 
    integer err = db_open(cmd[3], DB_LOCK_SHARED) 
    if err != DB_OK then 
        abort(1) 
    end if 
	 
--if db_select("iptable") != DB_OK then 
--    puts(2, "Could not select iptable database\n") 
--end if 
 
sequence names = db_table_list() 
for i = 1 to length(names) do 
    puts(1, names[i] & '\n') 
end for 
 
--if db_select("ip") != DB_OK then 
--    puts(2, "Could not select ip\n") 
--end if 
 
if db_select_table("ip") != DB_OK then 
    puts(2, "Couldn't find ip table!\n") 
    abort(1) 
end if 
 
for i = 1 to db_table_size() do 
	puts(1, db_record_data(i) & '\t' & db_record_key(i) & '\n') 
    --if db_record_key(i) = DB_OK then 
    --	puts(1, "0 key found\n") 
    --	exit 
    --end if 
end for 

Am I hitting a hard limit in EDB files? Meanwhile, the other program is still running and adding to the edb.

new topic     » goto parent     » topic index » view message » categorize

6. Re: Why doesn't the db grow?

Belay that! Must have been a bad copy (running on Azure server, copied to local drive). Working fine now.

new topic     » goto parent     » topic index » view message » categorize

Search



Quick Links

User menu

Not signed in.

Misc Menu