OpenEuphoria

1. Adding to or Updating the standard library

• Posted by Jeremy Cowgar <jeremy at cow?ar.co?> May 08, 2008
• 701 views

When adding to or updating the standard library, there are few things which we
should all do consistently.

#1... When altering the standard library at any time, the unit tests should be
run to ensure things still work after your changes. If not, they should be fixed
before committing.

#2... When fixing a bug in the standard library, a unit test should first be
created that exposes the bug. Unit tests should be run that that test should
fail. The bug should then be fixed and the unit test should start working.

#3... When adding a new function, we need an easy way to track it's status on
two elements, 1. Has it been tested? 2. Has it been documented? So, What I've
done from the start was added a comment to the top of any function that has 1 or
more of these things not done:

-- TODO: document, test

Choose which one, or both needs to be done.

However, when adding a function to the standard library, it should really be
tested before being committed. Please take this step serious.

#4.... (again) run unit tests.

To run unit tests on Linux/FreeBSD

cd $EUDIR/tests
exu all.ex

To run unit tests on Windows

cd %EUDIR%\tests
exwc all.ex

--
Jeremy Cowgar
http://jeremy.cowgar.com

3. Re: Adding to or Updating the standard library

• Posted by Jeremy Cowgar <jeremy at c?wgar.?om> May 08, 2008
• 658 views

CChris wrote:
  
> Two questions:
> 1/ How to unit-test that something should crash?

I have not solved that yet. Let me work on that now.

> 2/ There are three folders related to documentation: doc, html and htx. Is it
> proper to only modify the .htx file, as the other two will be generated
> anyway?

doc and html are generated from the .htx.

> How would this procedure fit into ck's reworking the doc format?

It's the same process, however, the .htx structure has greatly changed. Until
it's done (next few days), I would, unfortunately, just add a comment to the
function that it needs documented. Any example you see in the current htx/ folder
is going to change and not by a small amount either, a huge change.

--
Jeremy Cowgar
http://jeremy.cowgar.com

5. Re: Adding to or Updating the standard library

• Posted by Shawn Pringle <shawn.pringle at gma?l.?om> May 11, 2008
• 653 views

Jeremy Cowgar wrote:
> 
> CChris wrote:
>   
> > Two questions:
> > 1/ How to unit-test that something should crash?
> 
> I have not solved that yet. Let me work on that now.
> 

Testing if something crashes requires re-thinking the way we think of program
control.  For my executable memory allocator, allocate_code() I wrote a test
that first tries to run code allocated by allocate_code() and then tries
the same with allocate().  

For allocate() in particular, I made a global flag variable and a crash_routine
handler.  The global flag is set to true, when we want it to intercept a crash
and say something like: "code executed from memory using allocate caused an 
exception and as it should."  When the flag is set to false the the handler
returns 0 and no success message is printed and EUPHORIA calls the other
handlers
if applicable.  I set the global flag before executing the code that is supposed
to cause the exception and then I clear the flag afterwards.  After clearing the
flag an error message is printed, for it means that the crash routine wasn't
executed.  At this point though, the mainline of the program is in a crash
handler!

In the case of allocate_code(), we do not want to see an exception.  So, in this
cause the crash handler prints a message saying the test has failed and then
terminates the program.  After calling code on allocate_code that could cause
an exception (if there is a bug) it prints a success message.

The program flow reminds me of goto from BASIC.

-- Test the system for :
include xalloc.e as dep
include joy.e as joy

function isspace( integer x )
	return find( x, "\t \n" )
end function

integer log
log = open( "wordwrap.txt", "w" )

function visual_slice( joy:string_of_atoms s, integer i1, integer i2, integer
line_length )
	integer c, m1, m2
	c = 0
	m1 = -1
	m2 = -1
	for i3 = 1 to length(s) do
		if s[i3] = '\n' then
			c += line_length
		elsif s[i3] = '\t' then
			c += 8
		else
			c += 1
		end if
		if m1 = -1 then
			if c > i1 then
				m1 = c-1			
			elsif c = i1 then
				m1 = c
			end if
		end if
		if m2 = -1 then
			if c > i2 then
				m2 = c-1
			else
				m2 = c
			end if
		end if
		if m2 != -1 then
			exit
		end if
	end for
	return s[m1..m2]
end function
		
-- word_wrap function:  Please do not pass strings with TAB characters
function word_wrap( joy:string_of_integers s, integer line_length )
	sequence lines
	sequence haystack	
	integer whitelocation
	
	lines = {}
	while length(s) do
		haystack = s[1..joy:min({length(s),line_length})]
		puts( log, "haystack = "&haystack&10 )
		if length(s) <= line_length then
			lines = append( lines, s )
			s = ""
		else
			-- if no spaces Grab line_length of it
			-- if '\n' grab the first one, or the last ' ' 
			-- whichever comes first.
whitelocation = joy:min( {max({length(s),line_length}) + 1} & joy:remove_0s(
{ find( '\n', haystack ), joy:rfind( ' ', haystack ) } ) )

			puts( log, "s[1..whitelocation-1]=" & s[1..whitelocation-1] & 10 )
			lines = append( lines, s[1..whitelocation-1] )
			flush(log)
			
			if whitelocation+1 > length(s)+1 then
			   s = ""
			else
			  s = s[whitelocation+1..length(s)]
			end if
		end if
	end while
	return joy:join( "\n", lines )
end function

object line
procedure PETC()
	puts(1,"Press Enter to Continue")
	line = gets(0)
	puts(1,10)
end procedure

integer calling_memfunction
calling_memfunction = 0
function print_failure(object x)
    if calling_memfunction then
puts( 1, word_wrap( "We have found that the xalloc.e routines do not
        really make memory executable.\n", 80 ) )
puts( 1, word_wrap( "This conclusively tells us that there is a bug in the
allocate_code() function.  Please let me know about this:  Email me at
shawn.pringle at gmail.com", 80 ) )
	puts( 1, "Test result for allocate_code: FAILURE\n" )
	PETC()
	
        abort(0)
    end if
    return 0
end function

integer calling_dmemfunction
calling_dmemfunction = 0
function dep_is_enabled(object x)
    if calling_dmemfunction then
puts(1, word_wrap( "The code allocated with allocate(), however raised
        an exception and it SHOULD.  Therefore D.E.P. is enabled for the interpreter you
        used for this test on your system.  This means, your system is configured
        correctly to test allocate_code and that allocate_code works properly.\n", 80 ) )
	puts(1, "Test result for allocate_code: SUCCESS!\n" )
        PETC()
	abort(0)
    end if
    return 0
end function

atom code_space, data_space
sequence multiply_code
atom rexec, rdata
atom x,y
object void


-- machine code taken from callmach.ex
multiply_code = {
   -- int argument is at stack offset +4, double is at +8 
   #DB, #44, #24, #04,        -- fild  dword ptr +4[esp]
   #DC, #4C, #24, #08,        -- fmul  qword ptr +8[esp]
   #C2, #0C - 4 * (platform()=LINUX), #00  -- ret C -- pop 12 (or 8) bytes 
					   -- off the stack
    }

crash_routine(routine_id("print_failure"))
crash_routine(routine_id("dep_is_enabled"))
code_space = dep:allocate_code(multiply_code)
data_space = allocate(length(multiply_code))
poke( data_space, multiply_code )
rexec = define_c_func("", code_space, {C_INT, C_DOUBLE}, C_DOUBLE)
rdata = define_c_func("", data_space, {C_INT, C_DOUBLE}, C_DOUBLE ) 

x = 7
y = 8.5
puts(  1,  word_wrap( "We will now call a machine code function using memory
marked as executable.\n", 80 )  )
PETC()
calling_memfunction = 1
void = c_func(rexec, {x, y})
calling_memfunction = 0
printf(1, word_wrap( "The code allocated with allocate_code() was called without
an exception being raised.\nThe function allocate_code() works as it should.\n",
80 ), {} )
puts(1, "\n\n" )
puts(1, word_wrap( "We will now call a machine code function without using
memory marked as executable.  This should cause an exception.\n", 80 ) )
PETC()

function ifthenelse( integer condition, sequence s1, object x2 )
	if condition then return s1 else return x2 end if
end function


calling_dmemfunction = 1
void = c_func(rdata, {x,y})
calling_dmemfunction = 0
printf(1, word_wrap( "The code allocated with allocate() was also called " &
    "without an exception being "&
    "raised.  This means your system has no DEP for this interpreter " &
    " and these tests are " &
    "inconclusive.  " & 
    
    "You need to enable DEP in both your pre-OS hardware configuration screen" &
    " at " &
    "boot up and your Operating system.  "&
	     
"You probably can get into the hardware by pressing the delete key at boot
    up.  "&
    
    ifthenelse( platform() = WIN32,
    "In Windows XP, you can find the configuration for "&
    "hardware DEP by first, opening Control_Panel, next from there opening "&
"System, after that clicking the Advanced_Tab, and next clicking the
    Performance button and "&
    "finally clicking the Data execution Prevention tab. " &
    "There you can choose the option for enabling DEP for all processes.", "")&
    
    ifthenelse( platform() = LINUX, 
    "In Linux, you will need to download some kernel patches to allow, "&
    "this to work.", "" ) & 
    
    ifthenelse( platform() = DOS32, 
    "In DOS, there is no way to enable hardware DEP.", "" )&"\n", 80)
    , {} )
close(log)

puts(1, "Test result for allocate_code: INCONCLUSIVE!\n" )
PETC()

Shawn Pringle