1. upper()

data = repeat({rand(24)+97},1000000)
upperdata = repeat({rand(24)+97},1000000)

puts(1,"repeats done, press a key\n")
junk = wait_key()

  --for loop = 1 to length(data) do upperdata[loop] = upper(data[loop]) end for
  for loop = 1 to length(data) do upperdata[loop] = data[loop] end for


The commented out line uses 48megabytes of memory, and takes 170 
seconds to run.

The next line is the same, but without upper(), and executes instantly.

This line takes all night to run:
upperdata = upper(data)

Why, and how can i avoid the penalty of upper(), and still get the upperdata? 
Yes, i need both data and upperdata.

Incidently, OOEU runs the commented out line in 4 seconds, but takes all 
night to run parse() or gets(). Using define_c_proc(open_dll("msvcrt.dll"), 
"_strupr", {C_POINTER}) doesn't help in OOEU, but does in exw.exe. It's 
very frustrating.

The actual data (not the test set above) is a million ~30-character lines in a 
24megabyte file. I cannot run the app every 30 minutes anymore. Either 
parse(), gets(), or upper() take too long.

Kat

new topic     » topic index » view message » categorize

2. Re: upper()

The commented out line takes 1-2 seconds to run. What are you using?

JG

new topic     » goto parent     » topic index » view message » categorize

3. Re: upper()

Same here.  Uncommenting that line in this program:
include get.e
include wildcard.e

sequence data, upperdata
atom junk

data = repeat({rand(24)+97},1000000)
upperdata = repeat({rand(24)+97},1000000)
 puts(1,"repeats done, press a key\n")
junk = wait_key()
 for loop = 1 to length(data) do upperdata[loop] = upper(data[loop]) end for
  --for loop = 1 to length(data) do upperdata[loop] = data[loop] end for

Only takes a few seconds to run.

Jeremy

new topic     » goto parent     » topic index » view message » categorize

4. Re: upper()

Incidentally, dumping the sequence upperdata to a file gives a 4.76 MB, 4883
line file full of nothing but:
{{78},{78},{78},{78},{78},{78},{78},{78},{78},{78},{78},{78},{78},{78},{78},{78},{78},{78},{78},{78},{78},{78},{78},{78},{78},{78},{78},{78},{78},{78}
Altho the number 78 changes for each run.

Running the program uses about 40(approximately) MB of memory, like you said.
However, changing the line:
data = repeat({rand(24)+97},1000000)
upperdata = repeat({rand(24)+97},1000000)
--TO:
data = repeat({"get out of my face"},1000000)
upperdata = repeat({"get out of my face"},1000000)

Makes the program take about 12 seconds, the file outputted is 56641 lines,
55.3MB

In your test program it is simply running upper on atoms, which wouldn't take
very long.

Jeremy

new topic     » goto parent     » topic index » view message » categorize

5. Re: upper()

On 6 Aug 2006, at 1:35, Julio C. Galaret Viera wrote concerning:
Re: upper()

Using?

Kat

new topic     » goto parent     » topic index » view message » categorize

6. Re: upper()

Kat wrote:
> 
> data = repeat({rand(24)+97},1000000)
> upperdata = repeat({rand(24)+97},1000000)
> 
> puts(1,"repeats done, press a key\n")
> junk = wait_key()
> 
>   --for loop = 1 to length(data) do upperdata[loop] = upper(data[loop]) end
> for
>   for loop = 1 to length(data) do upperdata[loop] = data[loop] end for
> 
> 
> The commented out line uses 48megabytes of memory, and takes 170 
> seconds to run.
> 
> The next line is the same, but without upper(), and executes instantly.
> 
> This line takes all night to run:
> upperdata = upper(data)
> 
> Why, and how can i avoid the penalty of upper(), and still get the upperdata?
> 
> Yes, i need both data and upperdata.
.. ..
> Kat
> 
> 


Hi Kat,

the time you got seems to be out of the context.
Here are two routines to compare with your expression:
the RDS official 2.5; the beautiful one of Derek, 
(arranged here) that I use.

on my system it deals with 1 to 1,5 seconds.

	RDS's   elapsed Time  1.24
	used RAM                 45056
	
	Derek's elapsed Time  0.58
	used RAM                     0
-------	
	Derek's elapsed Time  2.12
	used RAM                663552
	
	                   ___________
	<space>

Note that the declared free memory can be due to other factors.

Antonio

global atom get_Mem, free_mem, min_free_mem, bg, en
include win32lib.ew
without warning
--- =========
constant
  lc = 
{#1,#2,#3,#4,#5,#6,#7,#8,#9,#A,#B,#C,#D,#E,#F,#10,#11,#12,#13,#14,#15,#16,#17,#18
,#19,#1A,#1B,#1C,#1D,#1E,#1F,#20,#21,#22,#23,#24,#25,#26,#27,#28,#29,#2A,#2B,#2C
,#2D,#2E,#2F,#30,#31,#32,#33,#34,#35,#36,#37,#38,#39,#3A,#3B,#3C,#3D,#3E,#3F,#40
,#61,#62,#63,#64,#65,#66,#67,#68,#69,#6A,#6B,#6C,#6D,#6E,#6F,#70,#71,#72,#73,#74
,#75,#76,#77,#78,#79,#7A,#5B,#5C,#5D,#5E,#5F,#60,#61,#62,#63,#64,#65,#66,#67,#68
,#69,#6A,#6B,#6C,#6D,#6E,#6F,#70,#71,#72,#73,#74,#75,#76,#77,#78,#79,#7A,#7B,#7C
,#7D,#7E,#7F,#80,#81,#82,#83,#84,#85,#86,#87,#88,#89,#9A,#8B,#9C,#8D,#9E,#8F,#90
,#91,#92,#93,#94,#95,#96,#97,#98,#99,#9A,#9B,#9C,#9D,#9E,#FF,#A0,#A1,#A2,#A3,#A4
,#A5,#A6,#A7,#A8,#A9,#AA,#AB,#AC,#AD,#AE,#AF,#B0,#B1,#B2,#B3,#B4,#B5,#B6,#B7,#B8
,#B9,#BA,#BB,#BC,#BD,#BE,#BF,#E0,#E1,#E2,#E3,#E4,#E5,#E6,#E7,#E8,#E9,#EA,#EB,#EC
,#ED,#EE,#EF,#F0,#F1,#F2,#F3,#F4,#F5,#F6,#D7,#F8,#F9,#FA,#FB,#FC,#FD,#FE,#DF,#E0
,#E1,#E2,#E3,#E4,#E5,#E6,#E7,#E8,#E9,#EA,#EB,#EC,#ED,#EE,#EF,#F0,#F1,#F2,#F3,#F4
,#F5,#F6,#F7,#F8,#F9,#FA,#FB,#FC,#FD,#FE,#FF}
 ,uc = 
{#1,#2,#3,#4,#5,#6,#7,#8,#9,#A,#B,#C,#D,#E,#F,#10,#11,#12,#13,#14,#15,#16,#17,#18
,#19,#1A,#1B,#1C,#1D,#1E,#1F,#20,#21,#22,#23,#24,#25,#26,#27,#28,#29,#2A,#2B,#2C
,#2D,#2E,#2F,#30,#31,#32,#33,#34,#35,#36,#37,#38,#39,#3A,#3B,#3C,#3D,#3E,#3F,#40
,#41,#42,#43,#44,#45,#46,#47,#48,#49,#4A,#4B,#4C,#4D,#4E,#4F,#50,#51,#52,#53,#54
,#55,#56,#57,#58,#59,#5A,#5B,#5C,#5D,#5E,#5F,#60,#41,#42,#43,#44,#45,#46,#47,#48
,#49,#4A,#4B,#4C,#4D,#4E,#4F,#50,#51,#52,#53,#54,#55,#56,#57,#58,#59,#5A,#7B,#7C
,#7D,#7E,#7F,#80,#81,#82,#83,#84,#85,#86,#87,#88,#89,#8A,#8B,#8C,#8D,#8E,#8F,#90
,#91,#92,#93,#94,#95,#96,#97,#98,#99,#8A,#9B,#8C,#9D,#8E,#9F,#A0,#A1,#A2,#A3,#A4
,#A5,#A6,#A7,#A8,#A9,#AA,#AB,#AC,#AD,#AE,#AF,#B0,#B1,#B2,#B3,#B4,#B5,#B6,#B7,#B8
,#B9,#BA,#BB,#BC,#BD,#BE,#BF,#C0,#C1,#C2,#C3,#C4,#C5,#C6,#C7,#C8,#C9,#CA,#CB,#CC
,#CD,#CE,#CF,#D0,#D1,#D2,#D3,#D4,#D5,#D6,#D7,#D8,#D9,#DA,#DB,#DC,#DD,#DE,#DF,#C0
,#C1,#C2,#C3,#C4,#C5,#C6,#C7,#C8,#C9,#CA,#CB,#CC,#CD,#CE,#CF,#D0,#D1,#D2,#D3,#D4
,#D5,#D6,#F7,#D8,#D9,#DA,#DB,#DC,#DD,#DE,#9F}

constant TO_LOWER = 'a' - 'A' 

global function EUupper(object x)
-- convert atom or sequence to upper case
    return x - (x >= 'a' and x <= 'z') * TO_LOWER
end function

--- ========
global function upper(object x)

    if integer(x) then
            x = uc[x +1]
    elsif sequence(x) then
        for i = 1 to length(x) do
            if x[i] then
            	x[i] = uc[x[i] +1]
            end if
        end for        
    end if
    return x
end function
--- ========
global function Obj_upper(object x)
    if integer(x) then
        if x >= 0 and x < length(uc) then
            x = uc[x+1]
        end if
    elsif sequence(x) then
        for i = 1 to length(x) do
            if integer(x[i]) then
                if x[i] >= 0 and x[i] < length(uc) then
                    x[i] = uc[x[i]+1]
                end if
            elsif sequence(x) then
                x[i] = Obj_upper(x[i])
            end if
        end for        
    end if
    return x
end function
--- ========

constant
    kernel32 = open_dll("kernel32.dll")      -- by H. W. Overman
,xGlobalMemoryStatus =
   define_c_proc(kernel32,"GlobalMemoryStatus",{C_POINTER})
   ,MEMORYSTATUS = allocate(32)     
    mem_set(MEMORYSTATUS, 0, 32)   

function get_freeMemory( )

	c_proc( xGlobalMemoryStatus, {MEMORYSTATUS})
	return peek4u( MEMORYSTATUS +12)

end function
get_Mem = routine_id( "get_freeMemory")
--- =========

atom tempo
sequence data, upperdata

constant iterations = 1


data = repeat({rand(24)+97},1000000)
upperdata = repeat({rand(24)+97},1000000)

	bg = get_freeMemory( )    
	    tempo=time()
	    for j = 1 to iterations do
	        for loop = 1 to length(data) do 
	    		upperdata[loop] = EUupper(data[loop]) 
	    	end for
	    end for
	
	puts(1, sprintf( "RDS's   elapsed Time  %2.2f\n", time()-tempo))
	puts(1, sprintf( "used RAM              %8d\n\n", bg -get_freeMemory( )))

	bg = get_freeMemory( )    
	
	    tempo=time()
	    for j = 1 to iterations do
	        for loop = 1 to length(data) do 
	    		upperdata[loop] = upper(data[loop]) 
	    	end for
	    end for
	
	puts(1, sprintf( "Derek's elapsed Time  %2.2f\n", time()-tempo))
	puts(1, sprintf( "used RAM              %8d\n\n", bg -get_freeMemory( )))
	
	bg = get_freeMemory( )    
	    tempo=time()
--	    for j = 1 to iterations do
	    	upperdata = Obj_upper(data) 
--	    end for
	
	puts(1, sprintf( "Derek's elapsed Time  %2.2f\n", time()-tempo))
	puts(1, sprintf( "used RAM              %8d\n\n", bg -get_freeMemory( )))


	puts(1,          "                   ___________\n")
	

puts(1, "<space>") puts(1, wait_key())


new topic     » goto parent     » topic index » view message » categorize

7. Re: upper()

Kat wrote:
> I cannot run the app every 30 minutes anymore. Either 
> parse(), gets(), or upper() take too long.

The real answer is to get a more modern machine than the one you are using. My
old (1998) machine runs this test at about 2 seconds. (P3 550Mhz)

However, as that is not a real option for you, you could improve the time by
using a faster case changing algorithm. Here is one that works for ASCII text ..

-- chgcase.e --

-- Routines for converting ASCII characters between upper and lower cases.

global sequence uc
global sequence lc

------------------------------------------------
procedure init()
------------------------------------------------
    uc = repeat(0, 256)
    for i = 1 to 256 do
        uc[i] = i-1
    end for
    lc = uc
    -- Adjust uppercase table
    for i = 'a' to 'z' do
        uc[i] = uc[i] + ('A' - 'a')
    end for
    -- Adjust lowercase table
    for i = 'A' to 'Z' do
        lc[i] = lc[i] + ('a' - 'A')
    end for
end procedure

------------------------------------------------
global function toUpper(sequence s)
------------------------------------------------
  for i = 1 to length(s) do
      s[i] = uc[s[i]+1]
  end for
  return s
end function
------------------------------------------------
global function toLower(sequence s)
------------------------------------------------
  for i = 1 to length(s) do
      s[i] = lc[s[i]+1]
  end for
  return s
end function
------------------------------------------------
global function toUpperAtom(atom s)
------------------------------------------------
  return uc[s+1]
end function
------------------------------------------------
global function toLowerAtom(atom s)
------------------------------------------------
  return lc[s+1]
end function
------------------------------------------------
global function toUpperObj(object s)
------------------------------------------------
    if atom(s) then
        return uc[s+1]
    else
        for i = 1 to length(s) do
            if atom(s[i]) then
                s[i] = uc[s[i]+1]
            else
                s[i] = toUpperObj(s[i])
            end if
        end for
    end if
end function
------------------------------------------------
global function toLowerObj(object s)
------------------------------------------------
    if atom(s) then
        return lc[s+1]
    else
        for i = 1 to length(s) do
            if atom(s[i]) then
                s[i] = lc[s[i]+1]
            else
                s[i] = toUpperObj(s[i])
            end if
        end for
        return s
    end if
end function

------------------------------------------------
------------------------------------------------
------------------------------------------------
init()
------------------------------------------------
------------------------------------------------
------------------------------------------------


This should halve the time. However you can shave a few more % points off it by
inlining the function...

-- test.ex --
include get.e
include misc.e
include chgcase.e
sequence data
data = repeat({rand(24)+97},1000000)
sequence upperdata
upperdata = repeat({rand(24)+97},1000000)
object junk


puts(1,"repeats done, press a key\n")
junk = wait_key()
atom e
sequence s
e = time()

for loop = 1 to length(data) do
   s = data[loop]
   for j = 1 to length(upperdata[loop]) do
      s[j] = uc[s[j]+1]
   end for
   upperdata[loop] = s
end for
printf(1, "%g\n", time()- e)
-------------




-- 
Derek Parnell
Melbourne, Australia
Skype name: derek.j.parnell

new topic     » goto parent     » topic index » view message » categorize

8. Re: upper()

For munging the 24megabyte data, i started the line:

upperurllist = upper(urllist)

And 45 minutes later, you can see the progress at
http://www.designerthinking.com/temp/UPPER1.GIF

It quickly ramped up to 320megabyes memory use at 5:30am, 
500megabytes at 5:50am, and at 6:20am it's at 200megabytes memory 
use,, still on that upper() line. (At 6:25am, it dropped to 100megs again, and 
is on the way up, fast, two mintes later it's at 551 megabytes.) This is 
unacceptable. It's still on the same line, with one call to upper().

I started 
http://www.designerthinking.com/temp/testupper.ew
20 minutes ago too, and have yet to get the first print of timing. It's at 
100megabytes memory use now.

Kat

new topic     » goto parent     » topic index » view message » categorize

9. Re: upper()

On 6 Aug 2006, at 4:41, Derek Parnell wrote concerning:
Re: upper()

> 
> 
> posted by: Derek Parnell <ddparnell at bigpond.com>
> 
> Kat wrote:
> > I cannot run the app every 30 minutes anymore. Either 
> > parse(), gets(), or upper() take too long.
> 
> The real answer is to get a more modern machine than the one you are using. My
> old (1998) machine runs this test at about 2 seconds. (P3 550Mhz)

This is running on a machine made last year, winxp on an AMD 3000+ and a 
gigabyte of ram.

> However, as that is not a real option for you, you could improve the time by
> using a faster case changing algorithm. Here is one that works for ASCII text
> ..

I'll try it as soon as the .... speak of the devil.... it's printing now.....

The code i posted on the webpage used 134 megabytes, and i copy/paste 
here the run:

--------------------------------------
0
3012
3
----
0
2
2
----
0
2
2
----
0
2
2
----
0
2
2
----
0
2
2
----
0
2
2
----
0
2
2
----
0
2
2
----
0
2
2
----
press 'q'

----------------------------------------

Note the first run of RDS Eu upper() took 50 minutes. In normal practice, i 
would not be running it again on the same data, so i don't know that i'd find 
the subsequent runs' timings to be useful. Also, like you said, those are all 
atoms, which should run very quickly, but that first upper() isn't quick at all.
Depending on how busy the computer is, some runs on the 24megabytes 
can take all day.

I don't recall the win95 computer (233mhz) being this slow.

Kat

new topic     » goto parent     » topic index » view message » categorize

10. Re: upper()

Kat wrote:
> 
> Incidently, OOEU runs the commented out line in 4 seconds, but takes all 
> night to run parse() or gets(). Using define_c_proc(open_dll("msvcrt.dll"),
> .. ..> 
> Kat
> 
> 

Sure it is not OOEU to give those problems?
I have met also, starting my application with it, waiting 45 seconds..

Antonio

new topic     » goto parent     » topic index » view message » categorize

11. Re: upper()

Hi Kat,


I ran the following program several times and the results are shown
at the end in comments...


include misc.e
include wildcard.e

sequence data,upperdata
atom t

data = repeat({rand(24)+97},1000000)
upperdata = repeat(0,1000000)

t=time()
upperdata = upper(data)
?time()-t

t=time()
for loop = 1 to length(data)
  do upperdata[loop] = upper(data[loop])
end for
?time()-t

t=time()
upperdata = upper(data)
?time()-t

sleep(50)

--Typical output of program:
-- 6.84
-- 2.29
-- 5.95


I have a rather old system too, under 1GHz speed.


Take care,
Al

E boa sorte com sua programacao Euphoria!


My bumper sticker: "I brake for LED's"

 From "Black Knight":
"I can live with losing the good fight,
 but i can not live without fighting it".
"Well on second thought, maybe not."

new topic     » goto parent     » topic index » view message » categorize

12. Re: upper()

On 6 Aug 2006, at 4:41, Derek Parnell wrote concerning:
Re: upper()

> ------------------------------------------------
> global function toUpperObj(object s)
> ------------------------------------------------
>     if atom(s) then
>         return uc[s+1]
>     else
>         for i = 1 to length(s) do
>             if atom(s[i]) then
>                 s[i] = uc[s[i]+1]
>             else
>                 s[i] = toUpperObj(s[i])
>             end if
>         end for
>     end if
> end function

attempt to exit a function without returning a value

Kat,
still working on it......

new topic     » goto parent     » topic index » view message » categorize

13. Re: upper()

Kat wrote:
> 
> On 6 Aug 2006, at 4:41, Derek Parnell wrote concerning:
> Re: upper()
> 
> > ------------------------------------------------
> > global function toUpperObj(object s)
> > ------------------------------------------------
> >     if atom(s) then
> >         return uc[s+1]
> >     else
> >         for i = 1 to length(s) do
> >             if atom(s[i]) then
> >                 s[i] = uc[s[i]+1]
> >             else
> >                 s[i] = toUpperObj(s[i])
> >             end if
> >         end for
> >     end if
> > end function
> 
> attempt to exit a function without returning a value
> 
> Kat,
> still working on it......
> 
> 

Give a look to my Obj_upper(object x).. the routine is complete and works.
You can just copy/paste and run my sample, that is complete and expressly 
made.

antonio

new topic     » goto parent     » topic index » view message » categorize

14. Re: upper()

Kat wrote:
> 
> On 6 Aug 2006, at 4:41, Derek Parnell wrote concerning:
> Re: upper()
> 
> > ------------------------------------------------
> > global function toUpperObj(object s)
> > ------------------------------------------------
> >     if atom(s) then
> >         return uc[s+1]
> >     else
> >         for i = 1 to length(s) do
> >             if atom(s[i]) then
> >                 s[i] = uc[s[i]+1]
> >             else
> >                 s[i] = toUpperObj(s[i])
> >             end if
> >         end for
> >     end if
> > end function
> 
> attempt to exit a function without returning a value
> 
> Kat,
> still working on it......

Well I just knocked it up in a couple of minutes.. gee!!! Give some slack here
okay! I suppose you can see the stupid error I made. And I suppose you fixed it
already. I only tested the toLower function so that's why I missed it.  No one is
forcing you to use this faster function. I just thought I'd try to help; sorry
for interrupting you.


-- 
Derek Parnell
Melbourne, Australia
Skype name: derek.j.parnell

new topic     » goto parent     » topic index » view message » categorize

15. Re: upper()

Kat wrote:
> 
> data = repeat({rand(24)+97},1000000)
> upperdata = repeat({rand(24)+97},1000000)
> 
> puts(1,"repeats done, press a key\n")
> junk = wait_key()
> 
>   --for loop = 1 to length(data) do upperdata[loop] = upper(data[loop]) end
> for
>   for loop = 1 to length(data) do upperdata[loop] = data[loop] end for
> 

I found some interesting results.  I tried 4 different ways:

1:}}}
<eucode> upperdata = upper(data) </eucode>
{{{

2:}}}
<eucode> for loop = 1 to length(data) do upperdata[loop] = upper(data[loop])
end for
3:}}}
<eucode> upperdata = c_func( c_upper, { data } )</eucode>
{{{
 (compiled wildcard.e
into a dll)
4:}}}
<eucode>
object x 
constant TO_LOWER = 'a' = 'A'
for loop = 1 to length(data) do
	x = data[loop]
	upperdata[loop] = x - (x >= 'a' and x <= 'z') * TO_LOWER
end for
</eucode>
{{{

5: Compiled this into a dll:
constant TO_LOWER = 'a' - 'A'

global function seq_upper(sequence seq)
-- convert atom or sequence to upper case
	sequence y, x
	y = repeat( 0, length(seq) )
	for loop = 1 to length(seq) do
		x = seq[loop]
		y[loop] = x - (x >= 'a' and x <= 'z') * TO_LOWER
	end for
    return y
end function

...and called it by...
upperdata = c_func( c_seq_upper, {data} )


I ran each method under exwc either natively, or using "exwc eu.ex" (ooeu):

   exwc   ooeu
1  4.06   3.85
2  1.10  66.97
3  4.18   4.35
4  1.00   5.83
5  0.97   1.08

(Run on a 2.4GHz Celeron with 512MB RAM.)

I really can't explain why ooeu beats exwc on #1.  The results of #2 and #4 
seem to make it clear that iterating through the sequence (and not forcing
more complicated sequence arithmetic) was faster, which took me to #5,
which definitely seems to be the best, especially for the ooeu case.  And 
for large datasets, it should more than compensate for the delayed startup
time if you don't have the registered translator.

Matt Lewis

new topic     » goto parent     » topic index » view message » categorize

16. Re: upper()

On 6 Aug 2006, at 8:28, Derek Parnell wrote concerning:
Re: upper()

> 
> 
> posted by: Derek Parnell <ddparnell at bigpond.com>
> 
> Kat wrote:
> > 
> > On 6 Aug 2006, at 4:41, Derek Parnell wrote concerning:
> > Re: upper()
> > 
> > > ------------------------------------------------
> > > global function toUpperObj(object s)
> > > ------------------------------------------------
> > >     if atom(s) then
> > >         return uc[s+1]
> > >     else
> > >         for i = 1 to length(s) do
> > >             if atom(s[i]) then
> > >                 s[i] = uc[s[i]+1]
> > >             else
> > >                 s[i] = toUpperObj(s[i])
> > >             end if
> > >         end for
> > >     end if
> > > end function
> > 
> > attempt to exit a function without returning a value
> > 
> > Kat,
> > still working on it......
> 
> Well I just knocked it up in a couple of minutes.. gee!!! Give some slack here
> okay! I suppose you can see the stupid error I made. And I suppose you fixed
> it
> already. I only tested the toLower function so that's why I missed it.  No one
> is forcing you to use this faster function. I just thought I'd try to help;
> sorry for interrupting you.

Oh,, well, actually, i thought there was a problem with listfilter deleteing
lines
or something. I actually went to the euforum webpage and checked there, 
and waited to see if anyone replied to your email and included your email 
with the same errors. Honestly, that *you* made an error was last on my 
mind. And i thought this was code you already had!

Kat

new topic     » goto parent     » topic index » view message » categorize

17. Re: upper()

On 6 Aug 2006, at 4:41, Derek Parnell wrote concerning:
Re: upper()

Thanks, Derek. I replaced RDS's upper() in the test run (on the webpage url i 
gave earlier) with your code. I called 
upperdata = toUpperObj(data)
since data is a nested sequence. These are the results:

0
363
3
----
0
0
2
----
0
1
3
----
0
1
3
----
0
1
3
----
0
1
3
----
0
1
3
----
0
1
3
----
0
1
3
----
0
1
3
----
press 'q'

Note again the fairly large time at the first run. Any idea what is causing
that?

Kat

new topic     » goto parent     » topic index » view message » categorize

18. Re: upper()

Kat wrote:
> 
> data = repeat({rand(24)+97},1000000)
> upperdata = repeat({rand(24)+97},1000000)
> 
> puts(1,"repeats done, press a key\n")
> junk = wait_key()
> 
>   --for loop = 1 to length(data) do upperdata[loop] = upper(data[loop]) end
> for
>   for loop = 1 to length(data) do upperdata[loop] = data[loop] end for
> 
> 
> The commented out line uses 48megabytes of memory, and takes 170 
> seconds to run.
> 
> The next line is the same, but without upper(), and executes instantly.
> 
> This line takes all night to run:
> upperdata = upper(data)
> 


-- this takes about 3-5 secs on my machine...
-- I suggest you execute with ex.exw and not ex.ex from ed.ex Esc->e.

include wildcard.e
sequence data, upperdata

    data = repeat({rand(24)+97},1000000)
    upperdata = repeat(-1, 1000000)

    upperdata = upper(data)

-- my P4 3.0x2-64 with 2Gb Duel Channel DDR 533 (a little bit forgiving).


but if you really need to use loops, 'routine_id()' speeds up things noticeably
in huge loop irrations.

constant id_upper  = routine_id("upper")

    for loop = 1 to length(data) do
        upperdata[loop] = call_func(id_upper, {data[loop]})
    end for


You also might want to try...
constant id_upper  = routine_id("upper")
integer junk

    for loop = 1 to length(data) do
        junk = data[loop]
        upperdata[loop] = call_func(id_upper, {junk})
    end for


> Why, and how can i avoid the penalty of upper(), and still get the upperdata?
> 
> Yes, i need both data and upperdata.
> 
> Incidently, OOEU runs the commented out line in 4 seconds, but takes all 
> night to run parse() or gets(). Using define_c_proc(open_dll("msvcrt.dll"),
> 
> "_strupr", {C_POINTER}) doesn't help in OOEU, but does in exw.exe. It's 
> very frustrating.
> 
> The actual data (not the test set above) is a million ~30-character lines in
> a 
> 24megabyte file. I cannot run the app every 30 minutes anymore. Either 
> parse(), gets(), or upper() take too long.
> 
> Kat
> 
> 



how many programmers does it take to change a light bulb?
none it's a hardware problem!

new topic     » goto parent     » topic index » view message » categorize

19. Re: upper()

On Sun, 6 Aug 2006 08:01:51 -0500, Kat <kat12 at coosahs.net> wrote:

>On 6 Aug 2006, at 4:41, Derek Parnell wrote concerning:
>Re: upper()
>
>> ------------------------------------------------
>> global function toUpperObj(object s)
>> ------------------------------------------------
>>     if atom(s) then
>>         return uc[s+1]
<snip>

>attempt to exit a function without returning a value

btw, you'll also get crashes passing -7 or 2006.
(Derek, I'm not trying to criticise, I just wanted to signal the
warning and point out that it isn't really fair to time-trial that
against anything else. Antonio's Obj_upper has the needed tests but,
perhaps understandably, his upper() does not.)

fwiw here, bltn, is my best stab:
sequence ucMap, lcMap

procedure initcase()
    ucMap = repeat(0,#FF)   -- #FF is highest valid 8-bit ascii (255)
    for i=1 to #FF do
        ucMap[i] = i
    end for
    lcMap = ucMap
    for i='A' to 'Z' do
        lcMap[i] = i+32
    end for
    lcMap['=C4'] = '=E4'
    lcMap['=D6'] = '=F6'
    lcMap['=DC'] = '=FC'
    for i='a' to 'z' do
        ucMap[i] = i-32
    end for
    ucMap['=E4'] = '=C4'
    ucMap['=F6'] = '=D6'
    ucMap['=FC'] = '=DC'
end procedure
initcase()

object c
global function Upper(object x)
    if integer(x) then
        if x >= 1 and x<=#FF then
            return ucMap[x]
        end if
    elsif sequence(x) then
        for i = 1 to length(x) do
            c = x[i]
            if integer(c) then
                if c >= 1 and c<=#FF then
                    x[i] = ucMap[c]
                end if
            elsif sequence(c) then
                x[i] = Upper(c)
            end if
        end for=20=20=20=20=20=20=20=20
    end if
    return x
end function

global function Lower(object x)
    if integer(x) then
        if x >= 1 and x<=#FF then
            return lcMap[x]
        end if
    elsif sequence(x) then
        for i = 1 to length(x) do
            c = x[i]
            if integer(c) then
                if c >= 1 and c<=#FF then
                    x[i] = lcMap[c]
                end if
            elsif sequence(c) then
                x[i] = Lower(c)
            end if
        end for=20=20=20=20=20=20=20=20
    end if
    return x
end function


Regards,
Pete

new topic     » goto parent     » topic index » view message » categorize

20. Re: upper()

Kat wrote:
> 
> On 6 Aug 2006, at 1:35, Julio C. Galaret Viera wrote concerning:
> Re: upper()
> 
> Using?

I meant box, cpu, etc.

The problem with upper() seems to be related, at least in part to the number of
sequences inside data/upperdata.

An alternative may be you do not remove '\n' or '\r' when reading from file so
you have only one long sequence of ~32.000.000 chars.

If I use upper() over this sequence, I quickly runs out of memory since I have
only 256 MB RAM. Even with enough memory it is too slow.

But if loop atom by atom I convert data to upper case in ~180 secs. That seems
to be the fastest way with upper().
Then you may write this sequence as a text file again.


JG

new topic     » goto parent     » topic index » view message » categorize

21. Re: upper()

> 
> 
> posted by: Julio C. Galaret Viera <galaret at adinet.com.uy>
> 
> Kat wrote:
> > 
> > On 6 Aug 2006, at 1:35, Julio C. Galaret Viera wrote concerning: Re:
> > upper()
> > 
> > Using?
> 
> I meant box, cpu, etc.

AMD Sempron 3000+ winxp sp2 1gigram

> The problem with upper() seems to be related, at least in part to the
> number of sequences inside data/upperdata.
> 
> An alternative may be you do not remove '\n' or '\r' when reading from
> file so you have only one long sequence of ~32.000.000 chars.

I parse(data,{10,13}) .

Kat

> If I use upper() over this sequence, I quickly runs out of memory
> since I have only 256 MB RAM. Even with enough memory it is too slow.
> 
> But if loop atom by atom I convert data to upper case in ~180 secs.
> That seems to be the fastest way with upper(). Then you may write this
> sequence as a text file again.
> 
> 
> JG
> 
> 
> 
>

new topic     » goto parent     » topic index » view message » categorize

22. Re: upper()

Kat wrote:
> 
> > 
> > posted by: Julio C. Galaret Viera <galaret at adinet.com.uy>
> > 
> > Kat wrote:
> > > 
> > > On 6 Aug 2006, at 1:35, Julio C. Galaret Viera wrote concerning: Re:
> > > upper()
> > > 
> > > Using?
> > 
> > I meant box, cpu, etc.
> 
> AMD Sempron 3000+ winxp sp2 1gigram
> 
> > The problem with upper() seems to be related, at least in part to the
> > number of sequences inside data/upperdata.
> > 
> > An alternative may be you do not remove '\n' or '\r' when reading from
> > file so you have only one long sequence of ~32.000.000 chars.
> 
> I parse(data,{10,13}) .
> 
> Kat

Perhaps due to the fact that English is not my native language, I am being
misunderstood. Sorry for my English.
What I mean is I advise you not to parse data initially so that you will manage
only one long sentence on which you can run upper() but instead of running it on
the sequence as a whole, run it recursively on each element of the sequence.
After that, if you need it, you may parse the sequence upperdata obtained,
getting rid of #10 and #13.

JG

> 
> > If I use upper() over this sequence, I quickly runs out of memory
> > since I have only 256 MB RAM. Even with enough memory it is too slow.
> > 
> > But if loop atom by atom I convert data to upper case in ~180 secs.
> > That seems to be the fastest way with upper(). Then you may write this
> > sequence as a text file again.
> > 
> > 
> > JG
> > 
> >

new topic     » goto parent     » topic index » view message » categorize

23. Re: upper()

Many people have demonstrated some nice ways but here's mine anyway...

theese funcs will only work on flat strings but they could easily be modified to
take an object and/or recurse a nested sequence.

nb. just a basic approach that will handle all characters.
    there's no arithmatic or condition testing.

include wildcard.e as euinc

sequence asciiLUT
    
    asciiLUT = repeat(-1, 256)
    for j = 0 to 255 by 1 do
        asciiLUT[(j+1)] = j
    end for

global function upper(sequence st)
    integer ch

    asciiLUT = euinc:upper(asciiLUT)
    for ps = 1 to length(st) by 1 do
        ch = st[ps]
        st[ps] = asciiLUT[(ch+1)]
    end for

    return st

end function

global function lower(sequence st)
    integer ch

    asciiLUT = euinc:lower(asciiLUT)
    for ps = 1 to length(st) by 1 do
        ch = st[ps]
        st[ps] = asciiLUT[(ch+1)]
    end for

    return st

end function


new topic     » goto parent     » topic index » view message » categorize

24. Re: upper()

On Thu, 10 Aug 2006 10:34:09 -0700, Hayden McKay
<guest at RapidEuphoria.com> wrote:

>}}}
<eucode>
>global function upper(sequence st)
>    asciiLUT = euinc:upper(asciiLUT)
>    ...
>end function
>
>global function lower(sequence st)
>    asciiLUT = euinc:lower(asciiLUT)
>    ...
>end function
></eucode>
{{{

Ouch! You clearly don't need to call euinc:lower/upper every single
time these replacements are called.

One other thing I feel a need to say here is that it seems apparent
that this approach will match the usual upper/lower 100%. It will not.
As a (daft) example, euinc:lower(75.123) will give 107.123 whereas the
replacement will typecheck on both 75.123 and {75.123}; and, if
"integer ch" is replaced with "atom ch", the latter will give {107}. 

No biggie, just saying don't claim/think 100% compatible when not.
I am of course slighting wildcard.e on that last point, not your code.

Barring those niggles, I think that is a clever approach.

Regards,
Pete

new topic     » goto parent     » topic index » view message » categorize

25. Re: upper()

> 
> 
> On Thu, 10 Aug 2006 10:34:09 -0700, Hayden McKay
> <guest at RapidEuphoria.com> wrote:
> 
> >}}}
<eucode>
> >global function upper(sequence st)
> >    asciiLUT = euinc:upper(asciiLUT)
> >    ...
> >end function
> >
> >global function lower(sequence st)
> >    asciiLUT = euinc:lower(asciiLUT)
> >    ...
> >end function
> ></eucode>
{{{

> Ouch! You clearly don't need to call euinc:lower/upper every single
> time these replacements are called.
> 
> One other thing I feel a need to say here is that it seems apparent
> that this approach will match the usual upper/lower 100%. It will not.
> As a (daft) example, euinc:lower(75.123) will give 107.123 whereas the
> replacement will typecheck on both 75.123 and {75.123}; and, if
> "integer ch" is replaced with "atom ch", the latter will give {107}. 
> 
> No biggie, just saying don't claim/think 100% compatible when not. I
> am of course slighting wildcard.e on that last point, not your code.
> 
> Barring those niggles, I think that is a clever approach.

I have been using the code Derek supplied, and run times have 
been ~6 sec to ~140 seconds, depending on how busy the 
machine is. 

This has all become academic. The neighbor's new dogs have been 
challenging me for right to access my property and mailbox, and 
the neighbor is now threatening to sue me for harrassment even tho 
i have constantly tried to avoid him. I'll be living in my car, in other 
states, after court on August 15th 2006. This is the only way i 
know to get further away from him and his dogs.

Kat

new topic     » goto parent     » topic index » view message » categorize

26. Re: upper()

Kat wrote:
> 
> > 
> > On Thu, 10 Aug 2006 10:34:09 -0700, Hayden McKay
> > <guest at RapidEuphoria.com> wrote:
> > 
> > >}}}
<eucode>
> > >global function upper(sequence st)
> > >    asciiLUT = euinc:upper(asciiLUT)
> > >    ...
> > >end function
> > >
> > >global function lower(sequence st)
> > >    asciiLUT = euinc:lower(asciiLUT)
> > >    ...
> > >end function
> > ></eucode>
{{{

> > Ouch! You clearly don't need to call euinc:lower/upper every single
> > time these replacements are called.
> > 
> > One other thing I feel a need to say here is that it seems apparent
> > that this approach will match the usual upper/lower 100%. It will not.
> > As a (daft) example, euinc:lower(75.123) will give 107.123 whereas the
> > replacement will typecheck on both 75.123 and {75.123}; and, if
> > "integer ch" is replaced with "atom ch", the latter will give {107}. 
> > 
> > No biggie, just saying don't claim/think 100% compatible when not. I
> > am of course slighting wildcard.e on that last point, not your code.
> > 
> > Barring those niggles, I think that is a clever approach.
> 
> I have been using the code Derek supplied, and run times have 
> been ~6 sec to ~140 seconds, depending on how busy the 
> machine is. 
> 
> This has all become academic. The neighbor's new dogs have been 
> challenging me for right to access my property and mailbox, and 
> the neighbor is now threatening to sue me for harrassment even tho 
> i have constantly tried to avoid him. I'll be living in my car, in other 
> states, after court on August 15th 2006. This is the only way i 
> know to get further away from him and his dogs.
> 
> Kat
> 
More trouble with the neighbor and his dogs?  After reading your previous posts
about this problem I have a suggestion:

First, purchase a silenced pistol - a Beretta M92 would do nicely, but a HK23
.45 calibur would be better.
You said in one of the other posts that you already owned a gun or two, so you
could just take it to a gunsmith and have a silencer installed.
But if there aren't that many people around you might not even need a silencer.

Also buy some kind of electrical stun gun that works on dogs as well as humans.
Next, wait till the neighbour leaves his house, go over and zap the dog(s)
unconcious, drag them over to your house and shoot it(them), then bury them
somewhere in your woods.
Neighbour comes home, dog(s) is(are) nowhere to be found and he doesn't have a
clue what happened to them.

The other way is to give it some poisoned meat when he attacks you, then repeat
burying him in the woods.
If he continues getting more dogs I suggest using C4 on his house:), or simply
hiring a detective to dig up some dirt on him and then blackmail him to move
away.

I might also suggest moving to another state, Alabama doesn't sound like a very
good one.

Jeremy

new topic     » goto parent     » topic index » view message » categorize

27. Re: upper()

Jeremy Peterson wrote:
> 
(snip)
> More trouble with the neighbor and his dogs?  After reading your previous
> posts
> about this problem I have a suggestion:
> 
> First, purchase a silenced pistol - a Beretta M92 would do nicely, but a HK23
> .45 calibur would be better.
> You said in one of the other posts that you already owned a gun or two, so you
> could just take it to a gunsmith and have a silencer installed.
> But if there aren't that many people around you might not even need a
> silencer.
> 
> Also buy some kind of electrical stun gun that works on dogs as well as
> humans.
> Next, wait till the neighbour leaves his house, go over and zap the dog(s)
> unconcious,
> drag them over to your house and shoot it(them), then bury them somewhere in
> your woods.
> Neighbour comes home, dog(s) is(are) nowhere to be found and he doesn't have
> a clue what happened to them.
> 
> The other way is to give it some poisoned meat when he attacks you, then
> repeat
> burying him in the woods.
> If he continues getting more dogs I suggest using C4 on his house:), or simply
> hiring a detective to dig up some dirt on him and then blackmail him to move
> away.  
> 
> I might also suggest moving to another state, Alabama doesn't sound like a
> very
> good one.
> 
> Jeremy

Err... While this may have been a joke I think that advocating illegal behavior
on a public mailing list is typically bad taste.

new topic     » goto parent     » topic index » view message » categorize

28. Re: upper()

> may have been a joke"
Well um yes...

>I think that advocating illegal behavior
>on a public mailing list is typically bad taste.
Well I don't.  It should be obvious that this is a intended to be a joke, and
most of you guys could do with a little humour now and then.

But jokes on this forum never seem to be appreciated, except by me and 1 or 2
others...

Take a look at this example from
http://www.listfilter.com/cgi-bin/esearch.exu?thread=1&fromMonth=6&fromYear=A&toMonth=8&toYear=A&keywords=%223.0+feature+request:+foreach%22

As you can see, nobody laughed at it.

>Date: 2005 Jul 13 7:41
>From: Dave Probert <zingo at purpletiger.com>
>Subject: Re: 3.0 feature request: foreach

Vincent wrote:
>> 
>> D. Newhall wrote:
> > >
> > >I think that the next release should have something new and useful to the
> > >languag
e.
--<snip>
> > > set to the next next element in "line". A consideration would be to
> > > optimize
expressions
> > >of the type "foreach obj in reverse(seq)do"
> > >
> >
> >Hi there... I don't think there needs to be any more loop methods. "while"
> >and
"for"
--<snip>
> >
> >Using UPX for EX.EXE and hardware floating point compiler flags, background
> >transpa
rent
> >icons, and a nice multitasking library & documentation dirived from Language
> >War's
> >Sched.e would make the nice addition for Euphoria 3.0.
> 
> >I'm looking forward to 3.0, I can't wait to try out the "Language War"
> >technique,
seeing
> >how much smaller EX.EXE is, and observe those new Windows icons on my
> >colorful
desktop
> >backgrounds.
> >
> >
> >Regards,
> >Vincent

>Sorry, gotta go with Vincent on this one - we really do need to have more
>>colourful
>icons for our desktop.

>Our programs will never be as good as ones written in the more complicated
>>languages
>without the colourful icons.  In fact I think Euphoria should drop many of >the
>
>current language features like - print, sprint, while, if, message_box, etc >in
>
>favour of having some extra functionality to provide better icon support.  

>Who needs other stuff anyway?

>:)
>Dave

>PS.  I actually vote FOR foreach - use it all the time in PHP and it rally
>>does save
>a huge amount of confusion - though mainly with associative arrays!

Hahahahahahahahahah! :)  Great joke Dave!

Jeremy

new topic     » goto parent     » topic index » view message » categorize

29. Re: upper()

Jeremy Peterson wrote:
> 
> > may have been a joke"
> Well um yes...
> 
> >I think that advocating illegal behavior
> >on a public mailing list is typically bad taste.
> Well I don't.  It should be obvious that this is a intended to be a joke, and
> most of you guys could do with a little humour now and then.
> 
> But jokes on this forum never seem to be appreciated, except by me and 1 or
> 2 others... 
> 
> Take a look at this example from
<<SNIP>>

Jeremy,

If you've really read Kat's description of her situation, then you should
be able to understand that it's NO JOKE, the stupid dogs could KILL her.

Stupid jokes are NO HELP at all.  

Dan Moyer

new topic     » goto parent     » topic index » view message » categorize

30. Re: upper()

And like I said, she should move away from that town and maybe even that state. 
That was a serious suggestion.

And I can bet someone is going to say
"Re: upper()" shove off, Jeremy"

Jeremy

new topic     » goto parent     » topic index » view message » categorize

31. Re: upper()

Jeremy Peterson wrote:
> 
> Kat wrote:
> > 
> > > 
> > > On Thu, 10 Aug 2006 10:34:09 -0700, Hayden McKay
> > > <guest at RapidEuphoria.com> wrote:
> > > 
> > > >}}}
<eucode>
> > > >global function upper(sequence st)
> > > >    asciiLUT = euinc:upper(asciiLUT)
> > > >    ...
> > > >end function
> > > >
> > > >global function lower(sequence st)
> > > >    asciiLUT = euinc:lower(asciiLUT)
> > > >    ...
> > > >end function
> > > ></eucode>
{{{

> > > Ouch! You clearly don't need to call euinc:lower/upper every single
> > > time these replacements are called.
> > > 
> > > One other thing I feel a need to say here is that it seems apparent
> > > that this approach will match the usual upper/lower 100%. It will not.
> > > As a (daft) example, euinc:lower(75.123) will give 107.123 whereas the
> > > replacement will typecheck on both 75.123 and {75.123}; and, if
> > > "integer ch" is replaced with "atom ch", the latter will give {107}. 
> > > 
> > > No biggie, just saying don't claim/think 100% compatible when not. I
> > > am of course slighting wildcard.e on that last point, not your code.
> > > 
> > > Barring those niggles, I think that is a clever approach.
> > 
> > I have been using the code Derek supplied, and run times have 
> > been ~6 sec to ~140 seconds, depending on how busy the 
> > machine is. 
> > 
> > This has all become academic. The neighbor's new dogs have been 
> > challenging me for right to access my property and mailbox, and 
> > the neighbor is now threatening to sue me for harrassment even tho 
> > i have constantly tried to avoid him. I'll be living in my car, in other 
> > states, after court on August 15th 2006. This is the only way i 
> > know to get further away from him and his dogs.
> > 
> > Kat
> > 
> More trouble with the neighbor and his dogs?  After reading your previous
> posts
> about this problem I have a suggestion:
> 
> First, purchase a silenced pistol - a Beretta M92 would do nicely, but a HK23
> .45 calibur would be better.
> You said in one of the other posts that you already owned a gun or two, so you
> could just take it to a gunsmith and have a silencer installed.
> But if there aren't that many people around you might not even need a
> silencer.
> 
> Also buy some kind of electrical stun gun that works on dogs as well as
> humans.
> Next, wait till the neighbour leaves his house, go over and zap the dog(s)
> unconcious,
> drag them over to your house and shoot it(them), then bury them somewhere in
> your woods.
> Neighbour comes home, dog(s) is(are) nowhere to be found and he doesn't have
> a clue what happened to them.
> 
> The other way is to give it some poisoned meat when he attacks you, then
> repeat
> burying him in the woods.
> If he continues getting more dogs I suggest using C4 on his house:), or simply
> hiring a detective to dig up some dirt on him and then blackmail him to move
> away.  
> 
> I might also suggest moving to another state, Alabama doesn't sound like a
> very
> good one.
> 
> Jeremy

I had a similar problem. All I do is ring the local council and tell them that
there is a 'big bitbull' wandering our street and that you think it belongs to
next door. The council dog catches will be there in about 10 mins
see the dog (even though it's probably not a bitbull) and take the dog away.

nb. The key is to mention 'bitbull', 'doverman' etc... don't tell them that it's
realy only a 'chowowa' jackrussle' etc...

new topic     » goto parent     » topic index » view message » categorize

32. Re: upper()

Jeremy Peterson wrote:
> 
> And like I said, she should move away from that town and maybe even that
> state.
>  That was a serious suggestion.
> 
> And I can bet someone is going to say
> "Re: upper()" shove off, Jeremy"
> 
> Jeremy

   Maybe Alabama needs Moore Governor.

Don Cole

new topic     » goto parent     » topic index » view message » categorize

Search



Quick Links

User menu

Not signed in.

Misc Menu