1. upper()
- Posted by "Kat" <Kat12 at coosahs.net> Aug 06, 2006
- 724 views
data = repeat({rand(24)+97},1000000) upperdata = repeat({rand(24)+97},1000000) puts(1,"repeats done, press a key\n") junk = wait_key() --for loop = 1 to length(data) do upperdata[loop] = upper(data[loop]) end for for loop = 1 to length(data) do upperdata[loop] = data[loop] end for The commented out line uses 48megabytes of memory, and takes 170 seconds to run. The next line is the same, but without upper(), and executes instantly. This line takes all night to run: upperdata = upper(data) Why, and how can i avoid the penalty of upper(), and still get the upperdata? Yes, i need both data and upperdata. Incidently, OOEU runs the commented out line in 4 seconds, but takes all night to run parse() or gets(). Using define_c_proc(open_dll("msvcrt.dll"), "_strupr", {C_POINTER}) doesn't help in OOEU, but does in exw.exe. It's very frustrating. The actual data (not the test set above) is a million ~30-character lines in a 24megabyte file. I cannot run the app every 30 minutes anymore. Either parse(), gets(), or upper() take too long. Kat
2. Re: upper()
- Posted by Julio C. Galaret Viera <galaret at adinet.com.uy> Aug 06, 2006
- 666 views
The commented out line takes 1-2 seconds to run. What are you using? JG
3. Re: upper()
- Posted by Jeremy Peterson <ptl99 at hotmail.com> Aug 06, 2006
- 689 views
Same here. Uncommenting that line in this program:
include get.e include wildcard.e sequence data, upperdata atom junk data = repeat({rand(24)+97},1000000) upperdata = repeat({rand(24)+97},1000000) puts(1,"repeats done, press a key\n") junk = wait_key() for loop = 1 to length(data) do upperdata[loop] = upper(data[loop]) end for --for loop = 1 to length(data) do upperdata[loop] = data[loop] end for
Only takes a few seconds to run. Jeremy
4. Re: upper()
- Posted by Jeremy Peterson <ptl99 at hotmail.com> Aug 06, 2006
- 687 views
Incidentally, dumping the sequence upperdata to a file gives a 4.76 MB, 4883 line file full of nothing but: {{78},{78},{78},{78},{78},{78},{78},{78},{78},{78},{78},{78},{78},{78},{78},{78},{78},{78},{78},{78},{78},{78},{78},{78},{78},{78},{78},{78},{78},{78} Altho the number 78 changes for each run. Running the program uses about 40(approximately) MB of memory, like you said. However, changing the line:
data = repeat({rand(24)+97},1000000) upperdata = repeat({rand(24)+97},1000000) --TO: data = repeat({"get out of my face"},1000000) upperdata = repeat({"get out of my face"},1000000)
Makes the program take about 12 seconds, the file outputted is 56641 lines, 55.3MB In your test program it is simply running upper on atoms, which wouldn't take very long. Jeremy
5. Re: upper()
- Posted by "Kat" <Kat12 at coosahs.net> Aug 06, 2006
- 681 views
On 6 Aug 2006, at 1:35, Julio C. Galaret Viera wrote concerning: Re: upper() Using? Kat
6. Re: upper()
- Posted by Antonio Alessi <a.admin at myway.it> Aug 06, 2006
- 684 views
Kat wrote: > > data = repeat({rand(24)+97},1000000) > upperdata = repeat({rand(24)+97},1000000) > > puts(1,"repeats done, press a key\n") > junk = wait_key() > > --for loop = 1 to length(data) do upperdata[loop] = upper(data[loop]) end > for > for loop = 1 to length(data) do upperdata[loop] = data[loop] end for > > > The commented out line uses 48megabytes of memory, and takes 170 > seconds to run. > > The next line is the same, but without upper(), and executes instantly. > > This line takes all night to run: > upperdata = upper(data) > > Why, and how can i avoid the penalty of upper(), and still get the upperdata? > > Yes, i need both data and upperdata. .. .. > Kat > > Hi Kat, the time you got seems to be out of the context. Here are two routines to compare with your expression: the RDS official 2.5; the beautiful one of Derek, (arranged here) that I use. on my system it deals with 1 to 1,5 seconds. RDS's elapsed Time 1.24 used RAM 45056 Derek's elapsed Time 0.58 used RAM 0 ------- Derek's elapsed Time 2.12 used RAM 663552 ___________ <space> Note that the declared free memory can be due to other factors. Antonio
global atom get_Mem, free_mem, min_free_mem, bg, en include win32lib.ew without warning --- ========= constant lc = {#1,#2,#3,#4,#5,#6,#7,#8,#9,#A,#B,#C,#D,#E,#F,#10,#11,#12,#13,#14,#15,#16,#17,#18 ,#19,#1A,#1B,#1C,#1D,#1E,#1F,#20,#21,#22,#23,#24,#25,#26,#27,#28,#29,#2A,#2B,#2C ,#2D,#2E,#2F,#30,#31,#32,#33,#34,#35,#36,#37,#38,#39,#3A,#3B,#3C,#3D,#3E,#3F,#40 ,#61,#62,#63,#64,#65,#66,#67,#68,#69,#6A,#6B,#6C,#6D,#6E,#6F,#70,#71,#72,#73,#74 ,#75,#76,#77,#78,#79,#7A,#5B,#5C,#5D,#5E,#5F,#60,#61,#62,#63,#64,#65,#66,#67,#68 ,#69,#6A,#6B,#6C,#6D,#6E,#6F,#70,#71,#72,#73,#74,#75,#76,#77,#78,#79,#7A,#7B,#7C ,#7D,#7E,#7F,#80,#81,#82,#83,#84,#85,#86,#87,#88,#89,#9A,#8B,#9C,#8D,#9E,#8F,#90 ,#91,#92,#93,#94,#95,#96,#97,#98,#99,#9A,#9B,#9C,#9D,#9E,#FF,#A0,#A1,#A2,#A3,#A4 ,#A5,#A6,#A7,#A8,#A9,#AA,#AB,#AC,#AD,#AE,#AF,#B0,#B1,#B2,#B3,#B4,#B5,#B6,#B7,#B8 ,#B9,#BA,#BB,#BC,#BD,#BE,#BF,#E0,#E1,#E2,#E3,#E4,#E5,#E6,#E7,#E8,#E9,#EA,#EB,#EC ,#ED,#EE,#EF,#F0,#F1,#F2,#F3,#F4,#F5,#F6,#D7,#F8,#F9,#FA,#FB,#FC,#FD,#FE,#DF,#E0 ,#E1,#E2,#E3,#E4,#E5,#E6,#E7,#E8,#E9,#EA,#EB,#EC,#ED,#EE,#EF,#F0,#F1,#F2,#F3,#F4 ,#F5,#F6,#F7,#F8,#F9,#FA,#FB,#FC,#FD,#FE,#FF} ,uc = {#1,#2,#3,#4,#5,#6,#7,#8,#9,#A,#B,#C,#D,#E,#F,#10,#11,#12,#13,#14,#15,#16,#17,#18 ,#19,#1A,#1B,#1C,#1D,#1E,#1F,#20,#21,#22,#23,#24,#25,#26,#27,#28,#29,#2A,#2B,#2C ,#2D,#2E,#2F,#30,#31,#32,#33,#34,#35,#36,#37,#38,#39,#3A,#3B,#3C,#3D,#3E,#3F,#40 ,#41,#42,#43,#44,#45,#46,#47,#48,#49,#4A,#4B,#4C,#4D,#4E,#4F,#50,#51,#52,#53,#54 ,#55,#56,#57,#58,#59,#5A,#5B,#5C,#5D,#5E,#5F,#60,#41,#42,#43,#44,#45,#46,#47,#48 ,#49,#4A,#4B,#4C,#4D,#4E,#4F,#50,#51,#52,#53,#54,#55,#56,#57,#58,#59,#5A,#7B,#7C ,#7D,#7E,#7F,#80,#81,#82,#83,#84,#85,#86,#87,#88,#89,#8A,#8B,#8C,#8D,#8E,#8F,#90 ,#91,#92,#93,#94,#95,#96,#97,#98,#99,#8A,#9B,#8C,#9D,#8E,#9F,#A0,#A1,#A2,#A3,#A4 ,#A5,#A6,#A7,#A8,#A9,#AA,#AB,#AC,#AD,#AE,#AF,#B0,#B1,#B2,#B3,#B4,#B5,#B6,#B7,#B8 ,#B9,#BA,#BB,#BC,#BD,#BE,#BF,#C0,#C1,#C2,#C3,#C4,#C5,#C6,#C7,#C8,#C9,#CA,#CB,#CC ,#CD,#CE,#CF,#D0,#D1,#D2,#D3,#D4,#D5,#D6,#D7,#D8,#D9,#DA,#DB,#DC,#DD,#DE,#DF,#C0 ,#C1,#C2,#C3,#C4,#C5,#C6,#C7,#C8,#C9,#CA,#CB,#CC,#CD,#CE,#CF,#D0,#D1,#D2,#D3,#D4 ,#D5,#D6,#F7,#D8,#D9,#DA,#DB,#DC,#DD,#DE,#9F} constant TO_LOWER = 'a' - 'A' global function EUupper(object x) -- convert atom or sequence to upper case return x - (x >= 'a' and x <= 'z') * TO_LOWER end function --- ======== global function upper(object x) if integer(x) then x = uc[x +1] elsif sequence(x) then for i = 1 to length(x) do if x[i] then x[i] = uc[x[i] +1] end if end for end if return x end function --- ======== global function Obj_upper(object x) if integer(x) then if x >= 0 and x < length(uc) then x = uc[x+1] end if elsif sequence(x) then for i = 1 to length(x) do if integer(x[i]) then if x[i] >= 0 and x[i] < length(uc) then x[i] = uc[x[i]+1] end if elsif sequence(x) then x[i] = Obj_upper(x[i]) end if end for end if return x end function --- ======== constant kernel32 = open_dll("kernel32.dll") -- by H. W. Overman ,xGlobalMemoryStatus = define_c_proc(kernel32,"GlobalMemoryStatus",{C_POINTER}) ,MEMORYSTATUS = allocate(32) mem_set(MEMORYSTATUS, 0, 32) function get_freeMemory( ) c_proc( xGlobalMemoryStatus, {MEMORYSTATUS}) return peek4u( MEMORYSTATUS +12) end function get_Mem = routine_id( "get_freeMemory") --- ========= atom tempo sequence data, upperdata constant iterations = 1 data = repeat({rand(24)+97},1000000) upperdata = repeat({rand(24)+97},1000000) bg = get_freeMemory( ) tempo=time() for j = 1 to iterations do for loop = 1 to length(data) do upperdata[loop] = EUupper(data[loop]) end for end for puts(1, sprintf( "RDS's elapsed Time %2.2f\n", time()-tempo)) puts(1, sprintf( "used RAM %8d\n\n", bg -get_freeMemory( ))) bg = get_freeMemory( ) tempo=time() for j = 1 to iterations do for loop = 1 to length(data) do upperdata[loop] = upper(data[loop]) end for end for puts(1, sprintf( "Derek's elapsed Time %2.2f\n", time()-tempo)) puts(1, sprintf( "used RAM %8d\n\n", bg -get_freeMemory( ))) bg = get_freeMemory( ) tempo=time() -- for j = 1 to iterations do upperdata = Obj_upper(data) -- end for puts(1, sprintf( "Derek's elapsed Time %2.2f\n", time()-tempo)) puts(1, sprintf( "used RAM %8d\n\n", bg -get_freeMemory( ))) puts(1, " ___________\n") puts(1, "<space>") puts(1, wait_key())
7. Re: upper()
- Posted by Derek Parnell <ddparnell at bigpond.com> Aug 06, 2006
- 688 views
Kat wrote: > I cannot run the app every 30 minutes anymore. Either > parse(), gets(), or upper() take too long. The real answer is to get a more modern machine than the one you are using. My old (1998) machine runs this test at about 2 seconds. (P3 550Mhz) However, as that is not a real option for you, you could improve the time by using a faster case changing algorithm. Here is one that works for ASCII text ..
-- chgcase.e -- -- Routines for converting ASCII characters between upper and lower cases. global sequence uc global sequence lc ------------------------------------------------ procedure init() ------------------------------------------------ uc = repeat(0, 256) for i = 1 to 256 do uc[i] = i-1 end for lc = uc -- Adjust uppercase table for i = 'a' to 'z' do uc[i] = uc[i] + ('A' - 'a') end for -- Adjust lowercase table for i = 'A' to 'Z' do lc[i] = lc[i] + ('a' - 'A') end for end procedure ------------------------------------------------ global function toUpper(sequence s) ------------------------------------------------ for i = 1 to length(s) do s[i] = uc[s[i]+1] end for return s end function ------------------------------------------------ global function toLower(sequence s) ------------------------------------------------ for i = 1 to length(s) do s[i] = lc[s[i]+1] end for return s end function ------------------------------------------------ global function toUpperAtom(atom s) ------------------------------------------------ return uc[s+1] end function ------------------------------------------------ global function toLowerAtom(atom s) ------------------------------------------------ return lc[s+1] end function ------------------------------------------------ global function toUpperObj(object s) ------------------------------------------------ if atom(s) then return uc[s+1] else for i = 1 to length(s) do if atom(s[i]) then s[i] = uc[s[i]+1] else s[i] = toUpperObj(s[i]) end if end for end if end function ------------------------------------------------ global function toLowerObj(object s) ------------------------------------------------ if atom(s) then return lc[s+1] else for i = 1 to length(s) do if atom(s[i]) then s[i] = lc[s[i]+1] else s[i] = toUpperObj(s[i]) end if end for return s end if end function ------------------------------------------------ ------------------------------------------------ ------------------------------------------------ init() ------------------------------------------------ ------------------------------------------------ ------------------------------------------------
This should halve the time. However you can shave a few more % points off it by inlining the function...
-- test.ex -- include get.e include misc.e include chgcase.e sequence data data = repeat({rand(24)+97},1000000) sequence upperdata upperdata = repeat({rand(24)+97},1000000) object junk puts(1,"repeats done, press a key\n") junk = wait_key() atom e sequence s e = time() for loop = 1 to length(data) do s = data[loop] for j = 1 to length(upperdata[loop]) do s[j] = uc[s[j]+1] end for upperdata[loop] = s end for printf(1, "%g\n", time()- e) -------------
-- Derek Parnell Melbourne, Australia Skype name: derek.j.parnell
8. Re: upper()
- Posted by "Kat" <Kat12 at coosahs.net> Aug 06, 2006
- 667 views
For munging the 24megabyte data, i started the line: upperurllist = upper(urllist) And 45 minutes later, you can see the progress at http://www.designerthinking.com/temp/UPPER1.GIF It quickly ramped up to 320megabyes memory use at 5:30am, 500megabytes at 5:50am, and at 6:20am it's at 200megabytes memory use,, still on that upper() line. (At 6:25am, it dropped to 100megs again, and is on the way up, fast, two mintes later it's at 551 megabytes.) This is unacceptable. It's still on the same line, with one call to upper(). I started http://www.designerthinking.com/temp/testupper.ew 20 minutes ago too, and have yet to get the first print of timing. It's at 100megabytes memory use now. Kat
9. Re: upper()
- Posted by "Kat" <Kat12 at coosahs.net> Aug 06, 2006
- 678 views
On 6 Aug 2006, at 4:41, Derek Parnell wrote concerning: Re: upper() > > > posted by: Derek Parnell <ddparnell at bigpond.com> > > Kat wrote: > > I cannot run the app every 30 minutes anymore. Either > > parse(), gets(), or upper() take too long. > > The real answer is to get a more modern machine than the one you are using. My > old (1998) machine runs this test at about 2 seconds. (P3 550Mhz) This is running on a machine made last year, winxp on an AMD 3000+ and a gigabyte of ram. > However, as that is not a real option for you, you could improve the time by > using a faster case changing algorithm. Here is one that works for ASCII text > .. I'll try it as soon as the .... speak of the devil.... it's printing now..... The code i posted on the webpage used 134 megabytes, and i copy/paste here the run: -------------------------------------- 0 3012 3 ---- 0 2 2 ---- 0 2 2 ---- 0 2 2 ---- 0 2 2 ---- 0 2 2 ---- 0 2 2 ---- 0 2 2 ---- 0 2 2 ---- 0 2 2 ---- press 'q' ---------------------------------------- Note the first run of RDS Eu upper() took 50 minutes. In normal practice, i would not be running it again on the same data, so i don't know that i'd find the subsequent runs' timings to be useful. Also, like you said, those are all atoms, which should run very quickly, but that first upper() isn't quick at all. Depending on how busy the computer is, some runs on the 24megabytes can take all day. I don't recall the win95 computer (233mhz) being this slow. Kat
10. Re: upper()
- Posted by Antonio Alessi <a.admin at myway.it> Aug 06, 2006
- 676 views
Kat wrote: > > Incidently, OOEU runs the commented out line in 4 seconds, but takes all > night to run parse() or gets(). Using define_c_proc(open_dll("msvcrt.dll"), > .. ..> > Kat > > Sure it is not OOEU to give those problems? I have met also, starting my application with it, waiting 45 seconds.. Antonio
11. Re: upper()
- Posted by Al Getz <Xaxo at aol.com> Aug 06, 2006
- 674 views
Hi Kat, I ran the following program several times and the results are shown at the end in comments... include misc.e include wildcard.e sequence data,upperdata atom t data = repeat({rand(24)+97},1000000) upperdata = repeat(0,1000000) t=time() upperdata = upper(data) ?time()-t t=time() for loop = 1 to length(data) do upperdata[loop] = upper(data[loop]) end for ?time()-t t=time() upperdata = upper(data) ?time()-t sleep(50) --Typical output of program: -- 6.84 -- 2.29 -- 5.95 I have a rather old system too, under 1GHz speed. Take care, Al E boa sorte com sua programacao Euphoria! My bumper sticker: "I brake for LED's" From "Black Knight": "I can live with losing the good fight, but i can not live without fighting it". "Well on second thought, maybe not."
12. Re: upper()
- Posted by "Kat" <Kat12 at coosahs.net> Aug 06, 2006
- 668 views
On 6 Aug 2006, at 4:41, Derek Parnell wrote concerning: Re: upper() > ------------------------------------------------ > global function toUpperObj(object s) > ------------------------------------------------ > if atom(s) then > return uc[s+1] > else > for i = 1 to length(s) do > if atom(s[i]) then > s[i] = uc[s[i]+1] > else > s[i] = toUpperObj(s[i]) > end if > end for > end if > end function attempt to exit a function without returning a value Kat, still working on it......
13. Re: upper()
- Posted by Antonio Alessi <a.admin at myway.it> Aug 06, 2006
- 685 views
Kat wrote: > > On 6 Aug 2006, at 4:41, Derek Parnell wrote concerning: > Re: upper() > > > ------------------------------------------------ > > global function toUpperObj(object s) > > ------------------------------------------------ > > if atom(s) then > > return uc[s+1] > > else > > for i = 1 to length(s) do > > if atom(s[i]) then > > s[i] = uc[s[i]+1] > > else > > s[i] = toUpperObj(s[i]) > > end if > > end for > > end if > > end function > > attempt to exit a function without returning a value > > Kat, > still working on it...... > > Give a look to my Obj_upper(object x).. the routine is complete and works. You can just copy/paste and run my sample, that is complete and expressly made. antonio
14. Re: upper()
- Posted by Derek Parnell <ddparnell at bigpond.com> Aug 06, 2006
- 679 views
Kat wrote: > > On 6 Aug 2006, at 4:41, Derek Parnell wrote concerning: > Re: upper() > > > ------------------------------------------------ > > global function toUpperObj(object s) > > ------------------------------------------------ > > if atom(s) then > > return uc[s+1] > > else > > for i = 1 to length(s) do > > if atom(s[i]) then > > s[i] = uc[s[i]+1] > > else > > s[i] = toUpperObj(s[i]) > > end if > > end for > > end if > > end function > > attempt to exit a function without returning a value > > Kat, > still working on it...... Well I just knocked it up in a couple of minutes.. gee!!! Give some slack here okay! I suppose you can see the stupid error I made. And I suppose you fixed it already. I only tested the toLower function so that's why I missed it. No one is forcing you to use this faster function. I just thought I'd try to help; sorry for interrupting you. -- Derek Parnell Melbourne, Australia Skype name: derek.j.parnell
15. Re: upper()
- Posted by Matt Lewis <matthewwalkerlewis at gmail.com> Aug 06, 2006
- 678 views
Kat wrote: > > data = repeat({rand(24)+97},1000000) > upperdata = repeat({rand(24)+97},1000000) > > puts(1,"repeats done, press a key\n") > junk = wait_key() > > --for loop = 1 to length(data) do upperdata[loop] = upper(data[loop]) end > for > for loop = 1 to length(data) do upperdata[loop] = data[loop] end for > I found some interesting results. I tried 4 different ways: 1:}}} <eucode> upperdata = upper(data) </eucode> {{{ 2:}}} <eucode> for loop = 1 to length(data) do upperdata[loop] = upper(data[loop]) end for 3:}}} <eucode> upperdata = c_func( c_upper, { data } )</eucode> {{{ (compiled wildcard.e into a dll) 4:}}} <eucode> object x constant TO_LOWER = 'a' = 'A' for loop = 1 to length(data) do x = data[loop] upperdata[loop] = x - (x >= 'a' and x <= 'z') * TO_LOWER end for </eucode> {{{ 5: Compiled this into a dll:
constant TO_LOWER = 'a' - 'A' global function seq_upper(sequence seq) -- convert atom or sequence to upper case sequence y, x y = repeat( 0, length(seq) ) for loop = 1 to length(seq) do x = seq[loop] y[loop] = x - (x >= 'a' and x <= 'z') * TO_LOWER end for return y end function
...and called it by...
upperdata = c_func( c_seq_upper, {data} )
I ran each method under exwc either natively, or using "exwc eu.ex" (ooeu): exwc ooeu 1 4.06 3.85 2 1.10 66.97 3 4.18 4.35 4 1.00 5.83 5 0.97 1.08 (Run on a 2.4GHz Celeron with 512MB RAM.) I really can't explain why ooeu beats exwc on #1. The results of #2 and #4 seem to make it clear that iterating through the sequence (and not forcing more complicated sequence arithmetic) was faster, which took me to #5, which definitely seems to be the best, especially for the ooeu case. And for large datasets, it should more than compensate for the delayed startup time if you don't have the registered translator. Matt Lewis
16. Re: upper()
- Posted by "Kat" <Kat12 at coosahs.net> Aug 06, 2006
- 669 views
On 6 Aug 2006, at 8:28, Derek Parnell wrote concerning: Re: upper() > > > posted by: Derek Parnell <ddparnell at bigpond.com> > > Kat wrote: > > > > On 6 Aug 2006, at 4:41, Derek Parnell wrote concerning: > > Re: upper() > > > > > ------------------------------------------------ > > > global function toUpperObj(object s) > > > ------------------------------------------------ > > > if atom(s) then > > > return uc[s+1] > > > else > > > for i = 1 to length(s) do > > > if atom(s[i]) then > > > s[i] = uc[s[i]+1] > > > else > > > s[i] = toUpperObj(s[i]) > > > end if > > > end for > > > end if > > > end function > > > > attempt to exit a function without returning a value > > > > Kat, > > still working on it...... > > Well I just knocked it up in a couple of minutes.. gee!!! Give some slack here > okay! I suppose you can see the stupid error I made. And I suppose you fixed > it > already. I only tested the toLower function so that's why I missed it. No one > is forcing you to use this faster function. I just thought I'd try to help; > sorry for interrupting you. Oh,, well, actually, i thought there was a problem with listfilter deleteing lines or something. I actually went to the euforum webpage and checked there, and waited to see if anyone replied to your email and included your email with the same errors. Honestly, that *you* made an error was last on my mind. And i thought this was code you already had! Kat
17. Re: upper()
- Posted by "Kat" <Kat12 at coosahs.net> Aug 06, 2006
- 674 views
On 6 Aug 2006, at 4:41, Derek Parnell wrote concerning: Re: upper() Thanks, Derek. I replaced RDS's upper() in the test run (on the webpage url i gave earlier) with your code. I called upperdata = toUpperObj(data) since data is a nested sequence. These are the results: 0 363 3 ---- 0 0 2 ---- 0 1 3 ---- 0 1 3 ---- 0 1 3 ---- 0 1 3 ---- 0 1 3 ---- 0 1 3 ---- 0 1 3 ---- 0 1 3 ---- press 'q' Note again the fairly large time at the first run. Any idea what is causing that? Kat
18. Re: upper()
- Posted by Hayden McKay <hmck1 at dodo.com.au> Aug 07, 2006
- 657 views
Kat wrote: > > data = repeat({rand(24)+97},1000000) > upperdata = repeat({rand(24)+97},1000000) > > puts(1,"repeats done, press a key\n") > junk = wait_key() > > --for loop = 1 to length(data) do upperdata[loop] = upper(data[loop]) end > for > for loop = 1 to length(data) do upperdata[loop] = data[loop] end for > > > The commented out line uses 48megabytes of memory, and takes 170 > seconds to run. > > The next line is the same, but without upper(), and executes instantly. > > This line takes all night to run: > upperdata = upper(data) >
-- this takes about 3-5 secs on my machine... -- I suggest you execute with ex.exw and not ex.ex from ed.ex Esc->e. include wildcard.e sequence data, upperdata data = repeat({rand(24)+97},1000000) upperdata = repeat(-1, 1000000) upperdata = upper(data) -- my P4 3.0x2-64 with 2Gb Duel Channel DDR 533 (a little bit forgiving).
but if you really need to use loops, 'routine_id()' speeds up things noticeably in huge loop irrations.
constant id_upper = routine_id("upper") for loop = 1 to length(data) do upperdata[loop] = call_func(id_upper, {data[loop]}) end for
You also might want to try...
constant id_upper = routine_id("upper") integer junk for loop = 1 to length(data) do junk = data[loop] upperdata[loop] = call_func(id_upper, {junk}) end for
> Why, and how can i avoid the penalty of upper(), and still get the upperdata? > > Yes, i need both data and upperdata. > > Incidently, OOEU runs the commented out line in 4 seconds, but takes all > night to run parse() or gets(). Using define_c_proc(open_dll("msvcrt.dll"), > > "_strupr", {C_POINTER}) doesn't help in OOEU, but does in exw.exe. It's > very frustrating. > > The actual data (not the test set above) is a million ~30-character lines in > a > 24megabyte file. I cannot run the app every 30 minutes anymore. Either > parse(), gets(), or upper() take too long. > > Kat > > how many programmers does it take to change a light bulb? none it's a hardware problem!
19. Re: upper()
- Posted by Pete Lomax <petelomax at blueyonder.co.uk> Aug 07, 2006
- 681 views
- Last edited Aug 08, 2006
On Sun, 6 Aug 2006 08:01:51 -0500, Kat <kat12 at coosahs.net> wrote: >On 6 Aug 2006, at 4:41, Derek Parnell wrote concerning: >Re: upper() > >> ------------------------------------------------ >> global function toUpperObj(object s) >> ------------------------------------------------ >> if atom(s) then >> return uc[s+1] <snip> >attempt to exit a function without returning a value btw, you'll also get crashes passing -7 or 2006. (Derek, I'm not trying to criticise, I just wanted to signal the warning and point out that it isn't really fair to time-trial that against anything else. Antonio's Obj_upper has the needed tests but, perhaps understandably, his upper() does not.) fwiw here, bltn, is my best stab:
sequence ucMap, lcMap procedure initcase() ucMap = repeat(0,#FF) -- #FF is highest valid 8-bit ascii (255) for i=1 to #FF do ucMap[i] = i end for lcMap = ucMap for i='A' to 'Z' do lcMap[i] = i+32 end for lcMap['=C4'] = '=E4' lcMap['=D6'] = '=F6' lcMap['=DC'] = '=FC' for i='a' to 'z' do ucMap[i] = i-32 end for ucMap['=E4'] = '=C4' ucMap['=F6'] = '=D6' ucMap['=FC'] = '=DC' end procedure initcase() object c global function Upper(object x) if integer(x) then if x >= 1 and x<=#FF then return ucMap[x] end if elsif sequence(x) then for i = 1 to length(x) do c = x[i] if integer(c) then if c >= 1 and c<=#FF then x[i] = ucMap[c] end if elsif sequence(c) then x[i] = Upper(c) end if end for=20=20=20=20=20=20=20=20 end if return x end function global function Lower(object x) if integer(x) then if x >= 1 and x<=#FF then return lcMap[x] end if elsif sequence(x) then for i = 1 to length(x) do c = x[i] if integer(c) then if c >= 1 and c<=#FF then x[i] = lcMap[c] end if elsif sequence(c) then x[i] = Lower(c) end if end for=20=20=20=20=20=20=20=20 end if return x end function
Regards, Pete
20. Re: upper()
- Posted by Julio C. Galaret Viera <galaret at adinet.com.uy> Aug 08, 2006
- 673 views
Kat wrote: > > On 6 Aug 2006, at 1:35, Julio C. Galaret Viera wrote concerning: > Re: upper() > > Using? I meant box, cpu, etc. The problem with upper() seems to be related, at least in part to the number of sequences inside data/upperdata. An alternative may be you do not remove '\n' or '\r' when reading from file so you have only one long sequence of ~32.000.000 chars. If I use upper() over this sequence, I quickly runs out of memory since I have only 256 MB RAM. Even with enough memory it is too slow. But if loop atom by atom I convert data to upper case in ~180 secs. That seems to be the fastest way with upper(). Then you may write this sequence as a text file again. JG
21. Re: upper()
- Posted by "Kat" <kat12 at coosahs.net> Aug 08, 2006
- 668 views
> > > posted by: Julio C. Galaret Viera <galaret at adinet.com.uy> > > Kat wrote: > > > > On 6 Aug 2006, at 1:35, Julio C. Galaret Viera wrote concerning: Re: > > upper() > > > > Using? > > I meant box, cpu, etc. AMD Sempron 3000+ winxp sp2 1gigram > The problem with upper() seems to be related, at least in part to the > number of sequences inside data/upperdata. > > An alternative may be you do not remove '\n' or '\r' when reading from > file so you have only one long sequence of ~32.000.000 chars. I parse(data,{10,13}) . Kat > If I use upper() over this sequence, I quickly runs out of memory > since I have only 256 MB RAM. Even with enough memory it is too slow. > > But if loop atom by atom I convert data to upper case in ~180 secs. > That seems to be the fastest way with upper(). Then you may write this > sequence as a text file again. > > > JG > > > >
22. Re: upper()
- Posted by Julio C. Galaret Viera <galaret at adinet.com.uy> Aug 08, 2006
- 672 views
Kat wrote: > > > > > posted by: Julio C. Galaret Viera <galaret at adinet.com.uy> > > > > Kat wrote: > > > > > > On 6 Aug 2006, at 1:35, Julio C. Galaret Viera wrote concerning: Re: > > > upper() > > > > > > Using? > > > > I meant box, cpu, etc. > > AMD Sempron 3000+ winxp sp2 1gigram > > > The problem with upper() seems to be related, at least in part to the > > number of sequences inside data/upperdata. > > > > An alternative may be you do not remove '\n' or '\r' when reading from > > file so you have only one long sequence of ~32.000.000 chars. > > I parse(data,{10,13}) . > > Kat Perhaps due to the fact that English is not my native language, I am being misunderstood. Sorry for my English. What I mean is I advise you not to parse data initially so that you will manage only one long sentence on which you can run upper() but instead of running it on the sequence as a whole, run it recursively on each element of the sequence. After that, if you need it, you may parse the sequence upperdata obtained, getting rid of #10 and #13. JG > > > If I use upper() over this sequence, I quickly runs out of memory > > since I have only 256 MB RAM. Even with enough memory it is too slow. > > > > But if loop atom by atom I convert data to upper case in ~180 secs. > > That seems to be the fastest way with upper(). Then you may write this > > sequence as a text file again. > > > > > > JG > > > >
23. Re: upper()
- Posted by Hayden McKay <hmck1 at dodo.com.au> Aug 10, 2006
- 686 views
Many people have demonstrated some nice ways but here's mine anyway... theese funcs will only work on flat strings but they could easily be modified to take an object and/or recurse a nested sequence. nb. just a basic approach that will handle all characters. there's no arithmatic or condition testing.
include wildcard.e as euinc sequence asciiLUT asciiLUT = repeat(-1, 256) for j = 0 to 255 by 1 do asciiLUT[(j+1)] = j end for global function upper(sequence st) integer ch asciiLUT = euinc:upper(asciiLUT) for ps = 1 to length(st) by 1 do ch = st[ps] st[ps] = asciiLUT[(ch+1)] end for return st end function global function lower(sequence st) integer ch asciiLUT = euinc:lower(asciiLUT) for ps = 1 to length(st) by 1 do ch = st[ps] st[ps] = asciiLUT[(ch+1)] end for return st end function
24. Re: upper()
- Posted by Pete Lomax <petelomax at blueyonder.co.uk> Aug 11, 2006
- 685 views
On Thu, 10 Aug 2006 10:34:09 -0700, Hayden McKay <guest at RapidEuphoria.com> wrote: >}}} <eucode> >global function upper(sequence st) > asciiLUT = euinc:upper(asciiLUT) > ... >end function > >global function lower(sequence st) > asciiLUT = euinc:lower(asciiLUT) > ... >end function ></eucode> {{{ Ouch! You clearly don't need to call euinc:lower/upper every single time these replacements are called. One other thing I feel a need to say here is that it seems apparent that this approach will match the usual upper/lower 100%. It will not. As a (daft) example, euinc:lower(75.123) will give 107.123 whereas the replacement will typecheck on both 75.123 and {75.123}; and, if "integer ch" is replaced with "atom ch", the latter will give {107}. No biggie, just saying don't claim/think 100% compatible when not. I am of course slighting wildcard.e on that last point, not your code. Barring those niggles, I think that is a clever approach. Regards, Pete
25. Re: upper()
- Posted by "Kat" <kat12 at coosahs.net> Aug 11, 2006
- 684 views
> > > On Thu, 10 Aug 2006 10:34:09 -0700, Hayden McKay > <guest at RapidEuphoria.com> wrote: > > >}}} <eucode> > >global function upper(sequence st) > > asciiLUT = euinc:upper(asciiLUT) > > ... > >end function > > > >global function lower(sequence st) > > asciiLUT = euinc:lower(asciiLUT) > > ... > >end function > ></eucode> {{{ > Ouch! You clearly don't need to call euinc:lower/upper every single > time these replacements are called. > > One other thing I feel a need to say here is that it seems apparent > that this approach will match the usual upper/lower 100%. It will not. > As a (daft) example, euinc:lower(75.123) will give 107.123 whereas the > replacement will typecheck on both 75.123 and {75.123}; and, if > "integer ch" is replaced with "atom ch", the latter will give {107}. > > No biggie, just saying don't claim/think 100% compatible when not. I > am of course slighting wildcard.e on that last point, not your code. > > Barring those niggles, I think that is a clever approach. I have been using the code Derek supplied, and run times have been ~6 sec to ~140 seconds, depending on how busy the machine is. This has all become academic. The neighbor's new dogs have been challenging me for right to access my property and mailbox, and the neighbor is now threatening to sue me for harrassment even tho i have constantly tried to avoid him. I'll be living in my car, in other states, after court on August 15th 2006. This is the only way i know to get further away from him and his dogs. Kat
26. Re: upper()
- Posted by Jeremy Peterson <ptl99 at hotmail.com> Aug 13, 2006
- 689 views
Kat wrote: > > > > > On Thu, 10 Aug 2006 10:34:09 -0700, Hayden McKay > > <guest at RapidEuphoria.com> wrote: > > > > >}}} <eucode> > > >global function upper(sequence st) > > > asciiLUT = euinc:upper(asciiLUT) > > > ... > > >end function > > > > > >global function lower(sequence st) > > > asciiLUT = euinc:lower(asciiLUT) > > > ... > > >end function > > ></eucode> {{{ > > Ouch! You clearly don't need to call euinc:lower/upper every single > > time these replacements are called. > > > > One other thing I feel a need to say here is that it seems apparent > > that this approach will match the usual upper/lower 100%. It will not. > > As a (daft) example, euinc:lower(75.123) will give 107.123 whereas the > > replacement will typecheck on both 75.123 and {75.123}; and, if > > "integer ch" is replaced with "atom ch", the latter will give {107}. > > > > No biggie, just saying don't claim/think 100% compatible when not. I > > am of course slighting wildcard.e on that last point, not your code. > > > > Barring those niggles, I think that is a clever approach. > > I have been using the code Derek supplied, and run times have > been ~6 sec to ~140 seconds, depending on how busy the > machine is. > > This has all become academic. The neighbor's new dogs have been > challenging me for right to access my property and mailbox, and > the neighbor is now threatening to sue me for harrassment even tho > i have constantly tried to avoid him. I'll be living in my car, in other > states, after court on August 15th 2006. This is the only way i > know to get further away from him and his dogs. > > Kat > More trouble with the neighbor and his dogs? After reading your previous posts about this problem I have a suggestion: First, purchase a silenced pistol - a Beretta M92 would do nicely, but a HK23 .45 calibur would be better. You said in one of the other posts that you already owned a gun or two, so you could just take it to a gunsmith and have a silencer installed. But if there aren't that many people around you might not even need a silencer. Also buy some kind of electrical stun gun that works on dogs as well as humans. Next, wait till the neighbour leaves his house, go over and zap the dog(s) unconcious, drag them over to your house and shoot it(them), then bury them somewhere in your woods. Neighbour comes home, dog(s) is(are) nowhere to be found and he doesn't have a clue what happened to them. The other way is to give it some poisoned meat when he attacks you, then repeat burying him in the woods. If he continues getting more dogs I suggest using C4 on his house:), or simply hiring a detective to dig up some dirt on him and then blackmail him to move away. I might also suggest moving to another state, Alabama doesn't sound like a very good one. Jeremy
27. Re: upper()
- Posted by D. Newhall <derek_newhall at yahoo.com> Aug 13, 2006
- 661 views
Jeremy Peterson wrote: > (snip) > More trouble with the neighbor and his dogs? After reading your previous > posts > about this problem I have a suggestion: > > First, purchase a silenced pistol - a Beretta M92 would do nicely, but a HK23 > .45 calibur would be better. > You said in one of the other posts that you already owned a gun or two, so you > could just take it to a gunsmith and have a silencer installed. > But if there aren't that many people around you might not even need a > silencer. > > Also buy some kind of electrical stun gun that works on dogs as well as > humans. > Next, wait till the neighbour leaves his house, go over and zap the dog(s) > unconcious, > drag them over to your house and shoot it(them), then bury them somewhere in > your woods. > Neighbour comes home, dog(s) is(are) nowhere to be found and he doesn't have > a clue what happened to them. > > The other way is to give it some poisoned meat when he attacks you, then > repeat > burying him in the woods. > If he continues getting more dogs I suggest using C4 on his house:), or simply > hiring a detective to dig up some dirt on him and then blackmail him to move > away. > > I might also suggest moving to another state, Alabama doesn't sound like a > very > good one. > > Jeremy Err... While this may have been a joke I think that advocating illegal behavior on a public mailing list is typically bad taste.
28. Re: upper()
- Posted by Jeremy Peterson <ptl99 at hotmail.com> Aug 13, 2006
- 679 views
- Last edited Aug 14, 2006
> may have been a joke" Well um yes... >I think that advocating illegal behavior >on a public mailing list is typically bad taste. Well I don't. It should be obvious that this is a intended to be a joke, and most of you guys could do with a little humour now and then. But jokes on this forum never seem to be appreciated, except by me and 1 or 2 others... Take a look at this example from http://www.listfilter.com/cgi-bin/esearch.exu?thread=1&fromMonth=6&fromYear=A&toMonth=8&toYear=A&keywords=%223.0+feature+request:+foreach%22 As you can see, nobody laughed at it. >Date: 2005 Jul 13 7:41 >From: Dave Probert <zingo at purpletiger.com> >Subject: Re: 3.0 feature request: foreach Vincent wrote: >> >> D. Newhall wrote: > > > > > >I think that the next release should have something new and useful to the > > >languag e. --<snip> > > > set to the next next element in "line". A consideration would be to > > > optimize expressions > > >of the type "foreach obj in reverse(seq)do" > > > > > > >Hi there... I don't think there needs to be any more loop methods. "while" > >and "for" --<snip> > > > >Using UPX for EX.EXE and hardware floating point compiler flags, background > >transpa rent > >icons, and a nice multitasking library & documentation dirived from Language > >War's > >Sched.e would make the nice addition for Euphoria 3.0. > > >I'm looking forward to 3.0, I can't wait to try out the "Language War" > >technique, seeing > >how much smaller EX.EXE is, and observe those new Windows icons on my > >colorful desktop > >backgrounds. > > > > > >Regards, > >Vincent >Sorry, gotta go with Vincent on this one - we really do need to have more >>colourful >icons for our desktop. >Our programs will never be as good as ones written in the more complicated >>languages >without the colourful icons. In fact I think Euphoria should drop many of >the > >current language features like - print, sprint, while, if, message_box, etc >in > >favour of having some extra functionality to provide better icon support. >Who needs other stuff anyway? >:) >Dave >PS. I actually vote FOR foreach - use it all the time in PHP and it rally >>does save >a huge amount of confusion - though mainly with associative arrays! Hahahahahahahahahah! :) Great joke Dave! Jeremy
29. Re: upper()
- Posted by Dan Moyer <danielmoyer at prodigy.net> Aug 14, 2006
- 680 views
Jeremy Peterson wrote: > > > may have been a joke" > Well um yes... > > >I think that advocating illegal behavior > >on a public mailing list is typically bad taste. > Well I don't. It should be obvious that this is a intended to be a joke, and > most of you guys could do with a little humour now and then. > > But jokes on this forum never seem to be appreciated, except by me and 1 or > 2 others... > > Take a look at this example from <<SNIP>> Jeremy, If you've really read Kat's description of her situation, then you should be able to understand that it's NO JOKE, the stupid dogs could KILL her. Stupid jokes are NO HELP at all. Dan Moyer
30. Re: upper()
- Posted by Jeremy Peterson <ptl99 at hotmail.com> Aug 14, 2006
- 673 views
And like I said, she should move away from that town and maybe even that state. That was a serious suggestion. And I can bet someone is going to say "Re: upper()" shove off, Jeremy" Jeremy
31. Re: upper()
- Posted by Hayden McKay <hmck1 at dodo.com.au> Aug 15, 2006
- 671 views
Jeremy Peterson wrote: > > Kat wrote: > > > > > > > > On Thu, 10 Aug 2006 10:34:09 -0700, Hayden McKay > > > <guest at RapidEuphoria.com> wrote: > > > > > > >}}} <eucode> > > > >global function upper(sequence st) > > > > asciiLUT = euinc:upper(asciiLUT) > > > > ... > > > >end function > > > > > > > >global function lower(sequence st) > > > > asciiLUT = euinc:lower(asciiLUT) > > > > ... > > > >end function > > > ></eucode> {{{ > > > Ouch! You clearly don't need to call euinc:lower/upper every single > > > time these replacements are called. > > > > > > One other thing I feel a need to say here is that it seems apparent > > > that this approach will match the usual upper/lower 100%. It will not. > > > As a (daft) example, euinc:lower(75.123) will give 107.123 whereas the > > > replacement will typecheck on both 75.123 and {75.123}; and, if > > > "integer ch" is replaced with "atom ch", the latter will give {107}. > > > > > > No biggie, just saying don't claim/think 100% compatible when not. I > > > am of course slighting wildcard.e on that last point, not your code. > > > > > > Barring those niggles, I think that is a clever approach. > > > > I have been using the code Derek supplied, and run times have > > been ~6 sec to ~140 seconds, depending on how busy the > > machine is. > > > > This has all become academic. The neighbor's new dogs have been > > challenging me for right to access my property and mailbox, and > > the neighbor is now threatening to sue me for harrassment even tho > > i have constantly tried to avoid him. I'll be living in my car, in other > > states, after court on August 15th 2006. This is the only way i > > know to get further away from him and his dogs. > > > > Kat > > > More trouble with the neighbor and his dogs? After reading your previous > posts > about this problem I have a suggestion: > > First, purchase a silenced pistol - a Beretta M92 would do nicely, but a HK23 > .45 calibur would be better. > You said in one of the other posts that you already owned a gun or two, so you > could just take it to a gunsmith and have a silencer installed. > But if there aren't that many people around you might not even need a > silencer. > > Also buy some kind of electrical stun gun that works on dogs as well as > humans. > Next, wait till the neighbour leaves his house, go over and zap the dog(s) > unconcious, > drag them over to your house and shoot it(them), then bury them somewhere in > your woods. > Neighbour comes home, dog(s) is(are) nowhere to be found and he doesn't have > a clue what happened to them. > > The other way is to give it some poisoned meat when he attacks you, then > repeat > burying him in the woods. > If he continues getting more dogs I suggest using C4 on his house:), or simply > hiring a detective to dig up some dirt on him and then blackmail him to move > away. > > I might also suggest moving to another state, Alabama doesn't sound like a > very > good one. > > Jeremy I had a similar problem. All I do is ring the local council and tell them that there is a 'big bitbull' wandering our street and that you think it belongs to next door. The council dog catches will be there in about 10 mins see the dog (even though it's probably not a bitbull) and take the dog away. nb. The key is to mention 'bitbull', 'doverman' etc... don't tell them that it's realy only a 'chowowa' jackrussle' etc...
32. Re: upper()
- Posted by don cole <doncole at pacbell.net> Aug 15, 2006
- 662 views
Jeremy Peterson wrote: > > And like I said, she should move away from that town and maybe even that > state. > That was a serious suggestion. > > And I can bet someone is going to say > "Re: upper()" shove off, Jeremy" > > Jeremy Maybe Alabama needs Moore Governor. Don Cole