1. RE: Defining long constants - do-able but not exactly elegant
What's wrong with :
constant testset=
-- "idfursf ujbdoc ujy negnxdnown idfu fun cxjkgdoc " &
-- "joy rggxnkkdbn ajo in wjoorf jggxnwdjfn fun "
-- "b hcn fba ioa caxw pbxlsbw ramc mpj scchk " &
-- "blmji mpbm pj rk ioaaran com cl mpj scchk"
-- "zaxabaprsanl cdrlt znlzdlsersanl nq zbntd " &
-- "fsabaprsanl nq skd brlu twedru nq habu baqd rhrv"
"ktj uz wjiwnlufza rnujw fv nyywzqfbnujcp zaj iztwug " &
"kjjyjw ugna fu nyyjnwv uz mj rgja czzdfah fauz fu"
--------
Derek
-----Original Message-----
From: petelomax at blueyonder.co.uk [mailto:petelomax at blueyonder.co.uk]
Sent: Thursday, 14 March 2002 9:38
To: EUforum
Subject: Defining long constants - do-able but not exactly elegant
After a few test runs with my fledgling cipher program, my source
contains something like this:
--constant ts1="idfursf ujbdoc ujy negnxdnown idfu fun cxjkgdoc ",
-- ts2="joy rggxnkkdbn ajo in wjoorf jggxnwdjfn fun ",
-- ts3="jzzjhdmdfl rz fun uronkf joy cnonxrsk ",
-- testset=ts1&ts2&ts3
--constant ts1="b hcn fba ioa caxw pbxlsbw ramc mpj scchk ",
-- ts2="blmji mpbm pj rk ioaaran com cl mpj scchk",
-- testset=ts1&ts2
--constant ts1="zaxabaprsanl cdrlt znlzdlsersanl nq zbntd ",
-- ts2="fsabaprsanl nq skd brlu twedru nq habu baqd rhrv",
-- testset=ts1&ts2
constant ts1="ktj uz wjiwnlufza rnujw fv nyywzqfbnujcp zaj iztwug ",
ts2="kjjyjw ugna fu nyyjnwv uz mj rgja czzdfah fauz fu",
testset=ts1&ts2
Obviously, I coded it that way because of line length problems, then
just carried on. I'd rather have one big constant & a loop.
Unfortunately Euphoria doesn't support \<eol>
I have a work-round (below), but before you read it, ask yourself how
you would do it, elegantly.
A gut of mine tells me that is not the best way to do it.
It's not very readable, really, & easy to make small slips, especially
when the set gets fairly large.
By the time you read this, I'll have coded:
testset={
t11&t12&t13,
t21&t22,
t31&t32,
t41&t42}
but I don't like all those extra names cluttering up my code. Was
there a t33, or not, t84? etc
Pete
PS I chopped a few of those ciphers to tidy up the post a bit.
Don't use em, if you want the originals I'll mail them direct.
2. RE: Defining long constants - do-able but not exactly elegant
What's wrong with simply gets()ing a text file of any length you want, edited
any way you want, and displayed any way you want, in any editor you
want, rather than cluttering up the code files?
Kat
On 14 Mar 2002, at 9:57, Derek Parnell wrote:
>
> What's wrong with :
>
> constant testset=
> -- "idfursf ujbdoc ujy negnxdnown idfu fun cxjkgdoc " &
> -- "joy rggxnkkdbn ajo in wjoorf jggxnwdjfn fun "
>
> -- "b hcn fba ioa caxw pbxlsbw ramc mpj scchk " &
> -- "blmji mpbm pj rk ioaaran com cl mpj scchk"
>
> -- "zaxabaprsanl cdrlt znlzdlsersanl nq zbntd " &
> -- "fsabaprsanl nq skd brlu twedru nq habu baqd rhrv"
>
> "ktj uz wjiwnlufza rnujw fv nyywzqfbnujcp zaj iztwug " &
> "kjjyjw ugna fu nyyjnwv uz mj rgja czzdfah fauz fu"
>
> --------
> Derek
>
> -----Original Message-----
> From: petelomax at blueyonder.co.uk [mailto:petelomax at blueyonder.co.uk]
> Sent: Thursday, 14 March 2002 9:38
> To: EUforum
> Subject: Defining long constants - do-able but not exactly elegant
>
>
> After a few test runs with my fledgling cipher program, my source
> contains something like this:
>
> --constant ts1="idfursf ujbdoc ujy negnxdnown idfu fun cxjkgdoc ",
> -- ts2="joy rggxnkkdbn ajo in wjoorf jggxnwdjfn fun ",
> -- ts3="jzzjhdmdfl rz fun uronkf joy cnonxrsk ",
> -- testset=ts1&ts2&ts3
> --constant ts1="b hcn fba ioa caxw pbxlsbw ramc mpj scchk ",
> -- ts2="blmji mpbm pj rk ioaaran com cl mpj scchk",
> -- testset=ts1&ts2
> --constant ts1="zaxabaprsanl cdrlt znlzdlsersanl nq zbntd ",
> -- ts2="fsabaprsanl nq skd brlu twedru nq habu baqd rhrv",
> -- testset=ts1&ts2
> constant ts1="ktj uz wjiwnlufza rnujw fv nyywzqfbnujcp zaj iztwug ",
> ts2="kjjyjw ugna fu nyyjnwv uz mj rgja czzdfah fauz fu",
> testset=ts1&ts2
>
> Obviously, I coded it that way because of line length problems, then
> just carried on. I'd rather have one big constant & a loop.
>
> Unfortunately Euphoria doesn't support \<eol>
>
> I have a work-round (below), but before you read it, ask yourself how
> you would do it, elegantly.
>
> A gut of mine tells me that is not the best way to do it.
> It's not very readable, really, & easy to make small slips, especially
> when the set gets fairly large.
>
>
> By the time you read this, I'll have coded:
> testset={
> t11&t12&t13,
> t21&t22,
> t31&t32,
> t41&t42}
> but I don't like all those extra names cluttering up my code. Was
> there a t33, or not, t84? etc
>
> Pete
> PS I chopped a few of those ciphers to tidy up the post a bit.
> Don't use em, if you want the originals I'll mail them direct.
>
>
>
>
3. RE: Defining long constants - do-able but not exactly elegant
Kat wrote:
>
> What's wrong with simply gets()ing a text file of any length you want,
> edited
> any way you want, and displayed any way you want, in any editor you
> want, rather than cluttering up the code files?
>
Yeah, that's what I was gonna say. The rules for #3 don't preclude you
from loading in any file you want (except machine code, or something
like that). The program I'm working on loads in a "pre-hashed"
dictionary, along with some other stuff, instead of wasting time on that
everytime it starts....
4. RE: Defining long constants - do-able but not exactly elegant
On 14 Mar 2002, at 0:55, Andy Serpa wrote:
>
>
> Kat wrote:
> >
> > What's wrong with simply gets()ing a text file of any length you want,
> > edited
> > any way you want, and displayed any way you want, in any editor you
> > want, rather than cluttering up the code files?
> >
>
> Yeah, that's what I was gonna say. The rules for #3 don't preclude you
> from loading in any file you want (except machine code, or something
> like that). The program I'm working on loads in a "pre-hashed"
> dictionary, along with some other stuff, instead of wasting time on that
> everytime it starts....
I found loading the pre-munged data took way longer than grabbing the plain
dictionary. Near as i can tell, it's the stuffing into 3-deep nested sequences
that is eating the time. Maybe i should try making them a set length in each
nest with the repeat() from the start. But munging dictionaries took forever
too. Anyhoo, like i said, i am out of the contest, i can write the code, but it
won't execute in time. It's so wierd, because the grammar parser runs faster
than loading the dictionary for this contest.
What bugs me a little about real-world apps using these cypto-solvers is that
anyone with a reason to use them would have more than one paragraph to
solve, and loading and dictionary munging time would be inconsequential,
because they'd be turned on and left running, looking for input. (I leave one
mining program running sometimes, it goes active when the other programs
feed it an url.) But then, these guys have load time down to sub-second
times, so,, errr, nevermind.
Kat
5. RE: Defining long constants - do-able but not exactly elegant
- Posted by bensler at mail.com
Mar 14, 2002
What are you using to read in the file data?
get(), gets() or getc()?
Can you give an example of what is taking so long?
I used to play with Eu on my 486DX 100mhz, and I never noticed any
significant file loading times. Granted, I never tried opening any 50Mb
files.
Chris
Kat wrote:
> On 14 Mar 2002, at 0:55, Andy Serpa wrote:
>
> >
> > Kat wrote:
> > >
> > > What's wrong with simply gets()ing a text file of any length you want,
> > > edited
> > > any way you want, and displayed any way you want, in any editor you
> > > want, rather than cluttering up the code files?
> > >
> >
> > Yeah, that's what I was gonna say. The rules for #3 don't preclude you
> > from loading in any file you want (except machine code, or something
> > like that). The program I'm working on loads in a "pre-hashed"
> > dictionary, along with some other stuff, instead of wasting time on that
> >
> > everytime it starts....
>
> I found loading the pre-munged data took way longer than grabbing the
> plain
> dictionary. Near as i can tell, it's the stuffing into 3-deep nested
> sequences
> that is eating the time. Maybe i should try making them a set length in
> each
> nest with the repeat() from the start. But munging dictionaries took
> forever
> too. Anyhoo, like i said, i am out of the contest, i can write the code,
> but it
> won't execute in time. It's so wierd, because the grammar parser runs
> faster
> than loading the dictionary for this contest.
>
> What bugs me a little about real-world apps using these cypto-solvers is
> that
> anyone with a reason to use them would have more than one paragraph to
> solve, and loading and dictionary munging time would be inconsequential,
>
> because they'd be turned on and left running, looking for input. (I
> leave one
> mining program running sometimes, it goes active when the other programs
>
> feed it an url.) But then, these guys have load time down to sub-second
> times, so,, errr, nevermind.
>
> Kat
>
>
6. RE: Defining long constants - do-able but not exactly elegant
On 14 Mar 2002, at 8:07, bensler at mail.com wrote:
>
> What are you using to read in the file data?
> get(), gets() or getc()?
I tried all 3.
> Can you give an example of what is taking so long?
dictionary = repeat({},200) -- we are going to organize the words by size
time1 = time()
for ocindex = 1 to length(ocypher) do
if length(ocypher[ocindex]) then
junk_i = length(ocypher[ocindex])
if not equal(ocypher[ocindex],{}) and equal(dictionary[junk_i],{}) then
dfilename = "D:\\Gertie\\decypher\\dfile"&sprintf("%d",junk_i)&".txt"
puts(1,sprintf("%d",ocindex)&" "&dfilename&"\n")
readfile = open(dfilename,"r")
if not equal(readfile,-1) then
readline = gets(readfile)
while not equal(readline,-1) do
dictionary[junk_i] =
append(dictionary[junk_i],readline[1..length(readline)-1])
readline = gets(readfile)
end while
close(readfile)
end if
end if
end if
end for
time2 = time()
puts(1,"dfile load time: "&sprintf("%d",time2-time1)&" seconds\n")
> I used to play with Eu on my 486DX 100mhz, and I never noticed any
> significant file loading times. Granted, I never tried opening any 50Mb
> files.
In this case, i premunged the dictionary to separate word-sized files and
used gets() for each word. I had made one file using printf(), and one get(),
which took longer and made a *much* bigger file. The getc() version ran the
slowest.
Kat
>
>
> Chris
>
>
> Kat wrote:
> > On 14 Mar 2002, at 0:55, Andy Serpa wrote:
> >
> > >
> > > Kat wrote:
> > > >
> > > > What's wrong with simply gets()ing a text file of any length you want,
> > > > edited any way you want, and displayed any way you want, in any editor
> > > > you
> > > > want, rather than cluttering up the code files?
> > > >
> > >
> > > Yeah, that's what I was gonna say. The rules for #3 don't preclude you
> > > from
> > > loading in any file you want (except machine code, or something like
> > > that).
> > > The program I'm working on loads in a "pre-hashed" dictionary, along with
> > > some other stuff, instead of wasting time on that
> > >
> > > everytime it starts....
> >
> > I found loading the pre-munged data took way longer than grabbing the
> > plain
> > dictionary. Near as i can tell, it's the stuffing into 3-deep nested
> > sequences
> > that is eating the time. Maybe i should try making them a set length in
> > each
> > nest with the repeat() from the start. But munging dictionaries took
> > forever
> > too. Anyhoo, like i said, i am out of the contest, i can write the code, but
> > it won't execute in time. It's so wierd, because the grammar parser runs
> > faster than loading the dictionary for this contest.
> >
> > What bugs me a little about real-world apps using these cypto-solvers is
> > that
> > anyone with a reason to use them would have more than one paragraph to
> > solve,
> > and loading and dictionary munging time would be inconsequential,
> >
> > because they'd be turned on and left running, looking for input. (I
> > leave one
> > mining program running sometimes, it goes active when the other programs
> >
> > feed it an url.) But then, these guys have load time down to sub-second
> > times, so,, errr, nevermind.
> >
> > Kat
> >
> >
>
>
>
7. RE: Defining long constants - do-able but not exactly elegant
> -----Original Message-----
> From: Kat [mailto:gertie at PELL.NET]
> In this case, i premunged the dictionary to separate
> word-sized files and
> used gets() for each word. I had made one file using
> printf(), and one get(),
> which took longer and made a *much* bigger file. The getc()
> version ran the
> slowest.
>
> Kat
I'm using a pre-munged dictionary file, and taking about 1.5 seconds to load
on a P-233, 64MB (~3200 sieves/sec). I put it into a 3-dimensioned sequence
( words[a][b][c] ). I know the range of 'b' based on 'a' (i.e., word
length--part of the pre-munging was based on length), and read a 4-byte
integer telling me the range of 'c' for the given [a][b] combination right
before those words. I use get_bytes( fn, a ) to read a word. I just run a
doubly nested loop, stopping when I get to the end of the file.
When I increment 'a':
words = append( words, repeat( {}, a ) )
Then, I read in the 4 byte integer (error checking omited):
range_c = bytes_to_int( get_bytes( fn, 4 ) )
words[a][b] = repeat( {}, range_c )
And then I read in range_c words:
for i = 1 to c do
words[a][b][c] = get_bytes( fn, a )
end for
Since I don't have line breaks, it reduced the size of words.txt (from
Junko's spell checker) by about 100K. I seem to recall someone (Derek?)
having come up with an optimization for get_bytes(), but most of the time
seems to be spent doing sequence manipulation.
I think it took about 42 seconds to do the munging (not including write
time). I'm sure I could improve that by pre-allocating sequence elements,
but this way was easier, and not really time critical. Using getc() was
definitely faster than gets(), which took about 180 seconds to do. The
writing process was just a nested loop running through all the subsequences
of words:
for i = 1 to length( words ) do
for j = 1 to i do
puts( fn, int_to_bytes( length( words[i][j] ) ) )
for k = 1 to length( words[i][j] ) do
puts( fn, words[i][j][k] )
end for
end for
end for
Matt Lewis
