1. Sequence slicing
I've made a program, and Ive noticed the slowest part of it being the
slicing. Could you tell me if there is a better(faster) way to slice. All
the values are unsigned chars so I could go the way of peeking and poking,
but dont really want to.
ex.
a=very big sequence of unsigned chars
a=a[1..length(a)-1500]--I'm called very very very often
Thanks in advance for the help
Daniel Kluss
2. Re: Sequence slicing
Daniel Kluss wrote:
> I've made a program, and Ive noticed the slowest part of it being the
> slicing.
Did you use:
with profile_time
?
> Could you tell me if there is a better(faster) way to slice. All
> the values are unsigned chars so I could go the way of peeking and poking,
> but dont really want to.
> ex.
> a=very big sequence of unsigned chars
>
> a=a[1..length(a)-1500]--I'm called very very very often
If a is very large, and there are no other references to
the same sequence, this statement will be extremely fast.
Euphoria will simply update the length of a,
without copying any elements or allocating/deallocating any space.
I think you need to give us some more context to this problem.
What are you trying to do? Why do you have to do it so often?
What is the typical length of a?
Regards,
Rob Craig
Rapid Deployment Software
http://www.RapidEuphoria.com
3. Re: Sequence slicing
I did profile and it takes up like 70+ percent most the time
"a" is the send buffer in a tcp stack
it goes like this, i send a bunch, then the other person says how much they
recved.
and so
"a" is always getting stuff added to the end of it and stuff taken from the
begining of it.
like
as "a" gets bigger like 1million in length, its deathly slow
while 1 do
a&=data
send(a[1..1500 to 5000 or more])
a=a[1..length(a)-acknowleged()]
end while
----- Original Message -----
From: "Robert Craig" <rds at RapidEuphoria.com>
To: <EUforum at topica.com>
Sent: Tuesday, October 21, 2003 3:59 PM
Subject: Re: Sequence slicing
>
>
> Daniel Kluss wrote:
> > I've made a program, and Ive noticed the slowest part of it being the
> > slicing.
>
> Did you use:
> with profile_time
> ?
>
> > Could you tell me if there is a better(faster) way to slice. All
> > the values are unsigned chars so I could go the way of peeking and
poking,
> > but dont really want to.
> > ex.
> > a=very big sequence of unsigned chars
> >
> > a=a[1..length(a)-1500]--I'm called very very very often
>
> If a is very large, and there are no other references to
> the same sequence, this statement will be extremely fast.
> Euphoria will simply update the length of a,
> without copying any elements or allocating/deallocating any space.
>
> I think you need to give us some more context to this problem.
> What are you trying to do? Why do you have to do it so often?
> What is the typical length of a?
>
> Regards,
> Rob Craig
> Rapid Deployment Software
> http://www.RapidEuphoria.com
>
>
>
> TOPICA - Start your own email discussion group. FREE!
>
>
4. Re: Sequence slicing
> while 1 do
> a&=data
> send(a[1..1500 to 5000 or more])
> a=a[1..length(a)-acknowleged()]
> end while
are you sure that calling acknowledged() isn't the bottle neck of your code?
try asigning it to a variable first, then profile it:
atom ack
while 1 do
a &= data
send( a[1..some_length] )
ack = acknowledged() -- does this take forever,
a = a[1..length(a)-ack] -- or this?
end while
5. Re: Sequence slicing
Heres a snipet, thats all one line by the way, and it takes forever
sockets[sock][SOCKET_DATA][tDEST][tDATA]=sockets[sock][SOCKET_DATA][tDEST][t
DATA][diff+1..length(sockets[sock][SOCKET_DATA][tDEST][tDATA])]
----- Original Message -----
From: "Greg Haberek" <g.haberek at comcast.net>
To: <EUforum at topica.com>
Sent: Tuesday, October 21, 2003 7:57 PM
Subject: Re: Sequence slicing
>
>
> > while 1 do
> > a&=data
> > send(a[1..1500 to 5000 or more])
> > a=a[1..length(a)-acknowleged()]
> > end while
>
> are you sure that calling acknowledged() isn't the bottle neck of your
code?
> try asigning it to a variable first, then profile it:
>
> atom ack
>
> while 1 do
> a &= data
> send( a[1..some_length] )
> ack = acknowledged() -- does this take forever,
> a = a[1..length(a)-ack] -- or this?
> end while
>
>
>
> TOPICA - Start your own email discussion group. FREE!
>
>
6. Re: Sequence slicing
Is it possible that a form of recorsion happens here i.e that the length()
operation is effected by the result of this line's execution ?
I'm only making a tentative suggestion for why it could take unusually long
(since really the length() function should be done as the situation before
the line was executed)
Peter Blonner
----- Original Message -----
From: "Daniel Kluss" <codepilot at netzero.net>
To: <EUforum at topica.com>
Sent: Wednesday, October 22, 2003 2:03 PM
Subject: Re: Sequence slicing
>
>
> Heres a snipet, thats all one line by the way, and it takes forever
>
>
sockets[sock][SOCKET_DATA][tDEST][tDATA]=sockets[sock][SOCKET_DATA][tDEST][t
> DATA][diff+1..length(sockets[sock][SOCKET_DATA][tDEST][tDATA])]
>
>
> ----- Original Message -----
> From: "Greg Haberek" <g.haberek at comcast.net>
> To: <EUforum at topica.com>
> Sent: Tuesday, October 21, 2003 7:57 PM
> Subject: Re: Sequence slicing
>
>
> > > while 1 do
> > > a&=data
> > > send(a[1..1500 to 5000 or more])
> > > a=a[1..length(a)-acknowleged()]
> > > end while
> >
> > are you sure that calling acknowledged() isn't the bottle neck of your
> code?
> > try asigning it to a variable first, then profile it:
> >
> > atom ack
> >
> > while 1 do
> > a &= data
> > send( a[1..some_length] )
> > ack = acknowledged() -- does this take forever,
> > a = a[1..length(a)-ack] -- or this?
> > end while
> >
> >
> > TOPICA - Start your own email discussion group. FREE!
> >
> >
>
>
> TOPICA - Start your own email discussion group. FREE!
>
>
7. Re: Sequence slicing
Wow. This is the first mailing I've gotten from the Euphoria board since 9/25.
I guess the folk on Topica remembered to flip a switch or something...
-- David Cuny
8. Re: Sequence slicing
I had a similar problem before, no email between 6/2 and 9/1 this year.
----- Original Message -----
From: "David Cuny" <dcuny at LANSET.COM>
To: <EUforum at topica.com>
Subject: Re: Sequence slicing
>
>
> Wow. This is the first mailing I've gotten from the Euphoria board since
9/25.
>
> I guess the folk on Topica remembered to flip a switch or something...
>
> -- David Cuny
>
>
>
> TOPICA - Start your own email discussion group. FREE!
>
>
9. Re: Sequence slicing
Daniel Kluss wrote:
>> Wow. This is the first mailing I've gotten from the Euphoria board since
>> 9/25.
> ...
> I had a similar problem before, no email between 6/2 and 9/1 this year.
And, irritatingly enough, I got your reply 5 minutes ago, but *still* can't
see my own post.
Go figure.
-- David Cuny
10. Re: Sequence slicing
Question for Rob, is the below accurate, sounds like it to me. Could this
interfere with the
"If a is very large, and there are no other references to
the same sequence, this statement will be extremely fast.
Euphoria will simply update the length of a,
without copying any elements or allocating/deallocating any space."
you said before.
What should I do about it, would just assigning the length to a var then the
slice make it faster? or not?
Daniel Kluss
----- Original Message -----
From: "PeterBlonner" <peter at blonner.com>
To: <EUforum at topica.com>
Sent: Tuesday, October 21, 2003 9:10 PM
Subject: Re: Sequence slicing
>
>
> Is it possible that a form of recorsion happens here i.e that the length()
> operation is effected by the result of this line's execution ?
> I'm only making a tentative suggestion for why it could take unusually
long
> (since really the length() function should be done as the situation before
> the line was executed)
>
> Peter Blonner
>
>
> ----- Original Message -----
> From: "Daniel Kluss" <codepilot at netzero.net>
> To: <EUforum at topica.com>
> Sent: Wednesday, October 22, 2003 2:03 PM
> Subject: Re: Sequence slicing
>
>
> > Heres a snipet, thats all one line by the way, and it takes forever
> >
> >
sockets[sock][SOCKET_DATA][tDEST][tDATA]=sockets[sock][SOCKET_DATA][tDEST][t
> > DATA][diff+1..length(sockets[sock][SOCKET_DATA][tDEST][tDATA])]
> >
> >
> > ----- Original Message -----
> > From: "Greg Haberek" <g.haberek at comcast.net>
> > To: <EUforum at topica.com>
> > Sent: Tuesday, October 21, 2003 7:57 PM
> > Subject: Re: Sequence slicing
> >
> >
> > > > while 1 do
> > > > a&=data
> > > > send(a[1..1500 to 5000 or more])
> > > > a=a[1..length(a)-acknowleged()]
> > > > end while
> > >
> > > are you sure that calling acknowledged() isn't the bottle neck of your
> > code?
> > > try asigning it to a variable first, then profile it:
> > >
> > > atom ack
> > >
> > > while 1 do
> > > a &= data
> > > send( a[1..some_length] )
> > > ack = acknowledged() -- does this take forever,
> > > a = a[1..length(a)-ack] -- or this?
> > > end while
> > >
> > >
> > > TOPICA - Start your own email discussion group. FREE!
> > >
> > >
> > TOPICA - Start your own email discussion group. FREE!
> >
> >
>
>
> TOPICA - Start your own email discussion group. FREE!
>
>
11. Re: Sequence slicing
Daniel Kluss wrote:
> Heres a snipet, thats all one line by the way, and it takes forever
>
> sockets[sock][SOCKET_DATA][tDEST][tDATA]=sockets[sock][SOCKET_DATA][tDEST][t
> DATA][diff+1..length(sockets[sock][SOCKET_DATA][tDEST][tDATA])]
Try using a temporary variable to avoid those repeated array lookups:
object temp
-- Assign
temp = sockets[sock][SOCKET_DATA][tDEST][tDATA]
-- Process
temp = temp[diff+1..length(temp)]
-- Reassign
sockets[sock][SOCKET_DATA][tDEST][tDATA] = temp
Euphoria uses pointers internally, so 'temp' will really only be a
pointer to that deep level of the 'sockets' array.
--
[ Carl R White == aka () = The Domain of Cyrek = ]
[ Cyrek the Illogical /\ www.cyreksoft.yorks.com ]
12. Re: Sequence slicing
> Daniel Kluss wrote:
>
> >> Wow. This is the first mailing I've gotten from the Euphoria board
since
> >> 9/25.
> > ...
> > I had a similar problem before, no email between 6/2 and 9/1 this year.
>
> And, irritatingly enough, I got your reply 5 minutes ago, but *still*
can't
> see my own post.
>
> Go figure.
>
> -- David Cuny
Great opportunity for me to again suggest
http://h2osoft.homeip.net
There is some activity on this forum that may interest some of you.
Keep in mind, Im still working on making it better.
Currently there are 35 registered Euphoria coders
Euman
13. Re: Sequence slicing
WIN98SE is the one im using, but its made for dos I use freedos and msdos
What issues are there for frags in dos?
Daniel Kluss
----- Original Message -----
From: "Andy Serpa" <ac at onehorseshy.com>
To: <EUforum at topica.com>
Subject: RE: Sequence slicing
>
>
> Al Getz wrote:
> >
> >
> > Daniel Kluss wrote:
> > >
> > >
> > > Heres a snipet,
> >
> >
> > Hi Daniel,
> >
> > I find that the way you use sequences affects the speed of
> > the program.
> > For example, if you have to make two sequences by parsing
> > data:
> >
> > seq1={}
> > seq2={}
> > while x do
> > GetDin1AndDin2()
> > seq1=append(seq1,Din1)
> > seq2=append(seq2,Din2)
> > end while
> >
> > Takes much longer (not just twice as long) then:
> >
> > seq1={}
> > while x do
> > GetDin1AndDin2()
> > seq1=append(seq1,Din1)
> > end while
> >
> > for some reason. I figured it had something to do with
> > the allocation of the sequences.
> >
> > What i gathered was that if you can find another way of
> > doing something using sequences, you might find a faster
> > method. Of course this assumes the sequence assignment is
> > what is causing the slowdown. The new method might be
> > extremely faster too, not just a little faster.
> >
> >
> Maybe this is the same problem I encountered with the WinME OS and
> really large fragmented sequences with Eu 2.4? If you are building up
> the sequence incrementally, and the adjusting the size all the time, it
> could be an allocation/fragmentation issue. What OS are you using?
>
>
>
> TOPICA - Start your own email discussion group. FREE!
>
>
14. Re: Sequence slicing
Daniel Kluss wrote:
> Question for Rob, is the below accurate, sounds like it to me. Could this
> interfere with the
> "If a is very large, and there are no other references to
> the same sequence, this statement will be extremely fast.
> Euphoria will simply update the length of a,
> without copying any elements or allocating/deallocating any space."
>
> you said before.
> What should I do about it, would just assigning the length to a var then the
> slice make it faster? or not?
You should look at:
http://www.rapideuphoria.com/perform.htm
in the section "Saving Results in Variables" about half way down.
Carl White suggested something similar.
Here's the example I give:
==============================================================
When you have a sequence with multiple levels of subscripting,
it is faster to change code like:
for i = 1 to 1000 do
y[a][i] = y[a][i]+1
end for
to:
ya = y[a]
for i = 1 to 1000 do
ya[i] = ya[i] + 1
end for
y[a] = ya
So you are doing 2 subscript operations per iteration of the loop,
rather than 4. The operations, ya = y[a] and y[a] = ya are very cheap.
They just copy a pointer. They don't copy a whole sequence.
==============================================================
In your case, you have 4 subscripts, so the payoff would be
even bigger. Also, the internal slicing optimization
I mentioned before might work better with a simpler statement.
Another idea is to avoid shortening the sequence all the time,
but just use a variable to remember how big it really
should be at the moment. That will cut down on the
frequent allocation/deallocation of space.
Regards,
Rob Craig
Rapid Deployment Software
http://www.RapidEuphoria.com
15. Re: Sequence slicing
This brings to mind the way the BIOS handles the keyboard buffer.
1. Decide how much space you need for your buffer and allocate a sequence that
size using repeat().
2. Declare two integers that are used as subscripts that mark the beginning and
ending of the data. When they are equal, the buffer is empty.
3. Make all subscript calculations modulo of the size. You have a circular
buffer, so that the byte after the last one is the first one.
The data keeps going round and round, but you don't have to free and allocate at
all. An enhancement of this technique would be code to detect when the buffer is
about to overflow, and in that case add more space to it (permanently).
Louis.
*********** REPLY SEPARATOR ***********
On 10/22/2003 at 4:05 PM Robert Craig wrote:
>
>
>Daniel Kluss wrote:
>> Question for Rob, is the below accurate, sounds like it to me. Could this
>> interfere with the
>> "If a is very large, and there are no other references to
>> the same sequence, this statement will be extremely fast.
>> Euphoria will simply update the length of a,
>> without copying any elements or allocating/deallocating any space."
>>
>> you said before.
>> What should I do about it, would just assigning the length to a var then
>the
>> slice make it faster? or not?
>
>You should look at:
>
> http://www.rapideuphoria.com/perform.htm
>
>in the section "Saving Results in Variables" about half way down.
>Carl White suggested something similar.
>
>Here's the example I give:
>==============================================================
>When you have a sequence with multiple levels of subscripting,
>it is faster to change code like:
> for i = 1 to 1000 do
> y[a][i] = y[a][i]+1
> end for
>
>to:
> ya = y[a]
> for i = 1 to 1000 do
> ya[i] = ya[i] + 1
> end for
> y[a] = ya
>
>So you are doing 2 subscript operations per iteration of the loop,
>rather than 4. The operations, ya = y[a] and y[a] = ya are very cheap.
>They just copy a pointer. They don't copy a whole sequence.
>==============================================================
>
>In your case, you have 4 subscripts, so the payoff would be
>even bigger. Also, the internal slicing optimization
>I mentioned before might work better with a simpler statement.
>
>Another idea is to avoid shortening the sequence all the time,
>but just use a variable to remember how big it really
>should be at the moment. That will cut down on the
>frequent allocation/deallocation of space.
>
>Regards,
> Rob Craig
> Rapid Deployment Software
> http://www.RapidEuphoria.com
