1. Passing sequences to functions & procedures / Memory limits in openEuphoria?
- Posted by ZNorQ Oct 04, 2013
- 2076 views
1) Are sequences passed by reference or by value in functions/procedures?
2) Is there a limit to the size of sequences (apart from the obvious physical limitation; RAM)?
ZNorQ aka. Kenneth
2. Re: Passing sequences to functions & procedures / Memory limits in openEuphoria?
- Posted by jimcbrown (admin) Oct 04, 2013
- 2035 views
1) Are sequences passed by reference or by value in functions/procedures?
Pass by value semantics is used, although this is implemented by passing pointers and doing copy-on-write.
2) Is there a limit to the size of sequences (apart from the obvious physical limitation; RAM)?
ZNorQ aka. Kenneth
On 32bit, a sequence may have a maximum of 2^31 elements (though each of those items may be a sequence with its own 2^31 limit on elements, and so on). On 64bit, the maximum is 2^63 elements.
edit: I made a mistake. It's still 2^31 elements on a 64bit system.
3. Re: Passing sequences to functions & procedures / Memory limits in openEuphoria?
- Posted by mattlewis (admin) Oct 04, 2013
- 2050 views
1) Are sequences passed by reference or by value in functions/procedures?
As a technical matter, they are passed by reference. However, they are reference counted, and a sequence passed as a parameter will have a reference count of at least 2, meaning that any modifications will trigger a copy on write. This means that if you do not intend to modify the sequence, passing even large amounts of data is very cheap.
Of course, this also means that if you're modifying sequence parameters a lot, this can be a significant bottleneck. There are ways around this, and things that can be done to reduce the burden. In std/map.e, for instance, we go to a lot of effort to avoid copy on writes, as they would really drag down map performance.
2) Is there a limit to the size of sequences (apart from the obvious physical limitation; RAM)?
The number of elements in a sequence is tracked by a signed 32-bit integer. Theoretically, a 64-bit machine and euphoria binary could have enough memory to store a sequence with more than 231 elements, but always on a 32-bit implementation, and in most practical cases under 64 bits, you'll run out of memory before you hit that barrier.
Matt
4. Re: Passing sequences to functions & procedures / Memory limits in openEuphoria?
- Posted by ghaberek (admin) Oct 04, 2013
- 2046 views
In std/map.e, for instance, we go to a lot of effort to avoid copy on writes, as they would really drag down map performance.
I think it should also be pointed out that std/map.e is based on std/eumem.e of which, "one use is to emulate PBR [pass-by-reference], such as Euphoria's map and stack types."
One can use std/eumem.e for a great and many thing! 
-Greg
5. Re: Passing sequences to functions & procedures / Memory limits in openEuphoria?
- Posted by ZNorQ Oct 07, 2013
- 1856 views
PBR; Thanks, this is very good news as I *usually* just perform read on large sequences; I was under the impression that passing any object was done by value. However, I would like to see the possibility to use pass-by-reference and having the ability to write on it without the copy. Is that something you are considering (I'm sorry if the question has been addressed a multitude of times before..)?
Memory usage; So, if I understand this correctly; I can have a sequence with an arbitrary collection of sequences and atoms that can amount to 2,147,483,648 items (2^31), and each of these "cells" can have a sequence with the same amount of objects, and this can go on on for each level..? If so, this is certainly good news, and I know now how to solve some of my memory-problems when handling big loads of data.
Signed integer; Why is the integer tracking the size of a sequence signed?
Last; I've been studying C plusplus for some time now, and absolutely love it, but guess what; I ALWAYS fall back to good'ol Euphoria. I have to give credit to all you guys that are keeping the language afloat. I lack the technical knowledge to be able to contribute, but I do promote the language every way I can in the business-environment I work in. If I create an application for business-purposes, I always include an about that gives reference to openEuphoria webpage. I would love to see Euphoria as a main-stream language.
Again, thanks for the feedback, and keep it up.
ZNorQ aka Kenneth.
6. Re: Passing sequences to functions & procedures / Memory limits in openEuphoria?
- Posted by mattlewis (admin) Oct 07, 2013
- 1841 views
PBR; Thanks, this is very good news as I *usually* just perform read on large sequences; I was under the impression that passing any object was done by value. However, I would like to see the possibility to use pass-by-reference and having the ability to write on it without the copy. Is that something you are considering (I'm sorry if the question has been addressed a multitude of times before..)?
It has certainly been talked about and thought about. I experimented a bit with this in my ooeu experiments. There are some tricky issues involved in implementation.
Signed integer; Why is the integer tracking the size of a sequence signed?
This was how Rob implemented it. I'd guess that he figured it was big enough. And practically speaking, it should be.
Matt
7. Re: Passing sequences to functions & procedures / Memory limits in openEuphoria?
- Posted by ZNorQ Oct 07, 2013
- 1826 views
PBR; Thanks, this is very good news as I *usually* just perform read on large sequences; I was under the impression that passing any object was done by value. However, I would like to see the possibility to use pass-by-reference and having the ability to write on it without the copy. Is that something you are considering (I'm sorry if the question has been addressed a multitude of times before..)?
It has certainly been talked about and thought about. I experimented a bit with this in my ooeu experiments. There are some tricky issues involved in implementation.
I can only imagine. The way it's implemented today should be sufficient for me in most cases.
Signed integer; Why is the integer tracking the size of a sequence signed?
This was how Rob implemented it. I'd guess that he figured it was big enough. And practically speaking, it should be.
I agree, just found it a bit strange, that's all (a conclusion based on ignorance on my part).
ZNorQ aka Kenneth
8. Re: Passing sequences to functions & procedures / Memory limits in openEuphoria?
- Posted by LarryMiller Oct 07, 2013
- 1812 views
In a 32 bit OS the practical limit to how large a sequence you can use is imposed by the size of virtual address space that can be allocated. The size of the private virtual address space is 2 GB, independent of RAM size. The largest block that can be allocated will be substantially less than that, possible much less. The usage patterns of the sequence and RAM size will determine performance.
To exceed these limits you would need to use a 64 bit OS and 64 bit version of Euphoria.
In practice these limits are unlikely to be a problem.
9. Re: Passing sequences to functions & procedures / Memory limits in openEuphoria?
- Posted by jimcbrown (admin) Oct 07, 2013
- 1891 views
The size of the private virtual address space is 2 GB, independent of RAM size.
This varies by OS. It's 3GB in Linux/GNU for example: http://users.nccs.gov/fwang2/linux/lk_addressing.txt
It's possible to get a 4GB/4GB split and use up almost all of 4GB of ram in a single userspace program: http://lwn.net/Articles/39283/ http://stackoverflow.com/questions/6947261/4gb-4gb-kernel-vm-split (of course an Euphoria sequence couldn't use the full 4GB anyways as some space is taken up by libc et al).
In a 32 bit OS the practical limit to how large a sequence you can use is imposed by the size of virtual address space that can be allocated. The largest block that can be allocated will be substantially less than that, possible much less. The usage patterns of the sequence and RAM size will determine performance.
To exceed these limits you would need to use a 64 bit OS and 64 bit version of Euphoria.
In practice these limits are unlikely to be a problem.
Agreed in full.
10. Re: Passing sequences to functions & procedures / Memory limits in openEuphoria?
- Posted by mattlewis (admin) Oct 07, 2013
- 1809 views
In a 32 bit OS the practical limit to how large a sequence you can use is imposed by the size of virtual address space that can be allocated. The size of the private virtual address space is 2 GB, independent of RAM size. The largest block that can be allocated will be substantially less than that, possible much less. The usage patterns of the sequence and RAM size will determine performance.
To exceed these limits you would need to use a 64 bit OS and 64 bit version of Euphoria.
In practice these limits are unlikely to be a problem.
Yes. There's a lot of memory involved. Let's say you wanted to create a 3 billion element sequence of integers with 64-bit euphoria. Firstly, you have 24-bytes of overhead information stored by the sequence structure. This is pretty much nothing compared to where we're going, of course. Next, you'd have 24,000,000,008 bytes allocated to store your integers (one extra element is allocated as an end marker).
There are certainly machines that have a lot more RAM than these, but if you're throwing that much at a single data structure, I will say that you're doing it wrong and should look for a better way, because it's going to be painful.
Matt
11. Re: Passing sequences to functions & procedures / Memory limits in openEuphoria?
- Posted by EUWX Oct 07, 2013
- 1828 views
In short, for Windows:
http://msdn.microsoft.com/en-us/library/windows/desktop/aa366796%28v=vs.85%29.aspx
12. Re: Passing sequences to functions & procedures / Memory limits in openEuphoria?
- Posted by jimcbrown (admin) Oct 07, 2013
- 1827 views
In short, for Windows:
http://msdn.microsoft.com/en-us/library/windows/desktop/aa366796%28v=vs.85%29.aspx
Not really. Euphoria doesn't support using AWE to store its variables, so it's still limited to the virtual address space size.
