1. Who uses tasks?
- Posted by ryanj Sep 08, 2009
- 1506 views
I was just wondering, who among you uses euphoria's multi-tasking features?
I use them a lot for FluidAE and another program that communicates with serial devices. Tasks make it easy to manage slow conversations with slow devices while letting the rest of the program work.
I would love to hear of examples of how you are using tasks.
2. Re: Who uses tasks?
- Posted by ChrisB (moderator) Sep 08, 2009
- 1395 views
So, does anyone use tasks?
Chris
3. Re: Who uses tasks?
- Posted by useless Sep 08, 2009
- 1346 views
Chris, you did not edit the replies counter. It still shows 10.
useless
4. Re: Who uses tasks?
- Posted by jeremy (admin) Sep 08, 2009
- 1318 views
I removed them all, it's a bug in euWeb still. See ticket:5 for the reason why. ticket:6 is for the remove bug.
Jeremy
5. Re: Who uses tasks?
- Posted by ryanj Sep 08, 2009
- 1368 views
Aww, man! I thought there were 12 replies to this! 
6. Re: Who uses tasks?
- Posted by useless Sep 09, 2009
- 1343 views
Aww, man! I thought there were 12 replies to this!
I'm going as fast as i can , Ry!
This is a cross post, but just for the record, the new task.e message passing functions work!
http://imgbin.org/index.php?page=image&id=759
useless
7. Re: Who uses tasks?
- Posted by raseu Sep 10, 2009
- 1201 views
I do, extensively!
have written wrappers for betfair sports/games apis for scheduled data updates
however, tasks under v4 do not work when compiled into an .exe under ubuntu jaunty (crash with stack_size exception) but work fine with interpreter
ie:
Ubuntu Jaunty
- WORKS $ eui news.ex <search term>
- CRASH with stack size exception $ euc news.ex $./news <search term>
just tested news.ex under win32 using bundled watcom compiler. after changing stack size as option to euc, program compiles and runs ok.
eg:
Win32
C:\Euphoria\demo\euc -stack 1024000 news.ex
8. Re: Who uses tasks?
- Posted by mattlewis (admin) Sep 10, 2009
- 1200 views
I do, extensively!
have written wrappers for betfair sports/games apis for scheduled data updates
however, tasks under v4 do not work when compiled into an .exe under ubuntu jaunty (crash with stack_size exception) but work fine with interpreter
Yes, this is a known issue:
https://sourceforge.net/tracker/?func=detail&aid=2842518&group_id=182827&atid=902782
I have feeling that the OS may be doing something with the stack, messing up the stuff that euphoria leaves there. I think the translated task stuff does things it's probably not supposed to be doing. We may need to rewrite the task backend for translated code.
Matt
9. Re: Who uses tasks?
- Posted by jeremy (admin) Sep 10, 2009
- 1154 views
I have feeling that the OS may be doing something with the stack, messing up the stuff that euphoria leaves there. I think the translated task stuff does things it's probably not supposed to be doing. We may need to rewrite the task backend for translated code.
Did this always fail in translated code or just in 4.0?
Jeremy
10. Re: Who uses tasks?
- Posted by mattlewis (admin) Sep 10, 2009
- 1166 views
I have feeling that the OS may be doing something with the stack, messing up the stuff that euphoria leaves there. I think the translated task stuff does things it's probably not supposed to be doing. We may need to rewrite the task backend for translated code.
Did this always fail in translated code or just in 4.0?
I'm not sure when it started failing. I believe that it used to work. There hasn't been much that's changed in be_task.c. There were some changes needed for interpreted tasks when built with gcc, due to the way were were using a variable on the stack after we should have.
Again, that used to work, but either gcc or the kernel, or something changed to invalidate the way we were doing stuff. I don't know enough to say definitively, but the way that translated tasks are handled really seems hacky and dangerous to me.
Basically, we rely on the memory above the current stack pointer to stay the same, and we write some guard values in there. The space gets chopped up for use by the various tasks, and we watch for overflowing the boundaries that we've set.
Basically, when one of our guard values changes, we assume that a stack overflow has occurred by one of the tasks. This is what happens, except that it happens immediately, and isn't because one of our stacks went too deep in its call stack. So, since I don't believe that anything we actively did changed the value, I assume that it's something that the OS is doingand why shouldn't it, we never allocated that memory via malloc or the stack.
There is an attempt made to allocate more stack space by recursively calling a function whose purpose is solely to take up stack space (I think there is a way in windows to ask for lots of stack space ahead of time). I think we came across something that happened to work, but certainly wasn't guaranteed to work, and it no longer does.
That's why I've proposed rewriting it using threads so that we can allow the OS / thread library to manage our stacks, etc, and we can use mutexes or something to keep only a single task running at a time.
Matt
11. Re: Who uses tasks?
- Posted by jeremy (admin) Sep 10, 2009
- 1203 views
That's why I've proposed rewriting it using threads so that we can allow the OS / thread library to manage our stacks, etc, and we can use mutexes or something to keep only a single task running at a time.
What level of effort are we talking about? Sounds pretty high and it would seem to be something that needs to be fixed before we move into 4.0rc's.
Jeremy
12. Re: Who uses tasks?
- Posted by jaygade Sep 10, 2009
- 1214 views
I may be stepping out of my bounds (of knowledge) here, but isn't there a call which specifically allocates more stack space for a task/function/whatever? realloc()? Or alloca()?
13. Re: Who uses tasks?
- Posted by jimcbrown (admin) Sep 10, 2009
- 1205 views
I may be stepping out of my bounds (of knowledge) here, but isn't there a call which specifically allocates more stack space for a task/function/whatever? realloc()? Or alloca()?
realloc() isn't applicable, its part of the malloc() family and uses the heap.
alloca() will grow more stack space, but this is only temporary until the function that calls it returns, so unless we call it from main(), it won't help. Regardless, my man page for alloca(3) (from 2002) says that alloca() is buggy and compiler dependent. It has further notes which state that the gcc implementation typically does nothing more that set the stack pointer, and does not check for stack overflow (so if you overflow you just segfault, or worse).
There is still brk(2)/sbrk(2) but that only mentions the data segment, not the stack per se.
14. Re: Who uses tasks?
- Posted by mattlewis (admin) Sep 10, 2009
- 1135 views
What level of effort are we talking about? Sounds pretty high and it would seem to be something that needs to be fixed before we move into 4.0rc's.
Yes, I fully agree.
Matt
15. Re: Who uses tasks?
- Posted by jaygade Sep 10, 2009
- 1182 views
I may be stepping out of my bounds (of knowledge) here, but isn't there a call which specifically allocates more stack space for a task/function/whatever? realloc()? Or alloca()?
realloc() isn't applicable, its part of the malloc() family and uses the heap.
alloca() will grow more stack space, but this is only temporary until the function that calls it returns, so unless we call it from main(), it won't help. Regardless, my man page for alloca(3) (from 2002) says that alloca() is buggy and compiler dependent. It has further notes which state that the gcc implementation typically does nothing more that set the stack pointer, and does not check for stack overflow (so if you overflow you just segfault, or worse).
There is still brk(2)/sbrk(2) but that only mentions the data segment, not the stack per se.
Huh. I hadn't seen anything stating it was "buggy" but then I'm using Google and looking at what I find.
I only remembered alloca() because I had seen it in a window manager I was trying to understand several years ago.
I'm not familiar enough with the tasking implementation of Euphoria does it call subroutines or does it just jump around single-threaded (a la Forth) for each task?
Anyway, it was just a suggestion for growing the stack of certain code.
16. Re: Who uses tasks?
- Posted by jimcbrown (admin) Sep 10, 2009
- 1119 views
Anyway, it was just a suggestion for growing the stack of certain code.
And a very good one, I might add.
The problem isn't with growing the stack per se, though, but with detection of stack overflow.
Stack manipulation can get very complex, and Matt's threading idea is looking more and more attractive each time.
17. Re: Who uses tasks?
- Posted by jacques_desch Sep 10, 2009
- 1096 views
- Last edited Sep 11, 2009
I have no idea how tasks code works but if you want to watch the stack pointer position why don't you read the stack pointer register and compare it with a saved value of itself.
Does-it make any sense in this context?
Jacques
18. Re: Who uses tasks?
- Posted by mattlewis (admin) Sep 10, 2009
- 1102 views
- Last edited Sep 11, 2009
I have no idea how tasks code works but if you want to watch the stack pointer position why don't you read the stack pointer register and compare it with a saved value of itself.
Does-it make any sense in this context?
Not really. Or maybe. It's complicated. Translated routines are native C routines, meaning that they manipulate the stack in a 'normal' way, as opposed to interpreted code, which actually uses the heap for the equivalent of stack space. Tasks are a way for a euphoria program to cooperatively multitask among its routines.
So we are allowed to suspend one routine, and start/resume execution of another. So the way it works is that you start with a 'normal' stack, prior to creating any tasks. When you start creating tasks, the task management stuff from the runtime library starts slicing up the stack for your various tasks to use.
When a task is called, the stack pointer is changed to wherever that task's stack space is. When it yields, the stack pointer goes back so that the back end task management stuff can figure out which task should run next.
However, if we can't even be sure that the stack won't get corrupted, we certainly can't go back and use it that way.
Matt
19. Re: Who uses tasks?
- Posted by useless Sep 10, 2009
- 1145 views
- Last edited Sep 11, 2009
The way Matt just described it, it smells like how i did the C64's multitasking. The memory in the C64 was not lockable, even tho with care no pointers would point into the other tasks, their zeropage, or the stored register area. Can you solve this problem simply by telling the OS to software or hardware lock the memory space for each task individually?
useless
20. Re: Who uses tasks?
- Posted by jimcbrown (admin) Sep 10, 2009
- 1120 views
- Last edited Sep 11, 2009
The way Matt just described it, it smells like how i did the C64's multitasking. The memory in the C64 was not lockable, even tho with care no pointers would point into the other tasks, their zeropage, or the stored register area. Can you solve this problem simply by telling the OS to software or hardware lock the memory space for each task individually?
useless
Yes, this is doable with the mmap() function.
21. Re: Who uses tasks?
- Posted by useless Sep 10, 2009
- 1096 views
- Last edited Sep 11, 2009
The way Matt just described it, it smells like how i did the C64's multitasking. The memory in the C64 was not lockable, even tho with care no pointers would point into the other tasks, their zeropage, or the stored register area. Can you solve this problem simply by telling the OS to software or hardware lock the memory space for each task individually?
useless
Yes, this is doable with the mmap() function.
Then why is the memory being overwritten?
useless
22. Re: Who uses tasks?
- Posted by jimcbrown (admin) Sep 10, 2009
- 1098 views
- Last edited Sep 11, 2009
Then why is the memory being overwritten?
useless
mmap() isn't being used. Although it is possible, using mmap() in this way would still be very complex. (I'm of the opinion that Matt's idea would be easier to implement.)
23. Re: Who uses tasks?
- Posted by euphoric (admin) Sep 10, 2009
- 1122 views
- Last edited Sep 11, 2009
I'm of the opinion that Matt's idea would be easier to implement.
Make it so.
(I just like saying that.)
24. Re: Who uses tasks?
- Posted by jacques_desch Sep 10, 2009
- 1103 views
- Last edited Sep 11, 2009
Not really. Or maybe. It's complicated. Translated routines are native C routines, meaning that they manipulate the stack in a 'normal' way, as opposed to interpreted code, which actually uses the heap for the equivalent of stack space. Tasks are a way for a euphoria program to cooperatively multitask among its routines.
So we are allowed to suspend one routine, and start/resume execution of another. So the way it works is that you start with a 'normal' stack, prior to creating any tasks. When you start creating tasks, the task management stuff from the runtime library starts slicing up the stack for your various tasks to use.
When a task is called, the stack pointer is changed to wherever that task's stack space is. When it yields, the stack pointer goes back so that the back end task management stuff can figure out which task should run next.
However, if we can't even be sure that the stack won't get corrupted, we certainly can't go back and use it that way.
Matt
If I understand well, in translated code, when task switch occur, the CPU SP register is loaded with a new address? But by the time the program switch to the next task the memory pointed by the SP register for that task as been modified by te OS, because for the OS this memory is not in use, the virtual memory manager don't care to preserve it? is it right?
Jacques
