OpenEuphoria

1. Pass by Reference

• Posted by dcuny Dec 18, 2014
• 3620 views

I'm writing an application that has multiple digital filters. Calling the filter:

• Calculates a filtered result, and
• Updates the filter's history.

That is, two different things are being modified here - there's a result returned from the function call, and as a side-effect, a data structure gets updated.

My first approach was to write the filter as a namespace, and try to treat them like an object. Only.. namespaces aren't really objects, and things got ugly fast.

My second approach was to write something like:

function do_filter( sequence theFilter, atom theInput ) 
   atom theOutput = theFilter[2] * theFilter[1] * theInput 
   theFilter[2] = theFilter[1] 
   theFilter[1] = theInput 
 
   return theOutput 
end function 
 
atom theResult = do_filter( theFilter, theInput ) 

Of course, this doesn't work because Euphoria only passes parameters by value and not by reference. I started remembering why I'd worked on Py.

So what to do with the result of the filter?

One solution would be to pass a sequence back from the function, which is really clunky:

function do_filter( sequence theFilter, atom theInput ) 
   atom theOutput = theFilter[2] * theFilter[1] * theInput 
   theFilter[2] = theFilter[1] 
   theFilter[1] = theInput 
 
   return {theFilter, theOutput} 
end function 
 
sequence theResult = do_filter( theFilter, theInput ) 
theFilter = theResult[1] 
atom theOutput = theResult[2] 

Yes, it's the worst filter ever. I don't have the exact code in front of me, but it gets to the essence of the problem.

Another option is to store the value return inside theFilter, which is only slightly less clunky:

enum FILTER_HIST_1, FILTER_HIST_2, FILTER_RESULT 
 
function do_filter( sequence theFilter, atom theInput ) 
   atom theOutput = theFilter[FILTER_HIST_1] * theFilter[FILTER_HIST_2] * theInput 
   theFilter[FILTER_HIST_2] = theFilter[FILTER_HIST_1] 
   theFilter[FILTER_HIST_1] = theInput 
   theFilter[FILTER_RESULT] = theOutput 
 
   return theFilter 
end function 
 
-- pass the filter data and value to filter to the filtering routine 
theFilter = do_filter( theFilter, theInput ) 
 
-- save the filtered value 
atom result = theResult[FILTER_RESULT] 

But it raises the question: why do I have to jump through these hoops at all? Why can't OpenEuphoria let me pass values by reference like just about any other language?

- David

3. Re: Pass by Reference

• Posted by DerekParnell (admin) Dec 18, 2014
• 3510 views

Using the eumem.e library, one can emulate this less-than-safe programming practice. When using this library, you acquire a reference to some 'global' memory object and can pass that reference to routines.

include sys/eumem.e 
 
object aFilter = eumem:malloc({0,0}) -- Initialze new structure with a couple of zero values. 
 
 
function do_filter( object theFilter, atom theInput ) 
   atom theOutput = ram_space[theFilter][2] * ram_space[theFilter][1] * theInput 
   ram_space[theFilter][2] = ram_space[theFilter][1] 
   ram_space[theFilter][1] = theInput 
 
   return theOutput 
end function 
 
atom theResult = do_filter( aFilter, theInput ) 

Because its more well behaved than other languages.

5. Re: Pass by Reference

• Posted by dcuny Dec 18, 2014
• 3519 views

Thanks, Jim.

You could be right, but the documentation is entirely opaque to me. It looks like it's doing something roughly equivalent to declaring a globally visible heap. If that's the case, how is it much different from:

global sequence GLOBAL_MEMORY = {} 
 
-- get the next handle (index) to the global sequence 
global function malloc() 
    GLOBAL_MEMORY &= 0 
    return length(GLOBAL_MEMORY) 
end function 
 
-- get the value stored in the global sequence 
global function get(integer index) 
    return GLOBAL_MEMORY[index] 
end function 
 
-- set a value stored in a global sequence 
global procedure set(integer index, object value) 
    GLOBAL_MEMORY[index] = value 
end procedure 

But it doesn't answer the question why OpenEuphoria support pass by reference.

- David

7. Re: Pass by Reference

• Posted by DerekParnell (admin) Dec 18, 2014
• 3588 views

If one passes a reference to an item to a routine, without actually looking at the receiver's source code, can the coder know if the passed item will be modified or not?

If a language knows, without a doubt, that a passed item cannot be modified by the called routine, then many optimisation possibilities open up for it.

If you want to see this philosophy in action, take a look at the D Programming Language. That modern language does support PBR but that is not the default. The default is that declared items are read-only, even when passed by reference. This language forces the coder to make extremely explicit, any intention to modify items passed to routines. The resulting object code can be very optimised when the compiler knows about this.

In general, it is a safer practice to pass by value, and much less costly to maintain programs written this way. The performance of PBR is not generally an issue but in those situations where it is a factor, you can revert to less safe coding practices (using 'global' memory).

9. Re: Pass by Reference

• Posted by dcuny Dec 18, 2014
• 3504 views

And..?

That's the case with anything else. Without looking at the code you can't tell if a global value is being altered, either. Yet OpenEuphoria allows it. And that's actually the (really dirty) workaround being suggested here.

Yep. The purpose of a language isn't to be optimized, but to be useful.

IMNSHO, Euphoria is one of the most unsafe languages I've ever coded in.

It crashes on errors... by design.

It took forever for Robert to be convinced that something like crash_routine() even needed to exist.

The core datatype - the sequence - is inherently unsafe. It's trivial to get an index out of bounds error. (I speak from experience!)

I'd argue that Euphoria is unsafe by design.

- David

11. Re: Pass by Reference

• Posted by jaygade Dec 18, 2014
• 3484 views

The Euphoria way would be to use a global or file-level variable for large sequences that need to be manipulated in place. Just look at the source of the interpreter itself.

Not that that's a bad thing, it just doesn't scale beyond a certain point -- admittedly, a pretty large point.

Google's Go language does pass by value on everything, but has pointers, and slices are pointers to arrays. So with pointers and slices you can manipulate large data structures in place.

Then again, golang folks are as adamant of its features and/or lack of other features as we Euphorians are. *shrug*

The question is whether you are willing to perform your task in an idiomatic way or try to force the language to doing things your way.

EDIT Just like me and this damned creole markup. Grr.

13. Re: Pass by Reference

• Posted by jimcbrown (admin) Dec 19, 2014
• 3449 views

Right, but the translated version probably wouldn't have any of this information. (The shrouded version is somewhere in between these two extremes.)

Agreed. Adding something like a try/catch mechanism is on the TODO list for this very reason.

It does with eumem.... again that's the whole point of eumem. Why is using eumem (or your superior rewrite with its getter/setter routines) so much more of a problem compared to built-in PBR or pointers?

Agreed on the try/catch part. Disagree on the PBR part... but if you really want to add PBR support to Euphoria itself, I don't know of anything stopping you. (If done cleverly, you could add PBR support just by modifying the parser, removing the need to change any underlying C code.)

I imagine that, if done in the right style (ala D, so PBR is not the default and made very explicit), a proof-of-concept feature could be merged into the mainlane pretty easily, without much objection.

Since we already have eumem, which is "good enough", I don't see a compelling reason to work on PBR as a core language feature. But if someone provided a patch with the hard work already done, then it'd be a different story.

15. Re: Pass by Reference

• Posted by dcuny Dec 19, 2014
• 3459 views

I don't think my rewrite is any better... I just couldn't grok how eumem was supposed to work from the docs, and have been too busy flaming away to pull up any examples.

I truly appreciate the work that's gone into maintaining and extending Euphoria. But it's frustrating to see that some of the stupid stuff that drove me away from the language is still there. I just don't want to get sucked into that timesink again... But we'll see.

- David

17. Re: Pass by Reference

• Posted by _tom (admin) Dec 19, 2014
• 3404 views

Euphoria 4.1 lets you do this:

include std/sequence.e 
include std/console.e 
 
function do_filter( sequence theFilter, atom theInput ) 
   atom theOutput = theFilter[1] * theFilter[2] * theInput 
   theFilter[2] = theFilter[1] 
   theFilter[1] = theInput 
 
   return { theFilter, theOutput } 
end function 
 
atom theInput = 1 
sequence theFilter = {2,3} 
atom theResult = 0 
 
 
        printf(1, "START\n\t input: %d", theInput ) 
        printf(1, "\n\t filter: %d %d", theFilter ) 
        printf(1, "\n\t result %d", theResult ) 
 
 
{ theFilter, theResult } = do_filter( theFilter, theInput ) 
 
 
        printf(1, "\n\nFINISH\n\t input: %d", theInput ) 
        printf(1, "\n\t filter: %d %d", theFilter ) 
        printf(1, "\n\t result %d", theResult ) 
 
--START 
--     input: 1 
--     filter: 2 3 
--     result 0 
-- 
--FINISH 
--     input: 1 
--     filter: 1 2 
--     result 6 

The idea is that in Euphoria you can always identify the exact line where an object changes its value.

_tom

19. Re: Pass by Reference

• Posted by mattlewis (admin) Dec 19, 2014
• 3398 views

I started working on doing this before OpenEuphoria was a thing. I think PBR allows some very important features and optimizes some important cases. I'd really like to see it be possible. I made some progress, but I recall getting stuck on some things.

The really short answer is that it's a difficult problem to make it work and not break lots of stuff.

Matt

21. Re: Pass by Reference

• Posted by Steady Dec 21, 2014
• 3230 views

Although I have not used WxLua, your acceptance of it is good enough for me. However, the fact is that from a "Branding" point of view, everybody knows about "BASIC" language - the older generation knows it as "BASIC" and the younger generation knows it as "VISUAL-BASIC"; therefore, WxBasic would be a good language to work with - the promotion work has already been done for you.

23. Re: Pass by Reference

• Posted by dcuny Dec 23, 2014
• 3137 views

Yes, I believe that Ralf Peters is the person currently maintaining that version.

- David

25. Re: Pass by Reference

• Posted by Shian_Lee Jan 03, 2015
• 2951 views

I think that programming style fits to specific personality; Personally I really don't like 'Pass by Reference' - because it's less maintainable and very confusing. local and global variables I use with caution, when really necessary, with full documentation.

I think that Euphoria is very safe if you're a disciplined programmer. But I/O routines must not trigger a fatal error - instead they should return an error value. I don't see why an I/O error should crash a program, it's ridiculous.

David's 'filters' program can be written very easily in Euphoria, in many ways, as long as you are a disciplined and consistent programmer.

I've never learned programming officially. Yet I took the job from professional programmers because the programs I wrote were maintainable. My last boss realised very quick that maintainable programs are saving him a lot of money.

For writing maintainable programs, I try to avoid any advanced or confusing feature, such as 'Pass by Reference', overriding identifiers, etc.

27. Re: Pass by Reference

• Posted by jimcbrown (admin) Jan 04, 2015
• 2920 views

Not sure I agree with you here. The difference between

ref a = b 

and

ram_space[a] = b 

is largely just syntax sugar. A good preprocessor can convert the former to the latter, in fact my preprocessor (Dredge) did something just like this. Considering v4's builtin support for hooking into preprocessors, this sort of syntax sugar should be very easy to add to the main language itself - if only someone could find the time to write one....

29. Re: Pass by Reference

• Posted by jimcbrown (admin) Jan 04, 2015
• 2887 views

If I am understanding this correctly, you are stating that:

In order to protect the positive attributes of Euphoria - we must protect the negative attributes of Euphoria as well..

Also, you define 'simplicity' of Euphoria as a positive attribute, but 'pass by value' as a negative attribute.

Please correct me if I have misunderstood something here.

31. Re: Pass by Reference

• Posted by jimcbrown (admin) Jan 04, 2015
• 2868 views

Yep. I feel that simple is good, but too simple is bad. In other words, you can have too much of a good thing. All things in moderation, etc.

In the past, RapidEuphoria was too simple, and that held it back. OpenEuphoria has taken steps to improve the balance, but I don't think anyone could reasonably disagree that there's still room for additional improvements.

Earlier, I didn't quite agree that PBR was one of these things, seeing that it was so easily emulated. Looking at dcuny's attempts to wrap wxWidgets in an object-oriented way though, I'm starting to move towards PBR being a necessary feature for OO. (Sure, you can get away with emulated PBR, but the lack of necessary syntax sugar makes things clumsier and introduces more room for programmer error.)

33. Re: Pass by Reference

• Posted by jimcbrown (admin) Jan 04, 2015
• 2882 views

My point is that they're functionally identical, so if PBR gets out of control, PBR emulation by global variables will get out of control in precisely the same way.

But I don't think that PBR (true or by emulation with global variables) would get out of control anyways. As I said originally, it's all in the moderation:

35. Re: Pass by Reference

• Posted by dcuny Jan 04, 2015
• 2856 views

    {i,j,k} = thing(i,j,k) 

Why not

    i,j,k = thing(i,j,k) 

The first example implies you are trying to build a sequence, while the second looks like an assignment.

Plus, it matches the syntax of existing languages (such as Python) which already support this feature.

- David

37. Re: Pass by Reference

• Posted by mattlewis (admin) Jan 04, 2015
• 2862 views

Yes. This has been implemented for 4.1.

    i,j,k = {1,2,3} 

Yes, it simply seemed more euphoric, IIRC. Or maybe no one thought to have naked LHS sequences like that. It's been a couple of years at this point.

if ... then 
    return 1,2 
else 
    return 1,2,3 

I'm sure that even if we'd gone with that syntax, we'd have to have turned it into a sequence. Probably better not to start removing stuff like that. Then the language starts to look like ruby, or something.

Matt

39. Re: Pass by Reference

• Posted by ryanj Jan 05, 2015
• 2852 views

Personally, i have no problem with not having pass by reference. There are times when i would use it if it was available, but since it isn't, i just find other ways to do what i want. I actually like how Euphoria doesn't have lots of different ways of doing things with "syntax sugar", but has the power to emulate many things that other languages do, such as OOP for example. Even if it takes a little bit of extra typing, i like the fact that i am able to see exactly what the code is doing because it is all written out in sequences, slices, elements, enums, etc. I can clearly see when something is a reference to an index vs. an ID, and i can see exactly how the sequence stores everything, and i know nothing else is sneaking behind my back to do something.

As an example, in my Redy environment, i use pseudo-OOP all the time. I don't need to pass by reference, all the data is right there, in sequences. All i need to do is search for an ID to find the index of whatever object i want to access, and pass that idx to a routine if i want to. I can use enum to define valid values for certain types of data and i can create routines such as create and destroy and other routines to modify the object "properties" however i want to. I don't have to learn about polymorphism and stuff like that or follow tons of OOP rules and syntax to do whatever OOP kind of stuff i want to do. I'm not even sure how my code compares to how you would do things in an OOP languages, and i don't really care. I just follow some basic rules to access sequences with complete control and understanding of what is going on.

41. Re: Pass by Reference

• Posted by dcuny Jan 05, 2015
• 2923 views

The statement has value, but I usually start with the opposite question: why should I force other people to learn a new syntax? It's not that should never happen, but all things being equal, there should be a good reason for doing so.

I apologize for being "late to the party" on this, but consider the two options:

function foo() 
   return 1, 2, 3, 4 
end function 
 
a, b, c = foo() 

vs.

function foo() 
   return {1, 2, 3} 
end function 
 
{a, b, c, d} = foo() 

Obviously, the first can be verified as incorrect by the parser.

The second can only be determined to be wrong at runtime, and will result in a crash.

Unlike indexes, there's no way to check for this error - you can't see how many values a function is returning. So the use of this syntax is intrinsically unsafe, and there's no way that I see to make it safe.

Am I missing something here?

- David

43. Re: Pass by Reference

• Posted by DerekParnell (admin) Jan 05, 2015
• 2803 views

Of course, but only when seen in light of other things. It doesn't stand on its own; it has no intrinsic value. It needs supporting arguments.

So that every programming language is not the same. For instance, which syntax ought we base Euphoria on? There are lots of options - each with their own flavor of syntax.

Yes! A good reason is why we should be doing anything. Both in following the syntax pattern of another language or when striking out in new ways.

function foo() 
   return 1, 2, 3, 4 
end function 
 
a, b, c = foo() 

function foo() 
   return {1, 2, 3} 
end function 
 
{a, b, c, d} = foo() 

Yes, you might be missing out on something here.

Firstly, it is not intrinsically (sorry - that word again) an error or mistake to ignore some returned function values. It is also not necessarily a mistake to present more variables for assignment than are needed. The parser cannot guess the intention of the code writer.

Euphoria's implementation does this ...

• When given more right-hand-side (RHS) values than there are LHS variables, the extra values are silently ignored.
• When given more LHS variables than there are RHS values, the extra variables are silently ignored.
• It allows omitted variables in the LHS set, which permits the coder to ignore selected returned values.

This implementation gives capabilities and responsibility to the coder. We are on our way to avoid treating coders as children, in need of protection from themselves.

And this brings us back to the original topic ... PBR, when used responsibly, is as safe as any other coding paradigm. So why not allow it in Euphoria?

On the plus side, PBR can make some coding simpler to write, and can improve the performance of some algorithms. It could also make the porting of code from other languages to Euphoria easier.

On the down side, PBR will make optimization harder to do and probably make some optimization choices impossible, thus as a side-effect, reducing the current performance of applications.

PBR is never necessary. The current pass-by-value can achieve the required results. Though sometimes this would require more code to be written and may also hamper performance of some things.

The most compelling reason to disallow it in Euphoria is it would transform the language into something that is no longer Euphoria. So that would be a judgment call - an matter of opinion.

Another point, is that the current implementation of Euphoria would have to have a massive and major rewrite due to all the pass-by-value assumptions built into the back-end and the generated IL code. So even if we agreed to allow PBR, it might be impractical to implement it now. Unless someone wants to start a new implementation from scratch.

45. Re: Pass by Reference

• Posted by Shian_Lee Jan 05, 2015
• 2766 views

This is really comforting!

That's the best news I've heard lately.

47. Re: Pass by Reference

• Posted by Steady Jan 05, 2015
• 2734 views

function foo() 
   return 1, 2, 3, 4 
end function 
 
a, b, c = foo() 

function foo() 
   return {1, 2, 3} 
end function 
 
{a, b, c, d} = foo() 

First of all Euphoria is promoted as a language that handles sequences. At the time of its original implementation there were very few languages addressing the problem of manipulating and working on sequences and Euphoria and APL were notable exceptions. So the sequence handling ability and emphasis of Euphoria should be emphasised to the newcomer so he knows he is undertaking an excursion into manipulating sequence and he chooses euphoria with open Eyes.
Then it becomes obvious that he should not be encouraged (in the examples he can access) or in the manual to

  return 1, 2, 3, 4 

and do instead

     retvar = 1, 2, 3, 4 
  return retvar 

The next statement or function that wants to use retvar will be forced to look at its size. (BTW retvar would be previously defined as a sequence that would eventually be available.) In any case somebody else has already answered David Cuny that Euphoria allows a mismatch in size of sequence or number of variables by dropping a value or a varname.

49. Re: Pass by Reference

• Posted by dcuny Jan 07, 2015
• 2684 views

Sorry for the late response - I've been busy with other stuff.

Thanks, this clarifies a lot.

- David

51. Re: Pass by Reference

• Posted by Spock Jan 08, 2015
• 2616 views

Question:

For every write (eg, x[i] = n) does the internal routine check to see if "x" has a refcount [and if it has make a fresh copy] before setting the data?

Spock

53. Re: Pass by Reference

• Posted by Spock Jan 08, 2015
• 2632 views

The check might be quick but it's still overhead [that could be eliminated by some radical thinking]. For operations such as, eg sorting, the performance is bound to suffer somewhat.

If there were no refcounts but there was a smarter parser lots of write operations could, I'm sure, be inlined. An extreme example would be x[i] = n resolving to a couple of machine operations.

But this would mean .. a massive rewrite.

Nah, forget I said anything.

Spock

55. Re: Pass by Reference

• Posted by mattlewis (admin) Jan 09, 2015
• 2535 views

Yes, it would.

Matt

57. Re: Pass by Reference

• Posted by petelomax Jan 09, 2015
• 2554 views

Yes, though quite a few bugs have been fixed in that area but never released, and the next one is still some months off.
When it is a local variable, since that can never be seen/referenced by the code being called,

  local = somefunc(...,local,...) 

the lhs gets set to <unassigned> over the call. Similar handling exists for

  local[i][j] = somefunc(...,local[i][j],...) 

though in that case the element gets set to 0, and there is also multiple assignment to consider.

There is a pointless benchmark/test case I have somewhere showing a 400-fold performance gain, though to be honest real-world gains are likely to be extremely rare, and tiny.

Pete

59. Re: Pass by Reference

• Posted by Ekhnat0n Jan 12, 2015
• 2447 views

Let me summarize the discussion:

Both passing by value AND by reference would be a valuable addition to Euphoria
why not write code named:
PassByVal(any code)
AND
PassByRef(any code). And again like my question in the strengths of Euphoria:
Why not ask Robert Craig about his opinion?

61. Re: Pass by Reference

• Posted by mattlewis (admin) Jan 12, 2015
• 2465 views

Except that we do have optional parameters now, so there is some flexibility.

I'm a little confused by this stuff. You're using the "naked" syntax here, which we don't use. In our implemented syntax, there's a big hint, I think, that you're discarding stuff. IOW:

a = foo() 

Nothing being discarded (unless the user does it explicitly later).

{a,?,c} = foo() 

This time, b got discarded.

{a} = foo() 

Everything after a is gone. We don't know how much by looking, but the braces are a hint that we're getting a sequence back and only keeping the first element (which could be all there is).

That, I think, is a major drawback to your proposed "naked" syntax.

Now I'm really confused. The naked stuff seems to make things implicit. It seems pretty explicit the way it is right now. I suspect I'm not getting what you're trying to say.

Matt

63. Re: Pass by Reference

• Posted by mattlewis (admin) Jan 12, 2015
• 2440 views

Yes, I didn't even consider the return statement, since there were none in your examples.

Yep, I'm tracking now. Not convinced of the value of this yet, however...

Matt

65. Re: Pass by Reference

• Posted by jimcbrown (admin) Jan 12, 2015
• 2462 views

Agreed.

Preaching to the choir.

function foo(integer a, integer b, integer c) 
    return 12 
end function 
 
? foo() 

<0236>:: 'foo' needs 3 arguments 
? foo() 

Though you could have optional parameters like so:

function foo(integer a = 1, integer b = 2, integer c = 3) 
    return 12 
end function 
 
? foo() 

and try to run it, this is the result you get:

0 

<xxxx>:: 'foo' returns 2 values 
this = foo(this, 11) 

Actually, it'd be easy enough to require, and have the parser enforce, when desequencing is used with a function, that the return statement of the function be of the form "return {a, b, ...}" where the curly braces are not optional. (So you wouldn't be able to desequence a function that had "return repeat(0, 10)" for example.) Then on the other side, count out the number of values that are being desequenced in the LHS, and require any values that are being skipped to be explicitly represented with a question mark. So you'd have to have "{this, ?} = foo(this, 11)" or else it's an error. That takes care of the logic errors from "{this} = foo(this, 11)" ...

But what of "this = foo(this, 11)" ? This is how that logic error could be detected: if the function is ever used once by a desequencing operation, the parser keeps track of that and at the end goes back and spits outs an error on "this = foo(this, 11)" if it sees "{that, ?} = foo(that, 12)" somewhere else. Only if the function is never used by a desequencing call do we then allow "this = foo(this, 11)" to be valid.

You're right - it is easier to just have the developer specify intent and let the parser enforce it. But I think we can still keep it within desequencing:

function foo(sequence this, integer update_value) 
    integer prior_value = this[1] 
    this[1] = update_value 
    return using desequencing with size 2 values {this, prior_value} -- we can come up with a better syntax than this 
end function 
 
object spare 
sequence this = {50} 
this = foo(this, 11) -- parser blows up here since we aren't using desequencing, but we stated that we intended the caller of this function must use it. 
{this} = foo(this, 11) -- parser blows up here since it can tell that there's a size mismatch 
{this, ?} = foo(this, 11) -- parser allows this 
{this, spare} = foo(this, 11) -- parser allows this 

That's a really good point. Maybe we can just make the changes to the parser to allow multiple return values - so it looks the same as the other languages and the parser can check intent - but have it implemented behind-the-scenes by replacing it with the appropriate sequence creation and desequencing calls (once the parser has validated everything of course)...

67. Re: Pass by Reference

• Posted by jimcbrown (admin) Jan 13, 2015
• 2418 views

Agreed. If we are using desequencing behind the scenes, then getting the routine_id() of a function returning multiple values will just work, but you'd get a sequence back from call_func() and you'd have to manually desequence it.

Works for me.

Not sure why we need one....

But _ is a valid variable name since 4.0.0

69. Re: Pass by Reference

• Posted by ghaberek (admin) Jan 13, 2015
• 2385 views

function foo(sequence this, integer update_value) 
    integer prior_value = this[1] 
    this[1] = update_value 
    return using desequencing with size 2 values {this, prior_value} -- we can come up with a better syntax than this 
end function 
 
object spare 
sequence this = {50} 
this = foo(this, 11) -- parser blows up here since we aren't using desequencing, but we stated that we intended the caller of this function must use it. 
{this} = foo(this, 11) -- parser blows up here since it can tell that there's a size mismatch 
{this, ?} = foo(this, 11) -- parser allows this 
{this, spare} = foo(this, 11) -- parser allows this 

It seems like this might be a better syntax.

function foo(sequence this, integer update_value)  
    integer prior_value = this[1]  
    this[1] = update_value  
    return (2){this, prior_value} 
end function 

It can be read aloud as "return two values, this and prior_value."

-Greg

71. Re: Pass by Reference

• Posted by petelomax Jan 13, 2015
• 2439 views

Hmmm. Just some random thoughts:

There is a limit to how far you can or should mollycoddle the developer, but let's not get into a fight over that.

Not all functions return a fixed number/length of results. Some, of course, return an integer on error and a sequence on success.

One thing about

s = foo() 
{x,y,z} = s 

is that it can give extra debugging info.

If we say s = foo() generates an error, what about {s} = {foo()} and s = {fee(),foo()} and g(foo()) and return foo() and... ?

OpenEuphoria (wrongly, imo) also allows foo(), ie implicit discard of all results, without any warning. In Phix I use {} = foo() as the explicit discard.

Regarding extra decorations on function definitions, may I suggest:

function f() 
with multiple_returns 

or

function f() 
with forced_desequence 

or

  return[2] {new,old} 
end function 

or even

  return[2..8] <result built with append etc> 
end function 

any of which means there would be no extra errors or warnings for any legacy code, a definite plus point.

FYI, Phix (already) currently does this:

function f2() 
    return {1,2} 
end function 
 
integer a 
 
--  a = f2() 
--    ^ type error (storing sequence in atom) 

Needless to say, Phix already deduces function return types and therefore I am not as keen on explicit return types as some might be.

Were those thoughts random enough for you?
Pete

73. Re: Pass by Reference

• Posted by dcuny Jan 13, 2015
• 2346 views

I disagree for a number of reasons. (Of course, that doesn't mean I'm right ).

First, there's no need for adding square brackets. The syntax is already distinct enough for humans and compilers to recognize. Language like Lua and Python already demonstrate that.

Second, it breaks the existing idiom. We already have:

x = foo() 

Adding a comma extends the existing syntax:

x, y = foo() 

Whereas using the square brackets forces the user to use a new syntax without any obvious value.

After all, if you wrote by mistake:

x = foo() 

At compile time, the parser is going to throw an error like:

Error: Return count mismatch: foo() returns 2 values, but only assigns 1 
x = foo() 

Third, it uses an existing feature of syntax in an unrelated way. The square brackets already have a semantic meaning in Euphoria. Now you're overloading them with a new, unrelated meaning.

Hrm... I wonder why Creole things this paragraph needs to be indented.

Anyhoo, I think I'm going to take back my statement about how call_func() should operate. Given that the parser doesn't let the code compile if there's a mismatch, and call_func() throws an exception if there's a parameter length mismatch, I think the parameter count match should be enforced at runtime - that is, the left and right sides should be equal.

- David

75. Re: Pass by Reference

• Posted by mattlewis (admin) Jan 13, 2015
• 2325 views

I agree there's no need. Right now, it strikes me as a good idea because it's helping the person understand that something different is going on.

x = foo() 

x, y = foo() 

I'm seeing the square brackets as something more like the desequencing operation. And makes it more obvious to me that we're expecting multiple return values. Again, I like the emphasis. I'm not a fan of the current trend of (what seems to me) sneaky syntax that does a lot more than is obvious. This is a relatively mild example, but still.

Yes, this is where I probably am closest to agreeing with you. I just wanted something different from sequence brackets. Parentheses were obvious non-starters.

Yes, I don't see any other way. Well, in some cases we could detect the target function and deal with it right away, but we'd still need to do something at runtime when we can't.

Matt

77. Re: Pass by Reference

• Posted by mattlewis (admin) Jan 14, 2015
• 2319 views

I've returned multiple values in euphoria, and I've never felt the need to not pass them as a sequence.

Seriously, though, putting my Raymond Chen hat on and starting this feature at -100 points, I'm not seeing the need for another way to pass and receive multiple values.

I think someone was saying something like that, but I don't think it was me.

Matt

79. Re: Pass by Reference

• Posted by mattlewis (admin) Jan 14, 2015
• 2298 views

Yeah. I'm just not feeling it yet for this particular thing. YMMV

Matt

81. Re: Pass by Reference

• Posted by dcuny Jan 14, 2015
• 2248 views

Actually, it shouldn't even compile.

Hrm... Obviously, this is a bit more work for forward references, too. Not a big deal, but not to be forgotten about.

Because the compiler ensures the counts match, the only situation where a crash should be possible would be:

a, b, c = call_function( routine_id("foo"), {}) 

I'd initially agreed that this shouldn't be an error, but I no longer think that's the case.

Mainly, I figure that if you're trying to read more values back than a routine returns, it's a pretty strong hint that perhaps something is wrong.

After all, the parser treats it as a fatal error.

You've still go the de-sequencing operator to fall back to if you want to handle multiple values. But my assumption is that a routine should return a fixed number of parameters.

If you're just trying to avoid having Euphoria throw an exception, I figure better exception handling is the path to take.

Could you give an example of where you think you might encounter this use case?

- David

83. Re: Pass by Reference

• Posted by Spock Jan 14, 2015
• 2250 views

a, b, c = call_function( routine_id("foo"), {}) 

Just my 2c. If the requirement for multiple return values were driven largely by returning error/status codes then the suggestion (of mine in the Try/Catch thread) of using an implicit error variable would reduce a lot of this need. Therefore, that seems to simplify the issue to mere desequencing.

Spock

85. Re: Pass by Reference

• Posted by dcuny Jan 14, 2015
• 2241 views

With disparate values, why would you want to use multiple returns here? Wouldn't this make more sense (ignoring keywords, of course):

sequence result = get_operand() 
switch result[1] do 
   case P_REG then 
       { op, size, flags } = result 
   case P_VAR then 
       { op, size, n, type } = result 
   case P_MEM then 
       { op, size, scale, index, base, offset } = result 
   case P_LBL then 
       { op, lblidx } = result 
   case P_IMM then 
       { op, atom } = result 
end case 

When different kinds of things are returned, using a sequence and the de-sequence operation makes sense. I'm certainly not advocating getting rid of that.

No compile time error to be had, if f() is returning a single value. No runtime error, either... Assuming f() is returning a sequence.

On the other hand, if you'd written:

a,b[1],{c[2..3],{d,?,e[4]}} = f() 

and f() doesn't returns two values, you've got a compile time error.

- David

87. Re: Pass by Reference

• Posted by petelomax Jan 15, 2015
• 2194 views

sequence result = get_operand() 
switch result[1] do 
   case P_REG then 
       { op, size, flags } = result 
   case P_VAR then 
       { op, size, n, type } = result 
   case P_MEM then 
       { op, size, scale, index, base, offset } = result 
   case P_LBL then 
       { op, lblidx } = result 
   case P_IMM then 
       { op, atom } = result 
end case 

Yes, but you can also, and indeed in places the current code actually does, do something like this

{ op, size, reg } = get_operand(P_REG) 

or this

{ op, size, scale, index, base, offset } = get_operand(P_MEM) 

In Phix, you get a fatal error if there aren't enough values, whereas OpenEuphoria just quietly leaves some values as-is, and on both, no messages if there are too many.

If anything, slightly tongue in cheek, it would be nice for

{ P_REG, size, reg, $ } = get_operand(P_REG) 
{ P_MEM, size, scale, index, base, offset, $ } = get_operand(P_MEM) 

to check, because P_REG/P_MEM are constants, that res[1] is the expected value and deliver a fatal runtime crash if not, and (from the ",$") check that the exact number of values was returned.

Pete

89. Re: Pass by Reference

• Posted by dcuny Jan 15, 2015
• 2242 views

Sorry, I must not have been clear about this.

Similar to the de-sequencing operation, you don't need to use the value - you just need to have the count match.

So I would expect the following to be legal if value passed back two values:

?,v = value("1234") 

But it doesn't - it returns a sequence. Changing how value() behaves at this point would break a lot of code, so I don't see an easy way to do this without the de-sequence operator.

What I'm asking for is a level of compile-time safety. Obviously, there's no way to know the number of parameters that are being passed back with a sequence until runtime. I assume that with 4.0 you are (supposed to) be able to handle this with:

{?,v} = value("1234") 

Similarly:

i,,, = int_to_bytes(999) 

should be valid as well... If int_to_bytes actually passed back two values.

The distinction I'm trying to make is that a function has the option of passing back multiple values. You'd only do this for the purpose of additional compile-time safety.

Or maybe I'm misunderstanding that you'd like the comma syntax with backward compatibility? For that, I'll go back to my initial point: you're still only returning a single value under the old system.

- David

91. Re: Pass by Reference

• Posted by petelomax Jan 15, 2015
• 2151 views

... I think this must have got lost in all the shouting:

I took a couple of minutes to check, and in my own particular parlance:

• DoRoutineDef() has to initialise a few new static vars/flags
• DoReturn() has to check opstack for opMkSq,N or an all-literal-const-sequence and set those flags/lengths accordingly
• DoRoutineDef() has to save those vars/flags permanently in the symtab, and post-validate any prior fwd calls
• MultipleAssignment() checks opstack for a return var, and either
  • logs the forward call for DoRoutineDef() to later process, or
  • validates things there and then

As I see it, the only sticky points are
firstly that one = f() is just a plain (sequence) assignment and gets no validation whatsoever, but anything with the {one,...} = f() syntax gets checked, and
secondly that any return not of the return {...} syntax precludes any and all checks on calls-with-multiple-assignment to that particular routine.
Third and last, {} = f() does not trigger a mismatch, as it is an explicit discard.

Pete

93. Re: Pass by Reference

• Posted by petelomax Jan 15, 2015
• 2058 views

You are assuming that return 1,2,3 and a,b,c=f() has been accepted while I am assuming it has been rejected. I still don't feel the benefits have been proved, or have been so vaguely described (or more often perhaps using you cannot when I think you can and therefore not counted) that I don't think I could even list them, at least not distinct from a still-{}-populated world. Admittedly my last post had a missing implied "if you adhere to this style, maybe we could have some useful compile-time messages" theme, sorry about that. Plus, the fact there would be two semantically different forms of multiple assignment, that might happen to be implemented identically under the hood, might just be a little too difficult for me. I've only just grasped the full extent of the call_func() issue and yes that gets horribly messy because it has to drag all this stuff out of the parser and right into the runtime/translator, for very little benefit.

Pete

95. Re: Pass by Reference

• Posted by jimcbrown (admin) Jan 16, 2015
• 2035 views

Actually, there is a dev team, but before outright rejecting anything we typically take a vote... (which hasn't happened yet afaik).

OTOH, features can still be discussed in the future, after a rejection... even if something is rejected the first time around, the additional discussion might lead to it being accepted the second time around.

I'm glad you're making the effort (and I think you're quite good at it).

Erm, is there a better way to do it?

Well, dcuny was the one who pointed out that if we adhere to Lua-style multiple assignment, then we'd have some useful compile-time messages - so we'd fail faster (parser/compile time error instead of a run-time error or worst of all, a potentially hard to catch logic bug), and perhaps it'd be easier on the coder (less time required to scrutinize all those desequencing calls to make sure no one is doing anything silly).

I did point out that it's possible to get the same benefit in a different way, but, OTOH I find dcuny to be very persuasive...

If it were added, multiple assignment would probably be implemented under the hood in OE this way - via desequencing - so the only difference would need to be at the parser level. There's still complications with call_func() perhaps, but see below..

If desequencing simply mandated that the LHS and the RHS must have the same number of elements, or else it's a run-time error, then simply using desequencing with call_func() would be able to catch most of the errors that the parser's multiple assignment scanner would. Hmm.

97. Re: Pass by Reference

• Posted by jimcbrown (admin) Jan 16, 2015
• 2069 views

I guess this depends on the implementation.

p = {1,2,3} 
{i,j} = p 

Let's take that for the sake of argument.

{i,j} = call_func(...)  -- (oi, there be 3 values there, mate) 

It's ugly (and I mean really ugly), but it's possible to make the desequencing operator aware that the value comes from a call to call_func(), and behave differently in that case. So then,

{i,j} = p -- ok, ignore the third 
{i,j} = call_func(...)  -- run-time error 
 
function identity(object o) 
return o 
end function 
 
{i,j} = identity(call_func(...))  -- a trap! no run-time error!!! 

p = call_func(...)      -- (oi, where's me multiple assignment) 

You got me. This one I don't know how to handle....

I hadn't even thought of this. I was thinking of ways to do it without having to dynamically determine the difference at run-time (by having it all turn into desequencing at run-time, and then adding some special cases to desequencing to make it look more like multiple assignment).

99. Re: Pass by Reference

• Posted by petelomax Jan 17, 2015
• 2150 views

-- test.ex 
without inline 
 
function foo() 
	return {1, 2, 3} 
end function 
 
object a, b, c, d 
 
{a, b, c, d } = foo() 

$ eui test 
 
test.ex:10 
subscript value 4 is out of bounds, reading from a sequence of length 3 
test.ex:10 
Segmentation fault (core dumped) 

ticket #911 raised

Pete