OpenEuphoria

2. Re: Enhancements to the include system

• Posted by Pete Lomax <petelomax at blueyonder.co.uk> Dec 14, 2006
• 689 views

CChris wrote:
> 
> I wrote in some detail about enhancements I plan to make to the current 
> include system. This appears on the relevant  EuWiki page in the 
> "Requested functionalities" category as solution #2.
If talking about with package=myLib: var1, proc_xyz,... friend, share, 
restrict, redefine, then imo completely the wrong approach, see 
my comments (3 * "||" markers) on that page.

Regards,
Pete

4. Re: Enhancements to the include system

• Posted by Chris Bensler <bensler at nt.net> Dec 14, 2006
• 719 views

Pete Lomax wrote:
> 
> CChris wrote:
> > 
> > I wrote in some detail about enhancements I plan to make to the current 
> > include system. This appears on the relevant  EuWiki page in the 
> > "Requested functionalities" category as solution #2.
> If talking about with package=myLib: var1, proc_xyz,... friend, share, 
> restrict, redefine, then imo completely the wrong approach, see 
> my comments (3 * "||" markers) on that page.
> 
> Regards,
> Pete

Since you ask..

Sorry, but I have to agree with Pete concerning your solution, it's too
complicated. I don't think I agree with forcing the use of qualifiers though. The
concept of pkgs might be useful, but I'm not sure how at the moment. Mainly I
think the problem of global contamination can be resolved with a much simpler and
robust solution.

I saw your proposed solution before, but refrained from commenting because I'm
much too opinionated, especially regarding this topic and my opinion is rather
biased on the matter.

For those who are lost, the article is here:
<a
href="http://euwiki.ayo.biz/Improved_scoping">http://euwiki.ayo.biz/Improved_scoping</a>

I wrote that article stub btw (with the exception of solutions #1 and #2) and
have a complete plan laid out in my mind yet to be explained, including the use
of chained include statements, the use of include subfolders (already implemented
in 3.0) and relative include paths (likely to be implemented soon) among a few
other things. I'm finishing it in private when I have time, to avoid the
confusion of the different proposals.

In any case, it's clear that there isn't any kind of consensus on how to deal
with global contamination, so I would hate to see any solution rushed into,
including my own. I'm pretty sure Pete's implementation is unique from both of
ours and I can think of at least a handful of other people with their own unique
views on this too.


Pete: I didn't see your description of the implementation for Positive.
Can you reiterate the concept or point me?

Chris Bensler
~ The difference between ordinary and extraordinary is that little extra ~
http://empire.iwireweb.com - Empire for Euphoria

6. Re: Enhancements to the include system

• Posted by Chris Bensler <bensler at nt.net> Dec 15, 2006
• 721 views

Pete Lomax wrote:
> 
> Chris Bensler wrote:
> >
> > Pete: I didn't see your description of the implementation for Positive.
> > Can you reiterate the concept or point me?
> > 
> Rather than hand you a solution on a plate (fairly obviously it is not
> easy to rip some code out of one compiler and stuff it into another),
> the point is more that this can be solved in the spirit of Euphoria, 
> without any strange voodoo.
> 
> First, to recap, there are two associated problems:
> }}}
<eucode>
> include win32lib.ew as w32
> w32:getOpenFileName()
> </eucode>
{{{

> is invalid in Eu because getOpenFileName() is defined in w32file.ew, and
> 
> Derek's infamous alice/bob/chris/diane example, which is (in eric.exw):
> }}}
<eucode>
>    include bob.e
>      include alice.e
>        global constant name = "Alice"
>        <end of alice.e>
>      if name!="Alice" then ?9/0 end if
>      <end of bob.e>
>    include diane.e
>      include chris.e
>        global constant name = "Chris"
>        <end of chris.e>
>      if name!="Chris" then ?9/0 end if -- [**]
>      <end of diane.e>
>    <end of eric.exw>
> </eucode>
{{{

> 
> The point is that diane.e works for eric exactly the same as it did
> when diane created it, rather than crash over a missing namespace
> qualifier at [**]. If eric.exw itself referenced name, then you'd
> get the 'qualifier needed' message, but within diane.e, name gets
> automatically resolved to the one in chris.e because diane.e 
> included that file (and did not include alice.e).
> 
> It no longer matters that diane.e can "see" globals defined in alice.e
> as any defined by it's own includes take automatic preference.
> 
> What I actually wrote (admittedly I thought I'd gone into more detail)
> was, mainly in respose to the first problem:
> 1] Or use Positive, if I ever get round to a new release, which
> handles nested sub-includes, provided, obviously, that there is only
> one valid instance of the global in that include "sub-tree". Anyone
> can see how it works & use any source found to fix this bug in 3.0.0.
> FWIW, relevant code is in psym.e, in functions InTable() and
> makeScanTable(), and use of variable abort_set, though it'll all be
> very different in eu.ex. I should perhaps also admit that Positive's
> routine_id does not (yet) do anything similar with namespaces.
> 
> Feel free to ask me any questions about what you find.
> Regards,
> Pete

Ok, from what I understand there is no change in syntax. Just that when a
collision is encountered, the most closely 'related' definition is used to
resolve?

Well I do like that idea, it makes sense but it still only seems to cover a
portion of the problem. Globals will continue to be abused, namespacing just
hides the problem.

None the less, I think that it would be a good compliment to Eu's namespacing
system, even if it doesn't address the global contamination issue at all.

Personally, I'm interested in a solution that directly addresses the nessecity
to use globals in the first place. If we can reduce the need for globals, we will
put a much larger dent in the problem than by simply combatting the
contamination.

My thinking is long term..
In 5 or 10 or however many years, if or when eu has 100 or 1000x as many
programmers, will a namespace solution be sufficient to deal with global
contamination and still allow people to share code freely without alot of
conflict hassles? Don't get me wrong, I don't think we will ever eliminate global
contamination, but it can be hugely reduced.

The implementation you suggest is I think the only one I've heard which I would
agree should be implemented. It's not going to paint eu into a corner. I don't
see it interfering with future compatability and it's transparent. I still don't
think it actually addresses the issue though, just deals with it.

Chris Bensler
~ The difference between ordinary and extraordinary is that little extra ~
http://empire.iwireweb.com - Empire for Euphoria

8. Re: Enhancements to the include system

• Posted by CChris <christian.cuvier at agriculture.gouv.fr> Dec 15, 2006
• 701 views

Chris Bensler wrote:
> 
> Pete Lomax wrote:
> > 
> > CChris wrote:
> > > 
> > > I wrote in some detail about enhancements I plan to make to the current 
> > > include system. This appears on the relevant  EuWiki page in the 
> > > "Requested functionalities" category as solution #2.
> > If talking about with package=myLib: var1, proc_xyz,... friend, share, 
> > restrict, redefine, then imo completely the wrong approach, see 
> > my comments (3 * "||" markers) on that page.
> > 
> > Regards,
> > Pete
> 
> Since you ask..
> 
> Sorry, but I have to agree with Pete concerning your solution, it's too
> complicated.
> I don't think I agree with forcing the use of qualifiers though. The concept
> of pkgs might be useful, but I'm not sure how at the moment. Mainly I think
> the problem of global contamination can be resolved with a much simpler and
> robust solution.
> 
> I saw your proposed solution before, but refrained from commenting because I'm
> much too opinionated, especially regarding this topic and my opinion is rather
> biased on the matter.
> 
> For those who are lost, the article is here:
> <a
> href="http://euwiki.ayo.biz/Improved_scoping">http://euwiki.ayo.biz/Improved_scoping</a>
> 
> I wrote that article stub btw (with the exception of solutions #1 and #2) and
> have a complete plan laid out in my mind yet to be explained, including the
> use of chained include statements, the use of include subfolders (already
> implemented
> in 3.0) and relative include paths (likely to be implemented soon) among a few
> other things. I'm finishing it in private when I have time, to avoid the
> confusion
> of the different proposals.
> 
> In any case, it's clear that there isn't any kind of consensus on how to deal
> with global contamination, so I would hate to see any solution rushed into,
> including my own. I'm pretty sure Pete's implementation is unique from both
> of ours and I can think of at least a handful of other people with their own
> unique views on this too.
> 
> 
> Pete: I didn't see your description of the implementation for Positive.
> Can you reiterate the concept or point me?
> 
> Chris Bensler
> ~ The difference between ordinary and extraordinary is that little extra ~
> <a href="http://empire.iwireweb.com">http://empire.iwireweb.com</a> - Empire
> for Euphoria

I had thought for a while that using priority rules (defining what closeness 
is and resolving unqualifird symbols to the closest one known) was enough.
Indeed, this is how the specs for Open Euphoria (v1.2) were written. This 
schme definitely looks practical.

However, it was pointed out at the time (by Matt I think) that such a solution 
would induce an unwelcome sensitivity to the shape of the inclusion tree, 
and would make maintaining large libraries more complex because of subtle 
bugs creeping in, while the purpose was to make that very acttivity easier.
This is why the scheme(s) I described hardly rely on closeness rules now.
The only thing I kept was localness (if a global is define in the same 
file, use this symbol). So, while I trust your solution to be well thought, 
I'm afraid that it makes maintainance/evolution of packages needlessly tricky.

Why not chime in in the Wiki?
CChris


PS: I moved the comments in "Improved scoping" to the talk page of that 
article, since I understand it is the very purpose of these talk pages.

10. Re: Enhancements to the include system

• Posted by Pete Lomax <petelomax at blueyonder.co.uk> Dec 16, 2006
• 716 views

Chris Bensler wrote:
> Ok, from what I understand there is no change in syntax. Just that when a
> collision
> is encountered, the most closely 'related' definition is used to resolve?

The main idea is that if something works stand-alone, it will work the same 
when included in a larger application. All the information needed to do that
is embedded in the include statements needed to make it work stand-alone.

Barring, that is, stuff you cannot really ever expect to handle such as
conflicting library version dependencies, say liba only works with
win32lib 0.59.1 but your app or libb only works with >= 0.60.4.

Actually, I have introduced a new class of problem... I've allowed forward
routine calls, and automatic includes of standard things, so you can say

s=sort(s)

without getting "sort has not been defined".
The snaglette I've introduced is that if sort.e is automatically included
by libX standalone but has been overridden by the calling application, I'll
either get the same old namespace qualifier required error, or it may start
to use the sort() defined in the app, when by my rules it should not...
There is even/however a simple fix, write a "sortstub.e" file:

include sort.e
include libX.e

<snip>
>it still only seems to cover a portion of the problem.
<snip>
>even if it doesn't address the global contamination issue at all.
<snip> 
> In 5 or 10 or however many years, if or when eu has 100 or 1000x as many
> programmers,
> will a namespace solution be sufficient to deal with global contamination and
> still allow people to share code freely without alot of conflict hassles?

Well, in a word, yes! There are two types of problem to be dealt with:

a) You have to modify Your code to use libX, or a new version of it.
b) You cannot use libX without modifying it.
   (Case b covers the case of libY defining one or more globals with
    the same name as one(s) defined in libX and applies to libY if
    you were thinking of modifying that to avoid modifying libX.)

I am only talking about solving b). Moaning about a) is, well, just 
moaning .

Regards,
Pete

12. Re: Enhancements to the include system

• Posted by Chris Bensler <bensler at nt.net> Dec 17, 2006
• 706 views
• Last edited Dec 18, 2006

Pete Lomax wrote:
>
> Actually, I have introduced a new class of problem... I've allowed forward
> routine calls, and automatic includes of standard things, so you can say
> }}}
<eucode>
>     s=sort(s)
> </eucode>
{{{

> without getting "sort has not been defined".

I think automatic inclusion is a bad idea.
One of my golden rules for a programming language or most any program for that
matter, is that it should never make assumptions. Assumptions cause confusion at
the very least.

> <snip>
> >it still only seems to cover a portion of the problem.
> <snip>
> >even if it doesn't address the global contamination issue at all.
> <snip> 
> > In 5 or 10 or however many years, if or when eu has 100 or 1000x as many
> > programmers,
> > will a namespace solution be sufficient to deal with global contamination
> > and
> > still allow people to share code freely without alot of conflict hassles?
> 
> Well, in a word, yes! There are two types of problem to be dealt with:
> 
> a) You have to modify Your code to use libX, or a new version of it.
> b) You cannot use libX without modifying it.
>    (Case b covers the case of libY defining one or more globals with
>     the same name as one(s) defined in libX and applies to libY if
>     you were thinking of modifying that to avoid modifying libX.)
> 
> I am only talking about solving b). Moaning about a) is, well, just 
> moaning .

I agree, you should never have to modify third party code. It's one of my
principle philosophies for modular programming.

Here, again I must point out that you are addressing how to better resolve
collisions (while at the same time increasing confusion), not how to prevent them
in the first place (reducing confusion).

Currently the Euphoria codebase is pretty tiny. The more code that is available,
the more collisions there will be. The more collisions there are, the more
namespacing will be required.

Euphoria is very wasteful in it's use of globals. It creates alot of
contamination for no good reason and at the same time forces the programmer to
have to expose certain declarations that are only intended for cross-file use
among the API they are developing. A large portion of global declarations in an
API that consists of multiple files are for internal use only or they are just
index constants for some sequence structure.

Firstly, if Eu had at least some kind of ability to name the members of a
sequence, that would help tremendously. I'm unresolved on how to deal with
structured sequences, but my preference is to implement namespaced codeblocks and
aliases (in fact, if I had my way I would eliminate the include namespace
mechanism in favor of codeblock namespacing) and forego any ability for typed
sequence members. This would work for naming members of a structured sequence by
allowing a declaration in the namespace with the same name to be inherited.

namespace rect
 sequence rect
 alias left   rect[1]
 alias top    rect[2]
 alias right  rect[3]
 alias bottom rect[4]
end namespace

I'm not actually happy with how I've used alias in that example, since in my
mind an alias should simply be a way to create another name for a declaration,
but the example makes it a macro. I'm working on it.

Using namespaced codeblocks, if someone wanted to recreate the include
namespacing that Eu uses now, you would do something like..

namespace FOO
  include myFoo.e
end namespace

Notice that you would also be able to define multiple includes into a single
namespace using that method.

Second, a way to isolate how far a global declaration should extend *by the
developer who declared it*, would add a great deal of control. This has to do
with my idea for chained includes, or global chaining, in other words. It's too
much to explain here.

Third, the ability to gain access to 3rd party declarations that were never
intended to be global would create an additional way that declarations could be
internally shared among files of an API without exposing them globally or forcing
a programmer to modify foreign code. This is more for creating flexibility while
preserving code and is perhaps more of an enhancement than anything to do with
reducing global contamination, but I think it's related. Again, I'm not going to
get into my idea for implementation at the moment. The purpose of this post is to
point out the issues that need to be recognized. I'd like to encourage ideas and
proper resolution, not enforce my own.

I'm probably missing some other possibly better ways that global contamination
could be reduced but these are the ones that I think would work best. Not only
that, but they would enhance the language, not just fix problems.

Chris Bensler
~ The difference between ordinary and extraordinary is that little extra ~
http://empire.iwireweb.com - Empire for Euphoria

14. Re: Enhancements to the include system

• Posted by Pete Lomax <petelomax at blueyonder.co.uk> Dec 24, 2006
• 744 views

On Sun, 17 Dec 2006 13:47:50 -0800, Chris Bensler
<guest at RapidEuphoria.com> wrote:

This message just bounced on me ;-( , so I'll re-try.

(My apologies if you already got this/see it twice)

>I think automatic inclusion is a bad idea.
I think I asked for that particular slap in the face 
>it should never make assumptions.
It is NOT an assumption, it is part of the language definition.
Technically, there is no functional difference between say open()
invoking some machine code in the back-end or auto-including pfileio.e
Personally, it drives me nuts that open is pre-defined but seek is
not, ditto current_dir and chdir, get_key and wait_key, rand and
set_rand, print and sprint...
I guess if 99% of your programs start with eg include win32lib.ew,
then you don't notice this much.

Anyway, I'm quite happy that the gotcha I mentioned is a feature of
*overriding builtins* and is only of concern if such introduces an
*incompatibility*. Onto matters real:

<snip>
>forces the programmer to have to expose certain declarations that are 
>only intended for cross-file use among the API they are developing.
OK, I see.
Now you mention it I tend to cross fingers and hope on that one.
>
>Second, a way to isolate how far a global declaration should extend
>*by the developer who declared it*, would add a great deal of control. 
>This has to do with my idea for chained includes, or global chaining, 
>in other words. It's too much to explain here.
Aha!
I know it is complicated but that is more, not less reason to discuss.
>
>Third, the ability to gain access to 3rd party declarations that were 
>never intended to be global would create an additional way 
>that declarations could be internally shared among files of an API 
>without exposing them globally or forcing a programmer to modify 
>foreign code. This is more for creating flexibility while preserving 
>code and is perhaps more of an enhancement than anything to do with 
>reducing global contamination, but I think it's related. Again, I'm 
>not going to get into my idea for implementation at the moment. The 
>purpose of this post is to point out the issues that need to be 
>recognized. I'd like to encourage ideas and proper resolution, not 
>enforce my own.
>
Umm, even more interesting...

The way I see it is you want to "ringfence" stuff.
You want globals to work normally within this fence.
You want to limit "what gets out"; other globals remain within. 
You *may* want "qualified" access back inside.

How does this strike you:

global scope gsname
...
end scope [gsname] [export g1,g2,g3...]

Within the scope, globals behave normally, after the end scope,
globals specified in the list can be accessed unqualified, those not
in the list can be accessed as gsname:<id> and as you may expect,
non-globals within such a construct cannot be accessed externally,
period.

External view of nested gscopes depends *entirely* on the last export
statement. Nested gscopes cannot "hop over" their container.

<pause here; suggestion complete, what follows is technical>

Putting the export list on the end scope may make life simpler for the
compiler, though I guess it would not be that difficult to save
line/col start/end if the consensus is it looks better on the start.

I believe you could implement such a [nested] scheme quite
efficiently, the cost being that if the symtab lookup runs into an
unqualified global with scope no > 1, then it has to check if that
scope is still open [else skip it the same as a hash collision].
All the export statement has to do is set the scope no to that of the
including parent. The gsname entry in the symtab would be a little
like an existing namespace entry, except global, and ... erm the 
rest is probably a bit trial and error
[symtab needs a new uniq "scope no" field for globals, obvs]

Behaviour of still open gscopes is *undefined*, at least for now.
(ie/eg global scope x ... x:yyy() ... end scope x... the "x:" has no
real meaning within, far as my logic yet goes, and I think correct to
avoid seriously mental games within nested scopes.)

LIkewise, the above remains *theory*, or at least vapourware..

You could of course allow global scope noname to avoid unnecessarily
polluting the global namespace with global scope names 

Regards,
Pete