Re: export & public
- Posted by mattlewis (admin) Dec 03, 2008
- 1651 views
The current idea seems to be to include whatever files you want to include, within the files that you include (this is why its inelegant - its moved away from the beautiful simplicity of eu3). This has been explained in various ways (by means of foo, bar.e etc, many times, and yet when it comes to implement it (in Langwar for instance), it just seems to fall over.
I don't think that it falls over at all. It's true that 4.0 (including the standard library) is not fully backwards compatible with 3.1, and it does take a bit of work to migrate code like langwar. First, I'd say that we didn't break things lightly, but that we did look to see where things could really be improved, and where breaking some backwards compatibility was beneficial.
More on why I think the changes to the include system were beneficial below...
Derek's comment, that I was not supposed (or recommended) to include std.e will I suspect become one that will come back to haunt him time and again, as beginners, and lazy programmers (myself included), who simply want the job done quickly, without having to hunt through the st include library to find the function (and dependencies) that they want, will just include std.e
That's certainly one way to do things. It does increase the odds of namespace conflicts, requiring additional work to deconflict those issues. The new standard library is a lot bigger than the old one, and there are conflicts between different files in the standard library, and with some built-ins, such as find().
One of the key advantages of the new scoping and include system is that it's easier to insulate code in one file from interfering with others. It's probably true that your own code in a file is unlikely to conflict in a single application, but this also applies to 3rd party libraries.
Here is a contrived example. Suppose that for some reason, you had a project that used both win32lib and arwen. I haven't looked at arwen closely, but I would be surprised if there weren't some namespace conflicts with win32lib (and if I'm wrong, then imagine that they are two other libraries that you are usingmaybe you need to connect to two different types of database). In 3.1, it was very possible to have two 3rd party libraries conflict with each other. Due to the global scoping of euphoria, the only way to resolve such a problem (remember, the namespace error is occurring within the libraries, not in any code that you wrote!) was to edit one or both of the libraries.
The old eu include system, while it had its faults, was quick, and simple to use, and any namespace problems could be resolved by changing the name of the function, or by using namespace qualifiers. (specifically, put the includes for the entire program in the head of the main program, and then allow allow functions etc, in the main program, and the includes, find the functions etc called below them)
The new way of doing things really isn't much more difficult. It's true that you'll need to put the include statements in each file. But in addition to the benefit of not having 3rd party conflicts, you have the additional benefit of more self documenting code, in that it's clearer what dependencies for a particular file are. It also reduces the potential for namespace conflicts, so code is more likely to 'just work' without having to qualify symbols.
To have to specify the scope of functions, procedures etc within include files, and then to have to specify that the include is public, and it is allowed to access it, and then to make sure add include directives to any sub includes is wrong (IMHO), and certainly a step away from the original goals of euphoria to be a simple to write programming language.
I think that a problem is that the intention of the system hasn't been explained and demonstrated. During development, we noted that we needed to explain and educate a wider audience to understand how the new things worked and to explain the benefits and the drawbacks. That hasn't really happened, though now that we've actually released a version to the public, we need to get moving on that (and here is one instance 
).
The "public include" mechanism is really meant for library developers. First, one assumption behind 4.0 was that the global scope was too open, and invites problems - many of which become apparent mainly in large projects, and especially those that use multiple 3rd party libraries.
While some libraries work well as single files, there are others, such as win32lib, that are easier to maintain and work with if they are multiple files. However, it is often desirable to present a clean interface to consumers of the library. Not all win32lib routines called by a win32lib app actually reside in the Win32Lib.ew file. With global scope, this isn't a problem in exposing functionality to the consumer, though now we can have lots of namespace issues.
Initially (in 4.0 development), we had just the public scope in addition to global. So we could expose a symbol from a file, and also allow the scope of the symbol to propagate through a "public include" statement. This would allow the developer to put API routines anywhere in his multi-file library. But in this system, there is no way to limit the scope of a symbol to only be used within the files of win32lib.
Enter the export scope. This declares symbols that cannot be propagated through additional include statements. So a library author could have internal library implementation that would remain hidden to the consumer of his library, but could be visible to multiple files within his library.
So that's a quick history of how the current scopes came to be, and some basic rationale for their existence.
If you come back and say, that's not how it is, lets explain it to you again, then this just goes to illustrate my point. I certainly do not hope to attain the heady heights of computer programming attained by Derek and Matt, but I am no newby with either programming or euphoria, and I suspect that if I am having problems, then many other people will be too.
I guess I don't understand what you're illustrating here. There are some new concepts, and I agree that we need to communicate better about this. Undoubtedly, the documentation could be improved, and there's probably FAQ or wiki content that could/should be created. For the typical user, however, I think that it's still pretty easy.
If you're just using other people's code (including the standard library) then usually all you need to do is to put a regular old include statement in your file, and then use the code. If you're designing libraries for other people to use, then there are some additional things you need to think about, because 4.0 offers some more powerful tools in the way of scoping the contents of your library.
My request is therefore this - continue with forward referencing of functions and procedures and the language keyword list, but remove the include directives, public and export, and allow multiple level includes includes again, or add another directive that allows the 'sloppy' include method.
The 'sloppy' include method still works. Well, it still works so long as you are using global symbols. After due consideration on the development list, and in the same process that gave us the scopes that we now have, the decision was made to reduce the reliance on global, mainly for the reasons I gave above, but I'm sure there were others.
In short, the cost is that you type some additional include statements. The benefit is that you will eliminate namespace conflicts between 3rd party libraries, and also reduce the frequency of namespace conflicts and the requirement to qualify references in your own code.
Derek asked me if I had downloaded the SVN version - in this instance I think it important that I do not, but I will continue to download, and rigorously punish any alpha release that comes along, as I will have no preconceived notion of what I should and should not be doing, that the devs will have acquired through there day to day involvement with the language.
That's fine. I think Derek's point was that he'd made some fixes in response to your report, and wanted you to test them out to see if they fixed the issues. You're absolutely correct that it's important to get outside reviews and testing. We've knocked down an awful lot of bugs, but there are lots of little corners out there that we probably haven't touched. It's worth noting that we have written lots of unit tests for euphoria 4.0, and continue to add them as issues come up. It's helped us a lot, and will continue to help us.
I would like to stress that the developers have done a great job to get eu 4 to where it is, most of the programs I test run with little or no mods, and some work arounds, (DOS programs too) but that include system is going to be a nightmare. The programs that include files are generally only include level deep, and are the 3.1 includes.
I think that once you get a little more familiar with it, it won't be so bad. We've even got some tools to help migrate to the new standard library. They're in the source directory (dis.ex), and unfortunately not well (or at all!) documented, which needs to be remedied. There is actually an option to have dis.ex generate a graph showing you how all of your include files are related (NB: you'll need to install GraphViz to get the graphs).
Hope I haven't put too many noses out of joint. And of course, the whole development cycle may have gone too far to backtrack, but if this is the case, then I suspect that there will be drop off of current users, or at least a reluctance to leave 3.x behind, despite the advantage that 4 offers.
I think that there will definitely be a reluctance, as there is with every major version of any piece of software. But I think that once people get comfortable with the additions, they'll be very happy with [most] of what euphoria 4.0 has become.
And this has been much more civil than the developer list often becomes.
Matt
