1. Include system

Hi All

I've just had a thought about the include system, well an observation really.

A program in a sub folder of an include file can say something like:
include ..\file_one.e

Which sucessfully includes a file from one directory level up.

But if that file_one.e includes another file from the folder it is in, eg
file_two.e, the interpreter will not be able to locate it.

What is to stop the interpreter automatically searching the folder an include is
actually in for any includes that file looks for?

Or would this break things? (even more blink

I say this because of making library distributions--if you want to keep any
demos in a sub folder you would want to include ..\something.ew (or include
..\include\something.ew) but if that file in turn includes stuff then things get
broken.

Gary

new topic     » topic index » view message » categorize

2. Re: Include system

ags wrote:
> 
> Hi All
> 
> I've just had a thought about the include system, well an observation really.
> 
> A program in a sub folder of an include file can say something like:
> }}}
<eucode>
> include ..\file_one.e
> </eucode>
{{{

> Which sucessfully includes a file from one directory level up.
> 
> But if that file_one.e includes another file from the folder it is in, eg
> file_two.e,
> the interpreter will not be able to locate it.
> 
> What is to stop the interpreter automatically searching the folder an include
> is actually in for any includes that file looks for?
> 
> Or would this break things? (even more blink
> 
> I say this because of making library distributions--if you want to keep any
> demos in a sub folder you would want to include ..\something.ew (or include
> ..\include\something.ew) but if that file in turn includes stuff then things
> get broken.
> 
> Gary

No, it wouldn't break anything.
I beleive this issue is one of the final frontiers for the evolution of  Eu's
physical include system, and I've been suggesting (or wanting to, I can't
remember though I'm sure I mentioned it at least a few times) it for a long time.
I never really pushed it though since it first required being able to use
subfolders for includes, which is only officially possible since 3.0.

I've implemented it in my preprocessor and I'm trying to implement it in the Eu
opensource, if I can ever manage to spend enough time to figure out Robs code.

Here is how it works in my preprocessor to give an idea for those who may not
understand what is being talked about. Maybe someone else with more experience
with the Eu source code can implement it...

All I do in my preprocessor is keep track of the active include folder.

Currently Eu searches for includes
1. in the program folder
2. in the EUINC paths
3. in EUDIR/include path

With relative include paths Eu would search
1. in the active include folder
2. in the program folder
3. in the EUINC paths
4. in the EUDIR/include path

PS.
Just MNSHO, but I don't suggest using ..\ in an include statement, although I
can see the reason, it may not be avoidable and is not of particular concern if
the lib will only be used in a particular project.
Doing so means that the subfolder is bound to the parent folder.
I wouldn't say it's taboo, but it should be considered with the same weight as
using global declarations should.

Sorry, I can't fully explain why it's not desirable, but I can say that you can
do that sort of thing in PHP and I've found it to be problematic. I've been
avoiding having to code that way in PHP for a few years, so the issue is rather
vague now. Take it with a grain of salt.


Chris Bensler
~ The difference between ordinary and extraordinary is that little extra ~
http://empire.iwireweb.com - Empire for Euphoria

new topic     » goto parent     » topic index » view message » categorize

3. Re: Include system

Chris Bensler wrote:
> ags wrote:
> > I've just had a thought about the include system, well an observation
> > really.
> > 
> > A program in a sub folder of an include file can say something like:
> > }}}
<eucode>
> > include ..\file_one.e
> > </eucode>
{{{

> > Which sucessfully includes a file from one directory level up.
> > 
> > But if that file_one.e includes another file from the folder it is in, eg
> > file_two.e,
> > the interpreter will not be able to locate it.
> > 
> > What is to stop the interpreter automatically searching the folder an
> > include
> > is actually in for any includes that file looks for?
> > 
> > Or would this break things? (even more blink
> > 
> > I say this because of making library distributions--if you want to keep any
> > demos in a sub folder you would want to include ..\something.ew (or include
> > ..\include\something.ew) but if that file in turn includes stuff then things
> > get broken.
> > 
> > Gary
> 
> No, it wouldn't break anything.
> I beleive this issue is one of the final frontiers for the evolution of  Eu's
> physical include system, and I've been suggesting (or wanting to, I can't
> remember
> though I'm sure I mentioned it at least a few times) it for a long time. I
> never
> really pushed it though since it first required being able to use subfolders
> for includes, which is only officially possible since 3.0.
> 
> I've implemented it in my preprocessor and I'm trying to implement it in the
> Eu opensource, if I can ever manage to spend enough time to figure out Robs
> code.
> 
> Here is how it works in my preprocessor to give an idea for those who may not
> understand what is being talked about. Maybe someone else with more experience
> with the Eu source code can implement it...
> 
> All I do in my preprocessor is keep track of the active include folder.
> 
> Currently Eu searches for includes
> 1. in the program folder
> 2. in the EUINC paths
> 3. in EUDIR/include path
> 
> With relative include paths Eu would search
> 1. in the active include folder
> 2. in the program folder
> 3. in the EUINC paths
> 4. in the EUDIR/include path

I have no objections in principle to this change.
If you (Chris Bensler) would like to do it, I can give you
developer permissions on SourceForge. Maybe you should
let other people on EUforum comment on it first before
you proceed. You might also want to be more specific about the
algorithm before coding it. Since this is a front-end only issue,
someone could make this change without any knowledge 
of the C backend. This would be a good exercise for someone
who wants to get familiar with modifying Euphoria under 
SVN (subversion) control on SoureForge.

Regards,
   Rob Craig
   Rapid Deployment Software
   http://www.RapidEuphoria.com

new topic     » goto parent     » topic index » view message » categorize

4. Re: Include system

Robert Craig wrote:
> 
> I have no objections in principle to this change.
> If you (Chris Bensler) would like to do it, I can give you
> developer permissions on SourceForge. Maybe you should
> let other people on EUforum comment on it first before
> you proceed. You might also want to be more specific about the
> algorithm before coding it. Since this is a front-end only issue,
> someone could make this change without any knowledge 
> of the C backend. This would be a good exercise for someone
> who wants to get familiar with modifying Euphoria under 
> SVN (subversion) control on SoureForge.
> 
> Regards,
>    Rob Craig
>    Rapid Deployment Software
>    <a href="http://www.RapidEuphoria.com">http://www.RapidEuphoria.com</a>

If I were any good at explaining things clearly, I probably wouldn't feel like I
have Cassandra Syndrome ;)

In my preprocessor, I've implemented the include path search in a fairly
different way than the Eu front end is doing it, so my description may not be
entirely applicable. Instead of building candidate paths to test if a file will
open, as the front end does, my preprocessor pushes and pops the current
directory, which makes the actual search algorithm much simpler although
admittedly probably slower and more prone to error.

I can try though:
On program entry, a variable: PROGRAM_PATH stores the program path as described
by the second subscript of command_line().

The current directory should not be modified or it will affect CLI/drag 'n drop
applications.

Another variable: ACTIVE_PATH stores the path to the file that is currently
being included. Initially it will be the same as PROGRAM_PATH.

1. When an include file is encountered, build the full path to the file being
included.

Determine if an absolute path is specified.
An absolute path begins with a drive letter designation or a SLASH.
Technically, I think it should/could begin with a URL moniker (C:\ , ftp:\\,
http:\\, svn:\\, etc) or a SLASH, but Eu doesn't support networked projects.

If it is absolute path, then no further searching is needed. Just attempt to
open the file, fail if it can't. Proceed to step 2.

If it's not an absolute path, then each directory in the include search path
must be tried in the following order until the file is successfully opened. (note
that I've assumed EUINC is a preparsed list of paths)

A) ACTIVE_PATH & SLASH & INCLUDE_FILE
B) PROGRAM_PATH & SLASH & INCLUDE_FILE
C) EUINC[i] & SLASH & INCLUDE_FILE
D) EUDIR & SLASH & include & SLASH & INCLUDE_FILE

2. push the ACTIVE_PATH onto an include_path stack

3. update ACTIVE_PATH to the path we just resolved for the included file.

4. When an include file is finished being processed and is closed, ACTIVE_PATH
is popped from the include_path stack, restoring the previously active include
directory.

Hopefully that covers it.

----
I have to say Rob, I've noticed a change in your disposition/attitude lately
since Eu is opensource (as have others) and I like it. I hope you will find that
mine has too.

Chris Bensler
~ The difference between ordinary and extraordinary is that little extra ~
http://empire.iwireweb.com - Empire for Euphoria

new topic     » goto parent     » topic index » view message » categorize

5. Re: Include system

Chris Bensler wrote:
> In my preprocessor, I've implemented the include path search in a fairly
> different
> way than the Eu front end is doing it, so my description may not be entirely
> applicable. Instead of building candidate paths to test if a file will open,
> as the front end does, my preprocessor pushes and pops the current directory,
> which makes the actual search algorithm much simpler although admittedly
> probably
> slower and more prone to error.
> 
> I can try though:
> On program entry, a variable: PROGRAM_PATH stores the program path as
> described
> by the second subscript of command_line().
> 
> The current directory should not be modified or it will affect CLI/drag 'n
> drop
> applications.
> 
> Another variable: ACTIVE_PATH stores the path to the file that is currently
> being included. Initially it will be the same as PROGRAM_PATH.
> 
> 1. When an include file is encountered, build the full path to the file being
> included.
> 
> Determine if an absolute path is specified.
> An absolute path begins with a drive letter designation or a SLASH.
> Technically, I think it should/could begin with a URL moniker (C:\ , ftp:\\,
> http:\\, svn:\\, etc) or a SLASH, but Eu doesn't support networked projects.
> 
> If it is absolute path, then no further searching is needed. Just attempt to
> open the file, fail if it can't. Proceed to step 2.
> 
> If it's not an absolute path, then each directory in the include search path
> must be tried in the following order until the file is successfully opened.
> (note that I've assumed EUINC is a preparsed list of paths)
> 
> A) ACTIVE_PATH & SLASH & INCLUDE_FILE
> B) PROGRAM_PATH & SLASH & INCLUDE_FILE
> C) EUINC[i] & SLASH & INCLUDE_FILE
> D) EUDIR & SLASH & include & SLASH & INCLUDE_FILE
> 
> 2. push the ACTIVE_PATH onto an include_path stack
> 
> 3. update ACTIVE_PATH to the path we just resolved for the included file.
> 
> 4. When an include file is finished being processed and is closed, ACTIVE_PATH
> is popped from the include_path stack, restoring the previously active include
> directory.
> 
> Hopefully that covers it.

That seems reasonable to me.
Maybe others would like to comment.

I don't think efficiency will be an issue here. 
The performance hot spots in the front end are in actions 
that take place for each character (the scanner), or maybe 
each line of source, not each file (unless you call
open() many more times than you logically need to). 
Symbol look up (via hashing) also burns up a fair chunk of time.

> I have to say Rob, I've noticed a change in your disposition/attitude lately
> since Eu is opensource (as have others) and I like it. I hope you will find
> that mine has too.

Thanks. I expect I'll enjoy open source development 
more than closed source. In the past when you raised
this issue, I don't think I expressed any fundamental
disagreement. I simply had other things I was working on
that I thought were more important, so I probably discouraged 
this proposal. In general, I always found it a bit irritating to 
have to spend time discussing side issues that were not relevant 
to the few things I had time for in a given release. 
Now people had better be careful what they ask for, because they may 
be asked to implement it themselves.  smile

If you want to go ahead, I just need your SourceForge id,
so I can give you permission to "check in" source files.
Registration with SourceForge is free.
You can check out the latest source from the Euphoria SVN repository, 
make your changes, test them (well I hope!), and check them back in. 
I've found that TortoiseSVN for Windows is very easy to learn and use. 
If you have any questions, let me know here, so others can also 
learn how to do it. You should read up a bit on SVN if you haven't 
used it before. The concept is quite simple. It lets a group of 
developers work on the same source. It warns you if you try to 
"check in" a file that some other developer has modified 
since you "checked out" that file. The SourceForge site has lots 
of info on it.

   http://sourceforge.net/projects/rapideuphoria

Of course there is no rush on this.
Do it whenever you have time: days, weeks or months from now.

Thanks,
   Rob Craig
   Rapid Deployment Software
   http://www.RapidEuphoria.com

new topic     » goto parent     » topic index » view message » categorize

6. Re: Include system

Hello Chris and Rob,

I just wanted to inject a few comments on some of what is being discussed, and
offer a second opinion on something to enhance this include process.

Taking something from Ruby, which I find quite useful in situations like this. 
Ruby has a global variable that can be modified at run time called $:, it's an
Array (Sequence in Euphoria), which holds a list of Search paths, in which to
find libraries, and ruby irritates through them, to try and find files.  If this
was possible to implement in Euphoria, it may solve a lot of problems dealing
with polluting the system environment table, along with making things modifiable
through execution of things to be included, at least within the Euphoria code.

This to would enhance the ability to avoid problems as what has been having.

Just an idea.

Mario Steele
http://enchantedblade.trilake.net
Attaining World Domination, one byte at a time...

new topic     » goto parent     » topic index » view message » categorize

7. Re: Include system

Chris Bensler wrote:
> 
Hi Chris,

All good on the other points.

> Just MNSHO, but I don't suggest using ..\ in an include statement, although
> I can see the reason, it may not be avoidable and is not of particular concern
> if the lib will only be used in a particular project.
> Doing so means that the subfolder is bound to the parent folder.
> I wouldn't say it's taboo, but it should be considered with the same weight
> as using global declarations should.

Not only would I want to use ..\file.e I'd probably want to use ..\lib\file.e
smile

In the case of a library distribution it would be very helpful to keep the
folder structure cleaner, but you are right, it would make moving the sub folder
harder (or copying any examples elsewhere harder)

I like Mario Steele's idea better (same as in Perl) you can modify the include
path within the program, so if you have (eg) at the top of each example:

include_folder ..\lib

It would be easy to change (or remove) just that one line.

What do you think?

Gary

new topic     » goto parent     » topic index » view message » categorize

8. Re: Include system

Mario Steele wrote:
> 
> Hello Chris and Rob,
> 
> I just wanted to inject a few comments on some of what is being discussed, and
> offer a second opinion on something to enhance this include process.
> 
> Taking something from Ruby, which I find quite useful in situations like this.
>  Ruby has a global variable that can be modified at run time called $:,
> it's an Array (Sequence in Euphoria), which holds a list of Search paths, in
> which to find libraries, and ruby irritates through them, to try and find
> files.
>  If this was possible to implement in Euphoria, it may solve a lot of problems
> dealing with polluting the system environment table, along with making things
> modifiable through execution of things to be included, at least within the
> Euphoria
> code.
> 
> This to would enhance the ability to avoid problems as what has been having.
> 
> Just an idea.
> 
> Mario Steele
> <a
> href="http://enchantedblade.trilake.net">http://enchantedblade.trilake.net</a>
> Attaining World Domination, one byte at a time...

I could see it being problematic. Same sort of maintenance issue as goto.
I personally don't like to base my language design decisions on protecting
programmers from bad programmers though, however I don't see any real benefit.
The only advantage I can see is the ability to designate arbitrary paths external
to the project and the eu environment. Both of which I don't beleive is good
programming practice.

Can you give me a practical example of how you would use $: ?

You did remind me of a similar idea I have had though, which is to provide
command line options to be able to specify the eu environment for any specific
instance of the interpetter. Addionally, a config file option could be provided
to be able to script the Eu environment setup.
This would basically provide the same ability as $:, but a person could rely on
the configurations they see being governing, instead of having to inspect the
entire source code to be sure of what is happening.

Would that be satisfactory?


Chris Bensler
~ The difference between ordinary and extraordinary is that little extra ~
http://empire.iwireweb.com - Empire for Euphoria

new topic     » goto parent     » topic index » view message » categorize

9. Re: Include system

> Subject: Re: Include system
> 
> 
> posted by: Chris Bensler <bensler at nt.net>
> 
> ags wrote:
> > 
> > Hi All
> > 
> > I've just had a thought about the include system, well an observation
> > really.
> > 
> > A program in a sub folder of an include file can say something like:
> > }}}
<eucode>
> > include ..\file_one.e
> > </eucode>
{{{

> > Which sucessfully includes a file from one directory level up.
> > 
> > But if that file_one.e includes another file from the folder it is in, eg
> > file_two.e,
> > the interpreter will not be able to locate it.
> > 
> > What is to stop the interpreter automatically searching the folder an
> > include
> > is actually in for any includes that file looks for?
> > 
> > Or would this break things? (even more blink
> > 
> > I say this because of making library distributions--if you want to keep any
> > demos in a sub folder you would want to include ..\something.ew (or include
> > ..\include\something.ew) but if that file in turn includes stuff then things
> > get broken.
> > 
> > Gary
> 
> No, it wouldn't break anything.
> I beleive this issue is one of the final frontiers for the evolution of 
> Eu's physical include system, and I've been suggesting (or wanting to, I
> can't remember though I'm sure I mentioned it at least a few times) it
> for a long time. I never really pushed it though since it first required
> being able to use subfolders for includes, which is only officially
> possible since 3.0.
> 
> I've implemented it in my preprocessor and I'm trying to implement it in
> the Eu opensource, if I can ever manage to spend enough time to figure
> out Robs code.
> 
> Here is how it works in my preprocessor to give an idea for those who may
> not understand what is being talked about. Maybe someone else with more
> experience with the Eu source code can implement it...
> 
> All I do in my preprocessor is keep track of the active include folder.
> 
> Currently Eu searches for includes
> 1. in the program folder
> 2. in the EUINC paths
> 3. in EUDIR/include path
> 
> With relative include paths Eu would search
> 1. in the active include folder
> 2. in the program folder
> 3. in the EUINC paths
> 4. in the EUDIR/include path
> 
> PS.
> Just MNSHO, but I don't suggest using ..\ in an include statement,
> although I can see the reason, it may not be avoidable and is not of
> particular concern if the lib will only be used in a particular project.
> Doing so means that the subfolder is bound to the parent folder.
> I wouldn't say it's taboo, but it should be considered with the same
> weight as using global declarations should.
> 
> Sorry, I can't fully explain why it's not desirable, but I can say that
> you can do that sort of thing in PHP and I've found it to be problematic.
> I've been avoiding having to code that way in PHP for a few years, so the
> issue is rather vague now. Take it with a grain of salt.
> 
> 
> Chris Bensler
> ~ The difference between ordinary and extraordinary is that little extra
> ~
> http://empire.iwireweb.com - Empire for Euphoria
> 
> 

If I understand Chris' proposal correctly,, nothing would be broken.
ACTIVE_PATH would be set to the demo folder, but program folder wouldn't
change. So, if ..\file_one.e includes file_two.e, the attempt to search
ACTIVE_PATH will fail, but using the program folder will succeed as it
did when finding file_one.e

A safaty net could be to try all the include path stack items, but this
shouldn't be necessary I think.

As for using .. in include statements, it sure makes the library design
more rigid, since - for instance - the demo will always have to be in a
subfolder of the main include files folder. My take on this would be to
have environment based symbolic names available for inclusion. Like

system("set main_folder=..\\",2)
include $main_folder\file_one.e


The $ syntax exists in other languages; I'd go along with about any
other scheme which doesn't require too much typing, as Eu often forces
to.

CChris
-- 
  
  cchris005 at fastmail.fm

-- 
http://www.fastmail.fm - The way an email service should be

new topic     » goto parent     » topic index » view message » categorize

10. Re: Include system

cchris005 wrote:
> A safaty net could be to try all the include path stack items, but this
> shouldn't be necessary I think.
The flaw in logic here is that you might want to reorganise you app into 
subfolders, but say win32lib may also be reorganised into subfolders, and
it will all fall down if they can't co-exist.

Regards,
Pete

new topic     » goto parent     » topic index » view message » categorize

11. Re: Include system

Something I wanted to mention that is closely related to this topic is
the fail-safe default include path that Eu uses on windows when all the searches
fail.

I beleive it's "c:\euphoria" or "/euphoria".

For one thing, this has caused me grief in the past until I realized what Eu was
doing. As far as I know it's not documented.

Second, I beleive I have a better solution.
When developing my preprocessor and dealing with this situation, I didn't like
the idea of a hardcoded default. My solution was to derive the default include
path from the directory the interpetter was called from.
Eg. the path specified in the first subscript of command_line()
For all intent and purpose, I think this works the same as now, but with the
additional flexibility of using an educated guess instead of an assumption.

Maybe even better would be to just eliminate the default behaviour and have it
fail at that point.

Is there something I'm missing here that makes the hard coded default useful?

Chris Bensler
~ The difference between ordinary and extraordinary is that little extra ~
http://empire.iwireweb.com - Empire for Euphoria

new topic     » goto parent     » topic index » view message » categorize

12. Re: Include system

Chris Bensler wrote:
> Something I wanted to mention that is closely related to this topic is
> the fail-safe default include path that Eu uses on windows when all the
> searches
> fail.
> 
> I beleive it's "c:\euphoria" or "/euphoria".
> 
> For one thing, this has caused me grief in the past until I realized what Eu
> was doing. As far as I know it's not documented.

The last directory searched is always %EUDIR%\include
(or equivalent on Linux/FreeBSD) i.e. the standard Euphoria libraries, 
that most programs depend on.

The "fail-safe" that you are thinking about only
comes into play when EUDIR has not been set.
In this case, rather than having most programs fail to run at all, 
I set my internal value of EUDIR to "\\EUPHORIA", and hope for the best.
I only did this to help the poor newbie, who has failed
to install Euphoria properly, to at least get something working 
before he curses at Euphoria and moves on to some other language.
I don't think this is something that people should rely on,
so I didn't put it in the official language definition.

Regards,
   Rob Craig
   Rapid Deployment Software
   http://www.RapidEuphoria.com

new topic     » goto parent     » topic index » view message » categorize

13. Re: Include system

On Wed, 13 Dec 2006 19:17:07 -0800, Chris Bensler
<guest at RapidEuphoria.com> wrote:

>
>With relative include paths Eu would search
>1. in the active include folder
>2. in the program folder
>3. in the EUINC paths
>4. in the EUDIR/include path
>
FYI, In Positive I use a permanent table thus:
Initially, load it with:
  EUDIR/include
  EUINC
 (for Positive only, a /builtins entry)
  rootdir

where rootdir is current_dir() and/or command_line[2].

I also set an integer ('alwaysactive') to the number of used entries,
and setup a sister array 'active', which is used as booleans but
actually contains file numbers. The table is searched backwards:
for i=length(filepaths) to 1 by -1 do
        if i<=alwaysactive or active[i] then
            fn=open(filepaths[i]&name,"r")
            if fn!=-1 then
                thispath=filepaths[i]
                exit
            end if
        end if
    end for

The main include file additionally looks in PATH, after/if the above
scan fails.

After successfully opening a file I cleanup the path&name, moving any
relative directory info out of name (raw text from the include
statement) onto the end of thispath, check if lower(thispath) is
already in filepaths, if not add it, and set active[i] to file no.

On EOF, active[i] (found by searching for current file no) is set to 0

This filepath table is kept forever, and even ends up in the .exe for
possible appearances in the ex.err file.

I can tell you the above works beautifully, and passes every test I
have yet devised for it. Hopefully that dots a few i's for you.

BTW, can you do anything with command_line()? It really ought to use
full path names, eg if I run exw bench\bench.exw, then it creates the
file C:\test\bench\ex.err containing "bench\bench.exw:80" so of course
Edita tries to open C:\test\bench\bench\bench.exw...

>Just MNSHO, but I don't suggest using ..\ in an include statement, 
Indeed. It works fine until you want to move something. I guess the
fact that includes with any relative path info in them would
previously only work if the include file contained no nested include
statements (except, perhaps EUDIR/INC) meant you couldn't organise
sources into directories, once you can, it is probably inevitable to
start doing this. There also seems to be some analogy with real life
and the notion of your parents always being there that makes ../
harder to process properly in your brain that /xx.

I've got several dozen of these things now, cause it is such a good
'quick fix', but I kinda know they may cause me grief one day, and a
few of them already have (when I reorganised stuff).

Regards,
Pete

new topic     » goto parent     » topic index » view message » categorize

14. Re: Include system

> Subject: Re: Include system
> 
> 

Replying to both Pete and Chris

> 
> 
> posted by: Pete Lomax <petelomax at blueyonder.co.uk>
> 
> cchris005 wrote:
> > A safaty net could be to try all the include path stack items, but this
> > shouldn't be necessary I think.
> The flaw in logic here is that you might want to reorganise you app into 
> subfolders, but say win32lib may also be reorganised into subfolders, and
> it will all fall down if they can't co-exist.
> 
> Regards,
> Pete
> 
> 

They will coexist if the environment variables don't clobber one
another. Indeed, using "win32lib_main_folder" is better than
"main_folder", and a symbolic, shorter name could be given to the string
representing the decorated env variable name.

CChris
-- 
  
  cchris005 at fastmail.fm

-- 
http://www.fastmail.fm - One of many happy users:
  http://www.fastmail.fm/docs/quotes.html

new topic     » goto parent     » topic index » view message » categorize

15. Re: Include system

>posted by: ags <eu at 531pi.co.nz>
>
>
>Hi All
>
>I've just had a thought about the include system, well an observation
> really.
>
>A program in a sub folder of an include file can say something like:
>
>include ..\file_one.e
>
>Which sucessfully includes a file from one directory level up.
>
>But if that file_one.e includes another file from the folder it is in,
> eg file_two.e,
>the interpreter will not be able to locate it.


As we all know, included files that are expressed in relative terms rather than
absolute (full path) terms are all relative to a set of 'include' paths. These
paths are defined as ...
 
  ** The directory in which the executing program resides.
  ** The list of directories specified in the EUINC environment variable.
** The directory ${EUDIR}/include, where EUDIR is the environment variable
  that specifies where Euphoria was installed.
 
What would be useful, is a way to specify in the include file name that we want
it relative to the file which is doing the including. For example's sake, let's
use the symbol "^" to represent the including file's directory. Then in the case
you mention one could do this ...
 
  ==== FILE_ZERO.EX ====
  include ..\file_one.e  -- one level up from 'here'
 
 
  ==== FILE_ONE.E ===
  include ^\file_two.e  -- same level as FILE_ONE.E
 
IN my case, with Win32lib I'd like to code ...
 
  === WIN32LIB.EW ===
  include ^/w32keys.e  -- same level as *this* win32lib
  include ^/w32alloc.e -- same level as *this* win32lib
  ... etc ...

-- 
Derek Parnell
Melbourne, Australia
Skype name: derek.j.parnell

new topic     » goto parent     » topic index » view message » categorize

16. Re: Include system

Derek Parnell wrote:
> 
> >posted by: ags <eu at 531pi.co.nz>
> >
> >
> >Hi All
> >
> >I've just had a thought about the include system, well an observation
> > really.
> >
> >A program in a sub folder of an include file can say something like:
> >
> >include ..\file_one.e
> >
> >Which sucessfully includes a file from one directory level up.
> >
> >But if that file_one.e includes another file from the folder it is in,
> > eg file_two.e,
> >the interpreter will not be able to locate it.
> 
> 
> As we all know, included files that are expressed in relative terms rather
> than
> absolute (full path) terms are all relative to a set of 'include' paths. These
> paths are defined as ...
>  
>   ** The directory in which the executing program resides.
>   ** The list of directories specified in the EUINC environment variable.
>   ** The directory ${EUDIR}/include, where EUDIR is the environment variable
> that specifies where Euphoria was installed.
>  
> What would be useful, is a way to specify in the include file name that we
> want
> it relative to the file which is doing the including.
<SNIP>

The revision I made addresses this by ammending the search rules to default to
the directory the calling file resides in.

So if test/foo/bar.e includes fubar/foo.e
It will first look for foo.e in test/foo/fubar/
Else look in test/fubar/
Else look in EUINC/fubar/
Else look in EUDIR/include/fubar/

It's always a natural relation.
Is it necessary to enforce the new behaviour, or otherwise make it explicit?

Chris Bensler
~ The difference between ordinary and extraordinary is that little extra ~
http://empire.iwireweb.com - Empire for Euphoria

new topic     » goto parent     » topic index » view message » categorize

17. Re: Include system

Derek Parnell wrote:
> What would be useful, is a way to specify in the include file name that we >
> want
> it relative to the file which is doing the including. For example's sake, >
> let's
> use the symbol "^" to represent the including file's directory. Then in the
> case you mention one could do this ...
>   ==== FILE_ZERO.EX ====
>   include ..\file_one.e  -- one level up from 'here'
>  
>  
>   ==== FILE_ONE.E ===
>   include ^\file_two.e  -- same level as FILE_ONE.E
>  
> IN my case, with Win32lib I'd like to code ...
>  
>   === WIN32LIB.EW ===
>   include ^/w32keys.e  -- same level as *this* win32lib
>   include ^/w32alloc.e -- same level as *this* win32lib

I think with the modifications Chris Bensler made that would happen anyway; the
current folder of the referencing include is the first one searched.

This is satisfactory for now and there is no need to get too complicated about
things as far as I am concerned.

Already with this we can start on a standardised hierarchical include structure
without breaking existing code.

If refinements are needed later on then they can be added.  My main concern was
with the path separator, so if ALL include files use forward slashes then that to
me is cross platform; specific ports can handle the local path separator issue.

So the thing we are worrying about is whether the interpreter searches the
folder of the referencing include file first (and relative paths to that) or
starts in one of the EUINC folders.

Well, what if starting the file name with a slash suppressed the relative
search, so that if you say:

include w32keys.e -- same level as this win32lib, or 1st one in EUINC folder
include /win32/lib/w32alloc.e -- a standard include, no relative search

or

include get.e -- possibly *my* get.e
include /get.e -- *the* get.e
include /std/math/complex.e
include /net/http/useragent/simple.e :)
include /user/b.ryan/wmotor.ew

The issue here is what if you really do want something relative to the root
folder on the filesystem?  Well, should that actually be allowed?

If everything is contained to a standardised 'virtual' file system then I can
see, for example exw and exu going "hmm, I don't see a
/net/http/useragent/simple.e, would you like me to fetch it?"

For end users it means once they install Euphoria they can download any
(possibly post 3.7.1 :) program and not worry about having to chase down include
files.  For developers it means not having to worry about whether the end user
will have (eg) wxEuphoria installed as long as it is in the central repository.

I guess there would be problems with versioning so you'd have to have another
modifier to include as well, like

include /math/bignum.e                  -- the current version
include /gfx/drivers/framebuf.e =9.3.01 -- only this version
include /database/odbc.ew >=4.2         -- greater or = to this version
include /win32/winsock.ew <2.0.1        -- any version less than 2.0.1?

There would also have to be security for this to work, ie digitally signed MD5
checksums (someone wrap PGP? :) etc but I don't see that as being insurmountable
and possibly unneccesary for now.

That's my crazy rant for the end of the year, but I would definitely be into
contributing to a system such as this even if it is just a variation on the above
(like a manual download system).

Gary

new topic     » goto parent     » topic index » view message » categorize

18. Re: Include system

ags wrote:
> 
> Derek Parnell wrote:
> > What would be useful, is a way to specify in the include file name that we >
> > want
> > it relative to the file which is doing the including. For example's sake, >
> > let's
> > use the symbol "^" to represent the including file's directory. Then in the
> > case you mention one could do this ...
> >   ==== FILE_ZERO.EX ====
> >   include ..\file_one.e  -- one level up from 'here'
> >  
> >  
> >   ==== FILE_ONE.E ===
> >   include ^\file_two.e  -- same level as FILE_ONE.E
> >  
> > IN my case, with Win32lib I'd like to code ...
> >  
> >   === WIN32LIB.EW ===
> >   include ^/w32keys.e  -- same level as *this* win32lib
> >   include ^/w32alloc.e -- same level as *this* win32lib
> 
> I think with the modifications Chris Bensler made that would happen anyway;
> the current folder of the referencing include is the first one searched.
> 
> This is satisfactory for now and there is no need to get too complicated about
> things as far as I am concerned.
> 
> Already with this we can start on a standardised hierarchical include
> structure
> without breaking existing code.
> 
> If refinements are needed later on then they can be added.  My main concern
> was with the path separator, so if ALL include files use forward slashes then
> that to me is cross platform; specific ports can handle the local path
> separator
> issue.
> 
> So the thing we are worrying about is whether the interpreter searches the
> folder
> of the referencing include file first (and relative paths to that) or starts
> in one of the EUINC folders.
> 
> Well, what if starting the file name with a slash suppressed the relative
> search,
> so that if you say:
> 
> include w32keys.e -- same level as this win32lib, or 1st one in EUINC folder
> include /win32/lib/w32alloc.e -- a standard include, no relative search
> 
> or
> 
> include get.e -- possibly *my* get.e
> include /get.e -- *the* get.e
> include /std/math/complex.e
> include /net/http/useragent/simple.e :)
> include /user/b.ryan/wmotor.ew
> 
> The issue here is what if you really do want something relative to the root
> folder on the filesystem?  Well, should that actually be allowed?

Yes, because you may have several versions on your HD and want to benchmark 
or otherwise compare something using each of them in turn.

Why not just quote any absolute path?
include get.e -- possibly mine
include /get.e -- no relative search
include "/user/alfred/get.e" -- use this path verbatim
include /"my own get.e" -- disable relative search even with spaces in the 
-- name under Windows (and others?)
include "./my own get.e"" -- use this file from the current directory
include "my own get.e" -- search for it, including in the current directory.


It would be easy for the interpreter, if platform()=DOS, to convert forward 
slashes to backslashes. I understand that all Windows versions accept both.
What about Macs? Perhaps not even needed.

> 
> If everything is contained to a standardised 'virtual' file system then I can
> see, for example exw and exu going "hmm, I don't see a
> /net/http/useragent/simple.e,
> would you like me to fetch it?"
> 
> For end users it means once they install Euphoria they can download any
> (possibly
> post 3.7.1 :) program and not worry about having to chase down include files.
>  For developers it means not having to worry about whether the end user will
> have (eg) wxEuphoria installed as long as it is in the central repository.
> 
> I guess there would be problems with versioning so you'd have to have another
> modifier to include as well, like
> 
> include /math/bignum.e                  -- the current version
> include /gfx/drivers/framebuf.e =9.3.01 -- only this version
> include /database/odbc.ew >=4.2         -- greater or = to this version
> include /win32/winsock.ew <2.0.1        -- any version less than 2.0.1?
> 

Interesting, but possibly complex. This is because you'll have to specify 
somehow what the interpreter is to do if it cant find the right version, or 
what's the update policy.

* = and < are easy: just fail if available file isn't here.
* As for ">=" with a net path, you cant ask the program to check always for the 
latest version: it would make the load time unacceptable. You may want a
"with update" to enable remote version check.
* Now what about >= when the right version isn't here? You can either issue a 
warning ("You may continue, but a couple things may fail") or just fail.
Perhaps a >>= to request failure on insufficent version available?

Oh, by the way, how would the interpreter know about file versions?

> There would also have to be security for this to work, ie digitally signed MD5
> checksums (someone wrap PGP? :) etc but I don't see that as being
> insurmountable
> and possibly unneccesary for now.
> 
> That's my crazy rant for the end of the year, but I would definitely be into
> contributing to a system such as this even if it is just a variation on the
> above (like a manual download system).
> 
> Gary

CChris

new topic     » goto parent     » topic index » view message » categorize

19. Re: Include system

Derek Parnell wrote:
> What would be useful, is a way to specify in the include file name that we 
> want it relative to the file which is doing the including.
Like Chris Bensler I just assumed this was always the case. I can put the 20 or
so win32lib files in say w604/ and include w604/win32lib.ew without making any
changes either to the sources or path/euinc/eudir settings.

Not sure if Chris has released the source mods he made for this yet.
You'll have to let me know if you want a sneak preview of my 0.0.2.

> For example's sake, let's
> use the symbol "^" to represent the including file's directory.
CChris has already beat me to saying that "./" is a much more natural
(and backward compatible) way of expressing this intention. I considered
treating this specially but in the end decided against it.

If two user contributions "include .\series.e" then it would force users to
install both in the same directory and I can foresee a new "dll hell" if
installation must put things in EUINC dirs because it is no longer allowed to
have a private copy of some-but-not-all files.

I might put a handful of useful files in a local common/ and find everything
works fine until a new version of a lib comes out with "./"'s and forces an
unwanted re-(/un-)organisation of my directory structure.

Regards,
Pete

new topic     » goto parent     » topic index » view message » categorize

20. Re: Include system

Pete Lomax wrote:
> 
> Derek Parnell wrote:
> > What would be useful, is a way to specify in the include file name that we 
> > want it relative to the file which is doing the including.
> Like Chris Bensler I just assumed this was always the case. I can put the 20
> or so win32lib files in say w604/ and include w604/win32lib.ew without making
> any changes either to the sources or path/euinc/eudir settings.


I'm not sure if I'm understanding you here. Are you saying that this actually
works with Euphoria 3.0 right now? If so, I can't get it to work. This is the
message I get ...


C:\>exwc C:\projects\eu_proj\win32lib\win32libcurrent\Demo\bitmap.exw
C:\projects\eu_proj\win32lib\win32libcurrent\Demo\bitmap.exw:1
can't find win32lib.ew in C:\projects\eu_proj\win32lib\win32libcurrent\Demo
or in c:\projects\eu_proj\euinc;.\include;..\include
or in C:\EUPHORIA\include
include win32lib.ew
                   


> Not sure if Chris has released the source mods he made for this yet.
> You'll have to let me know if you want a sneak preview of my 0.0.2.


Or are you saying that this is yet-to-be-released update to Euphoria?

> > For example's sake, let's
> > use the symbol "^" to represent the including file's directory.
> CChris has already beat me to saying that "./" is a much more natural
> (and backward compatible) way of expressing this intention. I considered
> treating
> this specially but in the end decided against it.

I must strongly disagree with you on that point.

The path "." is already assigned a meaning. I means relative to the current
working directory of the shell. Thus something like ".\series.e" is saying that
"series.e" is expected to reside in the shell's current working directory.

Also, I read that some are suggesting the paths beginning with "\" should be
interpreted as being relative to the including file. HOwever, once again, this
already has a meaning. When a path begins with a slash, it means that it is
relative to the root directory of the shell's current drive. Thus
"\util\series.e" means that the file "series.e" is expected in "C:\util"
(assuming C: is the current drive) and not relative to the including file.

> If two user contributions "include .\series.e" then it would force users to
> install both in the same directory and I can foresee a new "dll hell" if
> installation
> must put things in EUINC dirs because it is no longer allowed to have a
> private
> copy of some-but-not-all files.

And that is why I see the need for either a new special symbol in the include
statement to indicate 'relative-to-the-including-file', or the search algorithm
must change to do this automatically for all include statements.


-- 
Derek Parnell
Melbourne, Australia
Skype name: derek.j.parnell

new topic     » goto parent     » topic index » view message » categorize

21. Re: Include system

CChris wrote:
> 
> ags wrote:
> > 
> > include get.e -- possibly *my* get.e
> > include /get.e -- *the* get.e
> > include /std/math/complex.e
> > include /net/http/useragent/simple.e :)
> > include /user/b.ryan/wmotor.ew
> > 
> > The issue here is what if you really do want something relative to the root
> > folder on the filesystem?  Well, should that actually be allowed?
> 
> Yes, because you may have several versions on your HD and want to benchmark
> 
> or otherwise compare something using each of them in turn.
> 
> Why not just quote any absolute path?
> }}}
<eucode>
> include get.e -- possibly mine
> include /get.e -- no relative search
> include "/user/alfred/get.e" -- use this path verbatim
> include /"my own get.e" -- disable relative search even with spaces in the 
> -- name under Windows (and others?)
> include "./my own get.e"" -- use this file from the current directory
> include "my own get.e" -- search for it, including in the current directory.
> </eucode>
{{{


Yeah, that looks good.  But I was thinking of it from a security point of view. 
What happens in the interpreter when you include "/etc/passwd" ?
(People are going to want to know these things).

> 
> It would be easy for the interpreter, if platform()=DOS, to convert forward
> 
> slashes to backslashes. I understand that all Windows versions accept both.
> What about Macs? Perhaps not even needed.

I think Mac is :: and no it won't be needed if there is no OS9 port.
> 
> > include /math/bignum.e                  -- the current version
> > include /gfx/drivers/framebuf.e =9.3.01 -- only this version
> > include /database/odbc.ew >=4.2         -- greater or = to this version
> > include /win32/winsock.ew <2.0.1        -- any version less than 2.0.1?
> > 
> 
> Interesting, but possibly complex. This is because you'll have to specify 
> somehow what the interpreter is to do if it cant find the right version, or
> 
> what's the update policy.

Yeah.  I was thinking when I suggested that that it that it might be difficult. 
But if the definition of "current version" really just means what is in the
include folder (with the highest version number) will get used.  I started that
post with "there is no need to make it complex" and suggest something even more
complex.  But ya get that...

> * = and < are easy: just fail if available file isn't here.
> * As for ">=" with a net path, you cant ask the program to check always for
> the 
> latest version: it would make the load time unacceptable. You may want a
> "with update" to enable remote version check.
> * Now what about >= when the right version isn't here? You can either issue
> a 
> warning ("You may continue, but a couple things may fail") or just fail.
> Perhaps a >>= to request failure on insufficent version available?

With with update the programmer codes the behaviour whereas it should really be
in the hands of the end user, so some kind of preference of the EU system
itself...maybe a default policy of asking before downloading, but also including
a large core of includes with the install.

> Oh, by the way, how would the interpreter know about file versions?

Hehe yeah, that's the kicker.  It's easy enough to invent a new standard, but
editing every include in the archive would surely suck.  And I don't think
versioning really works unless you use automated constraint like cvs.

I was just throwing that in from Perl and I don't think I've ever taken that
feature too seriously before (although I'm sure it's great...).

Happy New Year

Gary

new topic     » goto parent     » topic index » view message » categorize

22. Re: Include system

Derek Parnell wrote:
> 
> I'm not sure if I'm understanding you here. Are you saying that this actually
> works with Euphoria 3.0 right now? If so, I can't get it to work. This is the
> message I get ...
> 
> 
> C:\>exwc C:\projects\eu_proj\win32lib\win32libcurrent\Demo\bitmap.exw
> C:\projects\eu_proj\win32lib\win32libcurrent\Demo\bitmap.exw:1
> can't find win32lib.ew in C:\projects\eu_proj\win32lib\win32libcurrent\Demo
> or in c:\projects\eu_proj\euinc;.\include;..\include
> or in C:\EUPHORIA\include
> include win32lib.ew
>                    
> 
> 
> > Not sure if Chris has released the source mods he made for this yet.
> > You'll have to let me know if you want a sneak preview of my 0.0.2.
> 
> 
> Or are you saying that this is yet-to-be-released update to Euphoria?

I am using Chris Bensler's version 3.0.1

But even with that it would do the same thing.  It would try to include
win32lib.ew in the folder of bitmap.exw, otherwise go through the include path
list.

Hey, I just noticed... you have "./include" and "../include" in your EUINC path.
So why isn't that working anyway?

Gary

new topic     » goto parent     » topic index » view message » categorize

23. Re: Include system

Derek Parnell wrote:
> I'm not sure if I'm understanding you here. Are you saying that this 
> actually works with Euphoria 3.0 right now?
See
http://www.listfilter.com/cgi-bin/esearch.exu?thread=1&fromMonth=B&fromYear=B&toMonth=1&toYear=C&keywords=%22Include+system+(implementation)%22

Also, it must be over 8 months that I've had this in Positive, though a new
release of that is several weeks away yet.

> If so, I can't get it to work. This is the
> message I get ...
> 
> C:\>exwc C:\projects\eu_proj\win32lib\win32libcurrent\Demo\bitmap.exw
> C:\projects\eu_proj\win32lib\win32libcurrent\Demo\bitmap.exw:1
> can't find win32lib.ew in C:\projects\eu_proj\win32lib\win32libcurrent\Demo

Actually, that's a bit different, since win32lib.ew is not in the \Demo\ folder.
The test you want (I think) is to change
> include win32lib.ew
to
  include ..\include\win32lib.ew

ditto the final line (23) to include ..\include\w32Start.ew

It compiled cleanly for me using cb's exe, whereas it failed on line 501,
include w32Keys.e under 3.0.0


> > > For example's sake, let's
> > > use the symbol "^" to represent the including file's directory.
> > CChris has already beat me to saying that "./" is a much more natural
> 
> I must strongly disagree with you on that point.
> 
> The path "." is already assigned a meaning. It means relative to the 
> current working directory of the shell.
Hmmm.
> And that is why I see the need for either a new special symbol in the 
> include statement to indicate 'relative-to-the-including-file', or the 
> search algorithm must change to do this automatically for all include 
> statements.
As I said above, I put this in 8 months ago, and I think my concept of current
directory has changed, at least while I'm actually compiling a relative include
file. Obviously I don't mean that an open("icons.xpm")
statement within include xpm/xpmlib.ew should (somehow) open xpm/icons.xpm.
You would still have to clutter up the main project directory with files
like that to keep src & exe behaviour equal. (xpmlib is fictional, btw)

Anyway, play with cb's 3.0.1 a while and see what you think.

> 
> Also, I read that some are suggesting the paths beginning with "\" should 
> be interpreted as being relative to the including file.
...
> Thus "\util\series.e" means "series.e" is expected in "C:\util"
I can accept that.

Regards,
Pete

new topic     » goto parent     » topic index » view message » categorize

Search



Quick Links

User menu

Not signed in.

Misc Menu