Euphoria Ticket #43: Simplify the mainteance of maintaining the Euphoria makefiles/build system.

We have tried many things:

  1. Batch Files
  2. Makefiles
  3. CMake
  4. Jam


Type: Feature Request Severity: Normal Category: Build System
Assigned To: jeremy Status: On Hold Reported Release: r2812
Fixed in SVN #: 2813, 2815, 2847 View VCS: 2813, 2815, 2847 Milestone:

1. Comment by jeremy Sep 12, 2009

With the new stuff in euc, we should be able to do this pretty easily.

2. Comment by jeremy Sep 12, 2009

Current line count of our Makefile system:

    519    1507   17311 source\Makefile.gnu 
    554    1824   20158 source\makefile.wat 
    170     624    5249 source\configure 
    164     428    4565 source\configure.bat 
   1407    4383   47283 total 

This does include install targets, but does not include generating docs.

3. Comment by jeremy Sep 12, 2009

Current line count of our Jam system:

     95     390    2580 Jamfile 
    240     850    5418 Jamrules 
    247     837    5466 source\Jamfile 
      3      13      80 docs\Jamfile 
      3       9      52 tests\Jamfile 
    588    2099   13596 total 

This includes everything including install and docs.

4. Comment by jeremy Sep 12, 2009

Our CMake system was removed as it was not going to work. Our Batch Files were removed as they were clearly not the right option.

5. Comment by jeremy Sep 12, 2009

A new system based on Bakefiles which I think is the method we should probably go with.

     86     167    2578 euphoria.bkl 
     10      11     196 bake.bkl 
    117     166    2762 source\bake.bkl 
     23      36     640 docs\bake.bkl 
     18      23     427 tests\bake.bkl 
    254     403    6603 total 

This does not include installation yet but it does include everything else, eui, euc, eub, eubw, downloading of eudoc and creole, manual generation and tests.

The thing I am liking about bakefiles is that from one set of bakefiles you can generate platform/make dependent makefiles. For example:

C:\development\projects\euphoria> bakefile_gen 
[1/12] generating watcom from tests\bake.bkl 
[2/12] generating mingw from tests\bake.bkl 
writing tests\makefile.gcc 
writing tests\makefile.wat 
[3/12] generating gnu from tests\bake.bkl 
[4/12] generating watcom from source\bake.bkl 
writing tests\GNUmakefile 
[5/12] generating mingw from source\bake.bkl 
writing source\makefile.wat 
[6/12] generating gnu from source\bake.bkl 
writing source\makefile.gcc 
writing source\GNUmakefile 
[7/12] generating watcom from bake.bkl 
[8/12] generating mingw from bake.bkl 
writing makefile.gcc 
writing makefile.wat 
[9/12] generating gnu from bake.bkl 
[10/12] generating watcom from docs\bake.bkl 
writing GNUmakefile 
[11/12] generating mingw from docs\bake.bkl 
writing docs\makefile.wat 
[12/12] generating gnu from docs\bake.bkl 
writing docs\makefile.gcc 
writing docs\GNUmakefile 
12 files modified 

This means that the user has to have nothing special installed. They use the real, platform based Makefiles, whatever they may be for that platform.

6. Comment by jeremy Sep 12, 2009

Oh, the bakefile system has not been tested on anything but Windows yet.

7. Comment by ne1uno Sep 13, 2009

what was the problem with jam? discounting the enormous amount of time it must have taken track down answers and tweek the files. I guess thats the answer, as or more complicated than the makefiles?

8. Comment by jeremy Sep 13, 2009

There are two problems with Jam that I see it:

  1. It's a tool that must be installed on the users box in order to build. We can distribute that tool on Windows as an executable and on Linux, we could include the source with a simple Makefile to build jam, then use jam to build Euphoria.
  2. During the time I was testing with Jam, I ran across a bug. It was rather easy to track down and fix. I then asked about submitting my patch back to the FT-Jam project. About a week later, I got a response that basically said FreeType is no longer interested in maintaining Jam. They still use it, but it works fine for them. They then proceeded to tell me the best thing to do is start a project on, github or something similar and give people commit access who has patches for Jam. They then told me I can even keep the name FT-Jam for the new project. This is all well and good, but I don't really want to get into maintaining a build tool.

Both of those problems are not a sure thing for not using Jam, but when Matt had a renewed interest in Bakefiles due to our euc command becoming so much smarter, I decided to give it a try as well. Bakefile is maintained, it does the job and doesn't require any special tool on the users box and best of all, Bakefile didn't ask me to pickup on keeping the project going.

Those are my thoughts. I am interested in more feedback.

9. Comment by SDPringle Apr 15, 2010

This is not a "bug report" but a feature request. It is not something that isn't working that is being reported but a new and supposedly better way of doing things.

10. Comment by SDPringle Apr 25, 2010

How does one generate the makefiles from the bakefiles in with one command? I couldn't find bakefile_gen anywhere...

  • There needs to be a way so that the bakefile doesn't try to translate the C files when there is no interpreter on the system.
  • You should have all of the options available as on the current two systems. extra-stats, heapcheck, etc...

11. Comment by ne1uno Apr 26, 2010

>> How does one generate the makefiles from the bakefiles

bakefile_gen executable comes from the bakefile distribution. bakefile is not necessary for use of the generated makefiles. I think the idea is to generate makefiles as part of distribution.

to generate the makefiles, cd to eu4 root dir run bakefile_gen

the bakefiles are out of date with the latest round of defines, align4, assert and a few others. I had no luck adding ALIGN4 and instead modified the bakefile generated makefile directly.

bakefile does not support MSYS the minGW bash shell so I had to run a sed script to change all backslashed filenames to forward slash as well.

12. Comment by jimcbrown Apr 26, 2010

The lack of support for MSYS and MinGW is a blocker. I think we need to reconsider whether or not bakefiles are appropriate.

Maybe we can add an simple eu4 program that takes the bakefile generated Makefile and fixes it up to be MSYS/MinGW compatible..

13. Comment by jeremy Apr 26, 2010

I had no problems with Bakefiles, MinGW and MSYS.

14. Comment by jimcbrown Oct 31, 2010

Where do we stand on this?

15. Comment by jeremy Oct 31, 2010

I think that Bakefiles are our best bet however due to more important tasks and lack of interest in bakefiles (sent an email or ticket comment, forget), I copied the svn tree to a new branch bakefileized. See svn:3560.

This, this ticked is on hold.

16. Comment by jeremy Nov 04, 2010

On Hold, no active work is being done until post 4.0.

17. Comment by mattlewis Nov 04, 2010

Yes, I like bakefiles, but post 4.0.

18. Comment by jimcbrown Nov 16, 2010

I had a crazy idea. The current problem with maintaining the current Makefile system is not because it is so complex (even though it is).

We are trying to maintain two Makefile systems independently of each other, so we basically have to duplicate every addition, every bug fix, twice.

So, if we only had a single Makefile system that worked on all systems, that would cut our workload significantly.

We already know that Makefile.gnu works on Windows with Cygwin's make program as well as MSYS make and mingw-make. So Makefile.gnu is a good place to start.

Setting CC in config.gnu to wcc386 and a few other places would allow us to call the watcom compiler from that Makefile. We could leave in a Makefile.wat that simply called gnumake.exe Makefile.gnu for compatibility, and also include gnumake.exe in source svn to allow for a seemless transition.

19. Comment by jeremy Nov 16, 2010

This would require others to install additional tools when on Windows, but I'm fine with the idea. For the most part on Windows the developers compiling from sources are somewhat limited.

Hm. I wonder if this is actually the case as the #1 downloaded file is Euphoria bundled with OpenWatcom (message:113571).. At least for RC1. Maybe more developers and bleeding edge people are downloading RC1, so that may not be a fair assessment.

It would be interesting to try. I wonder if as a transition/testing time we could have a gnuconfig.bat file that writes the config.gnu, uses the Makefile.gnu, etc... for Windows?

20. Comment by jeremy Nov 16, 2010

Using upx, mingw-make.exe goes to 90,624b ... When exploring Jam, we were going to include jam.exe which was a bit smaller, IIRC ~60k, in the bin directory. Maybe we could include mingw-make.exe in the source directory?

21. Comment by jeremy Nov 16, 2010

we would still have to include two configure scripts, albeit much simpler than the makefiles.

22. Comment by mattlewis Nov 16, 2010

This is probably a worthwhile experiment, but ultimately the bakefiles are probably better. Suppose we switch over to MSVC (hey, it's the native compiler for windows). Bakefiles would have us covered.

23. Comment by jeremy Nov 16, 2010

One thing that is concerned to me for bakefiles is:

Version 0.2.8 was released. This version contains a critical bug fix for 0.2.7's msvc format 

11 Commits for the entire last year and 9 of those were from Feb 6-Feb 12, i.e. 1 week.

24. Comment by jimcbrown Nov 16, 2010

This is more complicated that I thought. Makefile.gnu uses rm, cp, etc .. which requires at least MSYS. Right now we don't have anything in the Makefile that'd compenstate for a lack of these POSIX commands.

On the other hand, if we can count on MSYS being available, then configure.bat just needs to call sh.exe configure and we'd have a single unified configuration script as well.

25. Comment by jeremy Nov 16, 2010

In the config script just set those commands

echo CP=copy >> config.gnu 
echo RM=del >> config.gnu 

or on *nix

echo CP=cp >> config.gnu 
echo RM=del >> config.gnu 

Then in the makefile, instead of cp, use:

$(CP) file.txt new_file.txt 

26. Comment by mattlewis Nov 16, 2010

The bakefile code is not being heavily developed, but that may be more a sign of stability than abandonment. The developer is one of the main devs of wxWidgets, and they use bakefiles there.

The wiki page also claims that WebKit uses bakefiles.

27. Comment by jeremy Nov 16, 2010

0.2.8? Doesn't sound like it's reached maturity?

28. Comment by jimcbrown Nov 16, 2010

From the FAQ at :

    What's the difference between make and mingw32-make? 
        The "native" (i.e.: MSVCRT dependent) port of make is lacking in some 
        functionality and has modified functionality due to the lack of POSIX 
        on Win32. There also exists a version of make in the MSYS distribution 
        that is dependent on the MSYS runtime. This port operates more as make 
        was intended to operate and gives less headaches during execution. 
        Based on this, the MinGW developers/maintainers/packagers decided it 
        would be best to rename the native version so that both the "native" 
        version and the MSYS version could be present at the same time without 
        file name collision. 

It's generally recommended to use MSYS make wherever possible instead of mingw-make. Also, while dual maintaing configure.bat and configure might not be as bad as Makefile.gnu vs Makefile.wat, configure.bat will need quite a bit of work to handle the CYPTRUNKDIR stuff that configure does right now.

29. Comment by jeremy Nov 16, 2010

Jim, are you thus saying that it's not really a viable option?

30. Comment by jimcbrown Nov 16, 2010

We probably don't need all of MSYS.

I think the solution is to include MSYS make and MSYS bash / sh.exe

configure.bat runs sh.exe configure, and Makefile.wat calls MSYS make against Makefile.gnu

31. Comment by jeremy Nov 16, 2010

I think it's getting too complex getlost

32. Comment by SDPringle Nov 28, 2010

The bakefiles made makefiles that worked like batch files. That is, it didn't matter whether you had modified anything things were all rebuilt from sources every time.

You could do this in the eudoc and creolehtml Makefiles. One needs to carefully setup whether things are 'PHONY' and what the dependencies are. It probably wouldn't matter to most people whether or not those functioned as batch files.

33. Comment by jimcbrown Nov 28, 2010

If that's true (everything is rebuilt regardless), then that is a serious strike against using bakefiles. :/

34. Comment by mattlewis Nov 28, 2010

I don't recall the bakefiles being as bad as all that, though I don't think we got the dynamic nature of the translated files working correctly.

35. Comment by jimcbrown Nov 28, 2010

We don't have the dynamic nature working now. But that's really a translator issue, not a makefile/build system issue.

36. Comment by jeremy Nov 28, 2010

The bake files are not that bad, and if they are currently, it's a simple matter of fixing the bakefile. The bakefiles generate Makefiles for the native make tool. It does so in a much better way of automatic dependencies and such.

Bakefile is a great tool and should be pursued further along with the route of using just gmake.


Quick Links

User menu

Not signed in.

Misc Menu