Euphoria
Ticket #43:
Simplify the mainteance of maintaining the Euphoria makefiles/build system.
-
Reported by
jeremy
Sep 12, 2009
We have tried many things:
- Batch Files
- Makefiles
- CMake
- Jam
Details
1. Comment by jeremy
Sep 12, 2009
With the new stuff in euc, we should be able to do this pretty easily.
2. Comment by jeremy
Sep 12, 2009
Current line count of our Makefile system:
519 1507 17311 source\Makefile.gnu
554 1824 20158 source\makefile.wat
170 624 5249 source\configure
164 428 4565 source\configure.bat
1407 4383 47283 total
This does include install targets, but does not include generating docs.
3. Comment by jeremy
Sep 12, 2009
Current line count of our Jam system:
95 390 2580 Jamfile
240 850 5418 Jamrules
247 837 5466 source\Jamfile
3 13 80 docs\Jamfile
3 9 52 tests\Jamfile
588 2099 13596 total
This includes everything including install and docs.
4. Comment by jeremy
Sep 12, 2009
Our CMake system was removed as it was not going to work. Our Batch Files were removed as they were clearly not the right option.
5. Comment by jeremy
Sep 12, 2009
A new system based on Bakefiles which I think is the method we should probably go with.
86 167 2578 euphoria.bkl
10 11 196 bake.bkl
117 166 2762 source\bake.bkl
23 36 640 docs\bake.bkl
18 23 427 tests\bake.bkl
254 403 6603 total
This does not include installation yet but it does include everything else, eui, euc, eub, eubw, downloading of eudoc and creole, manual generation and tests.
The thing I am liking about bakefiles is that from one set of bakefiles you can generate platform/make dependent makefiles. For example:
C:\development\projects\euphoria> bakefile_gen
[1/12] generating watcom from tests\bake.bkl
[2/12] generating mingw from tests\bake.bkl
writing tests\makefile.gcc
writing tests\makefile.wat
[3/12] generating gnu from tests\bake.bkl
[4/12] generating watcom from source\bake.bkl
writing tests\GNUmakefile
[5/12] generating mingw from source\bake.bkl
writing source\makefile.wat
[6/12] generating gnu from source\bake.bkl
writing source\makefile.gcc
writing source\GNUmakefile
[7/12] generating watcom from bake.bkl
[8/12] generating mingw from bake.bkl
writing makefile.gcc
writing makefile.wat
[9/12] generating gnu from bake.bkl
[10/12] generating watcom from docs\bake.bkl
writing GNUmakefile
[11/12] generating mingw from docs\bake.bkl
writing docs\makefile.wat
[12/12] generating gnu from docs\bake.bkl
writing docs\makefile.gcc
writing docs\GNUmakefile
12 files modified
This means that the user has to have nothing special installed. They use the real, platform based Makefiles, whatever they may be for that platform.
6. Comment by jeremy
Sep 12, 2009
Oh, the bakefile system has not been tested on anything but Windows yet.
7. Comment by ne1uno
Sep 13, 2009
what was the problem with jam? discounting the enormous amount of time it must have taken track down answers and tweek the files. I guess thats the answer, as or more complicated than the makefiles?
8. Comment by jeremy
Sep 13, 2009
There are two problems with Jam that I see it:
- It's a tool that must be installed on the users box in order to build. We can distribute that tool on Windows as an executable and on Linux, we could include the source with a simple Makefile to build jam, then use jam to build Euphoria.
- During the time I was testing with Jam, I ran across a bug. It was rather easy to track down and fix. I then asked about submitting my patch back to the FT-Jam project. About a week later, I got a response that basically said FreeType is no longer interested in maintaining Jam. They still use it, but it works fine for them. They then proceeded to tell me the best thing to do is start a project on SF.net, github or something similar and give people commit access who has patches for Jam. They then told me I can even keep the name FT-Jam for the new project. This is all well and good, but I don't really want to get into maintaining a build tool.
Both of those problems are not a sure thing for not using Jam, but when Matt had a renewed interest in Bakefiles due to our euc command becoming so much smarter, I decided to give it a try as well. Bakefile is maintained, it does the job and doesn't require any special tool on the users box and best of all, Bakefile didn't ask me to pickup on keeping the project going.
Those are my thoughts. I am interested in more feedback.
9. Comment by SDPringle
Apr 15, 2010
This is not a "bug report" but a feature request. It is not something that isn't working that is being reported but a new and supposedly better way of doing things.
10. Comment by SDPringle
Apr 25, 2010
How does one generate the makefiles from the bakefiles in with one command? I couldn't find bakefile_gen anywhere...
- There needs to be a way so that the bakefile doesn't try to translate the C files when there is no interpreter on the system.
- You should have all of the options available as on the current two systems.
extra-stats, heapcheck, etc...
11. Comment by ne1uno
Apr 26, 2010
>> How does one generate the makefiles from the bakefiles
bakefile_gen executable comes from the bakefile distribution. bakefile is not necessary for use of the generated makefiles. I think the idea is to generate makefiles as part of distribution.
to generate the makefiles, cd to eu4 root dir run bakefile_gen
the bakefiles are out of date with the latest round of defines, align4, assert and a few others. I had no luck adding ALIGN4 and instead modified the bakefile generated makefile directly.
bakefile does not support MSYS the minGW bash shell so I had to run a sed script to change all backslashed filenames to forward slash as well.
12. Comment by jimcbrown
Apr 26, 2010
The lack of support for MSYS and MinGW is a blocker. I think we need to reconsider whether or not bakefiles are appropriate.
Maybe we can add an simple eu4 program that takes the bakefile generated Makefile and fixes it up to be MSYS/MinGW compatible..
13. Comment by jeremy
Apr 26, 2010
I had no problems with Bakefiles, MinGW and MSYS.
14. Comment by jimcbrown
Oct 31, 2010
Where do we stand on this?
15. Comment by jeremy
Oct 31, 2010
I think that Bakefiles are our best bet however due to more important tasks and lack of interest in bakefiles (sent an email or ticket comment, forget), I copied the svn tree to a new branch bakefileized. See svn:3560.
This, this ticked is on hold.
16. Comment by jeremy
Nov 04, 2010
On Hold, no active work is being done until post 4.0.
17. Comment by mattlewis
Nov 04, 2010
Yes, I like bakefiles, but post 4.0.
18. Comment by jimcbrown
Nov 16, 2010
I had a crazy idea. The current problem with maintaining the current Makefile system is not because it is so complex (even though it is).
We are trying to maintain two Makefile systems independently of each other, so we basically have to duplicate every addition, every bug fix, twice.
So, if we only had a single Makefile system that worked on all systems, that would cut our workload significantly.
We already know that Makefile.gnu works on Windows with Cygwin's make program as well as MSYS make and mingw-make. So Makefile.gnu is a good place to start.
Setting CC in config.gnu to wcc386 and a few other places would allow us to call the watcom compiler from that Makefile. We could leave in a Makefile.wat that simply called gnumake.exe Makefile.gnu for compatibility, and also include gnumake.exe in source svn to allow for a seemless transition.
19. Comment by jeremy
Nov 16, 2010
This would require others to install additional tools when on Windows, but I'm fine with the idea. For the most part on Windows the developers compiling from sources are somewhat limited.
Hm. I wonder if this is actually the case as the #1 downloaded file is Euphoria bundled with OpenWatcom (message:113571).. At least for RC1. Maybe more developers and bleeding edge people are downloading RC1, so that may not be a fair assessment.
It would be interesting to try. I wonder if as a transition/testing time we could have a gnuconfig.bat file that writes the config.gnu, uses the Makefile.gnu, etc... for Windows?
20. Comment by jeremy
Nov 16, 2010
Using upx, mingw-make.exe goes to 90,624b ... When exploring Jam, we were going to include jam.exe which was a bit smaller, IIRC ~60k, in the bin directory. Maybe we could include mingw-make.exe in the source directory?
21. Comment by jeremy
Nov 16, 2010
we would still have to include two configure scripts, albeit much simpler than the makefiles.
22. Comment by mattlewis
Nov 16, 2010
This is probably a worthwhile experiment, but ultimately the bakefiles are probably better. Suppose we switch over to MSVC (hey, it's the native compiler for windows). Bakefiles would have us covered.
23. Comment by jeremy
Nov 16, 2010
One thing that is concerned to me for bakefiles is:
2009-10-24
Version 0.2.8 was released. This version contains a critical bug fix for 0.2.7's msvc format
11 Commits for the entire last year and 9 of those were from Feb 6-Feb 12, i.e. 1 week.
24. Comment by jimcbrown
Nov 16, 2010
This is more complicated that I thought. Makefile.gnu uses rm, cp, etc .. which requires at least MSYS. Right now we don't have anything in the Makefile that'd compenstate for a lack of these POSIX commands.
On the other hand, if we can count on MSYS being available, then configure.bat just needs to call sh.exe configure and we'd have a single unified configuration script as well.
25. Comment by jeremy
Nov 16, 2010
In the config script just set those commands
echo CP=copy >> config.gnu
echo RM=del >> config.gnu
or on *nix
echo CP=cp >> config.gnu
echo RM=del >> config.gnu
Then in the makefile, instead of cp, use:
$(CP) file.txt new_file.txt
26. Comment by mattlewis
Nov 16, 2010
The bakefile code is not being heavily developed, but that may be more a sign of stability than abandonment. The developer is one of the main devs of wxWidgets, and they use bakefiles there.
http://www.bakefile.org/wiki/BakefileUsers
The wiki page also claims that WebKit uses bakefiles.
27. Comment by jeremy
Nov 16, 2010
0.2.8? Doesn't sound like it's reached maturity?
28. Comment by jimcbrown
Nov 16, 2010
From the FAQ at http://www.mingw.org/wiki/FAQ :
What's the difference between make and mingw32-make?
The "native" (i.e.: MSVCRT dependent) port of make is lacking in some
functionality and has modified functionality due to the lack of POSIX
on Win32. There also exists a version of make in the MSYS distribution
that is dependent on the MSYS runtime. This port operates more as make
was intended to operate and gives less headaches during execution.
Based on this, the MinGW developers/maintainers/packagers decided it
would be best to rename the native version so that both the "native"
version and the MSYS version could be present at the same time without
file name collision.
It's generally recommended to use MSYS make wherever possible instead of mingw-make. Also, while dual maintaing configure.bat and configure might not be as bad as Makefile.gnu vs Makefile.wat, configure.bat will need quite a bit of work to handle the CYPTRUNKDIR stuff that configure does right now.
29. Comment by jeremy
Nov 16, 2010
Jim, are you thus saying that it's not really a viable option?
30. Comment by jimcbrown
Nov 16, 2010
We probably don't need all of MSYS.
I think the solution is to include MSYS make and MSYS bash / sh.exe
configure.bat runs sh.exe configure, and Makefile.wat calls MSYS make against Makefile.gnu
31. Comment by jeremy
Nov 16, 2010
I think it's getting too complex 
32. Comment by SDPringle
Nov 28, 2010
The bakefiles made makefiles that worked like batch files. That is, it didn't matter whether you had modified anything things were all rebuilt from sources every time.
You could do this in the eudoc and creolehtml Makefiles. One needs to carefully setup whether things are 'PHONY' and what the dependencies are. It probably wouldn't matter to most people whether or not those functioned as batch files.
33. Comment by jimcbrown
Nov 28, 2010
If that's true (everything is rebuilt regardless), then that is a serious strike against using bakefiles. :/
34. Comment by mattlewis
Nov 28, 2010
I don't recall the bakefiles being as bad as all that, though I don't think we got the dynamic nature of the translated files working correctly.
35. Comment by jimcbrown
Nov 28, 2010
We don't have the dynamic nature working now. But that's really a translator issue, not a makefile/build system issue.
36. Comment by jeremy
Nov 28, 2010
The bake files are not that bad, and if they are currently, it's a simple matter of fixing the bakefile. The bakefiles generate Makefiles for the native make tool. It does so in a much better way of automatic dependencies and such.
Bakefile is a great tool and should be pursued further along with the route of using just gmake.
GitHub