Wiki Diff Roadmap42, revision #6 to tip
= Roadmap to Euphoria 4.2 =
=== To-do List ===
**Audit documentation**
* Ensure docs build correctly
* Proofread and correct errors
* Streamline publishing to web
**Audit tickets**
* Close any completed/obsolete tickets
* Reassign open tickets to active users
* Address and close remaining tickets
**Improve the build system**
* Simplify Makefile rules
* Proper multi-job support
* Avoid rebuilding too often
* Eliminate need for configure
=== New features ===
--##memstruct##-- //Skipping ##memstruct## for 4.2. See ##std/ffi.e## below.//
=== Standard libraries ===
##std/ctypes.e##
Add routines for defining and managing C types in memory.
##std/json.e##
Migrate [[https://github.com/OpenEuphoria/euphoria-mvc/blob/master/include/mvc/json.e|mvc/json.e]] from Euphoria MVC
##std/log.e##
Migrate [[https://github.com/OpenEuphoria/euphoria-mvc/blob/master/include/mvc/logger.e|mvc/logger.e]] from Euphoria MVC
##std/struct.e##
Add routines for defining and managing C structures and unions in memory.
##std/template.e##
Migrate [[https://github.com/OpenEuphoria/euphoria-mvc/blob/master/include/mvc/template.e|mvc/template.e]] from Euphoria MVC
##std/uuid.e##
Provide a simple library for creating and parsing [[https://en.wikipedia.org/wiki/Universally_unique_identifier|UUIDs]].
##std/net/http.e##
* Add support for HTTPS using CURL (see below).
* Should we keep existing code for HTTP or just rely entirely on CURL? maybe use ~WinInet on Windows? (see FAQ)
* Add other [[https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods|methods]] such as ##http_put()##, etc.
=== Built-in libraries ===
These are third-party libraries compiled directly into the backend of Euphoria, so they're always available.
**Note:** If anyone can offer native Euphoria replacements for these, we'd gladly accept those in their place!
##std/ffi.e##
* Compile [[https://sourceware.org/libffi/|libffi]] into the backend.
* Provide same API as ##std/dll.e## using FFI instead of legacy code.
* Mark all routines in ##std/dll.e## as ##deprecate##.
##std/hash.e##
* Compile Greg's [[https://github.com/ghaberek/freehash|freehash]] library into the backend.
* Remove ##hash()## as a built-in and make it a ##machine_func()## instead.
##std/markdown.e##
* Compile [[https://github.com/mity/md4c|MD4C]] into the backend.
* This provides a simple Markdown parser for getting off Creole.
##std/regex.e##
* Upgrade [[https://pcre.org/|PCRE]] library to the latest release (version 8.45 as of 2021-06-22).
* This is still the "old" PCRE (not PCRE2) but it's still being supported with patches and bug fixes.
* Many improvements between our current version 8.10 and latest 8.45. See [[https://pcre.org/original/changelog.txt|ChangeLog]] for details.
##std/xml.e##
* Compile [[https://github.com/michaelrsweet/mxml|mxml]] directly into the backend and expose all routines as wrappers in the std library.
* This provides a complete XML parser for reading and writing files.
##std/zip.e##
* Compile [[https://github.com/richgel999/miniz|miniz]] directly into the backend and expose all routines as wrappers in the std library.
* This provides access to GZIP, DEFLATE, and ZIP file routines.
=== External libraries ===
These are third-party libraries bundled with the Euphoria installation, but require sharing the library files (.dll, etc.) as well.
##std/curl.e##
* Provide a complete wrapper for [[https://curl.haxx.se/|libcurl]] and include prebuilt binaries in the ##bin/## directory on all platforms.
* CURL is pretty much the Swiss Army tool of Internet-related communication, and can be the basis for several other std libraries like HTTP, FTP, etc.
##std/ssh.e##
* Provide a complete wrapper for [[https://www.libssh2.org/|libssh2]] and include prebuilt binaries in the ##bin/## directory on all platforms.
* This provides out-of-the-box support for SSH, SCP, and SFTP protocols.
##std/sqlite3.e##
* Provide a complete wrapper for [[https://www.sqlite.org/index.html|SQLite3]] and include prebuilt binaries in the ##bin/## directory on all platforms.
* This provides out-of-the-box support for //simple// SQL-based applications. Larger SQL applications should still use MySQL, PostgreSQL, etc.
=== FAQ ===
\\
//**Q:** Should we use CURL for all HTTP functions?//
**A:** In an effort to maintain backward-compatibility with existing applications that already use ##http_get()## or ##http_post()##, maybe we should fall back to the original routines to prevent those applications from breaking if they upgrade to Euphoria 4.2. The original routines should eventually be removed in a future release, e.g. Euphoria 5.0.
//**Q:** Why compile some things into the backend and not others?//
**A:** We need to maintain some balance and avoid including "everything but the [[https://en.wikipedia.org/wiki/Scope_creep|kitchen sink]]" into the backend ~-- leading to a bloated and unportable executables. We should also avoid making //everything// an external library, which creates more strain on users to package and ship their own applications. Small libraries and high-priority libraries should be built into the backend. Large and low-priority libraries should be shipped separately.
= Roadmap to Euphoria 4.2 =
=== To-do List ===
\\
**Improve the build system**
* Simplify Makefile rules
* Proper multi-job support
* Avoid rebuilding too often
* Eliminate need for configure
**Audit documentation**
* Ensure docs build correctly
* Proofread and correct errors
* Streamline publishing to web
=== New features ===
\\
**C structures**\\
##//built-in//##
* Merge Matt's long-standing **struct** branch to provide these new features.
* Need to fix missing TODO items for outputting ##memstruct## blocks in translator.
* Need to update any additional std libraries that currently wrap structures manually.
=== Built-in libraries ===
\\
These are third-party libraries compiled directly into the backend of Euphoria, so they're always available.
**Note:** If anyone can offer working and complete Euphoria-based replacements for these libraries, we'd gladly accept those in their place.
**PCRE**\\
##std/regex.e##
* Upgrade [[https://pcre.org/|PCRE]] library to the latest release (version 8.43 as of 23-Feb-2019).
* This is still the "old" PCRE (not PCRE2) but it's still being supported with patches and bug fixes.
* Many improvements between our current version 8.10 and latest 8.43. See [[https://pcre.org/original/changelog.txt|ChangeLog]] for details.
**JSMN**\\
##std/json.e##
* Compile [[https://github.com/zserge/jsmn|jsmn]] directly into the backend for fast JSON parsing and provide additional native routines in the std library.
* JSMN only provides a single //parsing// function. We'll need to provide additional routines to read/write JSON files natively (see **jsmn** branch).
**MINIZ**\\
##std/zip.e##
* Compile [[https://github.com/richgel999/miniz|miniz]] directly into the backend and expose all routines as wrappers in the std library.
* This provides access to GZIP, DEFLATE, and ZIP file routines.
**MXML**\\
##std/xml.e##
* Compile [[https://github.com/michaelrsweet/mxml|mxml]] directly into the backend and expose all routines as wrappers in the std library.
* This provides a complete XML parser for reading and writing files.
**PICOHASH**\\
##std/hash.e##
* Compile [[https://github.com/kazuho/picohash|picohash]] directly into the backend and expose via ##hash()## function constants.
* Use this to extend ##hash()## and //actually// implement MD5, SHA256, etc.
=== External libraries ===
\\
These are third-party libraries bundled with the Euphoria installation, but require sharing the library files (.dll, etc.) as well.
**CURL**\\
##std/curl.e##
* Provide a complete wrapper for [[https://curl.haxx.se/|libcurl]] and include prebuilt binaries in the ##bin/## directory on all platforms.
* CURL is pretty much the Swiss Army knife of Internet-related communication, and can be the basis for several other std libraries.
**LIBUI**\\
##std/ui.e##
* Provide a complete wrapper for [[https://github.com/andlabs/libui|libui]] and include prebuilt binaries in the ##bin/## directory on all platforms.
* This provides out-of-the-box support for //simple// GUI applications. Larger GUI applications should still use ~Win32Lib, EuGTK, or wxEuphoria.
**LIBSSH2**\\
##std/libssh2.e##
* Provide a complete wrapper for [[https://www.libssh2.org/|libssh2]] and include prebuilt binaries in the ##bin/## directory on all platforms.
* This provides out-of-the-box support for SSH, SCP, and SFTP protocols.
**SQLITE**\\
##std/sqlite.e##
* Provide a complete wrapper for [[https://www.sqlite.org/index.html|SQLite]] and include prebuilt binaries in the ##bin/## directory on all platforms.
* This provides out-of-the-box support for //simple// SQL-based applications. Larger SQL applications should still use MySQL, PostgreSQL, etc.
=== Standard libraries ===
\\
Additions or extensions to libraries written in Euphoria.
**HTTP**\\
##std/net/http.e##
* Add support for HTTPS (see below).
* Add other [[https://restfulapi.net/http-methods/|HTTP methods]] such as ##http_put()##, etc.
In order to provide HTTPS support, we need to:
* Use CURL directly if it is found in the local directory, ##PATH##, or Euphoria ##bin/##
* ...or use [[https://docs.microsoft.com/en-us/windows/win32/wininet/about-wininet|WinInet]] on Windows if CURL is not found locally
* ...or use system-wide CURL in ##PATH## if CURL is not found locally
* ...or fall back to our original ##std/socket.e## based HTTP-only routines
* ...and then finally, return error ##ERR_HTTPS_UNAVAILABLE## for HTTPS requests.
**UUID support**\\
##std/uuid.e##
* Provide a simple library for creating and parsing [[https://en.wikipedia.org/wiki/Universally_unique_identifier|UUIDs]].
=== FAQ ===
\\
//**Q:** Why not use CURL for all HTTP functions?//
**A:** This an effort to maintain backward-compatibility with existing applications that already use ##http_get()## or ##http_post()##. By ensuring that we can fall back to the original routines, we'll prevent those applications from breaking if they upgrade to Euphoria 4.2. The original routines should eventually be removed in a future release, e.g. Euphoria 5.0.
//**Q:** Why compile some things into the backend and not others?//
**A:** We need to maintain some balance and avoid including "everything but the [[https://en.wikipedia.org/wiki/Scope_creep|kitchen sink]]" into the backend ~-- leading to a bloated and unportable executables. We should also avoid making //everything// an external library, which creates more strain on users to package and ship their own applications. Small libraries and high-priority libraries should be built into the backend. Large and low-priority libraries should be shipped separately.
=== To-do List ===
**Audit documentation**
* Ensure docs build correctly
* Proofread and correct errors
* Streamline publishing to web
**Audit tickets**
* Close any completed/obsolete tickets
* Reassign open tickets to active users
* Address and close remaining tickets
**Improve the build system**
* Simplify Makefile rules
* Proper multi-job support
* Avoid rebuilding too often
* Eliminate need for configure
=== New features ===
--##memstruct##-- //Skipping ##memstruct## for 4.2. See ##std/ffi.e## below.//
=== Standard libraries ===
##std/ctypes.e##
Add routines for defining and managing C types in memory.
##std/json.e##
Migrate [[https://github.com/OpenEuphoria/euphoria-mvc/blob/master/include/mvc/json.e|mvc/json.e]] from Euphoria MVC
##std/log.e##
Migrate [[https://github.com/OpenEuphoria/euphoria-mvc/blob/master/include/mvc/logger.e|mvc/logger.e]] from Euphoria MVC
##std/struct.e##
Add routines for defining and managing C structures and unions in memory.
##std/template.e##
Migrate [[https://github.com/OpenEuphoria/euphoria-mvc/blob/master/include/mvc/template.e|mvc/template.e]] from Euphoria MVC
##std/uuid.e##
Provide a simple library for creating and parsing [[https://en.wikipedia.org/wiki/Universally_unique_identifier|UUIDs]].
##std/net/http.e##
* Add support for HTTPS using CURL (see below).
* Should we keep existing code for HTTP or just rely entirely on CURL? maybe use ~WinInet on Windows? (see FAQ)
* Add other [[https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods|methods]] such as ##http_put()##, etc.
=== Built-in libraries ===
These are third-party libraries compiled directly into the backend of Euphoria, so they're always available.
**Note:** If anyone can offer native Euphoria replacements for these, we'd gladly accept those in their place!
##std/ffi.e##
* Compile [[https://sourceware.org/libffi/|libffi]] into the backend.
* Provide same API as ##std/dll.e## using FFI instead of legacy code.
* Mark all routines in ##std/dll.e## as ##deprecate##.
##std/hash.e##
* Compile Greg's [[https://github.com/ghaberek/freehash|freehash]] library into the backend.
* Remove ##hash()## as a built-in and make it a ##machine_func()## instead.
##std/markdown.e##
* Compile [[https://github.com/mity/md4c|MD4C]] into the backend.
* This provides a simple Markdown parser for getting off Creole.
##std/regex.e##
* Upgrade [[https://pcre.org/|PCRE]] library to the latest release (version 8.45 as of 2021-06-22).
* This is still the "old" PCRE (not PCRE2) but it's still being supported with patches and bug fixes.
* Many improvements between our current version 8.10 and latest 8.45. See [[https://pcre.org/original/changelog.txt|ChangeLog]] for details.
##std/xml.e##
* Compile [[https://github.com/michaelrsweet/mxml|mxml]] directly into the backend and expose all routines as wrappers in the std library.
* This provides a complete XML parser for reading and writing files.
##std/zip.e##
* Compile [[https://github.com/richgel999/miniz|miniz]] directly into the backend and expose all routines as wrappers in the std library.
* This provides access to GZIP, DEFLATE, and ZIP file routines.
=== External libraries ===
These are third-party libraries bundled with the Euphoria installation, but require sharing the library files (.dll, etc.) as well.
##std/curl.e##
* Provide a complete wrapper for [[https://curl.haxx.se/|libcurl]] and include prebuilt binaries in the ##bin/## directory on all platforms.
* CURL is pretty much the Swiss Army tool of Internet-related communication, and can be the basis for several other std libraries like HTTP, FTP, etc.
##std/ssh.e##
* Provide a complete wrapper for [[https://www.libssh2.org/|libssh2]] and include prebuilt binaries in the ##bin/## directory on all platforms.
* This provides out-of-the-box support for SSH, SCP, and SFTP protocols.
##std/sqlite3.e##
* Provide a complete wrapper for [[https://www.sqlite.org/index.html|SQLite3]] and include prebuilt binaries in the ##bin/## directory on all platforms.
* This provides out-of-the-box support for //simple// SQL-based applications. Larger SQL applications should still use MySQL, PostgreSQL, etc.
=== FAQ ===
\\
//**Q:** Should we use CURL for all HTTP functions?//
**A:** In an effort to maintain backward-compatibility with existing applications that already use ##http_get()## or ##http_post()##, maybe we should fall back to the original routines to prevent those applications from breaking if they upgrade to Euphoria 4.2. The original routines should eventually be removed in a future release, e.g. Euphoria 5.0.
//**Q:** Why compile some things into the backend and not others?//
**A:** We need to maintain some balance and avoid including "everything but the [[https://en.wikipedia.org/wiki/Scope_creep|kitchen sink]]" into the backend ~-- leading to a bloated and unportable executables. We should also avoid making //everything// an external library, which creates more strain on users to package and ship their own applications. Small libraries and high-priority libraries should be built into the backend. Large and low-priority libraries should be shipped separately.
