1. Euphoria MVC updates

I committed several improvements to Euphoria MVC today.

https://github.com/OpenEuphoria/euphoria-mvc

  • Built-in web server now supports GET and POST requests.
  • Model library can now query/insert/update/delete objects.
  • Template library does better at handling more expressions.
  • Route parser now accepts arbitrary regex patterns for type-checking.
  • Added libraries for parsing HTML and JSON documents.
  • Easily save and load configuration files (in INI format).
  • Wrappers for MySQL (or MariaDB), SQLite3, and libcurl.
  • Updated documentations!

Update 11/15/2020:

  • Added SQLite3 library and wrapper

Update 11/23/2020:

  • Added HTML parser

Update 11/28/2020:

  • Overhauled mvc/config.e

Update 10/24/2021:

  • Server now supports POST requests

There's still a lot to do. But I'm still working on it. I'm not dead yet!

-Greg

new topic     » topic index » view message » categorize

2. Re: Euphoria MVC updates

ghaberek said...

I committed several improvements to Euphoria MVC today.

https://github.com/OpenEuphoria/euphoria-mvc

  • Built-in web server now supports GET and POST requests.
  • Model library can now query/insert/update/delete objects.
  • Template library does better at handling more expressions.
  • Route parser now accepts arbitrary regex patterns for type-checking.
  • Updated documentations!

There's still a lot to do. But I'm still working on it. I'm not dead yet!

-Greg

Sounds great Greg. Hopefully thew new site will be up sometime this year.

new topic     » goto parent     » topic index » view message » categorize

3. Re: Euphoria MVC updates

Icy_Viking said...
ghaberek said...

I committed several improvements to Euphoria MVC today.

https://github.com/OpenEuphoria/euphoria-mvc

  • Built-in web server now supports GET and POST requests.
  • Model library can now query/insert/update/delete objects.
  • Template library does better at handling more expressions.
  • Route parser now accepts arbitrary regex patterns for type-checking.
  • Updated documentations!

There's still a lot to do. But I'm still working on it. I'm not dead yet!

-Greg

Sounds great Greg.

Agreed! Can't wait to see this when it's finally ready to be rolled out.

Icy_Viking said...

Hopefully thew new site will be up sometime this year.

Fingers crossed.

new topic     » goto parent     » topic index » view message » categorize

4. Re: Euphoria MVC updates

I added a JSON parser today! (Well, I actually wrote it a month ago.)

Source: https://github.com/OpenEuphoria/euphoria-mvc/blob/v1.8.0/include/mvc/json.e

Docs: https://github.com/OpenEuphoria/euphoria-mvc/blob/v1.8.0/docs/JSON.md

Edit: the docs reference Euphoria maps but I stripped all of that out to simplify the parser. Objects are just sequences of {key,{type,value}} items now. That's why I added json_fetch to look up values from keys. I'll update the documentation to remove references to maps soon.

-Greg

new topic     » goto parent     » topic index » view message » categorize

5. Re: Euphoria MVC updates

I added a CURL wrapper today! (Well, I actually wrote it several years ago to make the existing euweb work with reCAPTCHAv2.) Updated release to v1.9.0.

https://github.com/OpenEuphoria/euphoria-mvc/releases/tag/v1.9.0

-Greg

new topic     » goto parent     » topic index » view message » categorize

6. Re: Euphoria MVC updates

I added a lot more CURL functions and a few more JSON functions and updated the documentation a bit. Updated release to v1.10.0.

https://github.com/OpenEuphoria/euphoria-mvc/releases/tag/v1.10.0

-Greg

new topic     » goto parent     » topic index » view message » categorize

7. Re: Euphoria MVC updates

That's funny Greg. I built an importer that would load in the constants and functions from a C- header file and create a Euphoria E file for in order to create a curl wrapper. Is it necessary to use MVC in order to use the CURL function?

SD Pringle

new topic     » goto parent     » topic index » view message » categorize

8. Re: Euphoria MVC updates

SDPringle said...

That's funny Greg. I built an importer that would load in the constants and functions from a C- header file and create a Euphoria E file for in order to create a curl wrapper. Is it necessary to use MVC in order to use the CURL function?

Nope, it's entirely separate and in its own directory. It's just so integral to building web apps that I figured I would just include it directly.

I wrapped the whole thing by hand and made some improvements from when I originally wrote it a couple years ago.

Anyone who needs it can just copy the files from here: https://github.com/OpenEuphoria/euphoria-mvc/tree/master/include/curl

I also have some basic documentation here: https://github.com/OpenEuphoria/euphoria-mvc/blob/master/docs/CURL.md (including where best to get Windows binaries!)

-Greg

new topic     » goto parent     » topic index » view message » categorize

9. Re: Euphoria MVC updates

The failing t_net_http.e has been a thorn in my side for years now and it is a bigger waste of time to me to re-implement what CURL from (Euphoria MVC) already does in Euphoria than to leverage CURL it in order to fix t_net_http. I wouldn't mind if std/net/http.e went away completely at this point and I don't see anyone really using it. I am the kind of person who doesn't like to remove something from a library once it is already in there, though. What do you think?
Forked into: RM std/net/http.e?

new topic     » goto parent     » topic index » view message » categorize

10. Re: Euphoria MVC updates

I've added a SQLite3 wrapper and lots of little updates here and there.

https://github.com/OpenEuphoria/euphoria-mvc/releases/tag/v1.11.0

-Greg

new topic     » goto parent     » topic index » view message » categorize

11. Re: Euphoria MVC updates

I've added an HTML parser a few minor fixes.

https://github.com/OpenEuphoria/euphoria-mvc/releases/tag/v1.12.0

-Greg

new topic     » goto parent     » topic index » view message » categorize

12. Re: Euphoria MVC updates

It's really nice.

I found a bug with session cookies so I opened a new issue.

Regards

Jean-Marc

new topic     » goto parent     » topic index » view message » categorize

13. Re: Euphoria MVC updates

Where should local CSS files be located to be used with the development server ?

Jean-Marc

new topic     » goto parent     » topic index » view message » categorize

14. Re: Euphoria MVC updates

jmduro said...

It's really nice.

Thanks! That's good to hear. smile

jmduro said...

I found a bug with session cookies so I opened a new issue.

Thanks, I will look into this. Unfortunately some the older code is in need of better testing and cleanup.

FYI on GitHub you can enclose blocks of code in triple backticks ``` or by indenting at least four spaces.

-Greg

new topic     » goto parent     » topic index » view message » categorize

15. Re: Euphoria MVC updates

jmduro said...

Where should local CSS files be located to be used with the development server ?

Yeah I need to add a feature for serving files directly from a local directory. I'd also like to add an option for "base directory" so you can change it from current directory.

In the mean time, I've at least added send_file() and download_file() which are shorthand for verifying a file exists, looking up its mime-type, and then sending the required headers.

include mvc/app.e

send_file( sequence path ) - Return the contents of a file directly (and let the browser decide).
download_file( sequence filename, sequence path=filename ) - Return the contents of a file and prompt the browser to download it.

I pushed the commit for adding send_file() just now: https://github.com/OpenEuphoria/euphoria-mvc/releases/tag/v1.12.1

You can use this to add a static route handler to your application. Then just put everything in /static/, like /static/css/, /static/js/, etc.

Edit: Or you can add a route pattern for specific directories as I've indicated below.

include std/map.e 
include mvc/app.e 
 
include std/filesys.e -- for canonical_path() 
include std/search.e -- for search:begins() 
 
-- 
-- Route handler for files in /static/ 
-- 
function static( object request ) 
 
    -- get the path to the requested file 
    sequence path_info = map:get( request, "PATH_INFO" ) 
 
    -- 
    -- Note: path_info will always start with 
    -- "/static/" thanks to the route handler. 
    -- 
 
    -- ensure the local file is an actual full path 
    sequence local_file = canonical_path( "." & path_info ) 
 
    -- ensure the local file exists in current directory 
    if search:begins( current_dir(), local_file ) then 
 
        -- go ahead and send the file to the browser 
        return send_file( local_path ) 
 
    end if 
 
    -- tell the browser the file doesn't exist 
    return response_code( 404 ) 
end function 
-- 
-- Note: this pattern should resolve to "static" and find the function above. If not, you can call 
-- app:route("pattern", "func name") or even app:route("pattern", "func name", routine_id("func name")) 
-- 
app:route( "/static/.+" ) 
 
-- Or add a handler for specific directories like this: 
app:route( "/(css|js|fonts|images)/.+", "static" ) 
 

-Greg

new topic     » goto parent     » topic index » view message » categorize

16. Re: Euphoria MVC updates

jmduro said...

Where should local CSS files be located to be used with the development server ?

I should also point out for anyone reading along that if you're using Apache or Nginx, it should serve static files directly on your behalf. Routing them through your app would add unnecessary complications and slow everything down a bit.

-Greg

new topic     » goto parent     » topic index » view message » categorize

17. Re: Euphoria MVC updates

ghaberek said...
    sequence local_file = canonical_path( "." & path_info ) 
 
    if search:begins( current_dir(), local_file ) then 
 
        return send_file( local_path ) 
 
    end if 

local_path is not defined. I thought it could be local_file instead but that's wrong:

2020/11/26 11:41:13  INFO static@mysql_users.ex:22 path_info = "\"/static/css/w3.css\"" 
2020/11/26 11:41:13  INFO static@mysql_users.ex:31 local_file = "\"C:\\\\Users\\\\duro\\\\Documents\\\\mysql_users\\\\examples\\\\static\\\\css\\\\w3.css\"" 
2020/11/26 11:41:13  INFO client_handler@server.e:140 "127.0.0.1" "GET" "/static/css/w3.css" "404 Not Found" 
Jean-Marc

new topic     » goto parent     » topic index » view message » categorize

18. Re: Euphoria MVC updates

jmduro said...

local_path is not defined.

Yes, a typo on my part, sorry about that.

jmduro said...

I thought it could be local_file instead but that's wrong:

2020/11/26 11:41:13  INFO static@mysql_users.ex:22 path_info = "\"/static/css/w3.css\"" 
2020/11/26 11:41:13  INFO static@mysql_users.ex:31 local_file = "\"C:\\\\Users\\\\duro\\\\Documents\\\\mysql_users\\\\examples\\\\static\\\\css\\\\w3.css\"" 
2020/11/26 11:41:13  INFO client_handler@server.e:140 "127.0.0.1" "GET" "/static/css/w3.css" "404 Not Found" 

No, that's right. The problem in the example you provided is that "static" is not in the "examples" directory, so the file indeed does not exist.

This is the layout of the zip file you posted:

mysql_users\ 
  examples\ 
    mysql.e 
    mysql_users.ex 
  static\ 
    css\ 
      w3.css 
  templates\ 
    insert.html 
    login.html 
    main.html 
    users.html 
  libmariadb.dll 
  users.sql 

I recommend putting the main app file in the root directory for your project, and then other directories like "static" and "templates" can be easily referenced from there.

So your project should look more like this:

mysql_users\ 
  static\ 
    css\ 
      w3.css 
  templates\ 
    insert.html 
    login.html 
    main.html 
    users.html 
  libmariadb.dll 
  mysql.e 
  mysql_users.ex 
  users.sql 

And then run:

eui mysql_users.ex 

And your static route will find the correct path to "w3.css"

Logging Targets

The logger outputs to STDERR by default. You're changing this to "debug.log" which is helpful, but you can also target multiple outputs, like this:

set_log_output({ STDOUT, "debug.log" }) 

When writing to files, the logger defaults to append mode. If you want to overwrite the file each time, prepend the filename with an exclamation mark:

set_log_output({ STDOUT, "!debug.log" }) -- overwrite debug.log each time 

Part of the idea of the logger is to have it color-code its output to the console to make it easier to read in real-time while the application is running.

Logging Formats

You don't need to pretty-print data you send to the logger routines. They're already built to pretty-print anything that isn't already a string. So you're just needlessly double-escaping things like quotes and backslashes,

So you can change this:

sequence path_info = map:get( request, "PATH_INFO" ) 
log_info( "path_info = %s", {pretty_sprint(path_info, {2})} ) 

2020/11/26 11:41:13  INFO static@mysql_users.ex:22 path_info = "\"/static/css/w3.css\"" 
2020/11/26 11:41:13  INFO static@mysql_users.ex:31 local_file = "\"C:\\\\Users\\\\duro\\\\Documents\\\\mysql_users\\\\examples\\\\static\\\\css\\\\w3.css\"" 

To just this:

sequence path_info = map:get( request, "PATH_INFO" ) 
log_info( "path_info = %s", {path_info} ) 

2020/11/26 11:41:13  INFO static@mysql_users.ex:22 path_info = "/static/css/w3.css" 
2020/11/26 11:41:13  INFO static@mysql_users.ex:31 local_file = "C:\\Users\\duro\\Documents\\mysql_users\\examples\\static\\css\\w3.css" 

Part of the idea of Euphoria MVC is to reduce the friction on these kinds of things and help you write cleaner, simpler code.

-Greg

new topic     » goto parent     » topic index » view message » categorize

19. Re: Euphoria MVC updates

Thank you Greg,

I updated my MySQL example for euphoria-mvc: http://jean-marc.duro.pagesperso-orange.fr/mysql_users.zip

euphoria-mvc is a real game-changer in web programming for me.

Jean-Marc

new topic     » goto parent     » topic index » view message » categorize

20. Re: Euphoria MVC updates

jmduro said...

I updated my MySQL example for euphoria-mvc: http://jean-marc.duro.pagesperso-orange.fr/mysql_users.zip

Thanks, I'll take a look at it!

jmduro said...

euphoria-mvc is a real game-changer in web programming for me.

That's great to hear. I hope you get some good use out of it and please don't hesitate to ask questions here. Bug fixes and feature requests can be issues over on GitHub.

-Greg

new topic     » goto parent     » topic index » view message » categorize

21. Re: Euphoria MVC updates

I overhauled mvc/config.e today.

https://github.com/OpenEuphoria/euphoria-mvc/releases/tag/v1.13.0

If your configuration looks like this:

# app.cfg 
[database] 
name = mydatabase 
username = dbuser 
password = abc123 
host = db1.example.com 

You can load it in your app like this:

include mvc/config.e 
include mvc/logger.e 
 
if not load_config( "app.cfg" ) then 
    log_crash( "Failed to load configuration" ) 
end if 
 
sequence db_host, db_name, db_user, db_pass 
db_host = get_config( "database.host" ) 
db_name = get_config( "database.name" ) 
db_user = get_config( "database.username" ) 
db_pass = get_config( "database.password" ) 

And it will even preserve comments when saving!

-Greg

new topic     » goto parent     » topic index » view message » categorize

22. Re: Euphoria MVC updates

Euphoria MVC development server now supports POST requests, among several other small changes I've been hanging onto for the past year or so.

https://github.com/OpenEuphoria/euphoria-mvc/releases/tag/v1.14.0

I'm still working on an overhaul of the model library for cleaner object-relation mapping, as well as database instantiation and migrations.

Suggestions to all that are always welcome.

-Greg

new topic     » goto parent     » topic index » view message » categorize

23. Re: Euphoria MVC updates

Hi Greg,

To avoid any development server crashes and problems with web server versions, I decided to use lighttpd which is light, cross platform and allows to use dedicated configuration files per project.

In each project folder, I only need to create 4 sub-folders:

├───conf 
├───logs 
├───tmp 
└───www 

logs and tmp are empty. conf contains a copy of lighttpd conf subfolder with parameters specific to the project.

cgi.assign = (  
#-- Most relevant line:  
  ".ex"  => "C:\euphoria4.1_32\bin\eui",   
#-- Shrouded Euphoria scripts as CGI, runs faster:    
  ".il"  => "C:\euphoria4.1_32\bin\eub",   
  ".cgi" => "",  
)  

Euphoria files are located in www. In the project root folder, I create a file to launch lighttpd with the specific configuration.

C:\data\euphoria\v4\Perso\server>type lighttpd.bat 
"C:\WinApps\lighttpd\lighttpd.exe" -D 

It never crashes and reports OEU errors as usual.

Unfortunately, it shows errors with example3.ex on Windows:

C:\data\euphoria\v4\Perso\server>"C:\WinApps\lighttpd\lighttpd.exe" -D 
2021-10-26 08:41:18: (server.c.1083) trying to read configuration (test mode) 
2021-10-26 08:41:18: (server.c.1471) server started (lighttpd/1.4.48) 
C:\euphoria4.1_32\include\mvc\utils.e:188 
<0074>:: Errors resolving the following references: 
    'hwnd' (C:\euphoria4.1_32\include\mvc\utils.e:188) has not been declared. 
    'TRUE' (C:\euphoria4.1_32\include\mvc\utils.e:188) has not been declared. 
 
    c_func( xShellExecuteA, {hwnd,allocate_string("open",TRUE),allocate_string(url,TRUE),NULL,NULL,SW_SHOWNORMAL} ) 
                                                                                    ^ 
 
 
Press Enter 

With example4.ex and last line replaced by app:run(), there is one more error:

    'LOG_ALL' (example4.ex:45) has not been declared. 

If I comment the unneeded start_url procedure in mvc/utils.e, I get an HTTP 500 Internal error with following warnings first time:

2021/10/26 09:05:04  WARN getenv@utils.e:54 Environment variable not found: "SERVER_SIGNATURE" 
2021/10/26 09:05:04  WARN getenv@utils.e:54 Environment variable not found: "PATH_INFO" 
2021/10/26 09:05:04  WARN getenv@utils.e:54 Environment variable not found: "QUERY_STRING" 

On next attempts, warnings are not displayed anymore but this error:

2021-10-26 09:05:53: (mod_cgi.c.1031) CGI pid 6088 died with signal 11 

Jean-Marc

new topic     » goto parent     » topic index » view message » categorize

24. Re: Euphoria MVC updates

I think I've corrected all of these now. Please pull down the latest changes and try it again.

-Greg

new topic     » goto parent     » topic index » view message » categorize

25. Re: Euphoria MVC updates

Hello Greg,

I have no additional error now but still an HTTP 500 Error as response.

I have put on my Web page my testing code which contains Lighttpd (Windows, 9.39 MB):

http://jean-marc.duro.pagesperso-orange.fr/testMVC.zip

I expects OEU to be in c:\euphoria. This can be changed in conf/lighttpd.conf.

Regards

Jean-Marc

new topic     » goto parent     » topic index » view message » categorize

26. Re: Euphoria MVC updates

I found one of my errors: I was missing the templates folder. euphoria-mvc does not warn about missing files.

I added the templates folder with files layout.html, index.html and user.html and somme logging.

├───conf  
├───logs  
├───templates  
├───tmp  
└───www  

I still get the same "500 - Internal Server Error" with following code (modified example3.ex):

-- 
-- This is just a basic example of the path routing engine. 
-- 
 
include std/map.e 
include std/filesys.e 
include std/search.e 
include std/io.e 
include std/pretty.e 
include mvc/app.e 
include mvc/template.e 
include mvc/logger.e 
 
if search:ends( "examples", current_dir() ) then 
    -- make sure we can find our templates 
    -- if we're in the 'examples' directory 
    set_template_path( "../templates" ) 
end if 
 
function user_id( object request ) 
    log_info( "user_id: request: %s", {pretty_sprint(request, {2})} ) 
    integer id = map:get( request, "id", -1 ) 
    map response = map:new() 
    map:put( response, "title", "User" ) 
    map:put( response, "id", id ) 
 
    return render_template( "user.html", response ) 
end function 
app:route( "/user/<id:integer>", "user_id" ) 
 
function user_name( object request ) 
    log_info( "user_name: request: %s", {pretty_sprint(request, {2})} ) 
    sequence name = map:get( request, "name", "undefined" ) 
    map response = map:new() 
    map:put( response, "title", "User" ) 
    map:put( response, "name", name ) 
 
    return render_template( "user.html", response ) 
end function 
app:route( "/user/<name:string>", "user_name" ) 
 
function index( object request ) 
    log_info( "index: request: %s", {pretty_sprint(request, {2})} ) 
    map response = map:new() 
    map:put( response, "title", "Index" ) 
 
    return render_template( "index.html", response ) 
end function 
app:route( "/index", "index" ) 
app:route( "/", "index" ) 
 
set_log_output("!debug.log") -- overwrite debug.log each time  
set_log_level( LOG_DEBUG ) 
app:run() 

The debug log doesn't show much information:

2021/11/06 10:16:41  INFO set_log_output@logger.e:212 Logging to "debug.log" 
2021/11/06 10:16:41 DEBUG add_function@template.e:133 registered function "url_for" with params {{"name"},{"response",0}} at routine id 37 
2021/11/06 10:16:41 DEBUG add_function@template.e:133 registered function "get_current_route" with params "" at routine id 38 
2021/11/06 10:16:41 DEBUG add_function@template.e:133 registered function "get_current_path" with params "" at routine id 39 

Jean-Marc

new topic     » goto parent     » topic index » view message » categorize

27. Re: Euphoria MVC updates

With example4.ex and the development server, I finally understood what was going wrong: in example3.ex and example4.ex, there is a condition of beeing in examples/ sub-directory. I removed the condition to get example4.ex working:

set_template_path( "../templates" ) 

instead of:

if search:ends( "examples", current_dir() ) then 
    -- make sure we can find our templates 
    -- if we're in the 'examples' directory 
    set_template_path( "../templates" ) 
end if 

Now after reading once more Apache configuration on euphoria-mvc github page, I understand what I didn't before: to call euphoria-mvc code while adressing a different route, I need redirection in lighttpd configuration as mentionned under "# Send all non-existant paths to index.esp".

I thought it would be possible to call different cgi-scripts within same project folder but I understand it is not possible il I snd all non-existant paths to example3.ex. I expected a call to be http://127.0.0.1/example3.ex/index.html or better http://127.0.0.1/index.ex.

Jean-Marc

new topic     » goto parent     » topic index » view message » categorize

28. Re: Euphoria MVC updates

This is all good news!

Is Eu MVC at a point where you could make something like this: https://retroachievements.org/

I have a similar idea, but being able to use Euphoria for the database/web parts would be nice.

new topic     » goto parent     » topic index » view message » categorize

29. Re: Euphoria MVC updates

jmduro said...

With example4.ex and the development server, I finally understood what was going wrong: in example3.ex and example4.ex, there is a condition of beeing in examples/ sub-directory. I removed the condition to get example4.ex working:

set_template_path( "../templates" ) 

instead of:

if search:ends( "examples", current_dir() ) then 
    -- make sure we can find our templates 
    -- if we're in the 'examples' directory 
    set_template_path( "../templates" ) 
end if 

I tried to make sure it would pick up the "templates" directory if you ran it from either the root project directory or the "examples" directory. If that's still no working then it needs more attention. How exactly are you running the examples?

jmduro said...

Now after reading once more Apache configuration on euphoria-mvc github page, I understand what I didn't before: to call euphoria-mvc code while adressing a different route, I need redirection in lighttpd configuration as mentionned under "# Send all non-existant paths to index.esp".

I thought it would be possible to call different cgi-scripts within same project folder but I understand it is not possible il I snd all non-existant paths to example3.ex. I expected a call to be http://127.0.0.1/example3.ex/index.html or better http://127.0.0.1/index.ex.

That's not really the intent here. This isn't meant to be a traditional "CGI script" that is called directly by the web server. It's meant to be path-based routing framework, so the web server should send any or all requests directly to the application for processing. I think the problem with the examples is that they're meant to either be run by a properly-configured web server or act as their own development server, and I've not made that clear. I need to make it more obvious which is the case for each example.

Currently, you have to either call app:run() when you're using a web server, or call server:start() to run the development server. Ultimately, I'd like to make it entirely transparent: if you run your application directly, it should fire up the built-in web server or refuse to start, and if your app is run by a web server it should determine this and just handle the route, produce the output, and then exit.

This feedback and testing is all very helpful though. Thank you!

-Greg

new topic     » goto parent     » topic index » view message » categorize

30. Re: Euphoria MVC updates

Icy_Viking said...

This is all good news!

Is Eu MVC at a point where you could make something like this: https://retroachievements.org/

Probably! Most of the work would be in the design and layout of the front end and back end parts, mainly the page templates, CSS, etc. as well as the database layout and business logic.

Icy_Viking said...

I have a similar idea, but being able to use Euphoria for the database/web parts would be nice.

I wouldn't recommend the Euphoria EDS database but otherwise this is entirely doable. EDS is fine for local storage with single-user applications, but I'd avoid it for anything large-scale and/or multi-user. It's going to be a performance bottleneck, you'll have to set up a lock/polling mechanism to do writes, and it's not atomic you could very accidentally corrupt or overwrite the wrong data.

I'd still like to go back and rework both the database and model routines to better accommodate multiple database backends (currently MySQL is "mostly" supported and SQLite support is "started") and make for easier database instantiation and migrations from code. It's just not "there" yet and I'm open to input. But you could just use the database wrappers I've made to call MySQL or SQLite directly.

I think we're going to need a command line tool to do some of the heavy lifting and boilerplate things, like artisan or dotnet. Then you could do eumvc create [project] to create a new project from a template, and then eumvc migrate to create your tables from scratch or perform any required changes over time.

-Greg

new topic     » goto parent     » topic index » view message » categorize

31. Re: Euphoria MVC updates

ghaberek said...
Icy_Viking said...

This is all good news!

Is Eu MVC at a point where you could make something like this: https://retroachievements.org/

Probably! Most of the work would be in the design and layout of the front end and back end parts, mainly the page templates, CSS, etc. as well as the database layout and business logic.

Icy_Viking said...

I have a similar idea, but being able to use Euphoria for the database/web parts would be nice.

I wouldn't recommend the Euphoria EDS database but otherwise this is entirely doable. EDS is fine for local storage with single-user applications, but I'd avoid it for anything large-scale and/or multi-user. It's going to be a performance bottleneck, you'll have to set up a lock/polling mechanism to do writes, and it's not atomic you could very accidentally corrupt or overwrite the wrong data.

I'd still like to go back and rework both the database and model routines to better accommodate multiple database backends (currently MySQL is "mostly" supported and SQLite support is "started") and make for easier database instantiation and migrations from code. It's just not "there" yet and I'm open to input. But you could just use the database wrappers I've made to call MySQL or SQLite directly.

I think we're going to need a command line tool to do some of the heavy lifting and boilerplate things, like artisan or dotnet. Then you could do eumvc create [project] to create a new project from a template, and then eumvc migrate to create your tables from scratch or perform any required changes over time.

-Greg

Nice. My plan was to use the MySQL wrapper. I figured EDS was more for personal databases or for projects where you didn't need anything super fancy. I also figure rewriting EDS to be more flexible/powerful is probably a project of itself too.

new topic     » goto parent     » topic index » view message » categorize

32. Re: Euphoria MVC updates

Icy_Viking said...
ghaberek said...
Icy_Viking said...

This is all good news!

Is Eu MVC at a point where you could make something like this: https://retroachievements.org/

Probably! Most of the work would be in the design and layout of the front end and back end parts, mainly the page templates, CSS, etc. as well as the database layout and business logic.

Icy_Viking said...

I have a similar idea, but being able to use Euphoria for the database/web parts would be nice.

I wouldn't recommend the Euphoria EDS database but otherwise this is entirely doable. EDS is fine for local storage with single-user applications, but I'd avoid it for anything large-scale and/or multi-user. It's going to be a performance bottleneck, you'll have to set up a lock/polling mechanism to do writes, and it's not atomic you could very accidentally corrupt or overwrite the wrong data.

I'd still like to go back and rework both the database and model routines to better accommodate multiple database backends (currently MySQL is "mostly" supported and SQLite support is "started") and make for easier database instantiation and migrations from code. It's just not "there" yet and I'm open to input. But you could just use the database wrappers I've made to call MySQL or SQLite directly.

I think we're going to need a command line tool to do some of the heavy lifting and boilerplate things, like artisan or dotnet. Then you could do eumvc create [project] to create a new project from a template, and then eumvc migrate to create your tables from scratch or perform any required changes over time.

-Greg

Nice. My plan was to use the MySQL wrapper. I figured EDS was more for personal databases or for projects where you didn't need anything super fancy. I also figure rewriting EDS to be more flexible/powerful is probably a project of itself too.

It is about time Euphoria EDS was discarded in favour of something very close to SQL

new topic     » goto parent     » topic index » view message » categorize

33. Re: Euphoria MVC updates

Bhupen1277 said...

It is about time Euphoria EDS was discarded in favour of something very close to SQL

This might be a good idea, to have a more serious database system immediately available after install.

Heck, it would be a good idea to have a MySQL/PostgreSQL/Maria DB/Mongo DB interface immediately available after install.

Maybe there already is one. I don't know. grin


Forked into: Discarding EDS

new topic     » goto parent     » topic index » view message » categorize

34. Re: Euphoria MVC updates

ghaberek said...

How exactly are you running the examples?

My project structure is the following:

├───conf 
├───logs 
├───templates 
├───tmp 
└───www 

In the root folder, I run lighttpd.bat which contains following code:

"C:\WinApps\lighttpd\lighttpd.exe" -f conf\lighttpd.conf -D 

conf\lighttpd.conf contains this configuration:

server.modules              = ( 
                                "mod_access", 
                                "mod_accesslog", 
                                "mod_cgi", 
                                "mod_redirect", 
                                "mod_rewrite", 
                                "mod_status", 
                               ) 
 
## server root directory (default: 'CWD') 
var.server_root = CWD 
 
## logging directory (it isn't used by default) 
var.log_root    = server_root + "/logs" 
 
## directory for temporary files (e.g. cache, upload, etc.) 
var.temp_dir    = server_root + "/tmp" 
 
server.document-root        = server_root + "/www" 
 
server.upload-dirs          = ( temp_dir ) 
 
index-file.names            = ( "index.ex", "index.pl", "index.cgi", 
                                "index.html", "index.htm", "default.htm" ) 
 
server.event-handler        = "libev" 
 
url.access-deny = ( "~", ".inc", ".cfg", ".err", ".log", ".e" ) 
 
$HTTP["url"] =~ "\.pdf$" { 
  server.range-requests = "disable" 
} 
 
static-file.exclude-extensions = ( ".pl", ".cgi" ) 
 
server.errorlog = server_root + "/logs/lighttpd.errors" 
server.port                = 80 
 
dir-listing.activate        = "enable" 
dir-listing.encoding       = "utf-8" 
 
debug.log-request-header   = "enable" 
debug.log-response-header  = "enable" 
debug.log-request-handling = "enable" 
debug.log-file-not-found   = "enable" 
 
accesslog.filename          = server_root + "/logs/access.log" 
 
cgi.assign = (  
  ".esp"  => "C:/euphoria/bin/eui",   
  ".il"  => "C:/euphoria/bin/eub",   
  ".cgi" => "",  
)  
 
status.status-url           = "/server-status" 
status.config-url           = "/server-config" 
 
url.rewrite-if-not-file = ( 
	"^(.*)$" => "index.esp/$1" 
) 

When I want to access http://127.0.0.1/index, the browser asks to download index as a file but it cannot cause the file doesn't exist.

logs/debug.log contains following lines:

2021/11/09 10:57:19  INFO set_log_output@logger.e:212 Logging to "..\\logs\\debug.log" 
2021/11/09 10:57:19 DEBUG add_function@template.e:133 registered function "url_for" with params {{"name"},{"response",0}} at routine id 37 
2021/11/09 10:57:19 DEBUG add_function@template.e:133 registered function "get_current_route" with params "" at routine id 38 
2021/11/09 10:57:19 DEBUG add_function@template.e:133 registered function "get_current_path" with params "" at routine id 39 
2021/11/09 10:57:19  INFO index@index.esp:39 index: request: "13" 

Regards

Jean-Marc

new topic     » goto parent     » topic index » view message » categorize

35. Re: Euphoria MVC updates

logs/access.log containes:

127.0.0.1 127.0.0.1 - [09/Nov/2021:10:57:19 +0100] "GET /index HTTP/1.1" 200 18 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:94.0) Gecko/20100101 Firefox/94.0" 

logs/lighttpd.errors contains:

2021-11-09 10:57:18: (request.c.445) fd: 8 request-len: 461 \nGET /index HTTP/1.1\r\nHost: 127.0.0.1\r\nUser-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:94.0) Gecko/20100101 Firefox/94.0\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8\r\nAccept-Language: fr,fr-FR;q=0.8,en-US;q=0.5,en;q=0.3\r\nAccept-Encoding: gzip, deflate\r\nConnection: keep-alive\r\nUpgrade-Insecure-Requests: 1\r\nSec-Fetch-Dest: document\r\nSec-Fetch-Mode: navigate\r\nSec-Fetch-Site: none\r\nSec-Fetch-User: ?1\r\n\r\n  
2021-11-09 10:57:18: (response.c.388) -- splitting Request-URI  
2021-11-09 10:57:18: (response.c.389) Request-URI     :  /index  
2021-11-09 10:57:18: (response.c.390) URI-scheme      :  http  
2021-11-09 10:57:18: (response.c.391) URI-authority   :  127.0.0.1  
2021-11-09 10:57:18: (response.c.392) URI-path (raw)  :  /index  
2021-11-09 10:57:18: (response.c.393) URI-path (clean):  /index  
2021-11-09 10:57:18: (response.c.394) URI-query       :    
2021-11-09 10:57:18: (mod_access.c.159) -- mod_access_uri_handler called  
2021-11-09 10:57:18: (response.c.532) -- before doc_root  
2021-11-09 10:57:18: (response.c.533) Doc-Root     : D:/Data/Euphoria/testMVC/www  
2021-11-09 10:57:18: (response.c.534) Rel-Path     : /index  
2021-11-09 10:57:18: (response.c.535) Path         :   
2021-11-09 10:57:18: (response.c.587) -- after doc_root  
2021-11-09 10:57:18: (response.c.588) Doc-Root     : D:/Data/Euphoria/testMVC/www  
2021-11-09 10:57:18: (response.c.589) Rel-Path     : /index  
2021-11-09 10:57:18: (response.c.590) Path         : D:/Data/Euphoria/testMVC/www/index  
2021-11-09 10:57:18: (response.c.388) -- splitting Request-URI  
2021-11-09 10:57:18: (response.c.389) Request-URI     :  index.esp//index  
2021-11-09 10:57:18: (response.c.390) URI-scheme      :  http  
2021-11-09 10:57:18: (response.c.391) URI-authority   :  127.0.0.1  
2021-11-09 10:57:18: (response.c.392) URI-path (raw)  :  index.esp//index  
2021-11-09 10:57:18: (response.c.393) URI-path (clean):  /index.esp/index  
2021-11-09 10:57:18: (response.c.394) URI-query       :    
2021-11-09 10:57:18: (mod_access.c.159) -- mod_access_uri_handler called  
2021-11-09 10:57:18: (response.c.532) -- before doc_root  
2021-11-09 10:57:18: (response.c.533) Doc-Root     : D:/Data/Euphoria/testMVC/www  
2021-11-09 10:57:18: (response.c.534) Rel-Path     : /index.esp/index  
2021-11-09 10:57:18: (response.c.535) Path         :   
2021-11-09 10:57:18: (response.c.587) -- after doc_root  
2021-11-09 10:57:18: (response.c.588) Doc-Root     : D:/Data/Euphoria/testMVC/www  
2021-11-09 10:57:18: (response.c.589) Rel-Path     : /index.esp/index  
2021-11-09 10:57:18: (response.c.590) Path         : D:/Data/Euphoria/testMVC/www/index.esp/index  
2021-11-09 10:57:18: (response.c.607) -- logical -> physical  
2021-11-09 10:57:18: (response.c.608) Doc-Root     : D:/Data/Euphoria/testMVC/www  
2021-11-09 10:57:18: (response.c.609) Basedir      : D:/Data/Euphoria/testMVC/www  
2021-11-09 10:57:18: (response.c.610) Rel-Path     : /index.esp/index  
2021-11-09 10:57:18: (response.c.611) Path         : D:/Data/Euphoria/testMVC/www/index.esp/index  
2021-11-09 10:57:18: (response.c.623) -- handling physical path  
2021-11-09 10:57:18: (response.c.624) Path         : D:/Data/Euphoria/testMVC/www/index.esp/index  
2021-11-09 10:57:18: (response.c.631) -- handling subrequest  
2021-11-09 10:57:18: (response.c.632) Path         : D:/Data/Euphoria/testMVC/www/index.esp  
2021-11-09 10:57:18: (response.c.633) URI          : /index.esp  
2021-11-09 10:57:18: (response.c.634) Pathinfo     : /index  
2021-11-09 10:57:18: (mod_access.c.159) -- mod_access_uri_handler called  
2021-11-09 10:57:19: (response.c.122) Response-Header: \nHTTP/1.1 200 OK\r\nContent-Type: text/html\r\r\nContent-Length: 113\r\r\nDate: Tue, 09 Nov 2021 09:57:19 GMT\r\nServer: lighttpd/1.4.49\r\n\r\n  
2021-11-09 10:57:25: (server.c.1879) connection closed - keep-alive timeout: 8  

Jean-Marc

new topic     » goto parent     » topic index » view message » categorize

36. Re: Euphoria MVC updates

I'll be honest, I've done very little testing with Lighttpd or Windows. Each web server on each operating system creates a unique configuration to support, so with Windows and Linux (and maybe OS X) and Apache and Nginx (and maybe Lighttpd) we're looking at between four and nine unique combinations. That's a lot and I'm just one person with limited time.

If you can, it would help if you could get this working yourself and report back the solution when you find it. You can test running your app on Windows directly with a command script like this:

@ECHO OFF 
SETLOCAL 
SET REQUEST_URI=%1 
IF [%REQUEST_URI%] == [] (SET REQUEST_URI=/) 
SET QUERY_STRING=REQUEST_URI=%REQUEST_URI% 
C:\Euphoria\bin\eui.exe -batch index.esp 
ENDLOCAL 

Then just run index.bat (or whatever you named it) to test the default path / and you can run index.bat /path to test other paths.

Also, have you tried using the built-in development server?

-Greg

new topic     » goto parent     » topic index » view message » categorize

37. Re: Euphoria MVC updates

The script tells me "unexpected ]".

I tried the development server which works. I'd prefer lighttpd because in some cases when the script crashes I get no error with the development server while I get one with lighttpd.

Nginx does not manage CGI otherwise it would have been my preferate server.

Jean-Marc

new topic     » goto parent     » topic index » view message » categorize

38. Re: Euphoria MVC updates

jmduro said...

The script tells me "unexpected ]".

What version of Windows are you on? It worked fine for me on Windows 10. You can just replace the brackets with quotes:

@ECHO OFF 
SETLOCAL 
SET REQUEST_URI=%1 
IF "%REQUEST_URI%" == "" (SET REQUEST_URI=/) 
SET QUERY_STRING=REQUEST_URI=%REQUEST_URI% 
C:\Euphoria\bin\eui.exe -batch index.esp 
ENDLOCAL 

jmduro said...

I tried the development server which works. I'd prefer lighttpd because in some cases when the script crashes I get no error with the development server while I get one with lighttpd.

Interesting. Can you reproduce this with a small example? One thing I've been trying to ensure is that logging and crashing take priority for development.

jmduro said...

Nginx does not manage CGI otherwise it would have been my preferate server.

Yeah that's fair. I think when I was testing with Nginx I was using it as a reverse proxy for the development server. Ideally I'd like to support something like FastCGI or WSGI.

-Greg

new topic     » goto parent     » topic index » view message » categorize

39. Re: Euphoria MVC updates

I'm using Win10 but no Powershell. I dit not understand how to use the batch script with lighttpd.

Here is the failing example. Its provides and empty debug.log.

#!/usr/local/euphoria-4.1.0-Linux-x64/bin/eui 
 
include std/map.e 
include std/filesys.e  -- for canonical_path()  
include std/search.e   -- for search:begins()  
include std/text.e     -- for dequote 
include std/io.e       -- for STDOUT 
include std/pretty.e 
include mvc/app.e 
include mvc/server.e 
include mvc/template.e 
include mvc/logger.e 
include mvc/database.e 
include mvc/db_sqlite3.e 
 
public function list_users() 
  atom conn = db_connect( "sqlite3:test.sqlite" ) 
  atom result = db_query( "SELECT * FROM users ORDER BY login" ) 
  object row = {} 
  sequence user_list = {} 
  while sequence( row ) do 
    row = db_fetch( result ) 
    if sequence( row ) then 
      sequence user = { 
        { "id", row[1] }, 
        { "login", row[2] }, 
        { "password", row[3] } 
      } 
      log_info( "user: %s", {pretty_sprint(user, {2})} ) 
      user_list &= { map:new_from_kvpairs(user) } 
    end if 
  end while 
 
  db_free( result ) 
  db_disconnect() 
  return user_list 
end function 
 
public function insert_user(sequence login, sequence password) 
  log_info( "Inserting user %s", {login} ) 
  atom conn = db_connect( "sqlite3:test.sqlite" ) 
  atom result = db_query( sprintf("INSERT INTO users(login, password) VALUES ('%s','%s')", {login, password}) ) 
  integer nb = db_affected_rows() 
  db_disconnect() 
  return nb 
end function 
 
public function delete_user(sequence id) 
  log_info( "Deleting user %s", {id} ) 
  atom conn = db_connect( "sqlite3:test.sqlite" ) 
  atom result = db_query( sprintf("DELETE FROM users WHERE id=%s", {id}) ) 
  integer nb = db_affected_rows() 
  db_disconnect() 
  return nb 
end function 
 
public function check_user(sequence login, sequence password) 
  log_info( "Checking user %s", {login} ) 
  atom conn = db_connect( "sqlite3:test.sqlite" ) 
  atom result = db_query( sprintf("SELECT id FROM users WHERE login = '%s' and password = '%s'", {login, password}) ) 
  sequence user_list = {} 
  sequence row = db_fetch(result) 
  for i = 1 to length(row) do 
      sequence user = { 
        { "id", row[i] } 
      } 
      log_info( "user: %s", {pretty_sprint(user, {2})} ) 
      user_list &= { map:new_from_kvpairs(user) } 
  end for 
 
  db_free( result ) 
  db_disconnect() 
  return length(user_list) 
end function 
 
set_template_path( "../templates" ) 
 
function static( object request )  
  sequence path_info = map:get( request, "PATH_INFO" )  
  log_info( "path_info = %s", {path_info} ) 
  
  sequence local_file = canonical_path( "." & path_info )  
  log_info( "local_file = %s", {local_file} ) 
 
  if search:begins( current_dir(), local_file ) then  
    return send_file( local_file )  
  
  end if  
  
  return response_code( 404 )  
end function  
app:route( "/static/.+", "static" )  
 
function login( object request ) 
  sequence method = map:get(request, "REQUEST_METHOD", "") 
  object response = map:new() 
  sequence login = "", password = "",  err = "" 
   
  if equal(method, "POST") then 
    login = map:get(request, "login", "") 
    password = map:get(request, "password", "") 
    integer count = check_user(login, password) 
    if count = 1 then 
      map:put( response, "user", login ) 
      return render_template( "main.html", response ) 
    else 
      err = "Your Login Name or Password is invalid" 
    end if 
  end if 
 
  map:put( response, "user", login ) 
  map:put( response, "error", err ) 
  return render_template( "login.html", response ) 
end function 
app:route( "/", "login", {"GET","POST"} ) 
 
function ins( object request ) 
  object response = map:new() 
   
  map:put( response, "user", "test" ) 
  return render_template( "insert.html", response ) 
end function 
app:route( "/insert", "ins" ) 
 
function users( object request ) 
  log_info( "query = %s", {map:get(request, "QUERY_STRING", "")} ) 
  sequence method = map:get(request, "REQUEST_METHOD", "") 
  sequence err = "" 
  if equal(method, "POST") and map:has(request, "save") then 
    sequence login = map:get(request, "login", "") 
    sequence password = map:get(request, "password", "") 
    insert_user(login, password) 
  elsif  equal(method, "GET") and map:has(request, "del") then 
    sequence id = map:get( request, "del", "" ) 
    log_info( "id = %s", {id} ) 
    if length(id) then 
      delete_user(id) 
    else 
      err = "User ID not found" 
    end if 
  end if 
 
  object response = map:new() 
  sequence user_list = list_users() 
  map:put( response, "user", "test" ) 
  map:put( response, "user_list", user_list ) 
  map:put( response, "error", err ) 
  return render_template( "users.html", response ) 
end function 
app:route( "/users", "users", {"GET","POST"} ) 
 
set_log_output({ STDOUT, "!debug.log" }) -- overwrite debug.log each time  
set_log_level( LOG_DEBUG ) 
server:start("0.0.0.0", 8080) 

It may be related to the use of a DLL. When Euphoria crashes whle using a DLL, it usually behaves so (no trace).

Jean-Marc

new topic     » goto parent     » topic index » view message » categorize

40. Re: Euphoria MVC updates

It looks like a missing sqlite database caused the crash. When adding the file, the server runs and crashes with traces on the next error.

Jean-Marc

new topic     » goto parent     » topic index » view message » categorize

41. Re: Euphoria MVC updates

II also needed to replace "include std/map.e" with "include mvc/mapdbg.e as map"

new topic     » goto parent     » topic index » view message » categorize

42. Re: Euphoria MVC updates

jmduro said...

It looks like a missing sqlite database caused the crash. When adding the file, the server runs and crashes with traces on the next error.

Yeah nothing your code is creating a database so that needs to exist already. This is what I was talking about in the other thread about migrations. We need a way to "bootstrap" an application and its database.

I should probably include a call to flush() in the inner loop of log_message(). I have a feeling some things are getting logged but they're buffered so you never see them.

I also plan to add one or more crash routines to provide better crash output to the browser, close log files, and disconnect any databases.

I definitely think browser crash messages from the development server would be helpful. Right now the browser just spins waiting for a response while the server has crashed.

jmduro said...

II also needed to replace "include std/map.e" with "include mvc/mapdbg.e as map"

You shouldn't have to do that since, as its name implies, mapdbg is only for debugging maps. You have to have MAPDBG defined in eu.cfg or on the command line (-D MAPDBG) for it to work anyway. Otherwise it just acts as a passthrough to std/map.e.

namespace mapdbg 
 
ifdef not MAPDBG then 
 
public include std/map.e 
 
elsedef 
 
... 

What mapdbg does, if you're curious, is trace any calls to maps that are created so you can verify they're being cleaned up correctly. I was having issues with the development server early on, where continuous use would create maps without freeing them, resulting in a memory leak. I think I've got that under control now. It also provides a neat print_map() function for seeing the (possibly nested) structure of the maps you've created. There is also print_maps() which will display all of the maps it knows about at that time.

-Greg

new topic     » goto parent     » topic index » view message » categorize

43. Re: Euphoria MVC updates

Thank you Greg,

url_parse() may not be the best function to use with sqlite3 local files. I am trying many ways to indicate to db_connect() where my local sqlite3 database is but _connect() which is called by db_connect() alway fails because url_parse() returns name=0.

I am actually testing url_parse() with different syntaxes to find the right one to use, but I think regex would be best adapted to local files.

Jean-Marc

new topic     » goto parent     » topic index » view message » categorize

44. Re: Euphoria MVC updates

If I remove both "include std/map.e" and "include mvc/mapdbg.e as map" the development server runs. It crashes later while using the script.

If I use only "include std/map.e", the server crashes instantly without any debug trace.

If I use only "include mvc/mapdbg.e as map", the serevr runs and crashes later.

I found the right syntax for sqlite3 local files to comply with url_parse(), thanks to Firefox:

constant DB_URL = "sqlite3:file:///D:/Data/Euphoria/v4/Personnel/mysql_users/users.sqlite" 

proto = sqlite3, name = "///D:/Data/Euphoria/v4/Personnel/mysql_users/users.sqlite" 

However, the script crashes instantly with no debug trace. I then added flush() to log_messages() and here are the last debug lines after this change:

2021/11/13 10:54:46 DEBUG db_connect@database.e:83 url = "sqlite3:file:///D:/Data/Euphoria/v4/Personnel/mysql_users/users.sqlite", timeout = 5000 
2021/11/13 10:54:46 DEBUG _connect@db_sqlite3.e:24 name = "\"///D:/Data/Euphoria/v4/Personnel/mysql_users/users.sqlite\"" 

Jean-Marc

new topic     » goto parent     » topic index » view message » categorize

45. Re: Euphoria MVC updates

Here is a test script:

include std/net/url.e 
include std/io.e 
include std/console.e 
include db/sqlite3.e 
 
constant DB_URL = "sqlite3:file:///D:/Data/Euphoria/v4/Personnel/mysql_users/users.sqlite"  
 
atom result, db 
integer f_debug = open("debug.log", "w") 
object proto, host, user, passwd, name, port 
{proto,host,port,name,user,passwd, ?} = url:parse( DB_URL, 1 ) 
-- name = name[4..$] 
printf(f_debug, "proto = %s, name = %s\n", {proto, name} ) 
flush(f_debug) 
{result,db} = sqlite3_open( name ) 
puts(f_debug, "DB is opened!\n") 
sqlite3_close( db ) 
close(f_debug) 
 
maybe_any_key() 

It always fails on sqlite3_open(), even if I remove the "///" prefix. I checked with HEXAED: D:\Data\Euphoria\v4\Personnel\mysql_users\users.sqlite is an existing binary sqlite3 file.

Jean-Marc

new topic     » goto parent     » topic index » view message » categorize

46. Re: Euphoria MVC updates

I would suggest using following function instead of url_parse():

include std/regex.e as re 
include std/pretty.e 
include std/filesys.e 
include std/sequence.e 
include std/convert.e 
include std/console.e 
--include sqlite3.e 
 
constant URLS = { 
  "http://user:pass@www.debian.org:80/index.html?name=John&age=39", 
  "http://www.debian.org:80/index.html?name=John&age=39", 
  "http://www.debian.org/index.html?name=John&age=39", 
  "http://user:pass@www.debian.org/index.html?name=John&age=39", 
  "http://user:pass@www.debian.org:80", 
  "http://user:pass@www.debian.org:80/", 
  "http://user:pass@www.debian.org:80/index.html", 
  "http://user:pass@www.debian.org/index.html", 
  "http://www.debian.org:80/index.html", 
  "http://www.debian.org/index.html", 
  "file:///D:/Data/Euphoria/v4/Personnel/mysql_users/users.sqlite" 
} 
 
constant re_url = re:new("([a-z]+):(//+)(.*)") 
 
function parse_url(sequence url) 
  sequence s = "", user="", pass="", host="", path = "", query = "" 
  integer port=0 
  object matches = re:matches(re_url, url) 
  --pretty_print(1, matches, {2}) 
  --puts(1, "\n") 
  if equal("file", matches[2]) then 
    return {matches[2], mapping(matches[4], "/", {SLASH})} 
  else 
    if find('/', matches[4]) then 
      s = stdseq:split(matches[4], '/') 
      path = s[2] 
      url = s[1] 
    else 
      url = matches[4] 
    end if 
    if find('@', url) then 
      s = stdseq:split(url, '@') 
      {user, pass} = stdseq:split(s[1], ':') 
      url = s[2] 
    end if 
    if find(':', url) then 
      s = stdseq:split(url, ':') 
      host = s[1] 
      port = to_number(s[2]) 
    else 
      host = url 
    end if 
    if find('?', path) then 
      s = stdseq:split(path, '?') 
      path = s[1] 
      query = s[2] 
    else 
    end if 
    return {matches[2], user, pass, host, port, path, query} 
  end if 
end function 
 
for i = 1 to length(URLS) do 
  puts(1, URLS[i] & "\n") 
  sequence matches = parse_url(URLS[i]) 
  pretty_print(1, matches, {2}) 
  puts(1, "\n") 
/* 

  if equal(matches[1], "file") then 
    atom db = sqlite_open( matches[2] ) 
    puts(1, "DB is opened!\n") 
    sqlite_close( db ) 
  end if 
*/ 
end for 
 
maybe_any_key() 

Using my own sqlite3 wrapper, sqlite_open() does work.

If I use openeuphoria-mvc sqlite3_open(), database opening fails.

Jean-Marc

new topic     » goto parent     » topic index » view message » categorize

47. Re: Euphoria MVC updates

jmduro said...

If I remove both "include std/map.e" and "include mvc/mapdbg.e as map" the development server runs. It crashes later while using the script.

If I use only "include std/map.e", the server crashes instantly without any debug trace.

If I use only "include mvc/mapdbg.e as map", the serevr runs and crashes later.

I found the right syntax for sqlite3 local files to comply with url_parse(), thanks to Firefox:

constant DB_URL = "sqlite3:file:///D:/Data/Euphoria/v4/Personnel/mysql_users/users.sqlite" 

proto = sqlite3, name = "///D:/Data/Euphoria/v4/Personnel/mysql_users/users.sqlite" 

However, the script crashes instantly with no debug trace. I then added flush() to log_messages() and here are the last debug lines after this change:

2021/11/13 10:54:46 DEBUG db_connect@database.e:83 url = "sqlite3:file:///D:/Data/Euphoria/v4/Personnel/mysql_users/users.sqlite", timeout = 5000 
2021/11/13 10:54:46 DEBUG _connect@db_sqlite3.e:24 name = "\"///D:/Data/Euphoria/v4/Personnel/mysql_users/users.sqlite\"" 

Jean-Marc

Oh wow I didn't realize this is what you were doing until now. I haven't tried putting paths in URL parameter for SQLite3. The most I've done is "sqlite3://filename.db".

We'll need to update the connect handler in mvc/db_sqlite3.e to accommodate this better, as it's only pulling the name value from the parsed URL.

Does it work if you just give it a single file name without a path?

function _connect( sequence url, integer timeout ) 
 
    object proto, host, user, passwd, name, port 
    {proto,host,port,name,user,passwd,?} = url:parse( url ) 
 
    atom result, db 
 
    {result,db} = sqlite3_open( name ) 
 
    if result != SQLITE_OK then 
        return 0 
    end if 
 
    return db 
end function 
add_handler( SQLITE3, DB_CONNECT, routine_id("_connect") ) 

-Greg

new topic     » goto parent     » topic index » view message » categorize

48. Re: Euphoria MVC updates

With both "sqlite3:test.sqlite" and "sqlite3://test.sqlite", url:parse returns "filename = 0" on Windows 10.

Jean-Marc

new topic     » goto parent     » topic index » view message » categorize

49. Re: Euphoria MVC updates

Usually, when using a DLL with OpenEuphoria and a problem occurs while using the DLL, OEU crashes without traces.

I noticed using multitasking there is a way to avoid this. Let's take an empty SQLite3 database.

procedure task_update() 
  -- update a table here 
end procedure 
 
atom t1= task_create(routine_id("task_update", {}) 
task_schedule(t1, 1) 

If you try to update an non existing table with an SQLite3 wrapper, program will not crash as usual this way.

Jean-Marc

new topic     » goto parent     » topic index » view message » categorize

50. Re: Euphoria MVC updates

jmduro said...

With both "sqlite3:test.sqlite" and "sqlite3://test.sqlite", url:parse returns "filename = 0" on Windows 10.

I will do some more testing and make sure the database library knows how to handle this correctly. In the mean time, maybe SQLite3 isn't ready to be used with Euphoria MVC.

jmduro said...

Usually, when using a DLL with OpenEuphoria and a problem occurs while using the DLL, OEU crashes without traces.

This is correct. If something causes the DLL to crash, it takes the entire application down with it, and control never comes back to Euphoria to inform you of the crash. Not much can be done about this; it's just the way executables and shared libraries interact with each other. This may require debugging with gdb to determine where the crash occurs and why. Can you reproduce it with a small example using only the SQLite3 wrapper and nothing else?

jmduro said...

I noticed using multitasking there is a way to avoid this. Let's take an empty SQLite3 database.

procedure task_update() 
  -- update a table here 
end procedure 
 
atom t1= task_create(routine_id("task_update", {}) 
task_schedule(t1, 1) 

This... is a bizarre solution to the problem. I would advise trying to track down why and where the issue occurs. There's got to be a reason.

jmduro said...

If you try to update an non existing table with an SQLite3 wrapper, program will not crash as usual this way.

Is this the reason it's crashing? You need to ensure your tables are created; there's no working around that.

-Greg

new topic     » goto parent     » topic index » view message » categorize

51. Re: Euphoria MVC updates

I didn't explain well what I expect. I have intentionally provoked the crash by selecting a non-existing table in the SQLite database.

I'm using OpenEuphoria because of its debugging features. I need to get traces even when a DLL results in a crash. I expect getting an error message by avoiding an external crash caused by a DLL. Instead I want Euphoria to get a return code when possible and then to provoke a crash after logging a message. That is why I tested this "bizarre" method using multitasking. This is how:

integer return_code 
 
procedure task_update()  
  sequence cmd = sprintf("UPDATE \"devices\" SET \"%s\" = \"%s\" WHERE \"rowid\" = %d;", 
                         {"Last_update", deviceList[selected][LAST_UPDATE], selected}) 
  return_code =  sqlite_get_table(sql_db, cmd) 
end procedure  
  
atom t1= task_create(routine_id("task_update", {})  
task_schedule(t1, 1) 
if return_code != SQLITE_OK then 
  error:crash("Update failed!") 
end if 

Jean-Marc

new topic     » goto parent     » topic index » view message » categorize

52. Re: Euphoria MVC updates

jmduro said...

I didn't explain well what I expect. I have intentionally provoked the crash by selecting a non-existing table in the SQLite database.

I'm using OpenEuphoria because of its debugging features. I need to get traces even when a DLL results in a crash. I expect getting an error message by avoiding an external crash caused by a DLL. Instead I want Euphoria to get a return code when possible and then to provoke a crash after logging a message. That is why I tested this "bizarre" method using multitasking. This is how:

Oh okay, that's fair. You're right that we should get back a response from the DLL. Something else may be going on.

Which version and architecture of Euphoria are you using? I have been developing against Euphoria 4.1 64-bit.

-Greg

new topic     » goto parent     » topic index » view message » categorize

53. Re: Euphoria MVC updates

ghaberek said...

I think I've corrected all of these now. Please pull down the latest changes and try it again.

-Greg

c:\eu4\MY\euphoria-mvc-1.14.0\examples\example4.ex:45 
<0074>:: Errors resolving the following references: 
	'hwnd' (..\include\mvc\utils.e:188) has not been declared. 
	'TRUE' (..\include\mvc\utils.e:188) has not been declared. 
	'LOG_ALL' (example4.ex:45) has not been declared. 
 
set_log_level( LOG_ALL ) 
                       ^ 

new topic     » goto parent     » topic index » view message » categorize

Search



Quick Links

User menu

Not signed in.

Misc Menu