OpenEuphoria

1. json

• Posted by bugmagnet Aug 01, 2012
• 2929 views

I've noticed that this subject hasn't created a lot of interest. Pity. I need a json to euphoria translator.

Now that we have a "map" it would/should/could be a trivial task?

With that in mind, can maps be embedded in maps? How do you tell, when traversing a sequence, whether the next embedded sequence is a map or not?

Kind regards,
Bruce/bugmagnet

3. Re: json

• Posted by bugmagnet Aug 02, 2012
• 2981 views

Okay, I've found some other implementations (C, C#, javascript etc) and will give converting one them a try. The C# looks most promising at this point.

For those who want to beat me to it (and they will, given my workload), the ones I've found are as follows:
http://sourceforge.net/projects/mjson/
https://github.com/douglascrockford/JSON-js
http://techblog.procurios.nl/k/news/view/14605/14863/How-do-I-write-my-own-parser-for-JSON.html
http://lloyd.github.com/yajl/

Kind regards,
Bruce/bugmagnet

5. Re: json

• Posted by ghaberek (admin) Aug 02, 2012
• 2958 views

Ah ha! In reading this post, I did some Googling and started poking around with the code from that very same page.

Unfortunately, I had to backtrack a bit, as I feel one thing that would help make file/string parsing a bit easier in Euphoria would be more dynamic file and memory stream I/O, especially buffered I/O with a putback() or peek_next_char() function.

Here's a quick example I threw together in my head.

stream s = stream:open( filename, STREAM_READ ) 
 
while not stream:eof( s ) do 
 
    -- peek a single character (i.e. get the next character, but do not advance the 
    -- file position, so the next call to stream:getc() returns the same character) 
 
    char c = stream:peekc( s ) 
 
    switch c do 
 
        case '{' then 
            process_json_object( s ) 
 
        case '[' then 
            process_json_array( s ) 
 
        case else 
            puts( 2, "Expected '{' or '['\n" ) 
            abort( 1 ) 
 
    end switch 
 
end while 
 
stream:close( s ) 

It may also be advantageous to have the programmers of Euphoria itself add one of the open-source C-based processors to Euphoria directly. It would be nice to have a proper in-built XML parser as well.

-Greg

7. Re: json

• Posted by mattlewis (admin) Aug 02, 2012
• 2821 views

In fact, Jeremy has done some experimenting with adding Expat to the euphoria backend.

Matt

9. Re: json

• Posted by ghaberek (admin) Aug 02, 2012
• 2817 views

I've already started playing around with the concepts so I'll surely give it a whirl. At the very least, yes, they would be calls to the in-built file I/O routines, just with some "smart" buffering, etc. I would also like to be able to emulate file I/O in memory...

sequence my_data = {} -- obtain data 
atom ptr = allocate( length(my_data), 1 ) 
poke( ptr, my_data ) 
 
stream s = stream:open( ptr, STREAM_MEMORY ) 
 
-- perform I/O on this "stream" while buffers are moved 
-- and allocated to to accommodate the size of the data 
 
stream:close( s ) 

or just...

sequence my_data = {} -- obtain data 
stream s = stream:open( my_data, STREAM_MEMORY ) -- allocates data into memory with "cleanup" flag 
 
-- etc. 
 
stream:close( s ) 

-Greg

11. Re: json

• Posted by raseu Aug 02, 2012
• 2808 views

BTW, it is licensed under an opensource MIT licence

again, here it is : http://www.rapideuphoria.com/cjson.zip

ras

13. Re: json

• Posted by bugmagnet Aug 03, 2012
• 2740 views

I still haven't figured this out. How do I tell if the atom refers to a map? Is there a is_map() function?

Kind regards,
Bruce/bugmagnet

15. Re: json

• Posted by bugmagnet Aug 03, 2012
• 2718 views

if map(something) then ... 

Oh man, how obvious! Thanks, Derek.

Kind regards,
Bruce/bugmagnet

17. Re: json

• Posted by CoJaBo2 Aug 03, 2012
• 2701 views

The examples don't seem to run, so I couldn't test (it just loops forever); looking at the code, it doesn't seem to handle the escapes \b and \f, or decimal numbers (1.23).

Serializing, in particular, looks like it might be a nightmare.

Say you have the sequence "abc"; in Eu, this is equal to {97,98,99}. In JSON, however, the string "abc" and the array [97,98,99] are completely distinct values.

This also applies to decoding, where the distinction between "abc" and [97,98,99] is lost and has to be inferred by the application.

19. Re: json

• Posted by bugmagnet Aug 04, 2012
• 2611 views

That's due to an oddity with maps that I've since posted about separately.

Kind regards,
Bruce/bugmagnet

21. Re: json

• Posted by CoJaBo2 Aug 04, 2012
• 2618 views

It looks like writing your own types may be the only solution; e.g., JSONString, JSONNumber, JSONObject, JSONArray, JSONBoolean, JSONNull. As suggested in that post, those would be 2-element sequences, with the first element being the type and the second being the actual value.

You'd have to come up with a way to not have it terribly messy for the programmer to access and create these types though.

23. Re: json

• Posted by CoJaBo2 Aug 05, 2012
• 2613 views

Don't forget the context in which JSON is actually going to be used; e.g.

1. The server happens to be using maps for several different tasks, e.g., to return SQL data, to keep track of internal server settings such as the DB password and billing API keys, store deserialized JSON for the shopping cart, etc.
2. The user sets his cookie to {"Cart":[1,2,3,4,5,...,98,99,100]}, then adds some item to his cart.
3. The server deserializes the cart data, adds that item to the end, then serializes it again and sends it back and/or converts it to HTML for display.
4. The user's cookie now looks like this: {"Cart":[{/SENSITIVE DATA/},{/USELESS DATA/},{/SENSITIVE DATA/},{/USELESS DATA/},{/ITEM ADDED TO CART/},...,98,99,100,{/ITEM ADDED TO CART/}]}

Any assumption that "unlikely" circumstances like this will never happen could not be more invalid than when working with user-supplied data- it has to be bulletproof in all cases. The memoization hack fixes the crash, but nothing else. Another hack might fix this issue, but it would leave several more holes, and so on. It would also become needlessly convoluted.

The only way this can work reliably at all is with explicit typing, e.g., using the method above.

That would fix the map issue elegantly, as well as the issue of distinguishing arrays vs. strings and values such as nulls that cannot be represented in Eu.

25. Re: json

• Posted by CoJaBo2 Aug 06, 2012
• 2526 views

You'd probably want to use the map type to represent the JSON Objects (call them Objects, not some other term).

The actual representation would look something like this:

json=deserialize({ 
    "an_object": { 
        "an_array": [ 
            104,101,108,108,111 
        ], 
        "a_string": "hello", 
        "an_array_of_objects": [ 
            {},{},{} 
        ] 
    }, 
    "a_boolean": false, 
    "a_null": null}) 
--json={JSON_OBJECT,1} (1 simply being a map ID) 
 
an_object=get(json[2],"an_object") 
--object={JSON_OBJECT,2} (2 being another map ID) 
 
array=get(an_object[2],"an_array") 
--array={JSON_ARRAY,{{JSON_NUMBER,104},{JSON_NUMBER,101},{JSON_NUMBER,108},{JSON_NUMBER,108},{JSON_NUMBER,111}}} 
 
string=get(an_object[2],"a_string") 
--string={JSON_STRING,{104,101,108,108,111}} (or "hello") 
 
obj_array=get(an_object[2],"an_array_of_objects") 
--obj_array={JSON_ARRAY,{{JSON_OBJECT,3},{JSON_OBJECT,4},{JSON_OBJECT,5}}} 
 
boolean=get(json[2],"a_boolean") 
--boolean={JSON_BOOLEAN,0} (could probably define TRUE and FALSE constants if the std libs don't already) 
 
null=get(json[2],"a_null") 
--null={JSON_NULL,0} (keeping the "value" param for consistency, even though it is redundant) 
 

As you can see, this allows us to guarantee certain structural conditions-

• Every JSON value (Object, Array, String, Number, Boolean, Null) is represented as a 2-element sequence in the format {TYPE, VALUE}.
• Every Object is a Eu map with 0 or more string keys, each of which has a value that is a valid JSON value.
• Every Array is a Eu sequence of 0 or more valid JSON values.

An API could be layered on top of this to hide this typing complexity and simplify accessing things for the programmer; I have an idea for such an API, but don't have time to properly describe it right now.