1. json

I've noticed that this subject hasn't created a lot of interest. Pity. I need a json to euphoria translator.

Now that we have a "map" it would/should/could be a trivial task?

With that in mind, can maps be embedded in maps? How do you tell, when traversing a sequence, whether the next embedded sequence is a map or not?

Kind regards,
Bruce/bugmagnet

new topic     » topic index » view message » categorize

2. Re: json

bugmagnet said...

With that in mind, can maps be embedded in maps? How do you tell, when traversing a sequence, whether the next embedded sequence is a map or not?

Maps are not actually sequences. They are atoms, which are essentially pointers into a sequence (where the actual map data is stored inside a sequence). You can use explicit type checking to determine if you are dealing with a map. In theory, it's possible to get a false positive, but unlikely.

For easy embedding, see nested_put() and nested_get().

Matt

new topic     » goto parent     » topic index » view message » categorize

3. Re: json

Okay, I've found some other implementations (C, C#, javascript etc) and will give converting one them a try. The C# looks most promising at this point.

For those who want to beat me to it (and they will, given my workload), the ones I've found are as follows:
http://sourceforge.net/projects/mjson/
https://github.com/douglascrockford/JSON-js
http://techblog.procurios.nl/k/news/view/14605/14863/How-do-I-write-my-own-parser-for-JSON.html
http://lloyd.github.com/yajl/

Kind regards,
Bruce/bugmagnet

new topic     » goto parent     » topic index » view message » categorize

4. Re: json

I've created a github project for this at https://github.com/axtens/euJSON. There's nothing there but a readme at this point.

Kind regards
Bruce/bugmagnet

new topic     » goto parent     » topic index » view message » categorize

5. Re: json

bugmagnet said...

I've created a github project for this at https://github.com/axtens/euJSON. There's nothing there but a readme at this point.

Kind regards
Bruce/bugmagnet

Ah ha! In reading this post, I did some Googling and started poking around with the code from that very same page. smile

Unfortunately, I had to backtrack a bit, as I feel one thing that would help make file/string parsing a bit easier in Euphoria would be more dynamic file and memory stream I/O, especially buffered I/O with a putback() or peek_next_char() function.

Here's a quick example I threw together in my head.

stream s = stream:open( filename, STREAM_READ ) 
 
while not stream:eof( s ) do 
 
    -- peek a single character (i.e. get the next character, but do not advance the 
    -- file position, so the next call to stream:getc() returns the same character) 
 
    char c = stream:peekc( s ) 
 
    switch c do 
 
        case '{' then 
            process_json_object( s ) 
 
        case '[' then 
            process_json_array( s ) 
 
        case else 
            puts( 2, "Expected '{' or '['\n" ) 
            abort( 1 ) 
 
    end switch 
 
end while 
 
stream:close( s ) 

It may also be advantageous to have the programmers of Euphoria itself add one of the open-source C-based processors to Euphoria directly. It would be nice to have a proper in-built XML parser as well.

-Greg

new topic     » goto parent     » topic index » view message » categorize

6. Re: json

ghaberek said...

It may also be advantageous to have the programmers of Euphoria itself add one of the open-source C-based processors to Euphoria directly. It would be nice to have a proper in-built XML parser as well.

It would indeed be advantageous.

Kind regards,
Bruce/bugmagnet

new topic     » goto parent     » topic index » view message » categorize

7. Re: json

ghaberek said...

It would be nice to have a proper in-built XML parser as well.

In fact, Jeremy has done some experimenting with adding Expat to the euphoria backend.

Matt

new topic     » goto parent     » topic index » view message » categorize

8. Re: json

ghaberek said...

Unfortunately, I had to backtrack a bit, as I feel one thing that would help make file/string parsing a bit easier in Euphoria would be more dynamic file and memory stream I/O, especially buffered I/O with a putback() or peek_next_char() function.

Yes, this sounds like a good idea to me. You should totally code that up and contribute. smile

I suppose those could just be wrappers around regular getc() and seek() calls, no?

Matt

new topic     » goto parent     » topic index » view message » categorize

9. Re: json

mattlewis said...

Yes, this sounds like a good idea to me. You should totally code that up and contribute. smile

I suppose those could just be wrappers around regular getc() and seek() calls, no?

I've already started playing around with the concepts so I'll surely give it a whirl. At the very least, yes, they would be calls to the in-built file I/O routines, just with some "smart" buffering, etc. I would also like to be able to emulate file I/O in memory...

sequence my_data = {} -- obtain data 
atom ptr = allocate( length(my_data), 1 ) 
poke( ptr, my_data ) 
 
stream s = stream:open( ptr, STREAM_MEMORY ) 
 
-- perform I/O on this "stream" while buffers are moved 
-- and allocated to to accommodate the size of the data 
 
stream:close( s ) 

or just...

sequence my_data = {} -- obtain data 
stream s = stream:open( my_data, STREAM_MEMORY ) -- allocates data into memory with "cleanup" flag 
 
-- etc. 
 
stream:close( s ) 

-Greg

new topic     » goto parent     » topic index » view message » categorize

10. Re: json

see http://www.rapideuphoria.com/cjson.zip
in the rapideuphoria archives for a euphoria
wrapper for 'C' JSON library (with euphoria examples).

requires Euphoria v4.1 (memstruct)

ras

new topic     » goto parent     » topic index » view message » categorize

11. Re: json

BTW, it is licensed under an opensource MIT licence

again, here it is : http://www.rapideuphoria.com/cjson.zip

ras

new topic     » goto parent     » topic index » view message » categorize

12. Re: json

bugmagnet said...

I've created a github project for this at https://github.com/axtens/euJSON.

A bit more there now. I've got the conversion of the json decode about 90% there. Haven't touched the json encode as yet.

Can convert {"name": "bugmagnet"} and {"age", 51} to a map.
Can't yet convert {"name" : ["bugmagnet","attracts",100,"bugs"]}.

Kind regards,
Bruce/bugmagnet

P.S. This is a "pure euphoria" attempt, in the spirit of "pure perl http://www.perlmonks.org/?node_id=709757"

new topic     » goto parent     » topic index » view message » categorize

13. Re: json

mattlewis said...
bugmagnet said...

How do you tell, when traversing a sequence, whether the next embedded sequence is a map or not?

I still haven't figured this out. How do I tell if the atom refers to a map? Is there a is_map() function?

Kind regards,
Bruce/bugmagnet

new topic     » goto parent     » topic index » view message » categorize

14. Re: json

bugmagnet said...

I still haven't figured this out. How do I tell if the atom refers to a map? Is there a is_map() function?

if map(something) then ... 
new topic     » goto parent     » topic index » view message » categorize

15. Re: json

DerekParnell said...
if map(something) then ... 

Oh man, how obvious! Thanks, Derek.

Kind regards,
Bruce/bugmagnet

new topic     » goto parent     » topic index » view message » categorize

16. Re: json

bugmagnet said...
bugmagnet said...

I've created a github project for this at https://github.com/axtens/euJSON.

Please download, use and break it. It's sure to break somewhere. And tell me where it breaks.

I've got the json decode function pretty much working. I can decode {"client" : {"Name":"bugmagnet"}} into a map with an embedded map. I can also parse arrays within records. Still haven't started on the json encode.

Kind regards,
Bruce/bugmagnet

new topic     » goto parent     » topic index » view message » categorize

17. Re: json

The examples don't seem to run, so I couldn't test (it just loops forever); looking at the code, it doesn't seem to handle the escapes \b and \f, or decimal numbers (1.23).

Serializing, in particular, looks like it might be a nightmare.

Say you have the sequence "abc"; in Eu, this is equal to {97,98,99}. In JSON, however, the string "abc" and the array [97,98,99] are completely distinct values.

This also applies to decoding, where the distinction between "abc" and [97,98,99] is lost and has to be inferred by the application.

new topic     » goto parent     » topic index » view message » categorize

18. Re: json

CoJaBo2 said...

The examples don't seem to run, so I couldn't test (it just loops forever); looking at the code, it doesn't seem to handle the escapes \b and \f, or decimal numbers (1.23).

Noted. Thanks for the feedback.

CoJaBo2 said...

Serializing, in particular, looks like it might be a nightmare.

I've been thinking about it a fair bit, and yes, it may turn out to be rather a traumatic experience

Kind regards
Bruce/bugmagnet

new topic     » goto parent     » topic index » view message » categorize

19. Re: json

CoJaBo2 said...

The examples don't seem to run, so I couldn't test (it just loops forever);

That's due to an oddity with maps that I've since posted about separately.

Kind regards,
Bruce/bugmagnet

new topic     » goto parent     » topic index » view message » categorize

20. Re: json

CoJaBo2 said...

it doesn't seem to handle the escapes \b and \f, or decimal numbers (1.23).

Fixed ... I hope.

Kind regards,
Bruce/bugmagnet

new topic     » goto parent     » topic index » view message » categorize

21. Re: json

bugmagnet said...

That's due to an oddity with maps that I've since posted about separately.

Kind regards,
Bruce/bugmagnet

It looks like writing your own types may be the only solution; e.g., JSONString, JSONNumber, JSONObject, JSONArray, JSONBoolean, JSONNull. As suggested in that post, those would be 2-element sequences, with the first element being the type and the second being the actual value.

You'd have to come up with a way to not have it terribly messy for the programmer to access and create these types though.

new topic     » goto parent     » topic index » view message » categorize

22. Re: json

CoJaBo2 said...
bugmagnet said...

an oddity with maps that I've since posted about separately.

I've fixed the test routine, using memoization to handle the maps issue. I have my doubts, though, about using it in other contexts. As for serialisation, that's an Elm Street I haven't walked down yet.

Kind regards,
Bruce/bugmagnet

new topic     » goto parent     » topic index » view message » categorize

23. Re: json

Don't forget the context in which JSON is actually going to be used; e.g.

  1. The server happens to be using maps for several different tasks, e.g., to return SQL data, to keep track of internal server settings such as the DB password and billing API keys, store deserialized JSON for the shopping cart, etc.
  2. The user sets his cookie to {"Cart":[1,2,3,4,5,...,98,99,100]}, then adds some item to his cart.
  3. The server deserializes the cart data, adds that item to the end, then serializes it again and sends it back and/or converts it to HTML for display.
  4. The user's cookie now looks like this: {"Cart":[{/SENSITIVE DATA/},{/USELESS DATA/},{/SENSITIVE DATA/},{/USELESS DATA/},{/ITEM ADDED TO CART/},...,98,99,100,{/ITEM ADDED TO CART/}]}


Any assumption that "unlikely" circumstances like this will never happen could not be more invalid than when working with user-supplied data- it has to be bulletproof in all cases. The memoization hack fixes the crash, but nothing else. Another hack might fix this issue, but it would leave several more holes, and so on. It would also become needlessly convoluted.

The only way this can work reliably at all is with explicit typing, e.g., using the method above.

That would fix the map issue elegantly, as well as the issue of distinguishing arrays vs. strings and values such as nulls that cannot be represented in Eu.

new topic     » goto parent     » topic index » view message » categorize

24. Re: json

CoJaBo2 said...

The only way this can work reliably at all is with explicit typing, e.g., using the method above.

So doing this?

IN: {"key1" : {"array" : [1,2,3]}, "boolean" : false} 
OUT: {JSON_RECORD, { 
		{JSON_STRING, "key1"}, 
		{JSON_RECORD,  
			{ 
				{JSON_STRING, "array"},  
				{JSON_ARRAY, { 
					{JSON_NUMBER, 1}, 
					{JSON_NUMBER, 1}, 
					{JSON_NUMBER, 1} 
					} 
				} 
			} 
		}, 
		{JSON_STRING, "boolean"},  
		{JSON_BOOLEAN, 0} 
	} 
} 

new topic     » goto parent     » topic index » view message » categorize

25. Re: json

You'd probably want to use the map type to represent the JSON Objects (call them Objects, not some other term).

The actual representation would look something like this:

json=deserialize({ 
    "an_object": { 
        "an_array": [ 
            104,101,108,108,111 
        ], 
        "a_string": "hello", 
        "an_array_of_objects": [ 
            {},{},{} 
        ] 
    }, 
    "a_boolean": false, 
    "a_null": null}) 
--json={JSON_OBJECT,1} (1 simply being a map ID) 
 
an_object=get(json[2],"an_object") 
--object={JSON_OBJECT,2} (2 being another map ID) 
 
array=get(an_object[2],"an_array") 
--array={JSON_ARRAY,{{JSON_NUMBER,104},{JSON_NUMBER,101},{JSON_NUMBER,108},{JSON_NUMBER,108},{JSON_NUMBER,111}}} 
 
string=get(an_object[2],"a_string") 
--string={JSON_STRING,{104,101,108,108,111}} (or "hello") 
 
obj_array=get(an_object[2],"an_array_of_objects") 
--obj_array={JSON_ARRAY,{{JSON_OBJECT,3},{JSON_OBJECT,4},{JSON_OBJECT,5}}} 
 
boolean=get(json[2],"a_boolean") 
--boolean={JSON_BOOLEAN,0} (could probably define TRUE and FALSE constants if the std libs don't already) 
 
null=get(json[2],"a_null") 
--null={JSON_NULL,0} (keeping the "value" param for consistency, even though it is redundant) 
 

As you can see, this allows us to guarantee certain structural conditions-

  • Every JSON value (Object, Array, String, Number, Boolean, Null) is represented as a 2-element sequence in the format {TYPE, VALUE}.
  • Every Object is a Eu map with 0 or more string keys, each of which has a value that is a valid JSON value.
  • Every Array is a Eu sequence of 0 or more valid JSON values.

An API could be layered on top of this to hide this typing complexity and simplify accessing things for the programmer; I have an idea for such an API, but don't have time to properly describe it right now.

new topic     » goto parent     » topic index » view message » categorize

26. Re: json

CoJaBo2 said...

An API could be layered on top of this to hide this typing complexity and simplify accessing things for the programmer; I have an idea for such an API, but don't have time to properly describe it right now.

Well, given the lack of time I have right now to pursue this further, I think I'll have to leave it for you to finish. Sigh.

Kind regards,
Bruce/bugmagnet.

new topic     » goto parent     » topic index » view message » categorize

Search



Quick Links

User menu

Not signed in.

Misc Menu