delistify README

This commit is contained in:
alisdair sullivan 2012-05-26 18:21:50 -07:00
parent 0e53893fce
commit 56ff3b914e

View file

@ -25,74 +25,74 @@ jsx may be built using either [sinan][sinan] or [rebar][rebar]
## quickstart ##
* to build the library and run tests
#### to build the library and run tests ####
```bash
tanga:jsx alisdair$ sinan build
tanga:jsx alisdair$ sinan -r tests eunit
```
or
```bash
tanga:jsx alisdair$ rebar compile
tanga:jsx alisdair$ rebar eunit
```
```bash
tanga:jsx alisdair$ sinan build
tanga:jsx alisdair$ sinan -r tests eunit
```
or
```bash
tanga:jsx alisdair$ rebar compile
tanga:jsx alisdair$ rebar eunit
```
* to convert a utf8 binary containing a json string into an erlang term
#### to convert a utf8 binary containing a json string into an erlang term ####
```erlang
1> jsx:decode(<<"{\"library\": \"jsx\", \"awesome\": true}">>).
[{<<"library">>,<<"jsx">>},{<<"awesome">>,true}]
2> jsx:decode(<<"[\"a\",\"list\",\"of\",\"words\"]">>).
[<<"a">>, <<"list">>, <<"of">>, <<"words">>]
```
```erlang
1> jsx:decode(<<"{\"library\": \"jsx\", \"awesome\": true}">>).
[{<<"library">>,<<"jsx">>},{<<"awesome">>,true}]
2> jsx:decode(<<"[\"a\",\"list\",\"of\",\"words\"]">>).
[<<"a">>, <<"list">>, <<"of">>, <<"words">>]
```
* to convert an erlang term into a utf8 binary containing a json string
#### to convert an erlang term into a utf8 binary containing a json string ####
```erlang
1> jsx:encode([{<<"library">>,<<"jsx">>},{<<"awesome">>,true}]).
<<"{\"library\": \"jsx\", \"awesome\": true}">>
2> jsx:encode([<<"a">>, <<"list">>, <<"of">>, <<"words">>]).
<<"[\"a\",\"list\",\"of\",\"words\"]">>
```
```erlang
1> jsx:encode([{<<"library">>,<<"jsx">>},{<<"awesome">>,true}]).
<<"{\"library\": \"jsx\", \"awesome\": true}">>
2> jsx:encode([<<"a">>, <<"list">>, <<"of">>, <<"words">>]).
<<"[\"a\",\"list\",\"of\",\"words\"]">>
```
* to check if a binary or a term is valid json
#### to check if a binary or a term is valid json ####
```erlang
1> jsx:is_json(<<"[\"this is json\"]">>).
true
2> jsx:is_json("[\"this is not\"]").
false
3> jsx:is_term([<<"this is a term">>]).
true
4> jsx:is_term(["this is not"]).
false
```
```erlang
1> jsx:is_json(<<"[\"this is json\"]">>).
true
2> jsx:is_json("[\"this is not\"]").
false
3> jsx:is_term([<<"this is a term">>]).
true
4> jsx:is_term(["this is not"]).
false
```
* to minify some json
#### to minify some json ####
```erlang
1> jsx:minify(<<"{
\"a list\": [
1,
2,
3
]
}">>).
<<"{\"a list\":[1,2,3]}">>
```
```erlang
1> jsx:minify(<<"{
\"a list\": [
1,
2,
3
]
}">>).
<<"{\"a list\":[1,2,3]}">>
```
* to prettify some json
#### to prettify some json ####
```erlang
1> jsx:prettify(<<"{\"a list\":[1,2,3]}">>).
<<"{
\"a list\": [
1,
2,
3
]
}">>
```
```erlang
1> jsx:prettify(<<"{\"a list\":[1,2,3]}">>).
<<"{
\"a list\": [
1,
2,
3
]
}">>
```
## description ##
@ -108,7 +108,7 @@ the [spec][rfc4627] thinks json values must be wrapped in a json array or object
here is a table of how various json values map to erlang:
#### json &lt;-> erlang mapping ####
### json &lt;-> erlang mapping ###
**json** | **erlang**
--------------------------------|--------------------------------
@ -149,7 +149,7 @@ here is a table of how various json values map to erlang:
json objects are represented by erlang proplists. the empty object has the special representation `[{}]` to differentiate it from the empty list. ambiguities like `[true, false]` prevent the use of the shorthand form of property lists using atoms as properties so all properties must be tuples. all keys must be encoded as in `string` or as atoms (which will be escaped and converted to binaries for presentation to handlers). values should be valid json values
#### incomplete input ####
### incomplete input ###
jsx handles incomplete json texts. if a partial json text is parsed, rather than returning a term from your callback handler, jsx returns `{incomplete, F}` where `F` is a function with an identical API to the anonymous fun returned from `decoder/3`, `encoder/3` or `parser/3`. it retains the internal state of the parser at the point where input was exhausted. this allows you to parse as you stream json over a socket or file descriptor, or to parse large json texts without needing to keep them entirely in memory
@ -158,104 +158,100 @@ however, it is important to recognize that jsx is greedy by default. jsx will co
## data types ##
* `json_term()`
#### `json_term()` ####
```erlang
json_term() = [json_term()]
| [{binary() | atom(), json_term()}]
| true
| false
| null
| integer()
| float()
| binary()
```
```erlang
json_term() = [json_term()]
| [{binary() | atom(), json_term()}]
| true
| false
| null
| integer()
| float()
| binary()
```
the erlang representation of json. binaries should be `utf8` encoded, or close at least
the erlang representation of json. binaries should be `utf8` encoded, or close at least
* `json_text()`
#### `json_text()` ####
```erlang
json_text() = binary()
```
```erlang
json_text() = binary()
```
a utf8 encoded binary containing a json string
a utf8 encoded binary containing a json string
* `tokens()` & `token()`
#### `token()` ####
```erlang
tokens() = token() | [token()]
token() = start_object
| end_object
| start_array
| end_array
| {key, binary()}
| {string, binary()}
| binary()
| {number, integer() | float()}
| {integer, integer()}
| {float, float()}
| integer()
| float()
| {literal, true}
| {literal, false}
| {literal, null}
| true
| false
| null
| end_json
```
```erlang
token() = start_object
| end_object
| start_array
| end_array
| {key, binary()}
| {string, binary()}
| binary()
| {number, integer() | float()}
| {integer, integer()}
| {float, float()}
| integer()
| float()
| {literal, true}
| {literal, false}
| {literal, null}
| true
| false
| null
| end_json
```
the internal representation used during syntactic analysis
the internal representation used during syntactic analysis
* `events()` & `event()`
#### `event()` ####
```erlang
event() = start_object
| end_object
| start_array
| end_array
| {key, binary()}
| {string, binary()}
| {integer, integer()}
| {float, float()}
| {literal, true}
| {literal, false}
| {literal, null}
| end_json
```
```erlang
event() = start_object
| end_object
| start_array
| end_array
| {key, binary()}
| {string, binary()}
| {integer, integer()}
| {float, float()}
| {literal, true}
| {literal, false}
| {literal, null}
| end_json
```
the internal representation used during semantic analysis
the internal representation used during semantic analysis
* `options()` & `option()`
#### `option()` ####
```erlang
options() = [option()]
```erlang
option() = replaced_bad_utf8
| escaped_forward_slashes
| single_quoted_strings
| unescaped_jsonp
| comments
| escaped_strings
| dirty_strings
| ignored_bad_escapes
| relax
| explicit_end
```
option() = replaced_bad_utf8
| escaped_forward_slashes
| single_quoted_strings
| unescaped_jsonp
| comments
| escaped_strings
| dirty_strings
| ignored_bad_escapes
| relax
| explicit_end
```
jsx functions all take a common set of options. not all flags have meaning in all contexts, but they are always valid options. functions may have additional options beyond these. see [individual function documentation](#exports) for details
jsx functions all take a common set of options. not all flags have meaning in all contexts, but they are always valid options. functions may have additional options beyond these. see [individual function documentation](#exports) for details
- `replaced_bad_utf8`
- `replaced_bad_utf8`
json text input and json strings SHOULD be utf8 encoded binaries, appropriately escaped as per the json spec. attempts are made to replace invalid codepoints with `u+FFFD` as per the unicode spec when this option is present. this applies both to malformed unicode and disallowed codepoints
- `escaped_forward_slashes`
- `escaped_forward_slashes`
json strings are escaped according to the json spec. this means forward slashes (solidus) are optionally escaped. this option is only relevant for encoding; you may want to use this if you are embedding json directly into a html or xml document
- `single_quoted_strings`
- `single_quoted_strings`
some parsers allow double quotes (`u+0022`) to be replaced by single quotes (`u+0027`) to delimit keys and strings. this option allows json containing single quotes as structural characters to be parsed without errors. note that the parser expects strings to be terminated by the same quote type that opened it and that single quotes must, obviously, be escaped within strings delimited by single quotes
@ -263,31 +259,31 @@ however, it is important to recognize that jsx is greedy by default. jsx will co
the parser will never emit json with keys or strings delimited by single quotes
- `unescaped_jsonp`
- `unescaped_jsonp`
javascript interpreters treat the codepoints `u+2028` and `u+2029` as significant whitespace. json strings that contain either of these codepoints will be parsed incorrectly by some javascript interpreters. by default, these codepoints are escaped (to `\u2028` and `\u2029`, respectively) to retain compatibility. this option simply removes that escaping
- `comments`
- `comments`
json has no official comments but some parsers allow c style comments. anywhere whitespace is allowed this flag allows comments (both `// ...` and `/* ... */` style)
- `escaped_strings`
- `escaped_strings`
by default, both the encoder and decoder return strings as utf8 binaries appropriate for use in erlang. escape sequences that were present in decoded terms are converted into the appropriate codepoint while encoded terms are unaltered. this flag escapes strings as if for output in json, removing control codes and problematic codepoints and replacing them with the appropriate escapes
- `dirty_strings`
- `dirty_strings`
json escaping is lossy; it mutates the json string and repeated application can result in unwanted behaviour. if your strings are already escaped (or you'd like to force invalid strings into "json") use this flag to bypass escaping
- `ignored_bad_escapes`
- `ignored_bad_escapes`
during decoding, ignore unrecognized escape sequences and leave them as is in the stream. note that combining this option with `escaped_strings` will result in the escape character itself being escaped
- `explicit_end`
- `explicit_end`
this option treats all exhausted inputs as incomplete. the parser will not attempt to return a final state until the function is called with the value `end_stream`
- `relax`
- `relax`
relax is a synonym for `[replaced_bad_utf8, single_quoted_strings, comments, ignored_bad_escapes]` for when you don't care how janky and awful your json input is, you just want the parser to do the best it can
@ -295,245 +291,245 @@ however, it is important to recognize that jsx is greedy by default. jsx will co
## exports ##
* `encoder/3`, `decoder/3` & `parser/3`
#### `encoder/3`, `decoder/3` & `parser/3` ####
```erlang
decoder(Module, Args, Opts) -> Fun((JSONText) -> any())
encoder(Module, Args, Opts) -> Fun((JSONTerm) -> any())
parser(Module, Args, Opts) -> Fun((Tokens) -> any())
```erlang
decoder(Module, Args, Opts) -> Fun((JSONText) -> any())
encoder(Module, Args, Opts) -> Fun((JSONTerm) -> any())
parser(Module, Args, Opts) -> Fun((Tokens) -> any())
Module = atom()
Args = any()
Opts = options()
JSONText = json_text()
JSONTerm = json_term()
Tokens = tokens()
```
Module = atom()
Args = any()
Opts = [option()]
JSONText = json_text()
JSONTerm = json_term()
Tokens = token() | [token()]
```
jsx is a json compiler with distinct tokenizing, syntactic analysis and semantic analysis stages (actually, semantic analysis takes place during syntactic analysis, for efficiency). included are two tokenizers; one that handles json texts (`decoder/3`) and one that handles erlang terms (`encoder/3`). there is also an entry point to the syntactic analysis stage for use with user-defined tokenizers (`parser/3`)
jsx is a json compiler with interleaved tokenizing, syntactic analysis and semantic analysis stages. included are two tokenizers; one that handles json texts (`decoder/3`) and one that handles erlang terms (`encoder/3`). there is also an entry point to the syntactic analysis stage for use with user-defined tokenizers (`parser/3`)
all three functions return an anonymous function that takes the appropriate type of input and returns the result of performing semantic analysis, the tuple `{incomplete, F}` where `F` is a new anonymous function (see [incomplete input](#incomplete_input)) or a `badarg` error exception if syntactic analysis fails
all three functions return an anonymous function that takes the appropriate type of input and returns the result of performing semantic analysis, the tuple `{incomplete, F}` where `F` is a new anonymous function (see [incomplete input](#incomplete_input)) or a `badarg` error exception if syntactic analysis fails
`Module` is the name of the callback module
`Module` is the name of the callback module
`Args` is any term that will be passed to `Module:init/1` prior to syntactic analysis to produce an initial state
`Args` is any term that will be passed to `Module:init/1` prior to syntactic analysis to produce an initial state
`Opts` are detailed in [data types](#data_types)
`Opts` are detailed in [data types](#data_types)
see [callback exports](#callback_exports) for details on the callback module
check out [callback module documentation](#callback_exports) for details of the callback module interface
* `decode/1,2`
#### `decode/1,2` ####
```erlang
decode(JSON) -> Term
decode(JSON, Opts) -> Term
```erlang
decode(JSON) -> Term
decode(JSON, Opts) -> Term
JSON = json_text()
Term = json_term()
Opts = [option() | labels | {labels, Label} | {post_decode, F}]
Label = binary | atom | existing_atom
F = fun((any()) -> any())
```
JSON = json_text()
Term = json_term()
Opts = [option() | labels | {labels, Label} | {post_decode, F}]
Label = binary | atom | existing_atom
F = fun((any()) -> any())
```
`decode` parses a json text (a `utf8` encoded binary) and produces an erlang term (see [json <-> erlang mapping](#json---erlang-mapping))
`decode` parses a json text (a `utf8` encoded binary) and produces an erlang term
the option `labels` controls how keys are converted from json to erlang terms. `binary` does no conversion beyond normal escaping. `atom` converts keys to erlang atoms and results in a badarg error if the keys fall outside the range of erlang atoms. `existing_atom` is identical to `atom` except it will not add new atoms to the atom table
the option `labels` controls how keys are converted from json to erlang terms. `binary` does no conversion beyond normal escaping. `atom` converts keys to erlang atoms and results in a badarg error if the keys fall outside the range of erlang atoms. `existing_atom` is identical to `atom` except it will not add new atoms to the atom table
`{post_decode, F}` is a user defined function of arity 1 that is called on each output value (objects, arrays, strings, numbers and literals). it may return any value to be substituted in the returned term. for example:
`{post_decode, F}` is a user defined function of arity 1 that is called on each output value (objects, arrays, strings, numbers and literals). it may return any value to be substituted in the returned term. for example:
```erlang
1> F = fun(V) when is_list(V) -> V; (V) -> false end.
2> jsx:decode(<<"{\"a list\": [true, \"a string\", 1]}">>, [{post_decode, F}]).
[{<<"a list">>, [false, false, false]}]
```
```erlang
1> F = fun(V) when is_list(V) -> V; (V) -> false end.
2> jsx:decode(<<"{\"a list\": [true, \"a string\", 1]}">>, [{post_decode, F}]).
[{<<"a list">>, [false, false, false]}]
```
declaring more than one post-decoder will result in a `badarg` error exception
declaring more than one post-decoder will result in a `badarg` error exception
raises a `badarg` error exception if input is not valid json
raises a `badarg` error exception if input is not valid json
* `encode/1,2`
#### `encode/1,2` ####
```erlang
encode(Term) -> JSON
encode(Term, Opts) -> JSON
```erlang
encode(Term) -> JSON
encode(Term, Opts) -> JSON
Term = json_term()
JSON = json_text()
Opts = [option() | {pre_encode, F} | space | {space, N} | indent | {indent, N}]
F = fun((any()) -> any())
N = pos_integer()
```
Term = json_term()
JSON = json_text()
Opts = [option() | {pre_encode, F} | space | {space, N} | indent | {indent, N}]
F = fun((any()) -> any())
N = pos_integer()
```
`encode` parses a json text (a `utf8` encoded binary) and produces an erlang term (see [json <-> erlang mapping](#json---erlang-mapping))
`encode` parses a json text (a `utf8` encoded binary) and produces an erlang term
the option `{space, N}` inserts `N` spaces after every comma and colon in your json output. `space` is an alias for `{space, 1}`. the default is `{space, 0}`
the option `{space, N}` inserts `N` spaces after every comma and colon in your json output. `space` is an alias for `{space, 1}`. the default is `{space, 0}`
the option `{indent, N}` inserts a newline and `N` spaces for each level of indentation in your json output. note that this overrides spaces inserted after a comma. `indent` is an alias for `{indent, 1}`. the default is `{indent, 0}`
the option `{indent, N}` inserts a newline and `N` spaces for each level of indentation in your json output. note that this overrides spaces inserted after a comma. `indent` is an alias for `{indent, 1}`. the default is `{indent, 0}`
`{pre_encode, F}` is a user defined function of arity 1 that is called on each input value. it may return any valid json value to be substituted in the returned json. for example:
`{pre_encode, F}` is a user defined function of arity 1 that is called on each input value. it may return any valid json value to be substituted in the returned json. for example:
```erlang
1> F = fun(V) when is_list(V) -> V; (V) -> false end.
2> jsx:encode([{<<"a list">>, [true, <<"a string">>, 1]}], [{pre_encode, F}]).
<<"{\"a list\": [false, false, false]}">>
```
```erlang
1> F = fun(V) when is_list(V) -> V; (V) -> false end.
2> jsx:encode([{<<"a list">>, [true, <<"a string">>, 1]}], [{pre_encode, F}]).
<<"{\"a list\": [false, false, false]}">>
```
declaring more than one pre-encoder will result in a `badarg` error exception
declaring more than one pre-encoder will result in a `badarg` error exception
raises a `badarg` error exception if input is not a valid erlang representation of json
raises a `badarg` error exception if input is not a valid [erlang representation of json](#json---erlang-mapping)
* `format/1,2`
#### `format/1,2` ####
```erlang
format(JSON) -> JSON
format(JSON, Opts) -> JSON
```erlang
format(JSON) -> JSON
format(JSON, Opts) -> JSON
JSON = json_text()
Opts = [option() | space | {space, N} | indent | {indent, N}]
N = pos_integer()
```
JSON = json_text()
Opts = [option() | space | {space, N} | indent | {indent, N}]
N = pos_integer()
```
`format` parses a json text (a `utf8` encoded binary) and produces a new json text according to the format rules specified by `Opts`
`format` parses a json text (a `utf8` encoded binary) and produces a new json text according to the format rules specified by `Opts`
the option `{space, N}` inserts `N` spaces after every comma and colon in your json output. `space` is an alias for `{space, 1}`. the default is `{space, 0}`
the option `{indent, N}` inserts a newline and `N` spaces for each level of indentation in your json output. note that this overrides spaces inserted after a comma. `indent` is an alias for `{indent, 1}`. the default is `{indent, 0}`
the option `{space, N}` inserts `N` spaces after every comma and colon in your json output. `space` is an alias for `{space, 1}`. the default is `{space, 0}`
raises a `badarg` error exception if input is not valid json
the option `{indent, N}` inserts a newline and `N` spaces for each level of indentation in your json output. note that this overrides spaces inserted after a comma. `indent` is an alias for `{indent, 1}`. the default is `{indent, 0}`
raises a `badarg` error exception if input is not valid json
* `minify/1`
#### `minify/1` ####
```erlang
minify(JSON) -> JSON
```erlang
minify(JSON) -> JSON
JSON = json_text()
```
JSON = json_text()
```
`minify` parses a json text (a `utf8` encoded binary) and produces a new json text stripped of whitespace
`minify` parses a json text (a `utf8` encoded binary) and produces a new json text stripped of whitespace
raises a `badarg` error exception if input is not valid json
raises a `badarg` error exception if input is not valid json
* `prettify/1`
#### `prettify/1` ####
```erlang
prettify(JSON) -> JSON
```erlang
prettify(JSON) -> JSON
JSON = json_text()
```
JSON = json_text()
```
`prettify` parses a json text (a `utf8` encoded binary) and produces a new json text equivalent to `format(JSON, [{space, 1}, {indent, 2}])`
`prettify` parses a json text (a `utf8` encoded binary) and produces a new json text equivalent to `format(JSON, [{space, 1}, {indent, 2}])`
raises a `badarg` error exception if input is not valid json
raises a `badarg` error exception if input is not valid json
* `is_json/1,2`
#### `is_json/1,2` ####
```erlang
is_json(MaybeJSON) -> true | false
is_json(MaybeJSON, Opts) -> true | false
```erlang
is_json(MaybeJSON) -> true | false
is_json(MaybeJSON, Opts) -> true | false
MaybeJSON = any()
Opts = options()
```
MaybeJSON = any()
Opts = options()
```
returns true if input is a valid json text, false if not
returns true if input is a valid json text, false if not
what exactly constitutes valid json may be altered per the options, detailed in [data types](#data_types)
what exactly constitutes valid json may be altered per the options, detailed in [data types](#data_types)
* `is_term/1,2`
#### `is_term/1,2` ####
```erlang
is_term(MaybeJSON) -> true | false
is_term(MaybeJSON, Opts) -> true | false
```erlang
is_term(MaybeJSON) -> true | false
is_term(MaybeJSON, Opts) -> true | false
MaybeJSON = any()
Opts = options()
```
MaybeJSON = any()
Opts = options()
```
returns true if input is a valid erlang representation of json, false if not
returns true if input is a valid erlang representation of json, false if not
what exactly constitutes valid json may be altered per the options, detailed in [data types](#data_types)
what exactly constitutes valid json may be altered per the options, detailed in [data types](#data_types)
## callback exports ##
the following functions should be exported from a jsx callback module
* `Module:init/1`
#### `Module:init/1` ####
```erlang
Module:init(Args) -> InitialState
```erlang
Module:init(Args) -> InitialState
Args = any()
InitialState = any()
```
Args = any()
InitialState = any()
```
whenever any of `encoder/3`, `decoder/3` or `parser/3` are called, this function is called with the `Args` argument provided in the calling function to obtain `InitialState`
whenever any of `encoder/3`, `decoder/3` or `parser/3` are called, this function is called with the `Args` argument provided in the calling function to obtain `InitialState`
* `Module:handle_event/2`
#### `Module:handle_event/2` ####
```erlang
Module:handle_event(Event, State) -> NewState
```erlang
Module:handle_event(Event, State) -> NewState
Event = events()
State = any()
NewState = any()
```
Event = [event()]
State = any()
NewState = any()
```
semantic analysis is performed by repeatedly calling `handle_event/2` with a stream of events emitted by the tokenizer and the current state. the new state returned is used as the input to the next call to `handle_event/2`. the following events must be handled:
semantic analysis is performed by repeatedly calling `handle_event/2` with a stream of events emitted by the tokenizer and the current state. the new state returned is used as the input to the next call to `handle_event/2`. the following events must be handled:
- `start_object`
the start of a json object
- `start_object`
- `end_object`
the end of a json object
the start of a json object
- `start_array`
- `end_object`
the start of a json array
the end of a json object
- `end_array`
- `start_array`
the end of a json array
the start of a json array
- `{key, binary()}`
- `end_array`
a key in a json object. this is guaranteed to follow either `start_object` or a json value. it will usually be a `utf8` encoded binary. see the options under [data types](#data_types) for possible exceptions
the end of a json array
- `{string, binary()}`
- `{key, binary()}`
a json string. it will usually be a `utf8` encoded binary. see the options under [data types](#data_types) for possible exceptions
a key in a json object. this is guaranteed to follow either `start_object` or a json value. it will usually be a `utf8` encoded binary. see the options under [data types](#data_types) for possible exceptions
- `{integer, integer()}`
- `{string, binary()}`
an erlang integer (bignum)
a json string. it will usually be a `utf8` encoded binary. see the options under [data types](#data_types) for possible exceptions
- `{float, float()}`
- `{integer, integer()}`
an erlang float
an erlang integer (bignum)
- `{literal, true}`
- `{float, float()}`
the atom `true`
an erlang float
- `{literal, false}`
- `{literal, true}`
the atom `false`
the atom `true`
- `{literal, null}`
- `{literal, false}`
the atom `null`
the atom `false`
- `end_json`
- `{literal, null}`
this event is emitted when syntactic analysis is completed. you should do any cleanup and return the result of your semantic analysis
the atom `null`
- `end_json`
this event is emitted when syntactic analysis is completed. you should do any cleanup and return the result of your semantic analysis
## acknowledgements ##