first commit
This commit is contained in:
Josh Baker
commit
f4afb106da
1
.travis.yml
Normal file
1
.travis.yml
Normal file
|
|
@ -0,0 +1 @@
|
|||
language: go
|
||||
20
LICENSE
Normal file
20
LICENSE
Normal file
|
|
@ -0,0 +1,20 @@
|
|||
The MIT License (MIT)
|
||||
|
||||
Copyright (c) 2016 Josh Baker
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy of
|
||||
this software and associated documentation files (the "Software"), to deal in
|
||||
the Software without restriction, including without limitation the rights to
|
||||
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
|
||||
the Software, and to permit persons to whom the Software is furnished to do so,
|
||||
subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
|
||||
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
|
||||
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
|
||||
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
|
||||
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
||||
160
README.md
Normal file
160
README.md
Normal file
|
|
@ -0,0 +1,160 @@
|
|||
<p align="center">
|
||||
<img
|
||||
src="logo.png"
|
||||
width="240" height="78" border="0" alt="GJSON">
|
||||
<br>
|
||||
<a href="https://travis-ci.org/tidwall/gjson"><img src="https://img.shields.io/travis/tidwall/gjson.svg?style=flat-square" alt="Build Status"></a><!--
|
||||
<a href="http://gocover.io/github.com/tidwall/gjson"><img src="https://img.shields.io/badge/coverage-97%25-brightgreen.svg?style=flat-square" alt="Code Coverage"></a>
|
||||
-->
|
||||
<a href="https://godoc.org/github.com/tidwall/gjson"><img src="https://img.shields.io/badge/api-reference-blue.svg?style=flat-square" alt="GoDoc"></a>
|
||||
</p>
|
||||
|
||||
<p align="center">get a json value quickly</a></p>
|
||||
|
||||
GJSON is a Go package the provides a [very fast](#performance) and simple way to get a value from a json document. The reason for this library it to give efficent json indexing for the [BuntDB](https://github.com/tidwall/buntdb) project.
|
||||
|
||||
Getting Started
|
||||
===============
|
||||
|
||||
## Installing
|
||||
|
||||
To start using GJSON, install Go and run `go get`:
|
||||
|
||||
```sh
|
||||
$ go get -u github.com/tidwall/gjson
|
||||
```
|
||||
|
||||
This will retrieve the library.
|
||||
|
||||
## Get a value
|
||||
Get searches json for the specified path. A path is in dot syntax, such as "name.last" or "age". This function expects that the json is well-formed, and does not validate. Invalid json will not panic, but it may return back unexpected results. When the value is found it's returned immediately.
|
||||
|
||||
```go
|
||||
package main
|
||||
|
||||
import "github.com/tidwall/gjson"
|
||||
|
||||
const json = `{"name":{"first":"Janet","last":"Prichard"},"age":47}`
|
||||
|
||||
func main() {
|
||||
value := gjson.Get(json, "name.last")
|
||||
println(value.String())
|
||||
}
|
||||
```
|
||||
|
||||
This will print:
|
||||
|
||||
```
|
||||
Prichard
|
||||
```
|
||||
|
||||
A path is a series of keys seperated by a dot. A key may contain special wildcard characters '*' and '?'. To access an array value use the index as the key. To get the number of elements in an array use the '#' character.
|
||||
|
||||
```
|
||||
{
|
||||
"name": {"first": "Tom", "last": "Anderson"},
|
||||
"age":37,
|
||||
"children": ["Sara","Alex","Jack"]
|
||||
}
|
||||
"name.last" >> "Anderson"
|
||||
"age" >> 37
|
||||
"children.#" >> 3
|
||||
"children.1" >> "Alex"
|
||||
"child*.2" >> "Jack"
|
||||
"c?ildren.0" >> "Sara"
|
||||
```
|
||||
|
||||
|
||||
## Result Type
|
||||
|
||||
GJSON supports the json types `string`, `number`, `bool`, and `null`. Arrays and Objects are returned as their raw json types.
|
||||
|
||||
The `Result` type holds one of these types:
|
||||
|
||||
```
|
||||
bool, for JSON booleans
|
||||
float64, for JSON numbers
|
||||
Number, for JSON numbers
|
||||
string, for JSON string literals
|
||||
nil, for JSON null
|
||||
```
|
||||
|
||||
To get the value call the `Value()` method:
|
||||
|
||||
|
||||
```go
|
||||
result.Value() // interface{} which may be nil, string, float64, or bool
|
||||
|
||||
// Or just get the value in one step.
|
||||
gjson.Get(json, "name.last").Value()
|
||||
```
|
||||
|
||||
To directly access the value from its original type:
|
||||
|
||||
```go
|
||||
result.Type // can be String, Number, True, False, Null, or JSON
|
||||
result.Str // holds the string
|
||||
result.Num // holds the float64 number
|
||||
result.Raw // holds the raw json
|
||||
```
|
||||
|
||||
## Performance
|
||||
|
||||
Benchmarks of GJSON alongside [encoding/json](https://golang.org/pkg/encoding/json/), [ffjson](https://github.com/pquerna/ffjson), and [EasyJSON](https://github.com/mailru/easyjson).
|
||||
|
||||
```
|
||||
BenchmarkGJSONGet-8 3000000 477 ns/op 0 B/op 0 allocs/op
|
||||
BenchmarkJSONUnmarshalMap-8 600000 10738 ns/op 3176 B/op 69 allocs/op
|
||||
BenchmarkJSONUnmarshalStruct-8 600000 11635 ns/op 1960 B/op 69 allocs/op
|
||||
BenchmarkJSONDecoder-8 300000 17193 ns/op 4864 B/op 184 allocs/op
|
||||
BenchmarkFFJSONLexer-8 1500000 3773 ns/op 1024 B/op 8 allocs/op
|
||||
BenchmarkEasyJSONLexer-8 3000000 1134 ns/op 741 B/op 6 allocs/op
|
||||
```
|
||||
|
||||
The JSON document used was:
|
||||
|
||||
```json
|
||||
{
|
||||
"widget": {
|
||||
"debug": "on",
|
||||
"window": {
|
||||
"title": "Sample Konfabulator Widget",
|
||||
"name": "main_window",
|
||||
"width": 500,
|
||||
"height": 500
|
||||
},
|
||||
"image": {
|
||||
"src": "Images/Sun.png",
|
||||
"hOffset": 250,
|
||||
"vOffset": 250,
|
||||
"alignment": "center"
|
||||
},
|
||||
"text": {
|
||||
"data": "Click Here",
|
||||
"size": 36,
|
||||
"style": "bold",
|
||||
"vOffset": 100,
|
||||
"alignment": "center",
|
||||
"onMouseUp": "sun1.opacity = (sun1.opacity / 100) * 90;"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Each operation was rotated though one of the following search paths:
|
||||
|
||||
```
|
||||
widget.window.name
|
||||
widget.image.hOffset
|
||||
widget.text.onMouseUp
|
||||
```
|
||||
|
||||
|
||||
*These are the results from running the benchmarks on a MacBook Pro 15" 2.8 GHz Intel Core i7:*
|
||||
|
||||
## Contact
|
||||
Josh Baker [@tidwall](http://twitter.com/tidwall)
|
||||
|
||||
## License
|
||||
|
||||
GJSON source code is available under the MIT [License](/LICENSE).
|
||||
591
gjson.go
Normal file
591
gjson.go
Normal file
|
|
@ -0,0 +1,591 @@
|
|||
// Package gjson provides searching for json strings.
|
||||
package gjson
|
||||
|
||||
import "strconv"
|
||||
|
||||
// Type is Result type
|
||||
type Type byte
|
||||
|
||||
const (
|
||||
// Null is a null json value
|
||||
Null Type = iota
|
||||
// False is a json false boolean
|
||||
False
|
||||
// Number is json number
|
||||
Number
|
||||
// String is a json string
|
||||
String
|
||||
// True is a json true boolean
|
||||
True
|
||||
// JSON is a raw block of JSON
|
||||
JSON
|
||||
)
|
||||
|
||||
// Result represents a json value that is returned from Get().
|
||||
type Result struct {
|
||||
// Type is the json type
|
||||
Type Type
|
||||
// Raw is the raw json
|
||||
Raw string
|
||||
// Str is the json string
|
||||
Str string
|
||||
// Num is the json number
|
||||
Num float64
|
||||
}
|
||||
|
||||
// String returns a string representation of the value.
|
||||
func (t Result) String() string {
|
||||
switch t.Type {
|
||||
default:
|
||||
return "null"
|
||||
case False:
|
||||
return "false"
|
||||
case Number:
|
||||
return strconv.FormatFloat(t.Num, 'f', -1, 64)
|
||||
case String:
|
||||
return t.Str
|
||||
case JSON:
|
||||
return t.Raw
|
||||
case True:
|
||||
return "true"
|
||||
}
|
||||
}
|
||||
|
||||
// Value returns one of these types:
|
||||
//
|
||||
// bool, for JSON booleans
|
||||
// float64, for JSON numbers
|
||||
// Number, for JSON numbers
|
||||
// string, for JSON string literals
|
||||
// nil, for JSON null
|
||||
//
|
||||
func (t Result) Value() interface{} {
|
||||
switch t.Type {
|
||||
default:
|
||||
return nil
|
||||
case False:
|
||||
return false
|
||||
case Number:
|
||||
return t.Num
|
||||
case String:
|
||||
return t.Str
|
||||
case JSON:
|
||||
return t.Raw
|
||||
case True:
|
||||
return true
|
||||
}
|
||||
}
|
||||
|
||||
// Get searches json for the specified path.
|
||||
// A path is in dot syntax, such as "name.last" or "age".
|
||||
// This function expects that the json is well-formed, and does not validate.
|
||||
// Invalid json will not panic, but it may return back unexpected results.
|
||||
// When the value is found it's returned immediately.
|
||||
//
|
||||
// A path is a series of keys seperated by a dot.
|
||||
// A key may contain special wildcard characters '*' and '?'.
|
||||
// To access an array value use the index as the key.
|
||||
// To get the number of elements in an array use the '#' character.
|
||||
// {
|
||||
// "name": {"first": "Tom", "last": "Anderson"},
|
||||
// "age":37,
|
||||
// "children": ["Sara","Alex","Jack"]
|
||||
// }
|
||||
// "name.last" >> "Anderson"
|
||||
// "age" >> 37
|
||||
// "children.#" >> 3
|
||||
// "children.1" >> "Alex"
|
||||
// "child*.2" >> "Jack"
|
||||
// "c?ildren.0" >> "Sara"
|
||||
//
|
||||
func Get(json string, path string) Result {
|
||||
var i, s, depth int
|
||||
var squashed string
|
||||
var key string
|
||||
var stype byte
|
||||
var count int
|
||||
var wild bool
|
||||
var matched bool
|
||||
var parts = make([]string, 0, 4)
|
||||
var wilds = make([]bool, 0, 4)
|
||||
var keys = make([]string, 0, 4)
|
||||
var stypes = make([]byte, 0, 4)
|
||||
var counts = make([]int, 0, 4)
|
||||
|
||||
// do nothing when no path specified
|
||||
if len(path) == 0 {
|
||||
return Result{} // nothing
|
||||
}
|
||||
|
||||
depth = 1
|
||||
|
||||
// look for first delimiter
|
||||
for ; i < len(json); i++ {
|
||||
if json[i] > ' ' {
|
||||
if json[i] == '{' {
|
||||
stype = '{'
|
||||
} else if json[i] == '[' {
|
||||
stype = '['
|
||||
} else {
|
||||
// not a valid type
|
||||
return Result{}
|
||||
}
|
||||
i++
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
stypes = append(stypes, stype)
|
||||
counts = append(counts, count)
|
||||
|
||||
// parse the path. just split on the dot
|
||||
for i := 0; i < len(path); i++ {
|
||||
if path[i] == '.' {
|
||||
parts = append(parts, path[s:i])
|
||||
wilds = append(wilds, wild)
|
||||
if wild {
|
||||
wild = false
|
||||
}
|
||||
s = i + 1
|
||||
} else if path[i] == '*' || path[i] == '?' {
|
||||
wild = true
|
||||
}
|
||||
}
|
||||
parts = append(parts, path[s:])
|
||||
wilds = append(wilds, wild)
|
||||
|
||||
// search for key
|
||||
read_key:
|
||||
if stype == '[' {
|
||||
key = strconv.FormatInt(int64(count), 10)
|
||||
count++
|
||||
} else {
|
||||
for ; i < len(json); i++ {
|
||||
if json[i] == '"' {
|
||||
//read to end of key
|
||||
i++
|
||||
// readstr
|
||||
// the first double-quote has already been read
|
||||
s = i
|
||||
for ; i < len(json); i++ {
|
||||
if json[i] == '"' {
|
||||
key = json[s:i]
|
||||
i++
|
||||
break
|
||||
}
|
||||
if json[i] == '\\' {
|
||||
i++
|
||||
for ; i < len(json); i++ {
|
||||
if json[i] == '"' {
|
||||
// look for an escaped slash
|
||||
if json[i-1] == '\\' {
|
||||
n := 0
|
||||
for j := i - 2; j > s-1; j-- {
|
||||
if json[j] != '\\' {
|
||||
break
|
||||
}
|
||||
n++
|
||||
}
|
||||
if n%2 == 0 {
|
||||
continue
|
||||
}
|
||||
}
|
||||
break
|
||||
}
|
||||
}
|
||||
key = unescape(json[s:i])
|
||||
i++
|
||||
break
|
||||
}
|
||||
}
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
// end readstr
|
||||
|
||||
// we have a brand new key.
|
||||
// is it the key that we are looking for?
|
||||
if wilds[depth-1] {
|
||||
// it's a wildcard path element
|
||||
matched = wildcardMatch(key, parts[depth-1])
|
||||
} else {
|
||||
matched = parts[depth-1] == key
|
||||
}
|
||||
|
||||
// read to the value token
|
||||
// there's likely a colon here, but who cares. just burn past it.
|
||||
var val string
|
||||
var vc byte
|
||||
for ; i < len(json); i++ {
|
||||
switch json[i] {
|
||||
case 't', 'f', 'n': // true, false, null
|
||||
vc = json[i]
|
||||
s = i
|
||||
i++
|
||||
for ; i < len(json); i++ {
|
||||
// let's pick up any character. it doesn't matter.
|
||||
if json[i] < 'a' || json[i] > 'z' {
|
||||
break
|
||||
}
|
||||
}
|
||||
val = json[s:i]
|
||||
goto proc_val
|
||||
case '{': // open object
|
||||
i++
|
||||
vc = '{'
|
||||
goto proc_delim
|
||||
case '[': // open array
|
||||
i++
|
||||
vc = '['
|
||||
goto proc_delim
|
||||
case '"': // string
|
||||
i++
|
||||
// we read the val below
|
||||
vc = '"'
|
||||
goto proc_val
|
||||
case '-', '0', '1', '2', '3', '4', '5', '6', '7', '8', '9': // number
|
||||
vc = '0'
|
||||
s = i
|
||||
i++
|
||||
// look for characters that cannot be in a number
|
||||
for ; i < len(json); i++ {
|
||||
switch json[i] {
|
||||
default:
|
||||
continue
|
||||
case ' ', '\t', '\r', '\n', ',', ']', '}':
|
||||
}
|
||||
break
|
||||
}
|
||||
val = json[s:i]
|
||||
goto proc_val
|
||||
}
|
||||
}
|
||||
|
||||
// sanity check before we move on
|
||||
if i >= len(json) {
|
||||
return Result{}
|
||||
}
|
||||
|
||||
proc_delim:
|
||||
if (matched && depth == len(parts)) || !matched {
|
||||
// -- BEGIN SQUASH -- //
|
||||
// squash the value, ignoring all nested arrays and objects.
|
||||
s = i - 1
|
||||
// the first '[' or '{' has already been read
|
||||
depth := 1
|
||||
for ; i < len(json); i++ {
|
||||
if json[i] == '{' || json[i] == '[' {
|
||||
depth++
|
||||
} else if json[i] == '}' || json[i] == ']' {
|
||||
depth--
|
||||
if depth == 0 {
|
||||
i++
|
||||
break
|
||||
}
|
||||
} else if json[i] == '"' {
|
||||
i++
|
||||
s2 := i
|
||||
for ; i < len(json); i++ {
|
||||
if json[i] == '"' {
|
||||
// look for an escaped slash
|
||||
if json[i-1] == '\\' {
|
||||
n := 0
|
||||
for j := i - 2; j > s2-1; j-- {
|
||||
if json[j] != '\\' {
|
||||
break
|
||||
}
|
||||
n++
|
||||
}
|
||||
if n%2 == 0 {
|
||||
continue
|
||||
}
|
||||
}
|
||||
break
|
||||
}
|
||||
}
|
||||
if i == len(json) {
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
squashed = json[s:i]
|
||||
// -- END SQUASH -- //
|
||||
}
|
||||
|
||||
// process the value
|
||||
proc_val:
|
||||
if matched {
|
||||
// hit, that's good!
|
||||
if depth == len(parts) {
|
||||
var value Result
|
||||
value.Raw = val
|
||||
switch vc {
|
||||
case '{', '[':
|
||||
value.Raw = squashed
|
||||
value.Type = JSON
|
||||
case 'n':
|
||||
value.Type = Null
|
||||
case 't':
|
||||
value.Type = True
|
||||
case 'f':
|
||||
value.Type = False
|
||||
case '"':
|
||||
value.Type = String
|
||||
// readstr
|
||||
// the val has not been read yet
|
||||
// the first double-quote has already been read
|
||||
s = i
|
||||
for ; i < len(json); i++ {
|
||||
if json[i] == '"' {
|
||||
value.Str = json[s:i]
|
||||
i++
|
||||
break
|
||||
}
|
||||
if json[i] == '\\' {
|
||||
i++
|
||||
for ; i < len(json); i++ {
|
||||
if json[i] == '"' {
|
||||
// look for an escaped slash
|
||||
if json[i-1] == '\\' {
|
||||
n := 0
|
||||
for j := i - 2; j > s-1; j-- {
|
||||
if json[j] != '\\' {
|
||||
break
|
||||
}
|
||||
n++
|
||||
}
|
||||
if n%2 == 0 {
|
||||
continue
|
||||
}
|
||||
}
|
||||
break
|
||||
}
|
||||
}
|
||||
value.Str = unescape(json[s:i])
|
||||
i++
|
||||
break
|
||||
}
|
||||
}
|
||||
// end readstr
|
||||
case '0':
|
||||
value.Type = Number
|
||||
value.Num, _ = strconv.ParseFloat(val, 64)
|
||||
}
|
||||
return value
|
||||
//} else if vc != '{' {
|
||||
// can only deep search objects
|
||||
// return Result{}
|
||||
} else {
|
||||
stype = vc
|
||||
keys = append(keys, key)
|
||||
stypes = append(stypes, stype)
|
||||
counts = append(counts, count)
|
||||
depth++
|
||||
goto read_key
|
||||
}
|
||||
}
|
||||
if vc == '"' {
|
||||
// readstr
|
||||
// the val has not been read yet. we can read and throw away.
|
||||
// the first double-quote has already been read
|
||||
s = i
|
||||
for ; i < len(json); i++ {
|
||||
if json[i] == '"' {
|
||||
// look for an escaped slash
|
||||
if json[i-1] == '\\' {
|
||||
n := 0
|
||||
for j := i - 2; j > s-1; j-- {
|
||||
if json[j] != '\\' {
|
||||
break
|
||||
}
|
||||
n++
|
||||
}
|
||||
if n%2 == 0 {
|
||||
continue
|
||||
}
|
||||
}
|
||||
break
|
||||
}
|
||||
}
|
||||
i++
|
||||
// end readstr
|
||||
}
|
||||
|
||||
// read to the comma or end of object
|
||||
for ; i < len(json); i++ {
|
||||
switch json[i] {
|
||||
case '}', ']':
|
||||
if parts[depth-1] == "#" {
|
||||
return Result{Type: Number, Num: float64(count)}
|
||||
}
|
||||
// step the stack back
|
||||
depth--
|
||||
if depth == 0 {
|
||||
return Result{}
|
||||
}
|
||||
keys = keys[:len(keys)-1]
|
||||
stypes = stypes[:len(stypes)-1]
|
||||
counts = counts[:len(counts)-1]
|
||||
stype = stypes[len(stypes)-1]
|
||||
count = counts[len(counts)-1]
|
||||
case ',':
|
||||
i++
|
||||
goto read_key
|
||||
}
|
||||
}
|
||||
return Result{}
|
||||
}
|
||||
|
||||
// unescape unescapes a string
|
||||
func unescape(json string) string { //, error) {
|
||||
var str = make([]byte, 0, len(json))
|
||||
for i := 0; i < len(json); i++ {
|
||||
switch {
|
||||
default:
|
||||
str = append(str, json[i])
|
||||
case json[i] < ' ':
|
||||
return "" //, errors.New("invalid character in string")
|
||||
case json[i] == '\\':
|
||||
i++
|
||||
if i >= len(json) {
|
||||
return "" //, errors.New("invalid escape sequence")
|
||||
}
|
||||
switch json[i] {
|
||||
default:
|
||||
return "" //, errors.New("invalid escape sequence")
|
||||
case '\\':
|
||||
str = append(str, '\\')
|
||||
case '/':
|
||||
str = append(str, '/')
|
||||
case 'b':
|
||||
str = append(str, '\b')
|
||||
case 'f':
|
||||
str = append(str, '\f')
|
||||
case 'n':
|
||||
str = append(str, '\n')
|
||||
case 'r':
|
||||
str = append(str, '\r')
|
||||
case 't':
|
||||
str = append(str, '\t')
|
||||
case '"':
|
||||
str = append(str, '"')
|
||||
case 'u':
|
||||
if i+5 > len(json) {
|
||||
return "" //, errors.New("invalid escape sequence")
|
||||
}
|
||||
i++
|
||||
// extract the codepoint
|
||||
var code int
|
||||
for j := i; j < i+4; j++ {
|
||||
switch {
|
||||
default:
|
||||
return "" //, errors.New("invalid escape sequence")
|
||||
case json[j] >= '0' && json[j] <= '9':
|
||||
code += (int(json[j]) - '0') << uint(12-(j-i)*4)
|
||||
case json[j] >= 'a' && json[j] <= 'f':
|
||||
code += (int(json[j]) - 'a' + 10) << uint(12-(j-i)*4)
|
||||
case json[j] >= 'a' && json[j] <= 'f':
|
||||
code += (int(json[j]) - 'a' + 10) << uint(12-(j-i)*4)
|
||||
}
|
||||
}
|
||||
str = append(str, []byte(string(code))...)
|
||||
i += 3 // only 3 because we will increment on the for-loop
|
||||
}
|
||||
}
|
||||
}
|
||||
return string(str) //, nil
|
||||
}
|
||||
|
||||
// Less return true if a token is less than another token.
|
||||
// The caseSensitive paramater is used when the tokens are Strings.
|
||||
// The order when comparing two different type is:
|
||||
//
|
||||
// Null < False < Number < String < True < JSON
|
||||
//
|
||||
func (t Result) Less(token Result, caseSensitive bool) bool {
|
||||
if t.Type < token.Type {
|
||||
return true
|
||||
}
|
||||
if t.Type > token.Type {
|
||||
return false
|
||||
}
|
||||
switch t.Type {
|
||||
default:
|
||||
return t.Raw < token.Raw
|
||||
case String:
|
||||
if caseSensitive {
|
||||
return t.Str < token.Str
|
||||
}
|
||||
return stringLessInsensitive(t.Str, token.Str)
|
||||
case Number:
|
||||
return t.Num < token.Num
|
||||
}
|
||||
}
|
||||
|
||||
func stringLessInsensitive(a, b string) bool {
|
||||
for i := 0; i < len(a) && i < len(b); i++ {
|
||||
if a[i] >= 'A' && a[i] <= 'Z' {
|
||||
if b[i] >= 'A' && b[i] <= 'Z' {
|
||||
// both are uppercase, do nothing
|
||||
if a[i] < b[i] {
|
||||
return true
|
||||
} else if a[i] > b[i] {
|
||||
return false
|
||||
}
|
||||
} else {
|
||||
// a is uppercase, convert a to lowercase
|
||||
if a[i]+32 < b[i] {
|
||||
return true
|
||||
} else if a[i]+32 > b[i] {
|
||||
return false
|
||||
}
|
||||
}
|
||||
} else if b[i] >= 'A' && b[i] <= 'Z' {
|
||||
// b is uppercase, convert b to lowercase
|
||||
if a[i] < b[i]+32 {
|
||||
return true
|
||||
} else if a[i] > b[i]+32 {
|
||||
return false
|
||||
}
|
||||
} else {
|
||||
// neither are uppercase
|
||||
if a[i] < b[i] {
|
||||
return true
|
||||
} else if a[i] > b[i] {
|
||||
return false
|
||||
}
|
||||
}
|
||||
}
|
||||
return len(a) < len(b)
|
||||
}
|
||||
|
||||
// wilcardMatch returns true if str matches pattern. This is a very
|
||||
// simple wildcard match where '*' matches on any number characters
|
||||
// and '?' matches on any one character.
|
||||
func wildcardMatch(str, pattern string) bool {
|
||||
if pattern == "*" {
|
||||
return true
|
||||
}
|
||||
return deepMatch(str, pattern)
|
||||
}
|
||||
func deepMatch(str, pattern string) bool {
|
||||
for len(pattern) > 0 {
|
||||
switch pattern[0] {
|
||||
default:
|
||||
if len(str) == 0 || str[0] != pattern[0] {
|
||||
return false
|
||||
}
|
||||
case '?':
|
||||
if len(str) == 0 {
|
||||
return false
|
||||
}
|
||||
case '*':
|
||||
return wildcardMatch(str, pattern[1:]) ||
|
||||
(len(str) > 0 && wildcardMatch(str[1:], pattern))
|
||||
}
|
||||
str = str[1:]
|
||||
pattern = pattern[1:]
|
||||
}
|
||||
return len(str) == 0 && len(pattern) == 0
|
||||
}
|
||||
493
gjson_test.go
Normal file
493
gjson_test.go
Normal file
|
|
@ -0,0 +1,493 @@
|
|||
package gjson
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"io"
|
||||
"math/rand"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/mailru/easyjson/jlexer"
|
||||
fflib "github.com/pquerna/ffjson/fflib/v1"
|
||||
)
|
||||
|
||||
// TestRandomData is a fuzzing test that throughs random data at the Parse
|
||||
// function looking for panics.
|
||||
func TestRandomData(t *testing.T) {
|
||||
var lstr string
|
||||
defer func() {
|
||||
if v := recover(); v != nil {
|
||||
println("'" + hex.EncodeToString([]byte(lstr)) + "'")
|
||||
println("'" + lstr + "'")
|
||||
panic(v)
|
||||
}
|
||||
}()
|
||||
rand.Seed(time.Now().UnixNano())
|
||||
b := make([]byte, 200)
|
||||
for i := 0; i < 2000000; i++ {
|
||||
n, err := rand.Read(b[:rand.Int()%len(b)])
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
lstr = string(b[:n])
|
||||
Get(lstr, "zzzz")
|
||||
}
|
||||
}
|
||||
|
||||
func TestRandomValidStrings(t *testing.T) {
|
||||
rand.Seed(time.Now().UnixNano())
|
||||
b := make([]byte, 200)
|
||||
for i := 0; i < 100000; i++ {
|
||||
n, err := rand.Read(b[:rand.Int()%len(b)])
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
sm, err := json.Marshal(string(b[:n]))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var su string
|
||||
if err := json.Unmarshal([]byte(sm), &su); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
token := Get(`{"str":`+string(sm)+`}`, "str")
|
||||
if token.Type != String || token.Str != su {
|
||||
println("["+token.Raw+"]", "["+token.Str+"]", "["+su+"]", "["+string(sm)+"]")
|
||||
t.Fatal("string mismatch")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// this json block is poorly formed on purpose.
|
||||
var basicJSON = `{"age":100, "name":{"here":"B\\\"R"},
|
||||
"noop":{"what is a wren?":"a bird"},
|
||||
"happy":true,"immortal":false,
|
||||
"escaped\\\"":true,
|
||||
"arr":["1",2,"3",{"hello":"world"},"4",5],
|
||||
"vals":[1,2,3,{"sadf":sdf"asdf"}],"name":{"first":"tom","last":null}}`
|
||||
|
||||
func TestBasic(t *testing.T) {
|
||||
var token Result
|
||||
|
||||
token = Get(basicJSON, "name.here")
|
||||
if token.String() != "B\\\"R" {
|
||||
t.Fatal("expecting 'B\\\"R'", "got", token.String())
|
||||
}
|
||||
token = Get(basicJSON, "arr.#")
|
||||
if token.String() != "6" {
|
||||
t.Fatal("expecting '6'", "got", token.String())
|
||||
}
|
||||
token = Get(basicJSON, "arr.3.hello")
|
||||
if token.String() != "world" {
|
||||
t.Fatal("expecting 'world'", "got", token.String())
|
||||
}
|
||||
_ = token.Value().(string)
|
||||
token = Get(basicJSON, "name.first")
|
||||
if token.String() != "tom" {
|
||||
t.Fatal("expecting 'tom'", "got", token.String())
|
||||
}
|
||||
_ = token.Value().(string)
|
||||
token = Get(basicJSON, "name.last")
|
||||
if token.String() != "null" {
|
||||
t.Fatal("expecting 'null'", "got", token.String())
|
||||
}
|
||||
if token.Value() != nil {
|
||||
t.Fatal("should be nil")
|
||||
}
|
||||
token = Get(basicJSON, "age")
|
||||
if token.String() != "100" {
|
||||
t.Fatal("expecting '100'", "got", token.String())
|
||||
}
|
||||
_ = token.Value().(float64)
|
||||
token = Get(basicJSON, "happy")
|
||||
if token.String() != "true" {
|
||||
t.Fatal("expecting 'true'", "got", token.String())
|
||||
}
|
||||
_ = token.Value().(bool)
|
||||
token = Get(basicJSON, "immortal")
|
||||
if token.String() != "false" {
|
||||
t.Fatal("expecting 'false'", "got", token.String())
|
||||
}
|
||||
_ = token.Value().(bool)
|
||||
token = Get(basicJSON, "noop")
|
||||
if token.String() != `{"what is a wren?":"a bird"}` {
|
||||
t.Fatal("expecting '"+`{"what is a wren?":"a bird"}`+"'", "got", token.String())
|
||||
}
|
||||
_ = token.Value().(string)
|
||||
|
||||
if Get(basicJSON, "").Value() != nil {
|
||||
t.Fatal("should be nil")
|
||||
}
|
||||
|
||||
if !Get(basicJSON, "escaped\\\"").Value().(bool) {
|
||||
t.Fatal("could not escape")
|
||||
}
|
||||
|
||||
Get(basicJSON, "vals.hello")
|
||||
}
|
||||
|
||||
func TestUnescape(t *testing.T) {
|
||||
unescape(string([]byte{'\\', '\\', 0}))
|
||||
unescape(string([]byte{'\\', '/', '\\', 'b', '\\', 'f'}))
|
||||
}
|
||||
func assert(t testing.TB, cond bool) {
|
||||
if !cond {
|
||||
t.Fatal("assert failed")
|
||||
}
|
||||
}
|
||||
func TestLess(t *testing.T) {
|
||||
assert(t, !Result{Type: Null}.Less(Result{Type: Null}, true))
|
||||
assert(t, Result{Type: Null}.Less(Result{Type: False}, true))
|
||||
assert(t, Result{Type: Null}.Less(Result{Type: True}, true))
|
||||
assert(t, Result{Type: Null}.Less(Result{Type: JSON}, true))
|
||||
assert(t, Result{Type: Null}.Less(Result{Type: Number}, true))
|
||||
assert(t, Result{Type: Null}.Less(Result{Type: String}, true))
|
||||
assert(t, !Result{Type: False}.Less(Result{Type: Null}, true))
|
||||
assert(t, Result{Type: False}.Less(Result{Type: True}, true))
|
||||
assert(t, Result{Type: String, Str: "abc"}.Less(Result{Type: String, Str: "bcd"}, true))
|
||||
assert(t, Result{Type: String, Str: "ABC"}.Less(Result{Type: String, Str: "abc"}, true))
|
||||
assert(t, !Result{Type: String, Str: "ABC"}.Less(Result{Type: String, Str: "abc"}, false))
|
||||
assert(t, Result{Type: Number, Num: 123}.Less(Result{Type: Number, Num: 456}, true))
|
||||
assert(t, !Result{Type: Number, Num: 456}.Less(Result{Type: Number, Num: 123}, true))
|
||||
assert(t, !Result{Type: Number, Num: 456}.Less(Result{Type: Number, Num: 456}, true))
|
||||
assert(t, stringLessInsensitive("abcde", "BBCDE"))
|
||||
assert(t, stringLessInsensitive("abcde", "bBCDE"))
|
||||
assert(t, stringLessInsensitive("Abcde", "BBCDE"))
|
||||
assert(t, stringLessInsensitive("Abcde", "bBCDE"))
|
||||
assert(t, !stringLessInsensitive("bbcde", "aBCDE"))
|
||||
assert(t, !stringLessInsensitive("bbcde", "ABCDE"))
|
||||
assert(t, !stringLessInsensitive("Bbcde", "aBCDE"))
|
||||
assert(t, !stringLessInsensitive("Bbcde", "ABCDE"))
|
||||
assert(t, !stringLessInsensitive("abcde", "ABCDE"))
|
||||
assert(t, !stringLessInsensitive("Abcde", "ABCDE"))
|
||||
assert(t, !stringLessInsensitive("abcde", "ABCDE"))
|
||||
assert(t, !stringLessInsensitive("ABCDE", "ABCDE"))
|
||||
assert(t, !stringLessInsensitive("abcde", "abcde"))
|
||||
assert(t, !stringLessInsensitive("123abcde", "123Abcde"))
|
||||
assert(t, !stringLessInsensitive("123Abcde", "123Abcde"))
|
||||
assert(t, !stringLessInsensitive("123Abcde", "123abcde"))
|
||||
assert(t, !stringLessInsensitive("123abcde", "123abcde"))
|
||||
assert(t, !stringLessInsensitive("124abcde", "123abcde"))
|
||||
assert(t, !stringLessInsensitive("124Abcde", "123Abcde"))
|
||||
assert(t, !stringLessInsensitive("124Abcde", "123abcde"))
|
||||
assert(t, !stringLessInsensitive("124abcde", "123abcde"))
|
||||
assert(t, stringLessInsensitive("124abcde", "125abcde"))
|
||||
assert(t, stringLessInsensitive("124Abcde", "125Abcde"))
|
||||
assert(t, stringLessInsensitive("124Abcde", "125abcde"))
|
||||
assert(t, stringLessInsensitive("124abcde", "125abcde"))
|
||||
}
|
||||
|
||||
/*
|
||||
func TestTwitter(t *testing.T) {
|
||||
data, err := ioutil.ReadFile("twitter.json")
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
token := Get(string(data), "search_metadata.max_id")
|
||||
if token.Num != 505874924095815700 {
|
||||
t.Fatalf("expecting %d\n", 505874924095815700)
|
||||
}
|
||||
|
||||
}
|
||||
func BenchmarkTwitter(t *testing.B) {
|
||||
// the twitter.json file must be present
|
||||
data, err := ioutil.ReadFile("twitter.json")
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
json := string(data)
|
||||
t.ResetTimer()
|
||||
for i := 0; i < t.N; i++ {
|
||||
token := Get(json, "search_metadata.max_id")
|
||||
if token.Type != Number || token.Raw != "505874924095815700" || token.Num != 505874924095815700 {
|
||||
t.Fatal("invalid response")
|
||||
}
|
||||
}
|
||||
}
|
||||
*/
|
||||
|
||||
var exampleJSON = `
|
||||
{"widget": {
|
||||
"debug": "on",
|
||||
"window": {
|
||||
"title": "Sample Konfabulator Widget",
|
||||
"name": "main_window",
|
||||
"width": 500,
|
||||
"height": 500
|
||||
},
|
||||
"image": {
|
||||
"src": "Images/Sun.png",
|
||||
"hOffset": 250,
|
||||
"vOffset": 250,
|
||||
"alignment": "center"
|
||||
},
|
||||
"text": {
|
||||
"data": "Click Here",
|
||||
"size": 36,
|
||||
"style": "bold",
|
||||
"vOffset": 100,
|
||||
"alignment": "center",
|
||||
"onMouseUp": "sun1.opacity = (sun1.opacity / 100) * 90;"
|
||||
}
|
||||
}}
|
||||
`
|
||||
|
||||
type BenchStruct struct {
|
||||
Widget struct {
|
||||
Window struct {
|
||||
Name string `json:"name"`
|
||||
} `json:"window"`
|
||||
Image struct {
|
||||
HOffset int `json:"hOffset"`
|
||||
} `json:"image"`
|
||||
Text struct {
|
||||
OnMouseUp string `json:"onMouseUp"`
|
||||
} `json:"text"`
|
||||
} `json:"widget"`
|
||||
}
|
||||
|
||||
var benchPaths = []string{
|
||||
"widget.window.name",
|
||||
"widget.image.hOffset",
|
||||
"widget.text.onMouseUp",
|
||||
}
|
||||
|
||||
func BenchmarkGJSONGet(t *testing.B) {
|
||||
t.ReportAllocs()
|
||||
for i := 0; i < t.N; i++ {
|
||||
for j := 0; j < len(benchPaths); j++ {
|
||||
if Get(exampleJSON, benchPaths[j]).Type == Null {
|
||||
t.Fatal("did not find the value")
|
||||
}
|
||||
}
|
||||
}
|
||||
t.N *= len(benchPaths) // because we are running against 3 paths
|
||||
}
|
||||
|
||||
func BenchmarkJSONUnmarshalMap(t *testing.B) {
|
||||
t.ReportAllocs()
|
||||
for i := 0; i < t.N; i++ {
|
||||
for j := 0; j < len(benchPaths); j++ {
|
||||
parts := strings.Split(benchPaths[j], ".")
|
||||
var m map[string]interface{}
|
||||
if err := json.Unmarshal([]byte(exampleJSON), &m); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var v interface{}
|
||||
for len(parts) > 0 {
|
||||
part := parts[0]
|
||||
if len(parts) > 1 {
|
||||
m = m[part].(map[string]interface{})
|
||||
if m == nil {
|
||||
t.Fatal("did not find the value")
|
||||
}
|
||||
} else {
|
||||
v = m[part]
|
||||
if v == nil {
|
||||
t.Fatal("did not find the value")
|
||||
}
|
||||
}
|
||||
parts = parts[1:]
|
||||
}
|
||||
}
|
||||
}
|
||||
t.N *= len(benchPaths) // because we are running against 3 paths
|
||||
}
|
||||
|
||||
func BenchmarkJSONUnmarshalStruct(t *testing.B) {
|
||||
t.ReportAllocs()
|
||||
for i := 0; i < t.N; i++ {
|
||||
for j := 0; j < len(benchPaths); j++ {
|
||||
var s BenchStruct
|
||||
if err := json.Unmarshal([]byte(exampleJSON), &s); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
switch benchPaths[j] {
|
||||
case "widget.window.name":
|
||||
if s.Widget.Window.Name == "" {
|
||||
t.Fatal("did not find the value")
|
||||
}
|
||||
case "widget.image.hOffset":
|
||||
if s.Widget.Image.HOffset == 0 {
|
||||
t.Fatal("did not find the value")
|
||||
}
|
||||
case "widget.text.onMouseUp":
|
||||
if s.Widget.Text.OnMouseUp == "" {
|
||||
t.Fatal("did not find the value")
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
t.N *= len(benchPaths) // because we are running against 3 paths
|
||||
}
|
||||
|
||||
func BenchmarkJSONDecoder(t *testing.B) {
|
||||
t.ReportAllocs()
|
||||
for i := 0; i < t.N; i++ {
|
||||
for j := 0; j < len(benchPaths); j++ {
|
||||
dec := json.NewDecoder(bytes.NewBuffer([]byte(exampleJSON)))
|
||||
var found bool
|
||||
outer:
|
||||
for {
|
||||
tok, err := dec.Token()
|
||||
if err != nil {
|
||||
if err == io.EOF {
|
||||
break
|
||||
}
|
||||
t.Fatal(err)
|
||||
}
|
||||
switch v := tok.(type) {
|
||||
case string:
|
||||
if found {
|
||||
// break out once we find the value.
|
||||
break outer
|
||||
}
|
||||
switch benchPaths[j] {
|
||||
case "widget.window.name":
|
||||
if v == "name" {
|
||||
found = true
|
||||
}
|
||||
case "widget.image.hOffset":
|
||||
if v == "hOffset" {
|
||||
found = true
|
||||
}
|
||||
case "widget.text.onMouseUp":
|
||||
if v == "onMouseUp" {
|
||||
found = true
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
t.Fatal("field not found")
|
||||
}
|
||||
}
|
||||
}
|
||||
t.N *= len(benchPaths) // because we are running against 3 paths
|
||||
}
|
||||
|
||||
func BenchmarkFFJSONLexer(t *testing.B) {
|
||||
t.ReportAllocs()
|
||||
for i := 0; i < t.N; i++ {
|
||||
for j := 0; j < len(benchPaths); j++ {
|
||||
l := fflib.NewFFLexer([]byte(exampleJSON))
|
||||
var found bool
|
||||
outer:
|
||||
for {
|
||||
t := l.Scan()
|
||||
if t == fflib.FFTok_eof {
|
||||
break
|
||||
}
|
||||
if t == fflib.FFTok_string {
|
||||
b, _ := l.CaptureField(t)
|
||||
v := string(b)
|
||||
if found {
|
||||
// break out once we find the value.
|
||||
break outer
|
||||
}
|
||||
switch benchPaths[j] {
|
||||
case "widget.window.name":
|
||||
if v == "\"name\"" {
|
||||
found = true
|
||||
}
|
||||
case "widget.image.hOffset":
|
||||
if v == "\"hOffset\"" {
|
||||
found = true
|
||||
}
|
||||
case "widget.text.onMouseUp":
|
||||
if v == "\"onMouseUp\"" {
|
||||
found = true
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
t.Fatal("field not found")
|
||||
}
|
||||
}
|
||||
}
|
||||
t.N *= len(benchPaths) // because we are running against 3 paths
|
||||
}
|
||||
|
||||
func BenchmarkEasyJSONLexer(t *testing.B) {
|
||||
t.ReportAllocs()
|
||||
skipCC := func(l *jlexer.Lexer, n int) {
|
||||
for i := 0; i < n; i++ {
|
||||
l.Skip()
|
||||
l.WantColon()
|
||||
l.Skip()
|
||||
l.WantComma()
|
||||
}
|
||||
}
|
||||
skipGroup := func(l *jlexer.Lexer, n int) {
|
||||
l.WantColon()
|
||||
l.Delim('{')
|
||||
skipCC(l, n)
|
||||
l.Delim('}')
|
||||
l.WantComma()
|
||||
}
|
||||
for i := 0; i < t.N; i++ {
|
||||
for j := 0; j < len(benchPaths); j++ {
|
||||
l := &jlexer.Lexer{Data: []byte(exampleJSON)}
|
||||
l.Delim('{')
|
||||
if l.String() == "widget" {
|
||||
l.WantColon()
|
||||
l.Delim('{')
|
||||
switch benchPaths[j] {
|
||||
case "widget.window.name":
|
||||
skipCC(l, 1)
|
||||
if l.String() == "window" {
|
||||
l.WantColon()
|
||||
l.Delim('{')
|
||||
skipCC(l, 1)
|
||||
if l.String() == "name" {
|
||||
l.WantColon()
|
||||
if l.String() == "" {
|
||||
t.Fatal("did not find the value")
|
||||
}
|
||||
}
|
||||
}
|
||||
case "widget.image.hOffset":
|
||||
skipCC(l, 1)
|
||||
if l.String() == "window" {
|
||||
skipGroup(l, 4)
|
||||
}
|
||||
if l.String() == "image" {
|
||||
l.WantColon()
|
||||
l.Delim('{')
|
||||
skipCC(l, 1)
|
||||
if l.String() == "hOffset" {
|
||||
l.WantColon()
|
||||
if l.Int() == 0 {
|
||||
t.Fatal("did not find the value")
|
||||
}
|
||||
}
|
||||
}
|
||||
case "widget.text.onMouseUp":
|
||||
skipCC(l, 1)
|
||||
if l.String() == "window" {
|
||||
skipGroup(l, 4)
|
||||
}
|
||||
if l.String() == "image" {
|
||||
skipGroup(l, 4)
|
||||
}
|
||||
if l.String() == "text" {
|
||||
l.WantColon()
|
||||
l.Delim('{')
|
||||
skipCC(l, 5)
|
||||
if l.String() == "onMouseUp" {
|
||||
l.WantColon()
|
||||
if l.String() == "" {
|
||||
t.Fatal("did not find the value")
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
t.N *= len(benchPaths) // because we are running against 3 paths
|
||||
}
|
||||
Loading…
Reference in New Issue
Block a user