27 Commits

Author SHA1 Message Date
Josh Baker
01f00f1296 Merge pull request #66 from Hexilee/master
Result.Bool() can parse 'false' as false
2018-02-23 08:48:56 -07:00
Hexilee
9fa9086994 Result.Bool() can parse 'false' as false 2018-02-23 20:25:48 +08:00
Josh Baker
aff9dcea3c uncomment function 2018-02-18 10:09:03 -07:00
Josh Baker
c2e370e1b3 comment on Valid 2018-02-18 10:05:23 -07:00
Josh Baker
df0b0cce42 section on json validation 2018-02-18 09:49:01 -07:00
Josh Baker
46c712f1ce update playground badge 2018-02-10 11:31:49 -07:00
Josh Baker
54d114d487 minor updates 2018-02-10 09:30:30 -07:00
Josh Baker
6d43584b26 mention related projects 2018-02-10 09:10:40 -07:00
Josh Baker
4b8623b9d6 minor edits 2018-02-09 19:51:56 -07:00
Josh Baker
749b396bb9 Update README.md 2018-02-09 16:09:45 -07:00
Josh Baker
077965694f changed badge color 2018-02-09 16:08:32 -07:00
Josh Baker
b9953e2865 Update README.md 2018-02-09 16:04:47 -07:00
Josh Baker
a2f35b522e Added support for JSON Lines
Added support for JSON Lines (http://jsonlines.org) using the `..` prefix.
Which when specified, treats the multi-lined document as an array.

For example:

```
{"name": "Gilbert", "age": 61}
{"name": "Alexa", "age": 34}
{"name": "May", "age": 57}
{"name": "Deloise", "age": 44}
```

```
..#                   >> 4
..1                   >> {"name": "Alexa", "age": 34}
..3                   >> {"name": "Deloise", "age": 44}
..#.name              >> ["Gilbert","Alexa","May","Deloise"]
..#[name="May"].age   >> 57
```

Closes #60
2018-02-09 15:42:42 -07:00
Josh Baker
5fe9078c47 mention large integer support 2018-02-08 12:21:27 -07:00
Josh Baker
bff3f07fe7 mention large integer support 2018-02-08 12:21:08 -07:00
Josh Baker
87033efcae array query mismatch, fixes #58 2018-01-23 05:45:05 -07:00
Josh Baker
5cd723d566 simplify getmanybytes 2017-12-22 07:19:48 -07:00
Josh Baker
62ee2064df remove commented line 2017-12-22 07:17:19 -07:00
Josh Baker
09641abb33 update tagline 2017-12-22 07:16:29 -07:00
Josh Baker
e62d62a3e1 match GetMany and Get results, fixes #55 2017-12-22 06:58:04 -07:00
Josh Baker
080cd22816 fix mysterious missing result
fixes #54
2017-12-13 16:29:27 -07:00
Josh Baker
182ad76050 Array() from null becomes zero length Go array
fixes #53
2017-12-01 14:13:24 -07:00
Josh Baker
67e2a63ac7 Merge branch 'erikjohnston-fix-raw-false' 2017-11-20 11:50:05 -07:00
Erik Johnston
922b012d22 Fix bug where Result.Raw of literal 'false' was 'f' 2017-11-20 17:59:58 +00:00
Josh Baker
ac7b6ae6f2 deprecated unmarshalling 2017-10-18 05:19:20 -07:00
Josh Baker
5a69e67cfd GetMany result value missing, fixes #48 2017-09-25 04:39:06 -07:00
Josh Baker
3c91814cf6 GetMany result incorrect, fixes #47 2017-09-25 04:37:57 -07:00
3 changed files with 423 additions and 488 deletions

124
README.md
View File

@@ -5,15 +5,17 @@
<br>
<a href="https://travis-ci.org/tidwall/gjson"><img src="https://img.shields.io/travis/tidwall/gjson.svg?style=flat-square" alt="Build Status"></a>
<a href="https://godoc.org/github.com/tidwall/gjson"><img src="https://img.shields.io/badge/api-reference-blue.svg?style=flat-square" alt="GoDoc"></a>
<a href="http://tidwall.com/gjson-play"><img src="https://img.shields.io/badge/play-ground-orange.svg?style=flat-square" alt="GJSON Playground"></a>
<a href="http://tidwall.com/gjson-play"><img src="https://img.shields.io/badge/%F0%9F%8F%90-playground-9900cc.svg?style=flat-square" alt="GJSON Playground"></a>
</p>
<p align="center">get a json value quickly</a></p>
<p align="center">get json values quickly</a></p>
GJSON is a Go package that provides a [fast](#performance) and [simple](#get-a-value) way to get values from a json document.
It has features such as [one line retrieval](#get-a-value), [dot notation paths](#path-syntax), [iteration](#iterate-through-an-object-or-array).
It has features such as [one line retrieval](#get-a-value), [dot notation paths](#path-syntax), [iteration](#iterate-through-an-object-or-array), and [parsing json lines](#json-lines).
Also check out [SJSON](https://github.com/tidwall/sjson) for modifying json, and the [JJ](https://github.com/tidwall/jj) command line tool.
Getting Started
===============
@@ -29,7 +31,7 @@ $ go get -u github.com/tidwall/gjson
This will retrieve the library.
## Get a value
Get searches json for the specified path. A path is in dot syntax, such as "name.last" or "age". This function expects that the json is well-formed. Bad json will not panic, but it may return back unexpected results. When the value is found it's returned immediately.
Get searches json for the specified path. A path is in dot syntax, such as "name.last" or "age". When the value is found it's returned immediately.
```go
package main
@@ -95,6 +97,36 @@ friends.#[age>45]#.last >> ["Craig","Murphy"]
friends.#[first%"D*"].last >> "Murphy"
```
## JSON Lines
There's support for [JSON Lines](http://jsonlines.org/) using the `..` prefix, which treats a multilined document as an array.
For example:
```
{"name": "Gilbert", "age": 61}
{"name": "Alexa", "age": 34}
{"name": "May", "age": 57}
{"name": "Deloise", "age": 44}
```
```
..# >> 4
..1 >> {"name": "Alexa", "age": 34}
..3 >> {"name": "Deloise", "age": 44}
..#.name >> ["Gilbert","Alexa","May","Deloise"]
..#[name="May"].age >> 57
```
The `ForEachLines` function will iterate through JSON lines.
```go
gjson.ForEachLine(json, func(line gjson.Result) bool{
println(line.String())
return true
})
```
## Result Type
GJSON supports the json types `string`, `number`, `bool`, and `null`.
@@ -152,6 +184,15 @@ array >> []interface{}
object >> map[string]interface{}
```
### 64-bit integers
The `result.Int()` and `result.Uint()` calls are capable of reading all 64 bits, allowing for large JSON integers.
```go
result.Int() int64 // -9223372036854775808 to 9223372036854775807
result.Uint() int64 // 0 to 18446744073709551615
```
## Get nested array values
Suppose you want all the last names from the following json:
@@ -234,51 +275,17 @@ if gjson.Get(json, "name.last").Exists() {
}
```
## Unmarshalling
## Validate JSON
There's a `gjson.Unmarshal` function which loads json data into a value.
It's a general replacement for `json.Unmarshal` and you can typically
see a 2-3x boost in performance without the need for external generators.
This function works almost identically to `json.Unmarshal` except that
`gjson.Unmarshal` will automatically attempt to convert JSON values to any
Go type. For example, the JSON string "100" or the JSON number 100 can be
equally assigned to Go string, int, byte, uint64, etc. This rule applies to
all types.
The `Get*` and `Parse*` functions expects that the json is well-formed. Bad json will not panic, but it may return back unexpected results.
If you are consuming JSON from an unpredictable source then you may want to validate prior to using GJSON.
```go
package main
import (
"fmt"
"github.com/tidwall/gjson"
)
type Animal struct {
Type string `json:"type"`
Sound string `json:"sound"`
Age int `json:"age"`
if !gjson.Valid(json) {
return errors.New("invalid json")
}
var json = `{
"type": "Dog",
"Sound": "Bark",
"Age": "11"
}`
func main() {
var dog Animal
gjson.Unmarshal([]byte(json), &dog)
fmt.Printf("type: %s, sound: %s, age: %d\n", dog.Type, dog.Sound, dog.Age)
}
```
This will print:
```
type: Dog, sound: Bark, age: 11
value := gjson.Get(json, "name.last")
```
## Unmarshal to a map
@@ -318,7 +325,7 @@ This is a best-effort no allocation sub slice of the original json. This method
## Get multiple values at once
The `GetMany` function can be used to get multiple values at the same time, and is optimized to scan over a JSON payload once.
The `GetMany` function can be used to get multiple values at the same time.
```go
results := gjson.GetMany(json, "name.first", "name.last", "age")
@@ -338,7 +345,6 @@ and [json-iterator](https://github.com/json-iterator/go)
BenchmarkGJSONGet-8 3000000 372 ns/op 0 B/op 0 allocs/op
BenchmarkGJSONUnmarshalMap-8 900000 4154 ns/op 1920 B/op 26 allocs/op
BenchmarkJSONUnmarshalMap-8 600000 9019 ns/op 3048 B/op 69 allocs/op
BenchmarkJSONUnmarshalStruct-8 600000 9268 ns/op 1832 B/op 69 allocs/op
BenchmarkJSONDecoder-8 300000 14120 ns/op 4224 B/op 184 allocs/op
BenchmarkFFJSONLexer-8 1500000 3111 ns/op 896 B/op 8 allocs/op
BenchmarkEasyJSONLexer-8 3000000 887 ns/op 613 B/op 6 allocs/op
@@ -346,17 +352,6 @@ BenchmarkJSONParserGet-8 3000000 499 ns/op 21 B/op
BenchmarkJSONIterator-8 3000000 812 ns/op 544 B/op 9 allocs/op
```
Benchmarks for the `GetMany` function:
```
BenchmarkGJSONGetMany4Paths-8 4000000 303 ns/op 112 B/op 0 allocs/op
BenchmarkGJSONGetMany8Paths-8 8000000 208 ns/op 56 B/op 0 allocs/op
BenchmarkGJSONGetMany16Paths-8 16000000 156 ns/op 56 B/op 0 allocs/op
BenchmarkGJSONGetMany32Paths-8 32000000 127 ns/op 64 B/op 0 allocs/op
BenchmarkGJSONGetMany64Paths-8 64000000 117 ns/op 64 B/op 0 allocs/op
BenchmarkGJSONGetMany128Paths-8 128000000 109 ns/op 64 B/op 0 allocs/op
```
JSON document used:
```json
@@ -395,21 +390,6 @@ widget.image.hOffset
widget.text.onMouseUp
```
For the `GetMany` benchmarks these paths are used:
```
widget.window.name
widget.image.hOffset
widget.text.onMouseUp
widget.window.title
widget.image.alignment
widget.text.style
widget.window.height
widget.image.src
widget.text.data
widget.text.size
```
*These benchmarks were run on a MacBook Pro 15" 2.8 GHz Intel Core i7 using Go 1.8 and can be be found [here](https://github.com/tidwall/gjson-benchmarks).*

504
gjson.go
View File

@@ -96,7 +96,7 @@ func (t Result) Bool() bool {
case True:
return true
case String:
return t.Str != "" && t.Str != "0"
return t.Str != "" && t.Str != "0" && t.Str != "false"
case Number:
return t.Num != 0
}
@@ -177,8 +177,8 @@ func (t Result) Time() time.Time {
// If the result represents a non-existent value, then an empty array will be returned.
// If the result is not a JSON array, the return value will be an array containing one result.
func (t Result) Array() []Result {
if !t.Exists() {
return nil
if t.Type == Null {
return []Result{}
}
if t.Type != JSON {
return []Result{t}
@@ -192,7 +192,7 @@ func (t Result) IsObject() bool {
return t.Type == JSON && len(t.Raw) > 0 && t.Raw[0] == '{'
}
// IsObject returns true if the result value is a JSON array.
// IsArray returns true if the result value is a JSON array.
func (t Result) IsArray() bool {
return t.Type == JSON && len(t.Raw) > 0 && t.Raw[0] == '['
}
@@ -390,6 +390,11 @@ end:
}
// Parse parses the json and returns a result.
//
// This function expects that the json is well-formed, and does not validate.
// Invalid json will not panic, but it may return back unexpected results.
// If you are consuming JSON from an unpredictable source then you may want to
// use the Valid function first.
func Parse(json string) Result {
var value Result
for i := 0; i < len(json); i++ {
@@ -511,7 +516,7 @@ func tonum(json string) (raw string, num float64) {
func tolit(json string) (raw string) {
for i := 1; i < len(json); i++ {
if json[i] <= 'a' || json[i] >= 'z' {
if json[i] < 'a' || json[i] > 'z' {
return json[:i]
}
}
@@ -1077,7 +1082,7 @@ func queryMatches(rp *arrayPathResult, value Result) bool {
case "=":
return value.Num == rpvn
case "!=":
return value.Num == rpvn
return value.Num != rpvn
case "<":
return value.Num < rpvn
case "<=":
@@ -1128,7 +1133,7 @@ func parseArray(c *parseContext, i int, path string) (int, bool) {
partidx = int(n)
}
}
for i < len(c.json) {
for i < len(c.json)+1 {
if !rp.arrch {
pmatch = partidx == h
hit = pmatch && !rp.more
@@ -1137,8 +1142,16 @@ func parseArray(c *parseContext, i int, path string) (int, bool) {
if rp.alogok {
alog = append(alog, i)
}
for ; i < len(c.json); i++ {
switch c.json[i] {
for ; ; i++ {
var ch byte
if i > len(c.json) {
break
} else if i == len(c.json) {
ch = ']'
} else {
ch = c.json[i]
}
switch ch {
default:
continue
case '"':
@@ -1252,14 +1265,18 @@ func parseArray(c *parseContext, i int, path string) (int, bool) {
if rp.alogok {
var jsons = make([]byte, 0, 64)
jsons = append(jsons, '[')
for j, k := 0, 0; j < len(alog); j++ {
res := Get(c.json[alog[j]:], rp.alogkey)
if res.Exists() {
if k > 0 {
jsons = append(jsons, ',')
_, res, ok := parseAny(c.json, alog[j], true)
if ok {
res := res.Get(rp.alogkey)
if res.Exists() {
if k > 0 {
jsons = append(jsons, ',')
}
jsons = append(jsons, []byte(res.Raw)...)
k++
}
jsons = append(jsons, []byte(res.Raw)...)
k++
}
}
jsons = append(jsons, ']')
@@ -1290,16 +1307,32 @@ func parseArray(c *parseContext, i int, path string) (int, bool) {
return i, false
}
// ForEachLine iterates through lines of JSON as specified by the JSON Lines
// format (http://jsonlines.org/).
// Each line is returned as a GJSON Result.
func ForEachLine(json string, iterator func(line Result) bool) {
var res Result
var i int
for {
i, res, _ = parseAny(json, i, true)
if !res.Exists() {
break
}
if !iterator(res) {
return
}
}
}
type parseContext struct {
json string
value Result
calcd bool
lines bool
}
// Get searches json for the specified path.
// A path is in dot syntax, such as "name.last" or "age".
// This function expects that the json is well-formed, and does not validate.
// Invalid json will not panic, but it may return back unexpected results.
// When the value is found it's returned immediately.
//
// A path is a series of keys searated by a dot.
@@ -1326,19 +1359,28 @@ type parseContext struct {
// "c?ildren.0" >> "Sara"
// "friends.#.first" >> ["James","Roger"]
//
// This function expects that the json is well-formed, and does not validate.
// Invalid json will not panic, but it may return back unexpected results.
// If you are consuming JSON from an unpredictable source then you may want to
// use the Valid function first.
func Get(json, path string) Result {
var i int
var c = &parseContext{json: json}
for ; i < len(c.json); i++ {
if c.json[i] == '{' {
i++
parseObject(c, i, path)
break
}
if c.json[i] == '[' {
i++
parseArray(c, i, path)
break
if len(path) >= 2 && path[0] == '.' && path[1] == '.' {
c.lines = true
parseArray(c, 0, path[2:])
} else {
for ; i < len(c.json); i++ {
if c.json[i] == '{' {
i++
parseObject(c, i, path)
break
}
if c.json[i] == '[' {
i++
parseArray(c, i, path)
break
}
}
}
if len(c.value.Raw) > 0 && !c.calcd {
@@ -1595,400 +1637,22 @@ var ( // used for testing
testLastWasFallback bool
)
// areSimplePaths returns true if all the paths are simple enough
// to parse quickly for GetMany(). Allows alpha-numeric, dots,
// underscores, and the dollar sign. It does not allow non-alnum,
// escape characters, or keys which start with a numbers.
// For example:
// "name.last" == OK
// "user.id0" == OK
// "user.ID" == OK
// "user.first_name" == OK
// "user.firstName" == OK
// "user.0item" == BAD
// "user.#id" == BAD
// "user\.name" == BAD
func areSimplePaths(paths []string) bool {
for _, path := range paths {
var fi int // first key index, for keys with numeric prefix
for i := 0; i < len(path); i++ {
if path[i] >= 'a' && path[i] <= 'z' {
// a-z is likely to be the highest frequency charater.
continue
}
if path[i] == '.' {
fi = i + 1
continue
}
if path[i] >= 'A' && path[i] <= 'Z' {
continue
}
if path[i] == '_' || path[i] == '$' {
continue
}
if i > fi && path[i] >= '0' && path[i] <= '9' {
continue
}
return false
}
}
return true
}
// GetMany searches json for the multiple paths.
// The return value is a Result array where the number of items
// will be equal to the number of input paths.
func GetMany(json string, paths ...string) []Result {
if len(paths) < 4 {
if testWatchForFallback {
testLastWasFallback = false
}
switch len(paths) {
case 0:
// return nil when no paths are specified.
return nil
case 1:
return []Result{Get(json, paths[0])}
case 2:
return []Result{Get(json, paths[0]), Get(json, paths[1])}
case 3:
return []Result{Get(json, paths[0]), Get(json, paths[1]), Get(json, paths[2])}
}
func GetMany(json string, path ...string) []Result {
res := make([]Result, len(path))
for i, path := range path {
res[i] = Get(json, path)
}
var results []Result
var ok bool
var i int
if len(paths) > 512 {
// we can only support up to 512 paths. Is that too many?
goto fallback
}
if !areSimplePaths(paths) {
// If there is even one path that is not considered "simple" then
// we need to use the fallback method.
goto fallback
}
// locate the object token.
for ; i < len(json); i++ {
if json[i] == '{' {
i++
break
}
if json[i] <= ' ' {
continue
}
goto fallback
}
// use the call function table.
if len(paths) <= 8 {
results, ok = getMany8(json, i, paths)
} else if len(paths) <= 16 {
results, ok = getMany16(json, i, paths)
} else if len(paths) <= 32 {
results, ok = getMany32(json, i, paths)
} else if len(paths) <= 64 {
results, ok = getMany64(json, i, paths)
} else if len(paths) <= 128 {
results, ok = getMany128(json, i, paths)
} else if len(paths) <= 256 {
results, ok = getMany256(json, i, paths)
} else if len(paths) <= 512 {
results, ok = getMany512(json, i, paths)
}
if !ok {
// there was some fault while parsing. we should try the
// fallback method. This could result in performance
// degregation in some cases.
goto fallback
}
if testWatchForFallback {
testLastWasFallback = false
}
return results
fallback:
results = results[:0]
for i := 0; i < len(paths); i++ {
results = append(results, Get(json, paths[i]))
}
if testWatchForFallback {
testLastWasFallback = true
}
return results
return res
}
// GetManyBytes searches json for the specified path.
// If working with bytes, this method preferred over
// GetMany(string(data), paths...)
func GetManyBytes(json []byte, paths ...string) []Result {
if json == nil {
return GetMany("", paths...)
}
results := GetMany(*(*string)(unsafe.Pointer(&json)), paths...)
for i := range results {
results[i] = fromBytesGet(results[i])
}
return results
}
// parseGetMany parses a json object for keys that match against the callers
// paths. It's a best-effort attempt and quickly locating and assigning the
// values to the []Result array. If there are failures such as bad json, or
// invalid input paths, or too much recursion, the function will exit with a
// return value of 'false'.
func parseGetMany(
json string, i int,
level uint, kplen int,
paths []string, completed []bool, matches []uint64, results []Result,
) (int, bool) {
if level > 62 {
// The recursion level is limited because the matches []uint64
// array cannot handle more the 64-bits.
return i, false
}
// At this point the last character read was a '{'.
// Read all object keys and try to match against the paths.
var key string
var val string
var vesc, ok bool
next_key:
for ; i < len(json); i++ {
if json[i] == '"' {
// read the key
i, val, vesc, ok = parseString(json, i+1)
if !ok {
return i, false
}
if vesc {
// the value is escaped
key = unescape(val[1 : len(val)-1])
} else {
// just a plain old ascii key
key = val[1 : len(val)-1]
}
var hasMatch bool
var parsedVal bool
var valOrgIndex int
var valPathIndex int
for j := 0; j < len(key); j++ {
if key[j] == '.' {
// we need to look for keys with dot and ignore them.
if i, _, ok = parseAny(json, i, false); !ok {
return i, false
}
continue next_key
}
}
var usedPaths int
// loop through paths and look for matches
for j := 0; j < len(paths); j++ {
if completed[j] {
usedPaths++
// ignore completed paths
continue
}
if level > 0 && (matches[j]>>(level-1))&1 == 0 {
// ignore unmatched paths
usedPaths++
continue
}
// try to match the key to the path
// this is spaghetti code but the idea is to minimize
// calls and variable assignments when comparing the
// key to paths
if len(paths[j])-kplen >= len(key) {
i, k := kplen, 0
for ; k < len(key); k, i = k+1, i+1 {
if key[k] != paths[j][i] {
// no match
goto nomatch
}
}
if i < len(paths[j]) {
if paths[j][i] == '.' {
// matched, but there are still more keys in path
goto match_not_atend
}
}
if len(paths[j]) <= len(key) || kplen != 0 {
// matched and at the end of the path
goto match_atend
}
}
// no match, jump to the nomatch label
goto nomatch
match_atend:
// found a match
// at the end of the path. we must take the value.
usedPaths++
if !parsedVal {
// the value has not been parsed yet. let's do so.
valOrgIndex = i // keep track of the current position.
i, results[j], ok = parseAny(json, i, true)
if !ok {
return i, false
}
parsedVal = true
valPathIndex = j
} else {
results[j] = results[valPathIndex]
}
// mark as complete
completed[j] = true
// jump over the match_not_atend label
goto nomatch
match_not_atend:
// found a match
// still in the middle of the path.
usedPaths++
// mark the path as matched
matches[j] |= 1 << level
if !hasMatch {
hasMatch = true
}
nomatch: // noop label
}
if !parsedVal {
if hasMatch {
// we found a match and the value has not been parsed yet.
// let's find out if the next value type is an object.
for ; i < len(json); i++ {
if json[i] <= ' ' || json[i] == ':' {
continue
}
break
}
if i < len(json) {
if json[i] == '{' {
// it's an object. let's go deeper
i, ok = parseGetMany(json, i+1, level+1, kplen+len(key)+1, paths, completed, matches, results)
if !ok {
return i, false
}
} else {
// not an object. just parse and ignore.
if i, _, ok = parseAny(json, i, false); !ok {
return i, false
}
}
}
} else {
// Since there was no matches we can just parse the value and
// ignore the result.
if i, _, ok = parseAny(json, i, false); !ok {
return i, false
}
}
} else if hasMatch && len(results[valPathIndex].Raw) > 0 && results[valPathIndex].Raw[0] == '{' {
// The value was already parsed and the value type is an object.
// Rewind the json index and let's parse deeper.
i = valOrgIndex
for ; i < len(json); i++ {
if json[i] == '{' {
break
}
}
i, ok = parseGetMany(json, i+1, level+1, kplen+len(key)+1, paths, completed, matches, results)
if !ok {
return i, false
}
}
if usedPaths == len(paths) {
// all paths have been used, either completed or matched.
// we should stop parsing this object to save CPU cycles.
if level > 0 && i < len(json) {
i, _ = parseSquash(json, i)
}
return i, true
}
} else if json[i] == '}' {
// reached the end of the object. end it here.
return i + 1, true
}
}
return i, true
}
// Call table for GetMany. Using an isolated function allows for allocating
// arrays with know capacities on the stack, as opposed to dynamically
// allocating on the heap. This can provide a tremendous performance boost
// by avoiding the GC.
func getMany8(json string, i int, paths []string) ([]Result, bool) {
const max = 8
var completed = make([]bool, 0, max)
var matches = make([]uint64, 0, max)
var results = make([]Result, 0, max)
completed = completed[0:len(paths):max]
matches = matches[0:len(paths):max]
results = results[0:len(paths):max]
_, ok := parseGetMany(json, i, 0, 0, paths, completed, matches, results)
return results, ok
}
func getMany16(json string, i int, paths []string) ([]Result, bool) {
const max = 16
var completed = make([]bool, 0, max)
var matches = make([]uint64, 0, max)
var results = make([]Result, 0, max)
completed = completed[0:len(paths):max]
matches = matches[0:len(paths):max]
results = results[0:len(paths):max]
_, ok := parseGetMany(json, i, 0, 0, paths, completed, matches, results)
return results, ok
}
func getMany32(json string, i int, paths []string) ([]Result, bool) {
const max = 32
var completed = make([]bool, 0, max)
var matches = make([]uint64, 0, max)
var results = make([]Result, 0, max)
completed = completed[0:len(paths):max]
matches = matches[0:len(paths):max]
results = results[0:len(paths):max]
_, ok := parseGetMany(json, i, 0, 0, paths, completed, matches, results)
return results, ok
}
func getMany64(json string, i int, paths []string) ([]Result, bool) {
const max = 64
var completed = make([]bool, 0, max)
var matches = make([]uint64, 0, max)
var results = make([]Result, 0, max)
completed = completed[0:len(paths):max]
matches = matches[0:len(paths):max]
results = results[0:len(paths):max]
_, ok := parseGetMany(json, i, 0, 0, paths, completed, matches, results)
return results, ok
}
func getMany128(json string, i int, paths []string) ([]Result, bool) {
const max = 128
var completed = make([]bool, 0, max)
var matches = make([]uint64, 0, max)
var results = make([]Result, 0, max)
completed = completed[0:len(paths):max]
matches = matches[0:len(paths):max]
results = results[0:len(paths):max]
_, ok := parseGetMany(json, i, 0, 0, paths, completed, matches, results)
return results, ok
}
func getMany256(json string, i int, paths []string) ([]Result, bool) {
const max = 256
var completed = make([]bool, 0, max)
var matches = make([]uint64, 0, max)
var results = make([]Result, 0, max)
completed = completed[0:len(paths):max]
matches = matches[0:len(paths):max]
results = results[0:len(paths):max]
_, ok := parseGetMany(json, i, 0, 0, paths, completed, matches, results)
return results, ok
}
func getMany512(json string, i int, paths []string) ([]Result, bool) {
const max = 512
var completed = make([]bool, 0, max)
var matches = make([]uint64, 0, max)
var results = make([]Result, 0, max)
completed = completed[0:len(paths):max]
matches = matches[0:len(paths):max]
results = results[0:len(paths):max]
_, ok := parseGetMany(json, i, 0, 0, paths, completed, matches, results)
return results, ok
// GetManyBytes searches json for the multiple paths.
// The return value is a Result array where the number of items
// will be equal to the number of input paths.
func GetManyBytes(json []byte, path ...string) []Result {
return GetMany(string(json), path...)
}
var fieldsmu sync.RWMutex
@@ -2094,6 +1758,8 @@ var validate uintptr = 1
// UnmarshalValidationEnabled provides the option to disable JSON validation
// during the Unmarshal routine. Validation is enabled by default.
//
// Deprecated: Use encoder/json.Unmarshal instead
func UnmarshalValidationEnabled(enabled bool) {
if enabled {
atomic.StoreUintptr(&validate, 1)
@@ -2108,6 +1774,8 @@ func UnmarshalValidationEnabled(enabled bool) {
// gjson.Unmarshal will automatically attempt to convert JSON values to any Go
// type. For example, the JSON string "100" or the JSON number 100 can be equally
// assigned to Go string, int, byte, uint64, etc. This rule applies to all types.
//
// Deprecated: Use encoder/json.Unmarshal instead
func Unmarshal(data []byte, v interface{}) error {
if atomic.LoadUintptr(&validate) == 1 {
_, ok := validpayload(data, 0)
@@ -2377,6 +2045,12 @@ func validnull(data []byte, i int) (outi int, ok bool) {
}
// Valid returns true if the input is valid json.
//
// if !gjson.Valid(json) {
// return errors.New("invalid json")
// }
// value := gjson.Get(json, "name.last")
//
func Valid(json string) bool {
_, ok := validpayload([]byte(json), 0)
return ok

View File

@@ -7,6 +7,7 @@ import (
"fmt"
"math/rand"
"reflect"
"strconv"
"strings"
"testing"
"time"
@@ -147,7 +148,7 @@ func TestTimeResult(t *testing.T) {
func TestParseAny(t *testing.T) {
assert(t, Parse("100").Float() == 100)
assert(t, Parse("true").Bool())
assert(t, Parse("valse").Bool() == false)
assert(t, Parse("false").Bool() == false)
}
func TestManyVariousPathCounts(t *testing.T) {
@@ -1071,3 +1072,283 @@ func TestValidRandom(t *testing.T) {
validpayload(b[:n], 0)
}
}
func TestGetMany47(t *testing.T) {
json := `{"bar": {"id": 99, "mybar": "my mybar" }, "foo": {"myfoo": [605]}}`
paths := []string{"foo.myfoo", "bar.id", "bar.mybar", "bar.mybarx"}
expected := []string{"[605]", "99", "my mybar", ""}
results := GetMany(json, paths...)
if len(expected) != len(results) {
t.Fatalf("expected %v, got %v", len(expected), len(results))
}
for i, path := range paths {
if results[i].String() != expected[i] {
t.Fatalf("expected '%v', got '%v' for path '%v'", expected[i], results[i].String(), path)
}
}
}
func TestGetMany48(t *testing.T) {
json := `{"bar": {"id": 99, "xyz": "my xyz"}, "foo": {"myfoo": [605]}}`
paths := []string{"foo.myfoo", "bar.id", "bar.xyz", "bar.abc"}
expected := []string{"[605]", "99", "my xyz", ""}
results := GetMany(json, paths...)
if len(expected) != len(results) {
t.Fatalf("expected %v, got %v", len(expected), len(results))
}
for i, path := range paths {
if results[i].String() != expected[i] {
t.Fatalf("expected '%v', got '%v' for path '%v'", expected[i], results[i].String(), path)
}
}
}
func TestResultRawForLiteral(t *testing.T) {
for _, lit := range []string{"null", "true", "false"} {
result := Parse(lit)
if result.Raw != lit {
t.Fatalf("expected '%v', got '%v'", lit, result.Raw)
}
}
}
func TestNullArray(t *testing.T) {
n := len(Get(`{"data":null}`, "data").Array())
if n != 0 {
t.Fatalf("expected '%v', got '%v'", 0, n)
}
n = len(Get(`{}`, "data").Array())
if n != 0 {
t.Fatalf("expected '%v', got '%v'", 0, n)
}
n = len(Get(`{"data":[]}`, "data").Array())
if n != 0 {
t.Fatalf("expected '%v', got '%v'", 0, n)
}
n = len(Get(`{"data":[null]}`, "data").Array())
if n != 1 {
t.Fatalf("expected '%v', got '%v'", 1, n)
}
}
func TestRandomGetMany(t *testing.T) {
start := time.Now()
for time.Since(start) < time.Second*3 {
testRandomGetMany(t)
}
}
func testRandomGetMany(t *testing.T) {
rand.Seed(time.Now().UnixNano())
json, keys := randomJSON()
for _, key := range keys {
r := Get(json, key)
if !r.Exists() {
t.Fatal("should exist")
}
}
rkeysi := rand.Perm(len(keys))
rkeysn := 1 + rand.Int()%32
if len(rkeysi) > rkeysn {
rkeysi = rkeysi[:rkeysn]
}
var rkeys []string
for i := 0; i < len(rkeysi); i++ {
rkeys = append(rkeys, keys[rkeysi[i]])
}
mres1 := GetMany(json, rkeys...)
var mres2 []Result
for _, rkey := range rkeys {
mres2 = append(mres2, Get(json, rkey))
}
if len(mres1) != len(mres2) {
t.Fatalf("expected %d, got %d", len(mres2), len(mres1))
}
for i := 0; i < len(mres1); i++ {
mres1[i].Index = 0
mres2[i].Index = 0
v1 := fmt.Sprintf("%#v", mres1[i])
v2 := fmt.Sprintf("%#v", mres2[i])
if v1 != v2 {
t.Fatalf("\nexpected %s\n"+
" got %s", v2, v1)
}
}
}
func TestIssue54(t *testing.T) {
var r []Result
json := `{"MarketName":null,"Nounce":6115}`
r = GetMany(json, "Nounce", "Buys", "Sells", "Fills")
if strings.Replace(fmt.Sprintf("%v", r), " ", "", -1) != "[6115]" {
t.Fatalf("expected '%v', got '%v'", "[6115]", strings.Replace(fmt.Sprintf("%v", r), " ", "", -1))
}
r = GetMany(json, "Nounce", "Buys", "Sells")
if strings.Replace(fmt.Sprintf("%v", r), " ", "", -1) != "[6115]" {
t.Fatalf("expected '%v', got '%v'", "[6115]", strings.Replace(fmt.Sprintf("%v", r), " ", "", -1))
}
r = GetMany(json, "Nounce")
if strings.Replace(fmt.Sprintf("%v", r), " ", "", -1) != "[6115]" {
t.Fatalf("expected '%v', got '%v'", "[6115]", strings.Replace(fmt.Sprintf("%v", r), " ", "", -1))
}
}
func randomString() string {
var key string
N := 1 + rand.Int()%16
for i := 0; i < N; i++ {
r := rand.Int() % 62
if r < 10 {
key += string(byte('0' + r))
} else if r-10 < 26 {
key += string(byte('a' + r - 10))
} else {
key += string(byte('A' + r - 10 - 26))
}
}
return `"` + key + `"`
}
func randomBool() string {
switch rand.Int() % 2 {
default:
return "false"
case 1:
return "true"
}
}
func randomNumber() string {
return strconv.FormatInt(int64(rand.Int()%1000000), 10)
}
func randomObjectOrArray(keys []string, prefix string, array bool, depth int) (string, []string) {
N := 5 + rand.Int()%5
var json string
if array {
json = "["
} else {
json = "{"
}
for i := 0; i < N; i++ {
if i > 0 {
json += ","
}
var pkey string
if array {
pkey = prefix + "." + strconv.FormatInt(int64(i), 10)
} else {
key := randomString()
pkey = prefix + "." + key[1:len(key)-1]
json += key + `:`
}
keys = append(keys, pkey[1:])
var kind int
if depth == 5 {
kind = rand.Int() % 4
} else {
kind = rand.Int() % 6
}
switch kind {
case 0:
json += randomString()
case 1:
json += randomBool()
case 2:
json += "null"
case 3:
json += randomNumber()
case 4:
var njson string
njson, keys = randomObjectOrArray(keys, pkey, true, depth+1)
json += njson
case 5:
var njson string
njson, keys = randomObjectOrArray(keys, pkey, false, depth+1)
json += njson
}
}
if array {
json += "]"
} else {
json += "}"
}
return json, keys
}
func randomJSON() (json string, keys []string) {
return randomObjectOrArray(nil, "", false, 0)
}
func TestIssue55(t *testing.T) {
json := `{"one": {"two": 2, "three": 3}, "four": 4, "five": 5}`
results := GetMany(json, "four", "five", "one.two", "one.six")
expected := []string{"4", "5", "2", ""}
for i, r := range results {
if r.String() != expected[i] {
t.Fatalf("expected %v, got %v", expected[i], r.String())
}
}
}
func TestIssue58(t *testing.T) {
json := `{"data":[{"uid": 1},{"uid": 2}]}`
res := Get(json, `data.#[uid!=1]`).Raw
if res != `{"uid": 2}` {
t.Fatalf("expected '%v', got '%v'", `{"uid": 1}`, res)
}
}
func TestObjectGrouping(t *testing.T) {
json := `
[
true,
{"name":"tom"},
false,
{"name":"janet"},
null
]
`
res := Get(json, "#.name")
if res.String() != `["tom","janet"]` {
t.Fatalf("expected '%v', got '%v'", `["tom","janet"]`, res.String())
}
}
func TestJSONLines(t *testing.T) {
json := `
true
false
{"name":"tom"}
[1,2,3,4,5]
{"name":"janet"}
null
12930.1203
`
paths := []string{"..#", "..0", "..2.name", "..#.name", "..6", "..7"}
ress := []string{"7", "true", "tom", `["tom","janet"]`, "12930.1203", ""}
for i, path := range paths {
res := Get(json, path)
if res.String() != ress[i] {
t.Fatalf("expected '%v', got '%v'", ress[i], res.String())
}
}
json = `
{"name": "Gilbert", "wins": [["straight", "7♣"], ["one pair", "10♥"]]}
{"name": "Alexa", "wins": [["two pair", "4♠"], ["two pair", "9♠"]]}
{"name": "May", "wins": []}
{"name": "Deloise", "wins": [["three of a kind", "5♣"]]}
`
var i int
lines := strings.Split(strings.TrimSpace(json), "\n")
ForEachLine(json, func(line Result) bool {
if line.Raw != lines[i] {
t.Fatalf("expected '%v', got '%v'", lines[i], line.Raw)
}
i++
return true
})
if i != 4 {
t.Fatalf("expected '%v', got '%v'", 4, i)
}
}