Linux之jq

jq

jq官方文档

简介

jq可以对json数据进行分片、过滤、映射和转换，和sed、awk、grep等命令一样，都可以让你轻松地把玩文本。它能轻松地把你拥有的数据转换成你期望的格式，而且需要写的程序通常也比你期望的更加简短。

jq是用C编写，没有运行时依赖，所以几乎可以运行在任何系统上。预编译的二进制文件可以直接在Linux、OS X和windows系统上运行，当然在linux和OS X系统你需要赋与其可执行权限；在linux系统中也可以直接用yum安装。

安装

Mac

1
brew install jq

Linux

1
2
3
yum install jq

apt-get install jq

查看命令使用

1
man jq

json文档

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
{
	"store": {
		"book": [{
			"category": "reference",
			"author": "Nigel Rees",
			"title": "Sayings of the Century",
			"price": 8.95
		}, {
			"category": "fiction",
			"author": "Evelyn Waugh",
			"title": "Sword of Honour",
			"price": 12.99
		}, {
			"category": "fiction",
			"author": "Herman Melville",
			"title": "Moby Dick",
			"isbn": "0-553-21311-3",
			"price": 8.99
		}, {
			"category": "fiction",
			"author": "J. R. R. Tolkien",
			"title": "The Lord of the Rings",
			"isbn": "0-395-19395-8",
			"price": 22.99
		}],
		"bicycle": {
			"color": "red",
			"price": 19.95
		}
	}
}

使用

  • 使用jq将文件内容格式化为json

    1
    cat book.json | jq

  • 基本过滤 .

    • 最简单的过滤，.这是一个过滤器，它接受输入，并将其不变地生成为输出。
    1
    cat book.json | jq '.'

  • 获取book下所有的书籍

    1
    cat book.json | jq .store.book

  • 对象过滤 .key、.key1.key2、.["key"]

    • 可以通过.key或.key1.key2的方式来获取对应的value，这样的方式是不是有点像jsonpath?
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    jingdeMacBook-Pro:~ apple$ cat book.json | jq '.store.book'
    [
      {
        "category": "reference",
        "author": "Nigel Rees",
        "title": "Sayings of the Century",
        "price": 8.95
      },
      {
        "category": "fiction",
        "author": "Evelyn Waugh",
        "title": "Sword of Honour",
        "price": 12.99
      },
      {
        "category": "fiction",
        "author": "Herman Melville",
        "title": "Moby Dick",
        "isbn": "0-553-21311-3",
        "price": 8.99
      },
      {
        "category": "fiction",
        "author": "J. R. R. Tolkien",
        "title": "The Lord of the Rings",
        "isbn": "0-395-19395-8",
        "price": 22.99
      }
    ]

  • 索引过滤 .key[index]、.key[ startIndex: endIndex]

    • 其实就是在对象过滤的基础上加个索引而已，学过编程的都懂的~ 左闭右开，当然是针对数组的操纵，这里就以一种对象过滤写法基础上举例子，
    • .key[index]
    1
    2
    3
    4
    5
    6
    7
    jingdeMacBook-Pro:~ apple$ cat book.json | jq '.store.book[0]'
    {
      "category": "reference",
      "author": "Nigel Rees",
      "title": "Sayings of the Century",
      "price": 8.95
    }
    • .key[ startIndex: endIndex]
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    jingdeMacBook-Pro:~ apple$ cat book.json | jq '.store.book[0: 2]'
    [
      {
        "category": "reference",
        "author": "Nigel Rees",
        "title": "Sayings of the Century",
        "price": 8.95
      },
      {
        "category": "fiction",
        "author": "Evelyn Waugh",
        "title": "Sword of Honour",
        "price": 12.99
      }
    ]
    • .key[-index]
    1
    2
    3
    4
    5
    6
    7
    8
    jingdeMacBook-Pro:~ apple$ cat book.json | jq .store.book[-1]
    {
      "category": "fiction",
      "author": "J. R. R. Tolkien",
      "title": "The Lord of the Rings",
      "isbn": "0-395-19395-8",
      "price": 22.99
    }

  • 对象构建 {} 、[]

    • 可以利用{} 、[]构建新的对象(数组)
    • 修改key的值
    1
    2
    3
    4
    jingdeMacBook-Pro:~ apple$ cat book.json | jq '{ Java: .store.book[0].title}'
    {
      "Java": "Sayings of the Century"
    }
    • 如果其中一个表达式有多个结果，那么输出也会产生多个结果
    1
    2
    3
    4
    5
    6
    7
    8
    9
    jingdeMacBook-Pro:~ apple$ echo '{"user":"Jack", "titles":["Java", "Python"]}' | jq '{user, title:.titles[]}'
    {
      "user": "Jack",
      "title": "Java"
    }
    {
      "user": "Jack",
      "title": "Python"
    }
    • 如果key是不存在的，value将会赋值为null
    1
    2
    3
    4
    5
    6
    7
    8
    9
    jingdeMacBook-Pro:~ apple$ echo '{"user":"Jack", "titles":["Java", "Python"]}' | jq '{user1, title:.titles[]}'
    {
      "user1": null,
      "title": "Java"
    }
    {
      "user1": null,
      "title": "Python"
    }
    • 如果再键的周围加括号就意味着它将作为表达式进行计算
    1
    2
    3
    4
    5
    6
    7
    jingdeMacBook-Pro:~ apple$ echo '{"user":"Jack", "titles":["Java", "Python"]}' | jq '{(.user):.titles[]}'
    {
      "Jack": "Java"
    }
    {
      "Jack": "Python"
    }

  • 计算

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    jingdeMacBook-Pro:~ apple$ echo "10" | jq '(.+2)*5'
    60
    jingdeMacBook-Pro:~ apple$ echo null | jq '{a: 1} + {b: 2} + {c: 3} + {a: 42}'
    {
      "a": 42,
      "b": 2,
      "c": 3
    }
    jingdeMacBook-Pro:~ apple$ echo '["xml", "yaml", "json"]' | jq '. - ["xml"]'
    [
      "yaml",
      "json"
    ]
    jingdeMacBook-Pro:~ apple$ echo '{"a":5}' | jq '.a+=10'
    {
      "a": 15
    }

  • 获取长度

    1
    2
    3
    4
    5
    jingdeMacBook-Pro:~ apple$ echo '[[1,2], "string", {"a":2}, null]' | jq '.[] | length'
    2
    6
    1
    0

  • 数组运算

    • 并集('+')：有数组a,b; a+b就会得到一个包含所有a,b数组元素的新数组：
    1
    2
    3
    4
    5
    6
    7
    8
    9
    jingdeMacBook-Pro:~ apple$ echo '{"a":[1,2,3],"b":[3,4,5]}' | jq '.a+.b'
    [
      1,
      2,
      3,
      3,
      4,
      5
    ]

  • 差集('-')：有数组a,b; a-b就会得到一个只包含a中元素，不包含b中元素的新数组：

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    jingdeMacBook-Pro:~ apple$ echo '{"a":[1,2,3],"b":[3,4,5]}' | jq '.a-.b'
    [
      1,
      2
    ]
    jingdeMacBook-Pro:~ apple$ echo '{"a":[1,2,3],"b":[3,4,5]}' | jq '.b-.a'
    [
      4,
      5
    ]