15 JQ Command Examples

June 17, 2023 | by Arround The Web | No comments

JSON is a very useful data format that is used to transfer the data from one application to another application. The JSON data can be parsed in different ways. JQ is a command-line processor of Linux tools to transform the JSON data to another form very quickly. It is mainly used to filter the JSON data based on the input data. The JQ tool is not installed in Linux by default. So, you have to install it before using this command. The methods of using the JQ commands to read or search, and filter or convert the JSON data into another format are shown in the 15 JQ examples.

Prerequisites

You have to install the JQ tool and create a sample JSON file that will be used to check how the JQ command works.

A. Run the following command to install the JQ tool.

$ sudo apt install jq

B. Create a JSON file named department with the following content:

The “department.json” file contains a JSON array of three properties. These are “id”, “name”, and “semesters”. The “semesters” property contains another JSON array of four properties. These are “NO”, “Batch”, “Total_students”, and “Courses”. The “Courses” property contains another JSON array of three properties.

[
{
"id": "01",
"name": "CSE",
"Semesters":
[
{
"No": 1,
"Batch": 61,
"Total_students": 40,
"Courses": [
{"code": "CSE-101", "name": "Computer Fundamentals", "credit_hour": 2.0},
{"code": "MAT-101", "name": "Mathematics-1", "credit_hour": 2.0},
{"code": "ACC-101", "name": "Accounting", "credit_hour": 2.0}] },
{
"No": 2,
"Batch": 60,
"Total_students": 35,
"Courses": [
{"code": "CSE-201", "name": "Programming with C", "credit_hour": 3.0},
{"code": "MAT-201", "name": "Mathematics-2", "credit_hour": 2.0},
{"code": "PHY-201", "name": "Physics", "credit_hour": 2.0}] },
{
"No": 3,
"Batch": 59,
"Total_studeCnts": 37,
"Courses": [
{"code": "CSE-301", "name": "Data Structure", "credit_hour": 3.0},
{"code": "CSE-302", "name": "Object-oriented Programming", "credit_hour": 2.0},
{"code": "CSE-303", "name": null, "credit_hour": 2.0}] }
]

}
]

Syntax:

Different syntaxes of the “jq” command are shown in the following:

$ jq [options] [json_file] $ jq [options] --args [json_string]

The “jq” command can be used with the JSON data or with the content of the JSON file. Many types of operators are used with the “jq” command to filter the JSON data such as “.”, “.[]”, “|”, etc.

“JQ” Command Options

Some useful options of the “jq” command are mentioned in the following:

Option	Purpose
-c or –compact-output	The “jq” command prints the JSON output in the pretty print format by default. This option is used to print the JSON output by removing the unnecessary whitespace that minimizes the size of the JSON file.
-f filename or –from-file filename	Normally, the “jq” command is executed from the terminal. This option is used to read the “jq” command from the file.
-n or –null-input	The option is used to generate the JSON data without any input.
–raw-input or -R	It is used to take the input as a string instead of JSON.
–color-output or -C	It is used to generate the color output forcefully when the output is sent to a pipe or the file.
–ascii-output or -a	It is used to generate the ASCII output forcefully.
–sort-keys or -S	It is used to generate the sorted output based on the key.
–unbuffered	It is used to flush the output after printing the JSON object.
–version	It is used to print the “jq” version.

Basic Filters of “JQ”

The uses of some basic filters of the “jq” command are mentioned in the following:

Filter	Purpose
.	It is called the identity operator that is used to generate the unchanged output of the input.
.object	It is called the object identifier that is used to generate the value of the object.
.[]	It is called the array or object value iterator that returns all the elements of the array.
.[index]	It is called an array index that is used to read the key values based on the index.
.[5:10]	It is called the array slice or the string slice that is used to return a sub-array or the sub-string.

List of JQ Examples

Read the JSON File by Taking the JSON File as an Argument

The following “jq” command reads the full content of the “department.json” file and prints the content in the pretty print format that displays the content in a human-readable format. Here, “.” is used with the “jq” command.

$ jq . department.json

The last part of the output after executing the command is shown in the following image:

The same output can be generated for the “department.json” file using the following command. Here, “.[]” is used with the “jq” command:

$ jq .[] department.json

Read the JSON File by Taking the JSON Data from the “Cat” Command

The “cat” command can be used with the “jq” command with pipe(|) to read the content of the “department.json” file. The output of the following command is the same as the previous two commands:

$ cat department.json | jq .

Filter the JSON Data Based on a Single Key

The following command prints the value of the name key of the “department.json” file. The name key exists in two arrays of the JSON file. The first name key exists in the main JSON array and the second name key exists in the “Courses” array. The “.[].name” prints the value of the name key of the main array.

$ jq .[].name department.json

There is only one name key in the “department.json” file in the main array and the value of the key is CSE which is printed in the following output:

Filter the JSON Data Based on Multiple Keys

The JSON data can be filtered based on multiple key values where the key values are separated by a comma(,). The following command prints the values of the id and name keys. The “.[].id,.[].name” prints the value of the id and the name keys of the main array.

$ jq '.[].id,.[].name' department.json

The value of the id key is 01 and the name key is CSE in the main array of the “department.json” file that is printed in the output:

Filter the JSON Data Based on Nested Value

The semester array of the “department.json” file contains three-course information. Run the following “jq” command to read the key value of the third course of the first semester. The Semester [0] is used to define the first semester and the Courses [2] is used to define the third course because the array index starts from 0. The “.[].Semesters[0].Courses[2]” prints all key values of the nested array which is the last array element of the courses array of the first semesters array.

$ jq .[].Semesters[0].Courses[2] department.json

The following output appears based on the content of the “department.json” file after executing the command:

Filter the JSON Data Using “Select”

The data can be retrieved from the JSON file based on the particular condition using the select() function with the “jq” command. Run the following command to retrieve all course information from the “department.json” file where the value of the credit_hour key is more than 2.0. Two courses of 3.0 credit hours exist in the “department.json” file – these are “Programming with C” and the “Data Structure”. Here, the –c option of the “jq” command is used to generate the compact JSON output.

$ jq -c '.[].Semesters[].Courses[] | select(.credit_hour > 2.0)' department.json

The following output appears based on the content of the “department.json” file after executing the command. The courses of 3.0 credit hours are printed in the output:

Filter the JSON Data Based on NULL Value

The name key of the 3^rd course of the 3^rd semester is null in the “department.json” file. Run the following “jq” command to read the course information from the “department.json” file that contains a null value as the value of the name key of the course.

Only one course exists in the “department.json” file that contains the null value in the name key. The select() function is used in the command to filter the JSON data that contains the null value.

$ jq '.[].Semesters[].Courses[] | select(.name == null)' department.json

The following output appears based on the content of the “department.json” file after executing the command. The element of the “Courses” array that contains the null value is printed in the output:

Filter the Data from a JSON Array

The “jq” command can be applied to the array values to retrieve the particular values. The following command sends the array of four elements to the “jq” command using the “echo” command. The “jq” command prints the 2^nd and 3^rd values of the array using “.[1:3]”. Here, the starting position is counted from 0 and the length of the resulting array is counted from 1.

echo '["JSON","XML","SQL","MySQL"]' | jq '.[1:3]'

The following output appears based on the content of the array after executing the command. The second value of the array is “XML” and the third value of the array is “SQL”:

Sort the JSON Data Based on the Key

The sort_by() function is used to sort the output of the “jq” command based on the particular key of the JSON data. The following “jq” command selects those courses from the JSON key where the course name is not null and the credit hour is more than 2. The output is sorted based on the course name value. Two courses in the “department.json” file contain the not null value and the credit hour of those courses are more than 2.

$ jq '[.[].Semesters[].Courses[] | select(.name != null and .credit_hour > 2)] | sort_by(.name)' department.json

The following output appears based on the content of the “department.json” file after executing the command. The courses of 3.0 credit hour are printed in the output and the course names are sorted in ascending order:

Group the JSON Data Based on the Key

The group_by() function is used with the “jq” command to retrieve the data from the JSON content by grouping the data based on the JSON key. The following “jq” command retrieves all course information where the value of the course name is not null and the data is displayed based on the credit hour of the course:

$ jq '[.[].Semesters[].Courses[] | select(.name != null)] | group_by(.credit_hour)' department.json

After executing the command, the last part of the output looks like the following image. All courses of 2.0 credit hour are displayed in a group and all courses of 3.0 credit hour are displayed in another group:

Perform Arithmetic Operations on the JSON Data

Each course of each semester has a particular credit hour. The credit hour information of 9 courses of 3 semesters are stored in the “department.json” file. The following “jq” command counts the total credit hours of three courses of the first semester where the credit hour of each course is 2.0:

jq '.[].Semesters[0].Courses[0].credit_hour + .[].Semesters[0].Courses[1].credit_hour + .[].Semesters[0].Courses[2].credit_hour' department.json

The following output appears based on the content of the “department.json” file after executing the command. The sum of the credit hour of three courses of the first semester is 2.0+2.0+2.0. So, the total credit hour of the first semester is 6.0 which is printed in the output:

Map the Values of the JSON Array

The particular key-value pair of the JSON data can be mapped using the “jq” command. In the following command, the id and the name property values are mapped with the dept_id and dept_name keys in the following “jq” command:

jq '.[] | {dept_id: .id, dept_name: .name}' department.json

The following output appears after mapping the key with the content of the “department.json” file:

Add Value with Each Value of the JSON Array

The following “jq” command adds 5 with each value of the array. The “echo” command is used to pass the array to the “jq” command using pipe(|):

echo '[35,39,36,38,31]' | jq 'map(.+5)'

The following output appears after executing the command. The array values were 35, 39, 36, 38, and 31. The output returns the array of 40, 44, 41, 43, and 36 after adding 5:

Limit the Size of the JSON Data

The semester array of the “department.json” file contains four properties. These are “No.”, “Batch”, “Total_students”, and “Courses”. The limit() function can be used with the “jq” command to display the JSON content by limiting the size of the properties. In the following command, the first three properties of the semester array are retrieved by setting the first argument value of the limit() function to 3:

$ jq '.[].Semesters[] | limit(3;.[])' department.json

The following output appears based on the content of the “department.json” file after executing the command. The “semester no.”, “batch no.”, and “total number of students” of each semester are printed in the output. The batch no. of the first semester is 61 and the total number of students is 40. The batch no. of the second semester is 60 and the total number of students is 35. The batch no. of the third semester is 59 and the total number of students is 37. Here, only the property values are printed in the output:

Remove the Keys from the JSON Data

The “jq” command can be used to remove the particular content from the JSON file. The “department.json” file contains three semester information and each semester contains three course information. The following “jq” command removes the 3^rd course information from all semesters from the output using the del() function. This function does not remove the data permanently from the JSON file. It displays the content of the JSON file by omitting the 3^rd course information from each semester.

jq 'del(.[].Semesters[].Courses[2])' department.json

After executing the command, the last part of the output looks like the following image. The third course information of the semester is not printed in the output:

Conclusion

Some basic uses of the “jq” command are shown in this tutorial using the 15 simple “jq” examples and a JSON file. The purpose of using the “jq” command will be cleared for Linux users after reading this tutorial.

Source: linuxhint.com