I’m excited to announce the release of jc
version 1.20.0 available on github and pypi. jc
now supports over 100 standard and streaming parsers. Thank you to the Open Source community for making this possible!
jc
can be installed via pip
or through several official OS package repositories, including Debian, Ubuntu, Fedora, openSUSE, Arch Linux, NixOS Linux, Guix System Linux, FreeBSD, and macOS. For more information on how to get jc
, see the project README.
To upgrade with pip
:
$ pip3 install --upgrade jc
What’s New
- Add YAML output option with the
-y
option - Add
top -b
standard and streaming parsers tested on linux - Add
plugin_parser_count
,standard_parser_count
, andstreaming_parser_count
keys tojc -a
output - Add
is_compatible
function to theutils
module - Fix
pip-show
parser for packages with a multi-line license field - Fix ASCII Table parser for cases where centered headers cause mis-aligned fields
New Parsers
top -b
command parser
Support for the top -b
command. (Documentation)
$ top -b -n 3 | jc --top -p [ { "time": "11:20:43", "uptime": 118, "users": 2, "load_1m": 0.0, "load_5m": 0.01, "load_15m": 0.05, "tasks_total": 108, "tasks_running": 2, "tasks_sleeping": 106, "tasks_stopped": 0, "tasks_zombie": 0, "cpu_user": 5.6, "cpu_sys": 11.1, "cpu_nice": 0.0, "cpu_idle": 83.3, "cpu_wait": 0.0, "cpu_hardware": 0.0, "cpu_software": 0.0, "cpu_steal": 0.0, "mem_total": 3.7, "mem_free": 3.3, "mem_used": 0.2, "mem_buff_cache": 0.2, "swap_total": 2.0, "swap_free": 2.0, "swap_used": 0.0, "mem_available": 3.3, "processes": [ { "pid": 2225, "user": "kbrazil", "priority": 20, "nice": 0, "virtual_mem": 158.1, "resident_mem": 2.2, "shared_mem": 1.6, "status": "running", "percent_cpu": 12.5, "percent_mem": 0.1, "time_hundredths": "0:00.02", "command": "top", "parent_pid": 1884, "uid": 1000, "real_uid": 1000, "real_user": "kbrazil", "saved_uid": 1000, "saved_user": "kbrazil", "gid": 1000, "group": "kbrazil", "pgrp": 2225, "tty": "pts/0", "tty_process_gid": 2225, "session_id": 1884, "thread_count": 1, "last_used_processor": 0, "time": "0:00", "swap": 0.0, "code": 0.1, "data": 1.0, "major_page_fault_count": 0, "minor_page_fault_count": 736, "dirty_pages_count": 0, "sleeping_in_function": null, "flags": "..4.2...", "cgroups": "1:name=systemd:/user.slice/user-1000.+", "supplementary_gids": [ 10, 1000 ], "supplementary_groups": [ "wheel", "kbrazil" ], "thread_gid": 2225, "environment_variables": [ "XDG_SESSION_ID=2", "HOSTNAME=localhost" ], "major_page_fault_count_delta": 0, "minor_page_fault_count_delta": 4, "used": 2.2, "ipc_namespace_inode": 4026531839, "mount_namespace_inode": 4026531840, "net_namespace_inode": 4026531956, "pid_namespace_inode": 4026531836, "user_namespace_inode": 4026531837, "nts_namespace_inode": 4026531838 }, ... ] } ]
top -b
command streaming parser
Support for the top -b
command. This is a streaming parser and it outputs JSON Lines. (Documentation):
$ top -b | jc --top-s {"time":"11:24:50","uptime":2,"users":2,"load_1m":0.23,"load_5m":...} ...
v1.20.1 Updates
- Add
postconf -M
parser tested on linux - Update
asciitable
andasciitable-m
parsers to preserve case in key names when using the-r
orraw=True
options. - Add long options (e.g.
--help
,--about
,--pretty
, etc.) - Add shell completions for Bash and Zsh
- Fix
id
parser for cases where the user or group name is not present
postconf -M
command parser
Linux support for the postconf -m
command. (Documentation):
$ postconf -M | jc --postconf -p # or jc -p postconf -M [ { "service_name": "smtp", "service_type": "inet", "private": false, "unprivileged": null, "chroot": true, "wake_up_time": null, "process_limit": null, "command": "smtpd", "no_wake_up_before_first_use": null }, { "service_name": "pickup", "service_type": "unix", "private": false, "unprivileged": null, "chroot": true, "wake_up_time": 60, "process_limit": 1, "command": "pickup", "no_wake_up_before_first_use": false } ]
Long Options
jc
now supports long CLI options:
Options: -a, --about about jc -C, --force-color force color output even when using pipes (overrides -m) -d, --debug debug (double for verbose debug) -h, --help help (--help --parser_name for parser documentation) -m, --monochrome monochrome output -p, --pretty pretty print output -q, --quiet suppress warnings (double to ignore streaming errors) -r, --raw raw output -u, --unbuffer unbuffer output -v, --version version info -y, --yaml-out YAML output -B, --bash-comp gen Bash completion: jc -B > /etc/bash_completion.d/jc -Z, --zsh-comp gen Zsh completion: jc -Z > "${fpath[1]}/_jc"
Shell Completions
Bash and Zsh completions are now available for jc
! If your system is already set up for completions you can run the following to enable completions:
Bash
Linux
$ jc -B > /etc/bash_completion.d/jc
macOS
$ jc -B > /usr/local/etc/bash_completion.d/jc
Zsh
Linux and macOS
$ jc -Z > "${fpath[1]}/_jc"
v1.20.2 Updates
- Add
gpg --with-colons
command parser tested on linux - Add DER and PEM encoded X.509 Certificate parser
- Add Bash and Zsh completion scripts to DEB and RPM packages
gpg –with-colons command parser
Linux support for the gpg --with-colons
command. (Documentation):
$ gpg --with-colons --show-keys file.gpg | jc --gpg -p [ { "type": "pub", "validity": "f", "key_length": "1024", "pub_key_alg": "17", "key_id": "6C7EE1B8621CC013", "creation_date": "899817715", "expiration_date": "1055898235", "certsn_uidhash_trustinfo": null, "owner_trust": "m", "user_id": null, "signature_class": null, "key_capabilities": "scESC", ...
X.509 DER/PEM Certificate Files
Support for DER and PEM encoded certificate files (Documentation):
$ cat alice.crt | jc --x509-cert -p [ { "tbs_certificate": { "version": "v3", "serial_number": "01", "signature": { "algorithm": "sha1_rsa", "parameters": null }, "issuer": { "country_name": "FR", "state_or_province_name": "Alsace", "locality_name": "Strasbourg", "organization_name": "www.freelan.org", "organizational_unit_name": "freelan", "common_name": "Freelan Sample Certificate Authority", "email_address": "contact@freelan.org" }, "validity": { "not_before": 1335522678, "not_after": 1650882678, "not_before_iso": "2012-04-27T10:31:18+00:00", "not_after_iso": "2022-04-25T10:31:18+00:00" }, "subject": { "country_name": "FR", "state_or_province_name": "Alsace", "organization_name": "www.freelan.org", "organizational_unit_name": "freelan", "common_name": "alice", "email_address": "contact@freelan.org" }, "subject_public_key_info": { "algorithm": { "algorithm": "rsa", "parameters": null }, "public_key": { "modulus": "dd:6d:bd:f8:80:fa:d7:de:1b:1f:a7:a3:2e:b2:02:e2:16:f6:52:0a:3c:bf:a6:42:f8:ca:dc:93:67:4d:60:c3:4f:8d:c3:8a:00:1b:f1:c4:4b:41:6a:69:d2:69:e5:3f:21:8e:c5:0b:f8:22:37:ad:b6:2c:4b:55:ff:7a:03:72:bb:9a:d3:ec:96:b9:56:9f:cb:19:99:c9:32:94:6f:8f:c6:52:06:9f:45:03:df:fd:e8:97:f6:ea:d6:ba:bb:48:2b:b5:e0:34:61:4d:52:36:0f:ab:87:52:25:03:cf:87:00:87:13:f2:ca:03:29:16:9d:90:57:46:b5:f4:0e:ae:17:c8:0a:4d:92:ed:08:a6:32:23:11:71:fe:f2:2c:44:d7:6c:07:f3:0b:7b:0c:4b:dd:3b:b4:f7:37:70:9f:51:b6:88:4e:5d:6a:05:7f:8d:9b:66:7a:ab:80:20:fe:ee:6b:97:c3:49:7d:78:3b:d5:99:97:03:75:ce:8f:bc:c5:be:9c:9a:a5:12:19:70:f9:a4:bd:96:27:ed:23:02:a7:c7:57:c9:71:cf:76:94:a2:21:62:f6:b8:1d:ca:88:ee:09:ad:46:2f:b7:61:b3:2c:15:13:86:9f:a5:35:26:5a:67:f4:37:c8:e6:80:01:49:0e:c7:ed:61:d3:cd:bc:e4:f8:be:3f:c9:4e:f8:7d:97:89:ce:12:bc:ca:b5:c6:d2:e0:d9:b3:68:3c:2e:4a:9d:b4:5f:b8:53:ee:50:3d:bf:dd:d4:a2:8a:b6:a0:27:ab:98:0c:b3:b2:58:90:e2:bc:a1:ad:ff:bd:8e:55:31:0f:00:bf:68:e9:3d:a9:19:9a:f0:6d:0b:a2:14:6a:c6:4c:c6:4e:bd:63:12:a5:0b:4d:97:eb:42:09:79:53:e2:65:aa:24:34:70:b8:c1:ab:23:80:e7:9c:6c:ed:dc:82:aa:37:04:b8:43:2a:3d:2a:a8:cc:20:fc:27:5d:90:26:58:f9:b7:14:e2:9e:e2:c1:70:73:97:e9:6b:02:8e:d3:52:59:7b:00:ec:61:30:f1:56:3f:9c:c1:7c:05:c5:b1:36:c8:18:85:cf:61:40:1f:07:e8:a7:06:87:df:9a:77:0b:a9:64:72:03:f6:93:fc:e0:02:59:c1:96:ec:c0:09:42:3e:30:a2:7f:1b:48:2f:fe:e0:21:8f:53:87:25:0d:cb:ea:49:f5:4a:9b:d0:e3:5f:ee:78:18:e5:ba:71:31:a9:04:98:0f:b1:ad:67:52:a0:f2:e3:9c:ab:6a:fe:58:84:84:dd:07:3d:32:94:05:16:45:15:96:59:a0:58:6c:18:0e:e3:77:66:c7:b3:f7:99", "public_exponent": 65537 } }, "issuer_unique_id": null, "subject_unique_id": null, "extensions": [ { "extn_id": "basic_constraints", "critical": false, "extn_value": { "ca": false, "path_len_constraint": null } }, { "extn_id": "2.16.840.1.113730.1.13", "critical": false, "extn_value": "16:1d:4f:70:65:6e:53:53:4c:20:47:65:6e:65:72:61:74:65:64:20:43:65:72:74:69:66:69:63:61:74:65" }, { "extn_id": "key_identifier", "critical": false, "extn_value": "59:5f:c9:13:ba:1b:cc:b9:a8:41:4a:8a:49:79:6a:36:f6:7d:3e:d7" }, { "extn_id": "authority_key_identifier", "critical": false, "extn_value": { "key_identifier": "23:6c:2d:3d:3e:29:5d:78:b8:6c:3e:aa:e2:bb:2e:1e:6c:87:f2:53", "authority_cert_issuer": null, "authority_cert_serial_number": null } } ] }, "signature_algorithm": { "algorithm": "sha1_rsa", "parameters": null }, "signature_value": "13:e7:02:45:3e:a7:ab:bd:b8:da:e7:ef:74:88:ac:62:d5:dd:10:56:d5:46:07:ec:fa:6a:80:0c:b9:62:be:aa:08:b4:be:0b:eb:9a:ef:68:b7:69:6f:4d:20:92:9d:18:63:7a:23:f4:48:87:6a:14:c3:91:98:1b:4e:08:59:3f:91:80:e9:f4:cf:fd:d5:bf:af:4b:e4:bd:78:09:71:ac:d0:81:e5:53:9f:3e:ac:44:3e:9f:f0:bf:5a:c1:70:4e:06:04:ef:dc:e8:77:05:a2:7d:c5:fa:80:58:0a:c5:10:6d:90:ca:49:26:71:84:39:b7:9a:3e:e9:6f:ae:c5:35:b6:5b:24:8c:c9:ef:41:c3:b1:17:b6:3b:4e:28:89:3c:7e:87:a8:3a:a5:6d:dc:39:03:20:20:0b:c5:80:a3:79:13:1e:f6:ec:ae:36:df:40:74:34:87:46:93:3b:a3:e0:a4:8c:2f:43:4c:b2:54:80:71:76:78:d4:ea:12:28:d8:f2:e3:80:55:11:9b:f4:65:dc:53:0e:b4:4c:e0:4c:09:b4:dc:a0:80:5c:e6:b5:3b:95:d3:69:e4:52:3d:5b:61:86:02:e5:fd:0b:00:3a:fa:b3:45:cc:c9:a3:64:f2:dc:25:59:89:58:0d:9e:6e:28:3a:55:45:50:5f:88:67:2a:d2:e2:48:cc:8b:de:9a:1b:93:ae:87:e1:f2:90:50:40:d9:0f:44:31:53:46:ad:62:4e:8d:48:86:19:77:fc:59:75:91:79:35:59:1d:e3:4e:33:5b:e2:31:d7:ee:52:28:5f:0a:70:a7:be:bb:1c:03:ca:1a:18:d0:f5:c1:5b:9c:73:04:b6:4a:e8:46:52:58:76:d4:6a:e6:67:1c:0e:dc:13:d0:61:72:a0:92:cb:05:97:47:1c:c1:c9:cf:41:7d:1f:b1:4d:93:6b:53:41:03:21:2b:93:15:63:08:3e:2c:86:9e:7b:9f:3a:09:05:6a:7d:bb:1c:a7:b7:af:96:08:cb:5b:df:07:fb:9c:f2:95:11:c0:82:81:f6:1b:bf:5a:1e:58:cd:28:ca:7d:04:eb:aa:e9:29:c4:82:51:2c:89:61:95:b6:ed:a5:86:7c:7c:48:1d:ec:54:96:47:79:ea:fc:7f:f5:10:43:0a:9b:00:ef:8a:77:2e:f4:36:66:d2:6a:a6:95:b6:9f:23:3b:12:e2:89:d5:a4:c1:2c:91:4e:cb:94:e8:3f:22:0e:21:f9:b8:4a:81:5c:4c:63:ae:3d:05:b2:5c:5c:54:a7:55:8f:98:25:55:c4:a6:90:bc:19:29:b1:14:d4:e2:b0:95:e4:ff:89:71:61:be:8a:16:85" } ]
v1.20.4 Updates
- Add URL string parser
- Add Email Address string parser
- Add JWT string parser
- Add ISO 8601 Datetime string parser
- Add UNIX Epoch Timestamp string parser
- Add M3U/M3U8 file parser
- Add pager functionality to help (parser documentation only)
- Minor parser performance optimizations
jc
version 1.20.4 includes a few new string parsers that can be very useful in scripts.
The url
string parser not only allows you to pull out the specific parts of the URL you are interested in (e.g. path, query, hostname, etc.) but it also provides encoded and decoded versions of all of those values.
Similarly, the Email Address string parser allows you to quickly parse out the username and domain, even if Gmail “plus” addressing is used. The parser also allows you to separate out the username from the “plus” suffix.
JWT strings can now be parsed into their constituent Header, Payload, and Signature parts. The Payload is presented as a standard object.
And parsing time strings, including ISO 8601 Datetimes and Unix timestamps, just got easier. Both new parsers provide you detailed date information that you can use in your scripts. These parsers are a nice complement to the existing date
command parser.
Finally an M3U/M3U8 parser is included for media playlists. It includes the ability to parse extended information and, since these files are not usually well maintained, the parser fails gracefully for unparsable lines.
Other minor improvements include more/less paging when accessing parser documentation at the command line via jc --help --parser-name
.
For more details on each of the new parsers, see below.
URL string parser
This parser outputs Normalized, Encoded, and Decoded versions of the URL and all of the URL parts. (Documentation)
This allows you to pull specific information from the URL, including the scheme, netloc, user, password, hostname, port, path, path list, query, and fragment all three ways. For example, the following URL could be decoded:
$ echo 'http://%D0%BE%D0%B1%D0%BD%D0%BE%D0%B2%D0%BB%D0%B5%D0%BD%D0%B8%D0%B5%D0%BF%D0%BE%D0%B3%D0%BE%D0%B4%D1%8B.%72%75:%38%30' | jc --url | jq .decoded.hostname "обновлениепогоды.ru"
You can easily grab the path string and a path list:
$ echo 'https://example.com/this/is/a/path' | jc --url | jq .path "/this/is/a/path" $ echo 'https://example.com/this/is/a/path' | jc --url | jq .path_list [ "this", "is", "a", "path" ]
Or even the query string and object:
$ echo 'https://example.com?user=joe&selections=gardening&selections=plumbing' | jc --url | jq .query "user=joe&selections=gardening&selections=plumbing" $ echo 'https://example.com?user=joe&selections=gardening&selections=plumbing' | jc --url | jq .query_obj { "user": [ "joe" ], "selections": [ "gardening", "plumbing" ] }
There are many other use cases that the url
parser can help with. Here is a full example of the output:
$ echo 'https://www.example.com:443/mypath?q1=foo&q2=bar#heading-1' | jc --url -p { "url": "https://www.example.com:443/mypath?q1=foo&q2=bar#heading-1", "scheme": "https", "netloc": "www.example.com:443", "path": "/mypath", "path_list": [ "mypath" ], "query": "q1=foo&q2=bar", "query_obj": { "q1": [ "foo" ], "q2": [ "bar" ] }, "fragment": "heading-1", "username": null, "password": null, "hostname": "www.example.com", "port": 443, "encoded": { "url": "https://www.example.com:443/mypath?q1=foo&q2=bar#heading-1", "scheme": "https", "netloc": "www.example.com:443", "path": "/mypath", "path_list": [ "mypath" ], "query": "q1=foo&q2=bar", "fragment": "heading-1", "username": null, "password": null, "hostname": "www.example.com", "port": 443 }, "decoded": { "url": "https://www.example.com:443/mypath?q1=foo&q2=bar#heading-1", "scheme": "https", "netloc": "www.example.com:443", "path": "/mypath", "path_list": [ "mypath" ], "query": "q1=foo&q2=bar", "fragment": "heading-1", "username": null, "password": null, "hostname": "www.example.com", "port": 443 } }
Email Address string parser
The Email Address string parser allows you to easily pull the username and domain from an email address, even if it is using Gmail’s “plus” addressing. In those cases you can even pull the “plus” suffix. (Documentation)
$ echo 'joe.user+spam@example.com' | jc --email-address -p { "username": "joe.user", "domain": "example.com", "local": "joe.user+spam", "local_plus_suffix": "spam" }
JWT string parser
jc
can easily parse JWT strings into their constituent Header, Payload, and Signature parts. Note, the JWT parser does not check the integrity of the token. (Documentation)
$ echo 'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c' | jc --jwt -p { "header": { "alg": "HS256", "typ": "JWT" }, "payload": { "sub": "1234567890", "name": "John Doe", "iat": 1516239022 }, "signature": "49:f9:4a:c7:04:49:48:c7:8a:28:5d:90:4f:87:f0:a4:c7:89:7f:7e:8f:3a:4e:b2:25:5f:da:75:0b:2c:c3:97" }
ISO 8601 Datetime string parser
This parser explodes all of the relevant date and time fields when given an ISO 8601 datetime string. It will also provide a Unix timestamp and a normalized version of the ISO string. (Documentation)
$ echo "2022-07-20T14:52:45Z" | jc --iso-datetime -p { "year": 2022, "month": "Jul", "month_num": 7, "day": 20, "weekday": "Wed", "weekday_num": 3, "hour": 2, "hour_24": 14, "minute": 52, "second": 45, "microsecond": 0, "period": "PM", "utc_offset": "+0000", "day_of_year": 201, "week_of_year": 29, "iso": "2022-07-20T14:52:45+00:00", "timestamp": 1658328765 }
UNIX Epoch Timestamp string parser
In addition to the ISO 8601 Datetime string parser, the Timestamp parser takes in a 10+ digit epoch timestamp string and explodes it into all of the relevant date and time parts you might want to use, including a normalized ISO 8601 format string. Both Naive and timezone-aware UTC versions of the output are provided. (Documentation)
$ echo '1658599410' | jc --timestamp -p { "naive": { "year": 2022, "month": "Jul", "month_num": 7, "day": 23, "weekday": "Sat", "weekday_num": 6, "hour": 11, "hour_24": 11, "minute": 3, "second": 30, "period": "AM", "day_of_year": 204, "week_of_year": 29, "iso": "2022-07-23T11:03:30" }, "utc": { "year": 2022, "month": "Jul", "month_num": 7, "day": 23, "weekday": "Sat", "weekday_num": 6, "hour": 6, "hour_24": 18, "minute": 3, "second": 30, "period": "PM", "utc_offset": "+0000", "day_of_year": 204, "week_of_year": 29, "iso": "2022-07-23T18:03:30+00:00" } }
M3U/M3U8 file parser
jc
can now parse M3U files, including extended information. Unparsable lines are noted with a warning message unless the --quiet
flag is enabled. (Documentation)
$ cat playlist.m3u | jc --m3u -p [ { "runtime": 105, "display": "Example artist - Example title", "path": "C:\Files\My Music\Example.mp3" }, { "runtime": 321, "display": "Example Artist2 - Example title2", "path": "C:\Files\My Music\Favorites\Example2.ogg" } ]
Happy parsing!