JC Version 1.20.0 Released

I’m excited to announce the release of jc version 1.20.0 available on github and pypi. jc now supports over 100 standard and streaming parsers. Thank you to the Open Source community for making this possible!

jc can be installed via pip or through several official OS package repositories, including Debian, Ubuntu, Fedora, openSUSE, Arch Linux, NixOS Linux, Guix System Linux, FreeBSD, and macOS. For more information on how to get jc, see the project README.

To upgrade with pip:

$ pip3 install --upgrade jc

What’s New

• Add YAML output option with the -y option
• Add top -b standard and streaming parsers tested on linux
• Add plugin_parser_count, standard_parser_count, and streaming_parser_count keys to jc -a output
• Add is_compatible function to the utils module
• Fix pip-show parser for packages with a multi-line license field
• Fix ASCII Table parser for cases where centered headers cause mis-aligned fields

New Parsers

top -b command parser

Support for the top -b command. (Documentation)

$ top -b -n 3 | jc --top -p
[
  {
    "time": "11:20:43",
    "uptime": 118,
    "users": 2,
    "load_1m": 0.0,
    "load_5m": 0.01,
    "load_15m": 0.05,
    "tasks_total": 108,
    "tasks_running": 2,
    "tasks_sleeping": 106,
    "tasks_stopped": 0,
    "tasks_zombie": 0,
    "cpu_user": 5.6,
    "cpu_sys": 11.1,
    "cpu_nice": 0.0,
    "cpu_idle": 83.3,
    "cpu_wait": 0.0,
    "cpu_hardware": 0.0,
    "cpu_software": 0.0,
    "cpu_steal": 0.0,
    "mem_total": 3.7,
    "mem_free": 3.3,
    "mem_used": 0.2,
    "mem_buff_cache": 0.2,
    "swap_total": 2.0,
    "swap_free": 2.0,
    "swap_used": 0.0,
    "mem_available": 3.3,
    "processes": [
      {
        "pid": 2225,
        "user": "kbrazil",
        "priority": 20,
        "nice": 0,
        "virtual_mem": 158.1,
        "resident_mem": 2.2,
        "shared_mem": 1.6,
        "status": "running",
        "percent_cpu": 12.5,
        "percent_mem": 0.1,
        "time_hundredths": "0:00.02",
        "command": "top",
        "parent_pid": 1884,
        "uid": 1000,
        "real_uid": 1000,
        "real_user": "kbrazil",
        "saved_uid": 1000,
        "saved_user": "kbrazil",
        "gid": 1000,
        "group": "kbrazil",
        "pgrp": 2225,
        "tty": "pts/0",
        "tty_process_gid": 2225,
        "session_id": 1884,
        "thread_count": 1,
        "last_used_processor": 0,
        "time": "0:00",
        "swap": 0.0,
        "code": 0.1,
        "data": 1.0,
        "major_page_fault_count": 0,
        "minor_page_fault_count": 736,
        "dirty_pages_count": 0,
        "sleeping_in_function": null,
        "flags": "..4.2...",
        "cgroups": "1:name=systemd:/user.slice/user-1000.+",
        "supplementary_gids": [
          10,
          1000
        ],
        "supplementary_groups": [
          "wheel",
          "kbrazil"
        ],
        "thread_gid": 2225,
        "environment_variables": [
          "XDG_SESSION_ID=2",
          "HOSTNAME=localhost"
        ],
        "major_page_fault_count_delta": 0,
        "minor_page_fault_count_delta": 4,
        "used": 2.2,
        "ipc_namespace_inode": 4026531839,
        "mount_namespace_inode": 4026531840,
        "net_namespace_inode": 4026531956,
        "pid_namespace_inode": 4026531836,
        "user_namespace_inode": 4026531837,
        "nts_namespace_inode": 4026531838
      },
      ...
    ]
  }
]

top -b command streaming parser

Support for the top -b command. This is a streaming parser and it outputs JSON Lines. (Documentation):

$ top -b | jc --top-s
{"time":"11:24:50","uptime":2,"users":2,"load_1m":0.23,"load_5m":...}
...

v1.20.1 Updates

• Add postconf -M parser tested on linux
• Update asciitable and asciitable-m parsers to preserve case in key names when using the -r or raw=True options.
• Add long options (e.g. --help, --about, --pretty, etc.)
• Add shell completions for Bash and Zsh
• Fix id parser for cases where the user or group name is not present

postconf -M command parser

Linux support for the postconf -m command. (Documentation):

$ postconf -M | jc --postconf -p          # or jc -p postconf -M
[
  {
    "service_name": "smtp",
    "service_type": "inet",
    "private": false,
    "unprivileged": null,
    "chroot": true,
    "wake_up_time": null,
    "process_limit": null,
    "command": "smtpd",
    "no_wake_up_before_first_use": null
  },
  {
    "service_name": "pickup",
    "service_type": "unix",
    "private": false,
    "unprivileged": null,
    "chroot": true,
    "wake_up_time": 60,
    "process_limit": 1,
    "command": "pickup",
    "no_wake_up_before_first_use": false
  }
]

Long Options

jc now supports long CLI options:

Options:
    -a,  --about        about jc
    -C,  --force-color  force color output even when using pipes (overrides -m)
    -d,  --debug        debug (double for verbose debug)
    -h,  --help         help (--help --parser_name for parser documentation)
    -m,  --monochrome   monochrome output
    -p,  --pretty       pretty print output
    -q,  --quiet        suppress warnings (double to ignore streaming errors)
    -r,  --raw          raw output
    -u,  --unbuffer     unbuffer output
    -v,  --version      version info
    -y,  --yaml-out     YAML output
    -B,  --bash-comp    gen Bash completion: jc -B > /etc/bash_completion.d/jc
    -Z,  --zsh-comp     gen Zsh completion: jc -Z > "${fpath[1]}/_jc"

Shell Completions

Bash and Zsh completions are now available for jc! If your system is already set up for completions you can run the following to enable completions:

Bash

Linux

$ jc -B > /etc/bash_completion.d/jc

macOS

$ jc -B > /usr/local/etc/bash_completion.d/jc

Zsh

Linux and macOS

$ jc -Z > "${fpath[1]}/_jc"

v1.20.2 Updates

• Add gpg --with-colons command parser tested on linux
• Add DER and PEM encoded X.509 Certificate parser
• Add Bash and Zsh completion scripts to DEB and RPM packages

gpg –with-colons command parser

Linux support for the gpg --with-colons command. (Documentation):

$ gpg --with-colons --show-keys file.gpg | jc --gpg -p
[
  {
    "type": "pub",
    "validity": "f",
    "key_length": "1024",
    "pub_key_alg": "17",
    "key_id": "6C7EE1B8621CC013",
    "creation_date": "899817715",
    "expiration_date": "1055898235",
    "certsn_uidhash_trustinfo": null,
    "owner_trust": "m",
    "user_id": null,
    "signature_class": null,
    "key_capabilities": "scESC",
...

X.509 DER/PEM Certificate Files

Support for DER and PEM encoded certificate files (Documentation):

$ cat alice.crt | jc --x509-cert -p
[
  {
    "tbs_certificate": {
      "version": "v3",
      "serial_number": "01",
      "signature": {
        "algorithm": "sha1_rsa",
        "parameters": null
      },
      "issuer": {
        "country_name": "FR",
        "state_or_province_name": "Alsace",
        "locality_name": "Strasbourg",
        "organization_name": "www.freelan.org",
        "organizational_unit_name": "freelan",
        "common_name": "Freelan Sample Certificate Authority",
        "email_address": "contact@freelan.org"
      },
      "validity": {
        "not_before": 1335522678,
        "not_after": 1650882678,
        "not_before_iso": "2012-04-27T10:31:18+00:00",
        "not_after_iso": "2022-04-25T10:31:18+00:00"
      },
      "subject": {
        "country_name": "FR",
        "state_or_province_name": "Alsace",
        "organization_name": "www.freelan.org",
        "organizational_unit_name": "freelan",
        "common_name": "alice",
        "email_address": "contact@freelan.org"
      },
      "subject_public_key_info": {
        "algorithm": {
          "algorithm": "rsa",
          "parameters": null
        },
        "public_key": {
          "modulus": "dd:6d:bd:f8:80:fa:d7:de:1b:1f:a7:a3:2e:b2:02:e2:16:f6:52:0a:3c:bf:a6:42:f8:ca:dc:93:67:4d:60:c3:4f:8d:c3:8a:00:1b:f1:c4:4b:41:6a:69:d2:69:e5:3f:21:8e:c5:0b:f8:22:37:ad:b6:2c:4b:55:ff:7a:03:72:bb:9a:d3:ec:96:b9:56:9f:cb:19:99:c9:32:94:6f:8f:c6:52:06:9f:45:03:df:fd:e8:97:f6:ea:d6:ba:bb:48:2b:b5:e0:34:61:4d:52:36:0f:ab:87:52:25:03:cf:87:00:87:13:f2:ca:03:29:16:9d:90:57:46:b5:f4:0e:ae:17:c8:0a:4d:92:ed:08:a6:32:23:11:71:fe:f2:2c:44:d7:6c:07:f3:0b:7b:0c:4b:dd:3b:b4:f7:37:70:9f:51:b6:88:4e:5d:6a:05:7f:8d:9b:66:7a:ab:80:20:fe:ee:6b:97:c3:49:7d:78:3b:d5:99:97:03:75:ce:8f:bc:c5:be:9c:9a:a5:12:19:70:f9:a4:bd:96:27:ed:23:02:a7:c7:57:c9:71:cf:76:94:a2:21:62:f6:b8:1d:ca:88:ee:09:ad:46:2f:b7:61:b3:2c:15:13:86:9f:a5:35:26:5a:67:f4:37:c8:e6:80:01:49:0e:c7:ed:61:d3:cd:bc:e4:f8:be:3f:c9:4e:f8:7d:97:89:ce:12:bc:ca:b5:c6:d2:e0:d9:b3:68:3c:2e:4a:9d:b4:5f:b8:53:ee:50:3d:bf:dd:d4:a2:8a:b6:a0:27:ab:98:0c:b3:b2:58:90:e2:bc:a1:ad:ff:bd:8e:55:31:0f:00:bf:68:e9:3d:a9:19:9a:f0:6d:0b:a2:14:6a:c6:4c:c6:4e:bd:63:12:a5:0b:4d:97:eb:42:09:79:53:e2:65:aa:24:34:70:b8:c1:ab:23:80:e7:9c:6c:ed:dc:82:aa:37:04:b8:43:2a:3d:2a:a8:cc:20:fc:27:5d:90:26:58:f9:b7:14:e2:9e:e2:c1:70:73:97:e9:6b:02:8e:d3:52:59:7b:00:ec:61:30:f1:56:3f:9c:c1:7c:05:c5:b1:36:c8:18:85:cf:61:40:1f:07:e8:a7:06:87:df:9a:77:0b:a9:64:72:03:f6:93:fc:e0:02:59:c1:96:ec:c0:09:42:3e:30:a2:7f:1b:48:2f:fe:e0:21:8f:53:87:25:0d:cb:ea:49:f5:4a:9b:d0:e3:5f:ee:78:18:e5:ba:71:31:a9:04:98:0f:b1:ad:67:52:a0:f2:e3:9c:ab:6a:fe:58:84:84:dd:07:3d:32:94:05:16:45:15:96:59:a0:58:6c:18:0e:e3:77:66:c7:b3:f7:99",
          "public_exponent": 65537
        }
      },
      "issuer_unique_id": null,
      "subject_unique_id": null,
      "extensions": [
        {
          "extn_id": "basic_constraints",
          "critical": false,
          "extn_value": {
            "ca": false,
            "path_len_constraint": null
          }
        },
        {
          "extn_id": "2.16.840.1.113730.1.13",
          "critical": false,
          "extn_value": "16:1d:4f:70:65:6e:53:53:4c:20:47:65:6e:65:72:61:74:65:64:20:43:65:72:74:69:66:69:63:61:74:65"
        },
        {
          "extn_id": "key_identifier",
          "critical": false,
          "extn_value": "59:5f:c9:13:ba:1b:cc:b9:a8:41:4a:8a:49:79:6a:36:f6:7d:3e:d7"
        },
        {
          "extn_id": "authority_key_identifier",
          "critical": false,
          "extn_value": {
            "key_identifier": "23:6c:2d:3d:3e:29:5d:78:b8:6c:3e:aa:e2:bb:2e:1e:6c:87:f2:53",
            "authority_cert_issuer": null,
            "authority_cert_serial_number": null
          }
        }
      ]
    },
    "signature_algorithm": {
      "algorithm": "sha1_rsa",
      "parameters": null
    },
    "signature_value": "13:e7:02:45:3e:a7:ab:bd:b8:da:e7:ef:74:88:ac:62:d5:dd:10:56:d5:46:07:ec:fa:6a:80:0c:b9:62:be:aa:08:b4:be:0b:eb:9a:ef:68:b7:69:6f:4d:20:92:9d:18:63:7a:23:f4:48:87:6a:14:c3:91:98:1b:4e:08:59:3f:91:80:e9:f4:cf:fd:d5:bf:af:4b:e4:bd:78:09:71:ac:d0:81:e5:53:9f:3e:ac:44:3e:9f:f0:bf:5a:c1:70:4e:06:04:ef:dc:e8:77:05:a2:7d:c5:fa:80:58:0a:c5:10:6d:90:ca:49:26:71:84:39:b7:9a:3e:e9:6f:ae:c5:35:b6:5b:24:8c:c9:ef:41:c3:b1:17:b6:3b:4e:28:89:3c:7e:87:a8:3a:a5:6d:dc:39:03:20:20:0b:c5:80:a3:79:13:1e:f6:ec:ae:36:df:40:74:34:87:46:93:3b:a3:e0:a4:8c:2f:43:4c:b2:54:80:71:76:78:d4:ea:12:28:d8:f2:e3:80:55:11:9b:f4:65:dc:53:0e:b4:4c:e0:4c:09:b4:dc:a0:80:5c:e6:b5:3b:95:d3:69:e4:52:3d:5b:61:86:02:e5:fd:0b:00:3a:fa:b3:45:cc:c9:a3:64:f2:dc:25:59:89:58:0d:9e:6e:28:3a:55:45:50:5f:88:67:2a:d2:e2:48:cc:8b:de:9a:1b:93:ae:87:e1:f2:90:50:40:d9:0f:44:31:53:46:ad:62:4e:8d:48:86:19:77:fc:59:75:91:79:35:59:1d:e3:4e:33:5b:e2:31:d7:ee:52:28:5f:0a:70:a7:be:bb:1c:03:ca:1a:18:d0:f5:c1:5b:9c:73:04:b6:4a:e8:46:52:58:76:d4:6a:e6:67:1c:0e:dc:13:d0:61:72:a0:92:cb:05:97:47:1c:c1:c9:cf:41:7d:1f:b1:4d:93:6b:53:41:03:21:2b:93:15:63:08:3e:2c:86:9e:7b:9f:3a:09:05:6a:7d:bb:1c:a7:b7:af:96:08:cb:5b:df:07:fb:9c:f2:95:11:c0:82:81:f6:1b:bf:5a:1e:58:cd:28:ca:7d:04:eb:aa:e9:29:c4:82:51:2c:89:61:95:b6:ed:a5:86:7c:7c:48:1d:ec:54:96:47:79:ea:fc:7f:f5:10:43:0a:9b:00:ef:8a:77:2e:f4:36:66:d2:6a:a6:95:b6:9f:23:3b:12:e2:89:d5:a4:c1:2c:91:4e:cb:94:e8:3f:22:0e:21:f9:b8:4a:81:5c:4c:63:ae:3d:05:b2:5c:5c:54:a7:55:8f:98:25:55:c4:a6:90:bc:19:29:b1:14:d4:e2:b0:95:e4:ff:89:71:61:be:8a:16:85"
  }
]

v1.20.4 Updates

• Add URL string parser
• Add Email Address string parser
• Add JWT string parser
• Add ISO 8601 Datetime string parser
• Add UNIX Epoch Timestamp string parser
• Add M3U/M3U8 file parser
• Add pager functionality to help (parser documentation only)
• Minor parser performance optimizations

jc version 1.20.4 includes a few new string parsers that can be very useful in scripts.

The url string parser not only allows you to pull out the specific parts of the URL you are interested in (e.g. path, query, hostname, etc.) but it also provides encoded and decoded versions of all of those values.

Similarly, the Email Address string parser allows you to quickly parse out the username and domain, even if Gmail “plus” addressing is used. The parser also allows you to separate out the username from the “plus” suffix.

JWT strings can now be parsed into their constituent Header, Payload, and Signature parts. The Payload is presented as a standard object.

And parsing time strings, including ISO 8601 Datetimes and Unix timestamps, just got easier. Both new parsers provide you detailed date information that you can use in your scripts. These parsers are a nice complement to the existing date command parser.

Finally an M3U/M3U8 parser is included for media playlists. It includes the ability to parse extended information and, since these files are not usually well maintained, the parser fails gracefully for unparsable lines.

Other minor improvements include more/less paging when accessing parser documentation at the command line via jc --help --parser-name.

For more details on each of the new parsers, see below.

URL string parser

This parser outputs Normalized, Encoded, and Decoded versions of the URL and all of the URL parts. (Documentation)

This allows you to pull specific information from the URL, including the scheme, netloc, user, password, hostname, port, path, path list, query, and fragment all three ways. For example, the following URL could be decoded:

$ echo 'http://%D0%BE%D0%B1%D0%BD%D0%BE%D0%B2%D0%BB%D0%B5%D0%BD%D0%B8%D0%B5%D0%BF%D0%BE%D0%B3%D0%BE%D0%B4%D1%8B.%72%75:%38%30' | jc --url | jq .decoded.hostname
"обновлениепогоды.ru"

You can easily grab the path string and a path list:

$ echo 'https://example.com/this/is/a/path' | jc --url | jq .path
"/this/is/a/path"

$ echo 'https://example.com/this/is/a/path' | jc --url | jq .path_list
[
  "this",
  "is",
  "a",
  "path"
]

Or even the query string and object:

$ echo 'https://example.com?user=joe&selections=gardening&selections=plumbing' | jc --url | jq .query
"user=joe&selections=gardening&selections=plumbing"

$ echo 'https://example.com?user=joe&selections=gardening&selections=plumbing' | jc --url | jq .query_obj
{
  "user": [
    "joe"
  ],
  "selections": [
    "gardening",
    "plumbing"
  ]
}

There are many other use cases that the url parser can help with. Here is a full example of the output:

$ echo 'https://www.example.com:443/mypath?q1=foo&q2=bar#heading-1' | jc --url -p
{
  "url": "https://www.example.com:443/mypath?q1=foo&q2=bar#heading-1",
  "scheme": "https",
  "netloc": "www.example.com:443",
  "path": "/mypath",
  "path_list": [
    "mypath"
  ],
  "query": "q1=foo&q2=bar",
  "query_obj": {
    "q1": [
      "foo"
    ],
    "q2": [
      "bar"
    ]
  },
  "fragment": "heading-1",
  "username": null,
  "password": null,
  "hostname": "www.example.com",
  "port": 443,
  "encoded": {
    "url": "https://www.example.com:443/mypath?q1=foo&q2=bar#heading-1",
    "scheme": "https",
    "netloc": "www.example.com:443",
    "path": "/mypath",
    "path_list": [
      "mypath"
    ],
    "query": "q1=foo&q2=bar",
    "fragment": "heading-1",
    "username": null,
    "password": null,
    "hostname": "www.example.com",
    "port": 443
  },
  "decoded": {
    "url": "https://www.example.com:443/mypath?q1=foo&q2=bar#heading-1",
    "scheme": "https",
    "netloc": "www.example.com:443",
    "path": "/mypath",
    "path_list": [
      "mypath"
    ],
    "query": "q1=foo&q2=bar",
    "fragment": "heading-1",
    "username": null,
    "password": null,
    "hostname": "www.example.com",
    "port": 443
  }
}

Email Address string parser

The Email Address string parser allows you to easily pull the username and domain from an email address, even if it is using Gmail’s “plus” addressing. In those cases you can even pull the “plus” suffix. (Documentation)

$ echo 'joe.user+spam@example.com' | jc --email-address -p
{
  "username": "joe.user",
  "domain": "example.com",
  "local": "joe.user+spam",
  "local_plus_suffix": "spam"
}

JWT string parser

jc can easily parse JWT strings into their constituent Header, Payload, and Signature parts. Note, the JWT parser does not check the integrity of the token. (Documentation)

$ echo 'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c' | jc --jwt -p
{
  "header": {
    "alg": "HS256",
    "typ": "JWT"
  },
  "payload": {
    "sub": "1234567890",
    "name": "John Doe",
    "iat": 1516239022
  },
  "signature": "49:f9:4a:c7:04:49:48:c7:8a:28:5d:90:4f:87:f0:a4:c7:89:7f:7e:8f:3a:4e:b2:25:5f:da:75:0b:2c:c3:97"
}

ISO 8601 Datetime string parser

This parser explodes all of the relevant date and time fields when given an ISO 8601 datetime string. It will also provide a Unix timestamp and a normalized version of the ISO string. (Documentation)

$ echo "2022-07-20T14:52:45Z" | jc --iso-datetime -p
{
  "year": 2022,
  "month": "Jul",
  "month_num": 7,
  "day": 20,
  "weekday": "Wed",
  "weekday_num": 3,
  "hour": 2,
  "hour_24": 14,
  "minute": 52,
  "second": 45,
  "microsecond": 0,
  "period": "PM",
  "utc_offset": "+0000",
  "day_of_year": 201,
  "week_of_year": 29,
  "iso": "2022-07-20T14:52:45+00:00",
  "timestamp": 1658328765
}

UNIX Epoch Timestamp string parser

In addition to the ISO 8601 Datetime string parser, the Timestamp parser takes in a 10+ digit epoch timestamp string and explodes it into all of the relevant date and time parts you might want to use, including a normalized ISO 8601 format string. Both Naive and timezone-aware UTC versions of the output are provided. (Documentation)

$ echo '1658599410' | jc --timestamp -p
{
  "naive": {
    "year": 2022,
    "month": "Jul",
    "month_num": 7,
    "day": 23,
    "weekday": "Sat",
    "weekday_num": 6,
    "hour": 11,
    "hour_24": 11,
    "minute": 3,
    "second": 30,
    "period": "AM",
    "day_of_year": 204,
    "week_of_year": 29,
    "iso": "2022-07-23T11:03:30"
  },
  "utc": {
    "year": 2022,
    "month": "Jul",
    "month_num": 7,
    "day": 23,
    "weekday": "Sat",
    "weekday_num": 6,
    "hour": 6,
    "hour_24": 18,
    "minute": 3,
    "second": 30,
    "period": "PM",
    "utc_offset": "+0000",
    "day_of_year": 204,
    "week_of_year": 29,
    "iso": "2022-07-23T18:03:30+00:00"
  }
}

M3U/M3U8 file parser

jc can now parse M3U files, including extended information. Unparsable lines are noted with a warning message unless the --quiet flag is enabled. (Documentation)

$ cat playlist.m3u | jc --m3u -p
[
  {
    "runtime": 105,
    "display": "Example artist - Example title",
    "path": "C:\Files\My Music\Example.mp3"
  },
  {
    "runtime": 321,
    "display": "Example Artist2 - Example title2",
    "path": "C:\Files\My Music\Favorites\Example2.ogg"
  }
]

Happy parsing!