JC Version 1.18.1 Released

kellyjonbrazil

4 years ago

I’m excited to announce the release of jc version 1.18.1 available on github and pypi. This release includes several enhancements when using jc as a Python library. Enhancements include some higher-level APIs and improved documentation to simplify the use of jc in Python programs and scripts. Error message improvements have been made on the CLI as well.

jc can be installed via pip or through several official OS package repositories, including Debian, Ubuntu, Fedora, openSUSE, Arch Linux, NixOS Linux, Guix System Linux, FreeBSD, and macOS. For more information on how to get jc, see the project README.

To upgrade with pip:

$ pip3 install --upgrade jc

Sections

New Features

New high-level parse API that works for both builtin and custom plugin parsers

>>> import jc
>>> jc.parse('date', 'Thu Jan 27 11:40:00 PST 2022')
{'year': 2022, 'month': 'Jan', 'month_num': 1, 'day': 27, 'weekday': 'Thu', 'weekday_num': 4, 'hour': 11, 'hour_24': 11, 'minute': 40, 'second': 0, 'period': 'AM', 'timezone': 'PST', 'utc_offset': None, 'day_of_year': 27, 'week_of_year': 4, 'iso': '2022-01-27T11:40:00', 'epoch': 1643312400, 'epoch_utc': None, 'timezone_aware': False}

Several other high-level functions in jc.lib that allow you to gather detailed parser information:
- parser_mod_list() -> list
- plugin_parser_mod_list() -> list
- get_help(parser_module_name: str) -> None
Enhanced CLI error messages for certain OS errors that can happen when using the “magic syntax” (file permission errors, etc.)

v1.18.2 Updates

Enhanced documentation for public functions, including type annotations
Additional high-level convenience functions:
- parser_info(parser_module_name: str) -> dict
- all_parser_info() -> list[dict]
Enhanced CLI error message to suggest setting locale to C when parsing errors occur
Bug fix for plugin parsers with underscore(s) in the name

v1.18.3 Updates

Add rsync command and log file parser tested on linux and macOS
Add rsync command and log file streaming parser tested on linux and macOS
Add xrandr command parser tested on linux
Enhance timestamp performance with caching and format hints
Refactor ignore_exceptions functionality in streaming parsers
Fix man page in packages

`rsync` command parser

Linux and macOS support for the rsync command. (Documentation):

$ rsync -i -a source/ dest | jc --rsync -p          # or  jc -p rsync -i -a source/ dest
[
  {
    "summary": {
      "sent": 1708,
      "received": 8209,
      "bytes_sec": 19834.0,
      "total_size": 235,
      "speedup": 0.02
    },
    "files": [
      {
        "filename": "./",
        "metadata": ".d..t......",
        "update_type": "not updated",
        "file_type": "directory",
        "checksum_or_value_different": false,
        "size_different": false,
        "modification_time_different": true,
        "permissions_different": false,
        "owner_different": false,
        "group_different": false,
        "acl_different": false,
        "extended_attribute_different": false
      },
      ...
    ]
  }
]

`rsync` command streaming parser

Linux support for the rsync command. This is a streaming parser and it outputs JSON Lines. (Documentation):

$ rsync -i -a source/ dest | jc --rsync-s
{"type":"file","filename":"./","metadata":".d..t......","update_...}
...

`xrandr` command parser

Linux support for the xrandr command. (Documentation):

$ xrandr | jc --xrandr -p          # or  jc -p xrandr
{
  "screens": [
    {
      "screen_number": 0,
      "minimum_width": 8,
      "minimum_height": 8,
      "current_width": 1920,
      "current_height": 1080,
      "maximum_width": 32767,
      "maximum_height": 32767,
      "associated_device": {
        "associated_modes": [
          {
            "resolution_width": 1920,
            "resolution_height": 1080,
            "is_high_resolution": false,
            "frequencies": [
              {
                "frequency": 60.03,
                "is_current": true,
                "is_preferred": true
              },
              {
                "frequency": 59.93,
                "is_current": false,
                "is_preferred": false
              }
            ]
          },
          {
            "resolution_width": 1680,
            "resolution_height": 1050,
            "is_high_resolution": false,
            "frequencies": [
              {
                "frequency": 59.88,
                "is_current": false,
                "is_preferred": false
              }
            ]
          }
        ],
        "is_connected": true,
        "is_primary": true,
        "device_name": "eDP1",
        "resolution_width": 1920,
        "resolution_height": 1080,
        "offset_width": 0,
        "offset_height": 0,
        "dimension_width": 310,
        "dimension_height": 170
      }
    }
  ],
  "unassociated_devices": []
}

v1.18.4 Updates

Add nmcli command parser tested on linux
Enhance parse error messages at the cli
Add standard and streaming parser list functions to the public API
Enhance python developer documentation formatting

`nmcli` command parser

Linux support for the nmcli command. (Documentation):

$ nmcli connection show ens33 | jc --nmcli -p          # or  jc -p nmcli connection show ens33
[
  {
    "connection_id": "ens33",
    "connection_uuid": "d92ece08-9e02-47d5-b2d2-92c80e155744",
    "connection_stable_id": null,
    "connection_type": "802-3-ethernet",
    "connection_interface_name": "ens33",
    "connection_autoconnect": "yes",
    ...
    "ip4_address_1": "192.168.71.180/24",
    "ip4_gateway": "192.168.71.2",
    "ip4_route_1": {
      "dst": "0.0.0.0/0",
      "nh": "192.168.71.2",
      "mt": 100
    },
    "ip4_route_2": {
      "dst": "192.168.71.0/24",
      "nh": "0.0.0.0",
      "mt": 100
    },
    "ip4_dns_1": "192.168.71.2",
    "ip4_domain_1": "localdomain",
    "dhcp4_option_1": {
      "name": "broadcast_address",
      "value": "192.168.71.255"
    },
    ...
    "ip6_address_1": "fe80::c1cb:715d:bc3e:b8a0/64",
    "ip6_gateway": null,
    "ip6_route_1": {
      "dst": "fe80::/64",
      "nh": "::",
      "mt": 100
    }
  }
]

v1.18.5 Updates

Fix date parser to ensure AM/PM period string is always uppercase. Fixes broken tests in some locales

v1.18.6 Updates

Add pidstat command parser tested on linux
Add pidstat command streaming parser tested on linux
Add mpstat command parser tested on linux
Add mpstat command streaming parser tested on linux
Add single-line ASCII and Unicode table parser
Add multi-line ASCII and Unicode table parser
Add a documentation option to parser_info() and all_parser_info()

`pidstat` command parser

Linux support for the pidstat command. (Documentation):

$ pidstat -hl | jc --pidstat -p          # or  jc -p pidstat -hl
[
  {
    "time": 1646859134,
    "uid": 0,
    "pid": 1,
    "percent_usr": 0.0,
    "percent_system": 0.03,
    "percent_guest": 0.0,
    "percent_cpu": 0.03,
    "cpu": 0,
    "command": "/usr/lib/systemd/systemd --switched-root --system..."
  },
  {
    "time": 1646859134,
    "uid": 0,
    "pid": 6,
    "percent_usr": 0.0,
    "percent_system": 0.0,
    "percent_guest": 0.0,
    "percent_cpu": 0.0,
    "cpu": 0,
    "command": "ksoftirqd/0"
  },
  {
    "time": 1646859134,
    "uid": 0,
    "pid": 2263,
    "percent_usr": 0.0,
    "percent_system": 0.0,
    "percent_guest": 0.0,
    "percent_cpu": 0.0,
    "cpu": 0,
    "command": "kworker/0:0"
  }
]

`pidstat` command streaming parser

Linux support for the pidstat command. This is a streaming parser and it outputs JSON Lines. (Documentation):

$ pidstat -hl | jc --pidstat-s
{"time":1646859134,"uid":0,"pid":1,"percent_usr":0.0,"percent_syste...}
{"time":1646859134,"uid":0,"pid":6,"percent_usr":0.0,"percent_syste...}
{"time":1646859134,"uid":0,"pid":9,"percent_usr":0.0,"percent_syste...}
...

`asciitable` ASCII and Unicode table parser

Supports parsing various styles of plain text tables. (Documentation):

$ echo '
>     ╒══════════╤═════════╤════════╕
>     │ foo      │ bar     │ baz    │
>     ╞══════════╪═════════╪════════╡
>     │ good day │         │ 12345  │
>     ├──────────┼─────────┼────────┤
>     │ hi there │ abc def │ 3.14   │
>     ╘══════════╧═════════╧════════╛' | jc --asciitable -p
[
  {
    "foo": "good day",
    "bar": null,
    "baz": "12345"
  },
  {
    "foo": "hi there",
    "bar": "abc def",
    "baz": "3.14"
  }
]

`asciitable-m` multi-line ASCII and Unicode table parser

Supports parsing various styles of plain text tables with multi-line rows. (Documentation):

$ echo '
> +----------+---------+--------+
> | foo      | bar     | baz    |
> |          |         | buz    |
> +==========+=========+========+
> | good day | 12345   |        |
> | mate     |         |        |
> +----------+---------+--------+
> | hi there | abc def | 3.14   |
> |          |         |        |
> +==========+=========+========+' | jc --asciitable-m -p
[
  {
    "foo": "good day\nmate",
    "bar": "12345",
    "baz_buz": null
  },
  {
    "foo": "hi there",
    "bar": "abc def",
    "baz_buz": "3.14"
  }
]

v1.18.7 Updates

Add git-log command parser tested on linux and macOS
Add update-alternatives --query command parser tested on linux
Add update-alternatives --get-selections command parser tested on linux
Fix key/value and INI parsers to allow duplicate keys
Fix YAML file parser for files including timestamp objects
Update xrandr parser: add a rotation field
Fix failing tests by moving template files
Add python interpreter version and path to -v and -a output

`git-log` command parser

Linux support for the git log command. (Documentation):

$ git log --stat | jc --git-log -p          or:  jc -p git log --stat
[
  {
    "commit": "728d882ed007b3c8b785018874a0eb06e1143b66",
    "author": "Kelly Brazil",
    "author_email": "kellyjonbrazil@gmail.com",
    "date": "Wed Apr 20 09:50:19 2022 -0400",
    "stats": {
      "files_changed": 2,
      "insertions": 90,
      "deletions": 12,
      "files": [
        "docs/parsers/git_log.md",
        "jc/parsers/git_log.py"
      ]
    },
    "message": "add timestamp docs and examples",
    "epoch": 1650462619,
    "epoch_utc": null
  },
  {
    "commit": "b53e42aca623181aa9bc72194e6eeef1e9a3a237",
    "author": "Kelly Brazil",
    "author_email": "kellyjonbrazil@gmail.com",
    "date": "Wed Apr 20 09:44:42 2022 -0400",
    "stats": {
      "files_changed": 5,
      "insertions": 29,
      "deletions": 6,
      "files": [
        "docs/parsers/git_log.md",
        "docs/utils.md",
        "jc/parsers/git_log.py",
        "jc/utils.py",
        "man/jc.1"
      ]
    },
    "message": "add calculated timestamp",
    "epoch": 1650462282,
    "epoch_utc": null
  }
]

`update-alternatives --query` command parser

Linux support for the update-alternatives --query command. (Documentation):

$ update-alternatives --query editor | jc --update-alt-q -p          # or:  jc -p update-alternatives --query editor
{
  "name": "editor",
  "link": "/usr/bin/editor",
  "slaves": [
    {
      "name": "editor.1.gz",
      "path": "/usr/share/man/man1/editor.1.gz"
    },
    {
      "name": "editor.da.1.gz",
      "path": "/usr/share/man/da/man1/editor.1.gz"
    }
  ],
  "status": "auto",
  "best": "/bin/nano",
  "value": "/bin/nano",
  "alternatives": [
    {
      "name": "/bin/ed",
      "priority": -100,
      "slaves": [
        {
          "name": "editor.1.gz",
          "path": "/usr/share/man/man1/ed.1.gz"
        }
      ]
    },
    {
      "name": "/bin/nano",
      "priority": 40,
      "slaves": [
        {
          "name": "editor.1.gz",
          "path": "/usr/share/man/man1/nano.1.gz"
        }
      ]
    }
  ]
}

`update-alternatives --get-selections` command parser

Linux support for the update-alternatives --get-selections command. (Documentation):

$ update-alternatives --get-selections | jc --update-alt-gs -p          # or:  jc -p update-alternatives --get-selections
[
  {
    "name": "arptables",
    "status": "auto",
    "current": "/usr/sbin/arptables-nft"
  },
  {
    "name": "awk",
    "status": "auto",
    "current": "/usr/bin/gawk"
  }
]

v1.18.8 Updates

Fix update-alternatives --query parser for cases where slaves are not present
Fix UnicodeEncodeError on some systems where LANG=C is set and Unicode characters are in the output
Update history command parser: do not drop non-ASCII characters if the system is configured for UTF-8 encoding
Enhance “magic syntax” to always use UTF-8 encoding

New Features

v1.18.2 Updates

v1.18.3 Updates

rsync command parser

rsync command streaming parser

xrandr command parser

v1.18.4 Updates

nmcli command parser

v1.18.5 Updates

v1.18.6 Updates

pidstat command parser

pidstat command streaming parser

asciitable ASCII and Unicode table parser

asciitable-m multi-line ASCII and Unicode table parser