Try the
jc
web demo!
I’m excited to announce the release of jc
version 1.8.0 available on github and pypi. See below for more information on the new features and parsers.
To upgrade, run:
$ pip3 install --upgrade jc
New Parsers
jc
now includes 45 parsers! New parsers (tested on linux and OSX) include blkid
, last
, lastb
, who
, /etc/passwd
files, /etc/shadow
files, /etc/group
files, /etc/gshadow
files, and CSV
files.
Documentation and schemas for all parsers can be found here.
blkid
command parser
Linux support for the blkid
command:
$ blkid | jc --blkid -p # or: jc -p blkid [ { "device": "/dev/sda1", "uuid": "05d927ab-5875-49e4-ada1-7f46cb32c932", "type": "xfs" }, { "device": "/dev/sda2", "uuid": "3klkIj-w1kk-DkJi-0XBJ-y3i7-i2Ac-vHqWBM", "type": "LVM2_member" }, { "device": "/dev/mapper/centos-root", "uuid": "07d718ff-950c-4e5b-98f0-42a1147c77d9", "type": "xfs" }, { "device": "/dev/mapper/centos-swap", "uuid": "615eb89a-bcbf-46fd-80e3-c483ff5c931f", "type": "swap" } ] $ sudo blkid -o udev -ip /dev/sda2 | jc --blkid -p # or: sudo jc -p blkid -o udev -ip /dev/sda2 [ { "id_fs_uuid": "3klkIj-w1kk-DkJi-0XBJ-y3i7-i2Ac-vHqWBM", "id_fs_uuid_enc": "3klkIj-w1kk-DkJi-0XBJ-y3i7-i2Ac-vHqWBM", "id_fs_version": "LVM2\x20001", "id_fs_type": "LVM2_member", "id_fs_usage": "raid", "id_iolimit_minimum_io_size": 512, "id_iolimit_physical_sector_size": 512, "id_iolimit_logical_sector_size": 512, "id_part_entry_scheme": "dos", "id_part_entry_type": "0x8e", "id_part_entry_number": 2, "id_part_entry_offset": 2099200, "id_part_entry_size": 39843840, "id_part_entry_disk": "8:0" } ]
last
and lastb
command parsers
Linux and OSX support for the last
command. Linux support for the lastb
command.
$ last | jc --last -p # or: jc -p last [ { "user": "joeuser", "tty": "ttys002", "hostname": null, "login": "Thu Feb 27 14:31", "logout": "still logged in" }, { "user": "joeuser", "tty": "ttys003", "hostname": null, "login": "Thu Feb 27 10:38", "logout": "10:38", "duration": "00:00" }, { "user": "joeuser", "tty": "ttys003", "hostname": null, "login": "Thu Feb 27 10:18", "logout": "10:18", "duration": "00:00" }, ... ] $ sudo lastb | jc --last -p # or: sudo jc -p lastb [ { "user": "joeuser", "tty": "ssh:notty", "hostname": "127.0.0.1", "login": "Tue Mar 3 00:48", "logout": "00:48", "duration": "00:00" }, { "user": "joeuser", "tty": "ssh:notty", "hostname": "127.0.0.1", "login": "Tue Mar 3 00:48", "logout": "00:48", "duration": "00:00" }, { "user": "jouser", "tty": "ssh:notty", "hostname": "127.0.0.1", "login": "Tue Mar 3 00:48", "logout": "00:48", "duration": "00:00" } ]
who
command parser
Linux and OSX support for the who
command:
$ who | jc --who -p # or: jc -p who [ { "user": "joeuser", "tty": "ttyS0", "time": "2020-03-02 02:52" }, { "user": "joeuser", "tty": "pts/0", "time": "2020-03-02 05:15", "from": "192.168.71.1" } ] $ who -a | jc --who -p # or: jc -p who -a [ { "event": "reboot", "time": "Feb 7 23:31", "pid": 1 }, { "user": "joeuser", "writeable_tty": "-", "tty": "console", "time": "Feb 7 23:32", "idle": "old", "pid": 105 }, { "user": "joeuser", "writeable_tty": "+", "tty": "ttys000", "time": "Feb 13 16:44", "idle": ".", "pid": 51217, "comment": "term=0 exit=0" }, { "user": "joeuser", "writeable_tty": "?", "tty": "ttys003", "time": "Feb 28 08:59", "idle": "01:36", "pid": 41402 }, { "user": "joeuser", "writeable_tty": "+", "tty": "ttys004", "time": "Mar 1 16:35", "idle": ".", "pid": 15679, "from": "192.168.1.5" } ]
CSV File Parser
Convert generic CSV files to JSON. The parser will attempt to automatically detect the delimiter character. If it cannot detect the delimiter it will use the comma (‘,
‘) as the delimiter. The file must contain a header row as the first line:
$ cat homes.csv "Sell", "List", "Living", "Rooms", "Beds", "Baths", "Age", "Acres", "Taxes" 142, 160, 28, 10, 5, 3, 60, 0.28, 3167 175, 180, 18, 8, 4, 1, 12, 0.43, 4033 129, 132, 13, 6, 3, 1, 41, 0.33, 1471 ... $ cat homes.csv | jc --csv -p [ { "Sell": "142", "List": "160", "Living": "28", "Rooms": "10", "Beds": "5", "Baths": "3", "Age": "60", "Acres": "0.28", "Taxes": "3167" }, { "Sell": "175", "List": "180", "Living": "18", "Rooms": "8", "Beds": "4", "Baths": "1", "Age": "12", "Acres": "0.43", "Taxes": "4033" }, { "Sell": "129", "List": "132", "Living": "13", "Rooms": "6", "Beds": "3", "Baths": "1", "Age": "41", "Acres": "0.33", "Taxes": "1471" }, ... ]
/etc/passwd
, /etc/shadow
, /etc/group
, and /etc/gshadow
file parsers
Convert /etc/passwd
, /etc/shadow
, /etc/group
, and /etc/gshadow
files to JSON format:
$ cat /etc/passwd | jc --passwd -p [ { "username": "nobody", "password": "*", "uid": -2, "gid": -2, "comment": "Unprivileged User", "home": "/var/empty", "shell": "/usr/bin/false" }, { "username": "root", "password": "*", "uid": 0, "gid": 0, "comment": "System Administrator", "home": "/var/root", "shell": "/bin/sh" }, { "username": "daemon", "password": "*", "uid": 1, "gid": 1, "comment": "System Services", "home": "/var/root", "shell": "/usr/bin/false" }, ... ] $ sudo cat /etc/shadow | jc --shadow -p [ { "username": "root", "password": "*", "last_changed": 18113, "minimum": 0, "maximum": 99999, "warn": 7, "inactive": null, "expire": null }, { "username": "daemon", "password": "*", "last_changed": 18113, "minimum": 0, "maximum": 99999, "warn": 7, "inactive": null, "expire": null }, { "username": "bin", "password": "*", "last_changed": 18113, "minimum": 0, "maximum": 99999, "warn": 7, "inactive": null, "expire": null }, ... ] $ cat /etc/group | jc --group -p [ { "group_name": "nobody", "password": "*", "gid": -2, "members": [] }, { "group_name": "nogroup", "password": "*", "gid": -1, "members": [] }, { "group_name": "wheel", "password": "*", "gid": 0, "members": [ "root" ] }, { "group_name": "certusers", "password": "*", "gid": 29, "members": [ "root", "_jabber", "_postfix", "_cyrus", "_calendar", "_dovecot" ] }, ... ] $ cat /etc/gshadow | jc --gshadow -p [ { "group_name": "root", "password": "*", "administrators": [], "members": [] }, { "group_name": "adm", "password": "*", "administrators": [], "members": [ "syslog", "joeuser" ] }, ... ]
Updated Parsers
- The
ls
parser now supports filenames that contain newline characters when usingls -l
orls -b
. A warning message will be sent tostderr
if newlines are detected andls -l
orls -b
are not used:
$ ls | jc --ls jc: Warning - Newline characters detected. Filenames probably corrupted. Use ls -l or -b instead. [{"filename": "this file has"}, {"filename": "a newline inside"}, {"filename": "this file has"}, {"filename": "four contiguous newlines inside"}, ...]
- The
ls
parser now supports multiple directory listings, globbing, and recursive listings.
$ ls -R | jc --ls [{"filename": "centos-7.7"}, {"filename": "create_fixtures.sh"}, {"filename": "generic"}, {"filename": "osx-10.11.6"}, {"filename": "osx-10.14.6"}, ...]
Alternative “Magic” Syntax
jc
now accepts a simplified syntax for most command parsers. Instead of piping the data into jc
you can now also prepend “jc
” to the command you would like to convert. Note that command aliases are not supported:
$ jc dig www.example.com [{"id": 31113, "opcode": "QUERY", "status": "NOERROR", "flags": ["qr", "rd", "ra"], "query_num": 1, "answer_num": 1, "authority_num": 0, "additional_num": 1, "question": {"name": "www.example.com.", "class": "IN", "type": "A"}, "answer": [{"name": "www.example.com.", "class": "IN", "type": "A", "ttl": 35366, "data": "93.184.216.34"}], "query_time": 37, "server": "2600", "when": "Mon Mar 02 16:13:31 PST 2020", "rcvd": 60}]
You can also insert jc
options before the command:
$ jc -pqd dig www.example.com [ { "id": 7495, "opcode": "QUERY", "status": "NOERROR", "flags": [ "qr", "rd", "ra" ], "query_num": 1, "answer_num": 1, "authority_num": 0, "additional_num": 1, "question": { "name": "www.example.com.", "class": "IN", "type": "A" }, "answer": [ { "name": "www.example.com.", "class": "IN", "type": "A", "ttl": 36160, "data": "93.184.216.34" } ], "query_time": 40, "server": "2600", "when": "Mon Mar 02 16:15:21 PST 2020", "rcvd": 60 } ]
Schema Changes
There are no schema changes in this release.
Full Parser List
arp
blkid
crontab
crontab-u
CSV
df
dig
du
env
free
fstab
/etc/group
/etc/gshadow
history
/etc/hosts
id
ifconfig
INI
iptables
jobs
last
andlastb
ls
lsblk
lsmod
lsof
mount
netstat
/etc/passwd
pip list
pip show
ps
route
/etc/shadow
ss
stat
systemctl
systemctl list-jobs
systemctl list-sockets
systemctl list-unit-files
uname -a
uptime
w
who
XML
YAML
For more information on the motivations for creating
jc
, see my blog post.
Happy parsing!