Try the jc web demo!
I’m excited to announce the release of jc version 1.8.0 available on github and pypi. See below for more information on the new features and parsers.
To upgrade, run:
$ pip3 install --upgrade jc
New Parsers
jc now includes 45 parsers! New parsers (tested on linux and OSX) include blkid, last, lastb, who, /etc/passwd files, /etc/shadow files, /etc/group files, /etc/gshadow files, and CSV files.
Documentation and schemas for all parsers can be found here.
blkid command parser
Linux support for the blkid command:
$ blkid | jc --blkid -p # or: jc -p blkid
[
{
"device": "/dev/sda1",
"uuid": "05d927ab-5875-49e4-ada1-7f46cb32c932",
"type": "xfs"
},
{
"device": "/dev/sda2",
"uuid": "3klkIj-w1kk-DkJi-0XBJ-y3i7-i2Ac-vHqWBM",
"type": "LVM2_member"
},
{
"device": "/dev/mapper/centos-root",
"uuid": "07d718ff-950c-4e5b-98f0-42a1147c77d9",
"type": "xfs"
},
{
"device": "/dev/mapper/centos-swap",
"uuid": "615eb89a-bcbf-46fd-80e3-c483ff5c931f",
"type": "swap"
}
]
$ sudo blkid -o udev -ip /dev/sda2 | jc --blkid -p # or: sudo jc -p blkid -o udev -ip /dev/sda2
[
{
"id_fs_uuid": "3klkIj-w1kk-DkJi-0XBJ-y3i7-i2Ac-vHqWBM",
"id_fs_uuid_enc": "3klkIj-w1kk-DkJi-0XBJ-y3i7-i2Ac-vHqWBM",
"id_fs_version": "LVM2\x20001",
"id_fs_type": "LVM2_member",
"id_fs_usage": "raid",
"id_iolimit_minimum_io_size": 512,
"id_iolimit_physical_sector_size": 512,
"id_iolimit_logical_sector_size": 512,
"id_part_entry_scheme": "dos",
"id_part_entry_type": "0x8e",
"id_part_entry_number": 2,
"id_part_entry_offset": 2099200,
"id_part_entry_size": 39843840,
"id_part_entry_disk": "8:0"
}
]
last and lastb command parsers
Linux and OSX support for the last command. Linux support for the lastb command.
$ last | jc --last -p # or: jc -p last
[
{
"user": "joeuser",
"tty": "ttys002",
"hostname": null,
"login": "Thu Feb 27 14:31",
"logout": "still logged in"
},
{
"user": "joeuser",
"tty": "ttys003",
"hostname": null,
"login": "Thu Feb 27 10:38",
"logout": "10:38",
"duration": "00:00"
},
{
"user": "joeuser",
"tty": "ttys003",
"hostname": null,
"login": "Thu Feb 27 10:18",
"logout": "10:18",
"duration": "00:00"
},
...
]
$ sudo lastb | jc --last -p # or: sudo jc -p lastb
[
{
"user": "joeuser",
"tty": "ssh:notty",
"hostname": "127.0.0.1",
"login": "Tue Mar 3 00:48",
"logout": "00:48",
"duration": "00:00"
},
{
"user": "joeuser",
"tty": "ssh:notty",
"hostname": "127.0.0.1",
"login": "Tue Mar 3 00:48",
"logout": "00:48",
"duration": "00:00"
},
{
"user": "jouser",
"tty": "ssh:notty",
"hostname": "127.0.0.1",
"login": "Tue Mar 3 00:48",
"logout": "00:48",
"duration": "00:00"
}
]
who command parser
Linux and OSX support for the who command:
$ who | jc --who -p # or: jc -p who
[
{
"user": "joeuser",
"tty": "ttyS0",
"time": "2020-03-02 02:52"
},
{
"user": "joeuser",
"tty": "pts/0",
"time": "2020-03-02 05:15",
"from": "192.168.71.1"
}
]
$ who -a | jc --who -p # or: jc -p who -a
[
{
"event": "reboot",
"time": "Feb 7 23:31",
"pid": 1
},
{
"user": "joeuser",
"writeable_tty": "-",
"tty": "console",
"time": "Feb 7 23:32",
"idle": "old",
"pid": 105
},
{
"user": "joeuser",
"writeable_tty": "+",
"tty": "ttys000",
"time": "Feb 13 16:44",
"idle": ".",
"pid": 51217,
"comment": "term=0 exit=0"
},
{
"user": "joeuser",
"writeable_tty": "?",
"tty": "ttys003",
"time": "Feb 28 08:59",
"idle": "01:36",
"pid": 41402
},
{
"user": "joeuser",
"writeable_tty": "+",
"tty": "ttys004",
"time": "Mar 1 16:35",
"idle": ".",
"pid": 15679,
"from": "192.168.1.5"
}
]
CSV File Parser
Convert generic CSV files to JSON. The parser will attempt to automatically detect the delimiter character. If it cannot detect the delimiter it will use the comma (‘,‘) as the delimiter. The file must contain a header row as the first line:
$ cat homes.csv
"Sell", "List", "Living", "Rooms", "Beds", "Baths", "Age", "Acres", "Taxes"
142, 160, 28, 10, 5, 3, 60, 0.28, 3167
175, 180, 18, 8, 4, 1, 12, 0.43, 4033
129, 132, 13, 6, 3, 1, 41, 0.33, 1471
...
$ cat homes.csv | jc --csv -p
[
{
"Sell": "142",
"List": "160",
"Living": "28",
"Rooms": "10",
"Beds": "5",
"Baths": "3",
"Age": "60",
"Acres": "0.28",
"Taxes": "3167"
},
{
"Sell": "175",
"List": "180",
"Living": "18",
"Rooms": "8",
"Beds": "4",
"Baths": "1",
"Age": "12",
"Acres": "0.43",
"Taxes": "4033"
},
{
"Sell": "129",
"List": "132",
"Living": "13",
"Rooms": "6",
"Beds": "3",
"Baths": "1",
"Age": "41",
"Acres": "0.33",
"Taxes": "1471"
},
...
]
/etc/passwd, /etc/shadow, /etc/group, and /etc/gshadow file parsers
Convert /etc/passwd, /etc/shadow, /etc/group, and /etc/gshadow files to JSON format:
$ cat /etc/passwd | jc --passwd -p
[
{
"username": "nobody",
"password": "*",
"uid": -2,
"gid": -2,
"comment": "Unprivileged User",
"home": "/var/empty",
"shell": "/usr/bin/false"
},
{
"username": "root",
"password": "*",
"uid": 0,
"gid": 0,
"comment": "System Administrator",
"home": "/var/root",
"shell": "/bin/sh"
},
{
"username": "daemon",
"password": "*",
"uid": 1,
"gid": 1,
"comment": "System Services",
"home": "/var/root",
"shell": "/usr/bin/false"
},
...
]
$ sudo cat /etc/shadow | jc --shadow -p
[
{
"username": "root",
"password": "*",
"last_changed": 18113,
"minimum": 0,
"maximum": 99999,
"warn": 7,
"inactive": null,
"expire": null
},
{
"username": "daemon",
"password": "*",
"last_changed": 18113,
"minimum": 0,
"maximum": 99999,
"warn": 7,
"inactive": null,
"expire": null
},
{
"username": "bin",
"password": "*",
"last_changed": 18113,
"minimum": 0,
"maximum": 99999,
"warn": 7,
"inactive": null,
"expire": null
},
...
]
$ cat /etc/group | jc --group -p
[
{
"group_name": "nobody",
"password": "*",
"gid": -2,
"members": []
},
{
"group_name": "nogroup",
"password": "*",
"gid": -1,
"members": []
},
{
"group_name": "wheel",
"password": "*",
"gid": 0,
"members": [
"root"
]
},
{
"group_name": "certusers",
"password": "*",
"gid": 29,
"members": [
"root",
"_jabber",
"_postfix",
"_cyrus",
"_calendar",
"_dovecot"
]
},
...
]
$ cat /etc/gshadow | jc --gshadow -p
[
{
"group_name": "root",
"password": "*",
"administrators": [],
"members": []
},
{
"group_name": "adm",
"password": "*",
"administrators": [],
"members": [
"syslog",
"joeuser"
]
},
...
]
Updated Parsers
- The
ls parser now supports filenames that contain newline characters when using ls -l or ls -b. A warning message will be sent to stderr if newlines are detected and ls -l or ls -b are not used:
$ ls | jc --ls
jc: Warning - Newline characters detected. Filenames probably corrupted. Use ls -l or -b instead.
[{"filename": "this file has"}, {"filename": "a newline inside"}, {"filename": "this file has"}, {"filename": "four contiguous newlines inside"}, ...]
- The
ls parser now supports multiple directory listings, globbing, and recursive listings.
$ ls -R | jc --ls
[{"filename": "centos-7.7"}, {"filename": "create_fixtures.sh"}, {"filename": "generic"}, {"filename": "osx-10.11.6"}, {"filename": "osx-10.14.6"}, ...]
Alternative “Magic” Syntax
jc now accepts a simplified syntax for most command parsers. Instead of piping the data into jc you can now also prepend “jc” to the command you would like to convert. Note that command aliases are not supported:
$ jc dig www.example.com
[{"id": 31113, "opcode": "QUERY", "status": "NOERROR", "flags": ["qr", "rd", "ra"], "query_num": 1, "answer_num": 1, "authority_num": 0, "additional_num": 1, "question": {"name": "www.example.com.", "class": "IN", "type": "A"}, "answer": [{"name": "www.example.com.", "class": "IN", "type": "A", "ttl": 35366, "data": "93.184.216.34"}], "query_time": 37, "server": "2600", "when": "Mon Mar 02 16:13:31 PST 2020", "rcvd": 60}]
You can also insert jc options before the command:
$ jc -pqd dig www.example.com
[
{
"id": 7495,
"opcode": "QUERY",
"status": "NOERROR",
"flags": [
"qr",
"rd",
"ra"
],
"query_num": 1,
"answer_num": 1,
"authority_num": 0,
"additional_num": 1,
"question": {
"name": "www.example.com.",
"class": "IN",
"type": "A"
},
"answer": [
{
"name": "www.example.com.",
"class": "IN",
"type": "A",
"ttl": 36160,
"data": "93.184.216.34"
}
],
"query_time": 40,
"server": "2600",
"when": "Mon Mar 02 16:15:21 PST 2020",
"rcvd": 60
}
]
Schema Changes
There are no schema changes in this release.
Full Parser List
arpblkidcrontabcrontab-uCSVdfdigduenvfreefstab/etc/group/etc/gshadowhistory/etc/hostsidifconfigINIiptablesjobslast and lastblslsblklsmodlsofmountnetstat/etc/passwdpip listpip showpsroute/etc/shadowssstatsystemctlsystemctl list-jobssystemctl list-socketssystemctl list-unit-filesuname -auptimewwhoXMLYAML
For more information on the motivations for creating jc, see my blog post.
Happy parsing!
