I thought it would be fun to show the power of using JSON for rapid development of Bash scripts. When you don’t need to parse unstructured command output and common string types manually you are freed to focus on the code logic which can really reduce development and troubleshooting cycles.
To that end, I created a toy CLI application that scans the local subnet to see which hosts respond to ICMP requests. This application took less than a half-hour to write (plus some fine-tuning) and I think it demonstrates some cool concepts utilizing the JSON command output of several commands, including ifconfig
, ping
, arp
, date
, and wc
. We use jc
to convert the command output to JSON and jq
to grab the values we want from that output.
We will also be utilizing the ip-address
parser that comes with jc
. It acts like a subnet calculator but provides all of the values in a nice JSON schema for easy access with jq
.
Yes, this is not as efficient as pinging the broadcast address and checking the local ARP table, but this is arguably more fun!
Here is the Bash script of around 80 lines. Because there is no low-level command output parsing via grep
, cut
, sed
, awk
, etc. it is pretty simple to understand, but we’ll go through some of the interesting parts in more detail.
Here is some sample output when using the script:
% ./scansubnet.sh Please enter the interface name as command argument. Options: lo0 en0 % ./scansubnet.sh wrong-interface ifconfig: interface wrong-interface does not exist % ./scansubnet.sh lo0 Subnet is too large (16777214 IPs). Exiting. % ./scansubnet.sh en0 My IP: 192.168.1.221/24 Sending ICMP requests to 254 IPs: 192.168.1.1 - 192.168.1.254 Start Time: 2022-08-29T07:05:40 13.623 ms 192.168.1.249 f0:ef:86:f6:21:84 camera1.local 350.634 ms 192.168.1.72 f0:18:98:3:f8:39 laptop1.local 10.645 ms 192.168.1.243 fc:ae:34:a1:35:82 561.997 ms 192.168.1.188 18:3e:ef:d3:3f:82 laptop2.local 19.775 ms 192.168.1.254 fc:ae:34:a1:3a:80 router.local <snip> 27.917 ms 192.168.1.197 cc:a7:c1:5e:c3:f1 camera2.local 28.582 ms 192.168.1.235 56:c8:36:64:2a:8d camera3.local 38.199 ms 192.168.1.246 d8:30:62:2e:a6:cf extender.local 44.617 ms 192.168.1.242 50:14:79:1e:42:3e vacuum.local 5.350 ms 192.168.1.88 c8:d0:83:cd:f4:2d tv.local 0.087 ms 192.168.1.221 a4:83:e7:2d:62:4e laptop3.local Scanned 192.168.1.0/24 subnet in 27 seconds. 30 alive hosts found. End Time: 2022-08-29T07:06:07
Note: For best results, use
jc
version 1.21.1 or higher in this script.
Notice there are some basic checks and features:
- If you don’t specify an interface it will suggest any interfaces on the system with a configured IPv4 address
- If you select an interface attached to too large of a subnet (say a 127.0.0.1/8 address of a loopback) it will exit
- It will show the subnet information, including the number of hosts it will scan and the range of IP addresses to be scanned
- It will log the start and end time and calculate how long it took to complete the scan
- It will run the pings in the background for parallel processing
- It will report the round-trip time, IP, MAC address, and name (if known) of the hosts that respond
Let’s take a closer look!
Displaying valid interface options
We could make the user figure out the valid interface names that can be used as the only command argument. Instead, we can use the ifconfig
output passed through jc
and a simple jq
query to print valid options (that is, interfaces with an IPv4 address configured):
if [[ $1 == "" ]]; then echo "Please enter the interface name as command argument." echo echo "Options:" # Only show interfaces with an assigned IP address jc ifconfig | jq -r '.[] | select(.ipv4_addr != null) | .name' exit 1 fi
Once jc
converts the ifconfig
output to JSON we pipe it to jq
so it will filter only items with an ipv4_addr
field that is not set to null
. Pretty straightforward and easy to read.
Note: We could have gotten the same result by using the
ip
command with the JSON output option instead of parsing theifconfig
output to JSON viajc
, but this allows the script to be more cross-platform with macOS, BSD, etc.
Grab the selected interface IP and subnet mask
Once the user selects a valid interface, let’s grab the IP and Subnet:
interfaceInfo=$(jc ifconfig "$1") || exit 1 ip=$(jq -r '.[0].ipv4_addr' <<<"$interfaceInfo") mask=$(jq -r '.[0].ipv4_mask' <<<"$interfaceInfo")
Here you can see we parse the ifconfig
output to JSON with jc
in the first line. If the interface name from the user is invalid, ifconfig
will give us a non-zero exit code and quit the program with a helpful message. Then we do two variable assignments to pull the IP address and subnet mask via jq
queries.
Grab detailed subnet information for the IP/Mask
Next we take the Interface IP and Mask values and use the IP Address string parser in jc
to gather more detailed subnet information we will use later in the script.
ipInfo=$(jc --ip-address <<<"$ip/$mask") network=$(jq -r '.network' <<<"$ipInfo") numHosts=$(jq -r '.hosts' <<<"$ipInfo") cidrMask=$(jq -r '.cidr_netmask' <<<"$ipInfo") firstHostIp=$(jq -r '.first_host' <<<"$ipInfo") lastHostIp=$(jq -r '.last_host' <<<"$ipInfo") firstHost=$(jq -r '.int.first_host' <<<"$ipInfo") lastHost=$(jq -r '.int.last_host' <<<"$ipInfo")
The IP Address parser in jc
is nice because it acts like a subnet calculator and gives us a lot of data, including the subnet, number of hosts in the subnet, first host, and last host in different formats (decimal, hex, binary, and standard format). This will help us build a simple for
loop that will do most of the work. All we need to do is pick the fields we want with jq
. Here is an example of all of the information available with the jc
IP Address string parser (IPv6 is also supported):
% echo 192.168.1.10/255.255.255.0 | jc --ip-address -p { "version": 4, "max_prefix_length": 32, "ip": "192.168.1.10", "ip_compressed": "192.168.1.10", "ip_exploded": "192.168.1.10", "scope_id": null, "ipv4_mapped": null, "six_to_four": null, "teredo_client": null, "teredo_server": null, "dns_ptr": "10.1.168.192.in-addr.arpa", "network": "192.168.1.0", "broadcast": "192.168.1.255", "hostmask": "0.0.0.255", "netmask": "255.255.255.0", "cidr_netmask": 24, "hosts": 254, "first_host": "192.168.1.1", "last_host": "192.168.1.254", "is_multicast": false, "is_private": true, "is_global": false, "is_link_local": false, "is_loopback": false, "is_reserved": false, "is_unspecified": false, "int": { "ip": 3232235786, "network": 3232235776, "broadcast": 3232236031, "first_host": 3232235777, "last_host": 3232236030 }, "hex": { "ip": "c0:a8:01:0a", "network": "c0:a8:01:00", "broadcast": "c0:a8:01:ff", "hostmask": "00:00:00:ff", "netmask": "ff:ff:ff:00", "first_host": "c0:a8:01:01", "last_host": "c0:a8:01:fe" }, "bin": { "ip": "11000000101010000000000100001010", "network": "11000000101010000000000100000000", "broadcast": "11000000101010000000000111111111", "hostmask": "00000000000000000000000011111111", "netmask": "11111111111111111111111100000000", "first_host": "11000000101010000000000100000001", "last_host": "11000000101010000000000111111110" } }
In addition to accepting a CIDR subnet mask, the jc
IP Address string parser also accepts a dotted-quad subnet mask (that’s how ifconfig
gives it to us) and provides us the CIDR notation for it in the cidr_netmask
field. The IP Address string parser is fairly liberal in the IP formats it will accept.
We’ll see how having the first_host
and last_host
values in decimal makes for easy looping later.
Sanity check the subnet size
We can use the hosts
value from the jc
IP Address string parser to see if the subnet is a suitable size to scan. If the subnet supports any more than 1022 hosts (/22
) then we don’t want to bother spinning up that many ping
processes in the background for the scan. The following code does that sanity check for us:
if [[ $numHosts -gt 1022 ]]; then echo "Subnet is too large ($numHosts IPs). Exiting." exit 1 fi
Grab the start time in ISO and Unix Epoch format
Next we want to grab the start time. The date
command parser in jc
gives us the current time in ISO and Epoch formats that we can easily pull with jq
. This allows us to display the time in a nice, standard human readable format and also have the date-time information in an easy-to-use format for calculating the duration later.
startTime=$(jc date) startTimeIso=$(jq -r '.iso' <<<"$startTime") startTimeEpoch=$(jq -r '.epoch' <<<"$startTime")
Here are all of the fields available when running the date command through jc
:
% jc -p date { "year": 2022, "month": "Aug", "month_num": 8, "day": 28, "weekday": "Sun", "weekday_num": 7, "hour": 1, "hour_24": 13, "minute": 39, "second": 20, "period": "PM", "timezone": "PDT", "utc_offset": null, "day_of_year": 240, "week_of_year": 34, "iso": "2022-08-28T13:39:20", "epoch": 1661719160, "epoch_utc": null, "timezone_aware": false }
Show the user what is going to happen
Next we use a series of simple echo
commands to provide the subnet and time information back to the user before the scan:
echo "My IP: $ip/$cidrMask" echo "Sending ICMP requests to $numHosts IPs: $firstHostIp - $lastHostIp" echo "Start Time: $startTimeIso"
The main loop
Now comes the fun part – here is the main loop where we ping every host in the subnet and record the round-trip time, IP, MAC address, and name of each host that responds:
for (( ipDecimal=firstHost; ipDecimal<=lastHost; ipDecimal++ )); do # Do each ping in the background for parallel processing { # grab the packets received and rtt values from the ping output thisIp=$(jc --ip-address <<<"$ipDecimal" | jq -r '.ip') pingResult=$(ping -c1 "$thisIp" | jc --ping) packetsReceived=$(jq -r '.packets_received' <<<"$pingResult") rtTime=$(jq -r '.round_trip_ms_max' <<<"$pingResult") if [[ $packetsReceived -gt 0 ]]; then # Grab the MAC address and name for each alive host from the arp command thisIpArpInfo=$(arp -a | jc --arp | jq --arg myip "$thisIp" '.[] | select(.address == $myip)') thisIpMac=$(jq -r '.hwaddress // empty' <<<"$thisIpArpInfo") thisIpName=$(jq -r '.name // empty' <<<"$thisIpArpInfo") printf "%9.3f ms %-16s%-20s%s\n" "$rtTime" "$thisIp" "$thisIpMac" "$thisIpName" | tee -a "$tempFile" fi } & done wait
Let’s break this down a little bit:
for (( ipDecimal=$firstHost; ipDecimal<=$lastHost; ipDecimal++ )); do
We use a C-style for loop which allows us to use those decimal versions of the first and last host IP addresses. I told you those decimal values would come in handy!
thisIp=$(jc --ip-address <<<"$ipDecimal" | jq -r '.ip') pingResult=$(ping -c1 "$thisIp" | jc --ping) packetsReceived=$(jq -r '.packets_received' <<<"$pingResult") rtTime=$(jq -r '.round_trip_ms_max' <<<"$pingResult")
The decimal IP format is nice to loop over, but unfortunately the ping
and arp
commands do not seem to accept IP addresses in decimal format (at least not on all platforms). Not to worry – we simply send the decimal IP address to the jc
IP Address string parser and it will tell us what the IP address is in standard dotted-quad notation.
Then we give ping
that IP address and parse its output with jc
. We only care about the packets_received
and round_trip_ms_max
fields, so we assign them to Bash variables.
Next, let’s take a look at the if
block:
if [[ $packetsReceived -gt 0 ]]; then # Grab the MAC address and name for each alive host from the arp command thisIpArpInfo=$(arp -a | jc --arp | jq --arg myip "$thisIp" '.[] | select(.address == $myip)') thisIpMac=$(jq -r '.hwaddress // empty' <<<"$thisIpArpInfo") thisIpName=$(jq -r '.name // empty' <<<"$thisIpArpInfo") printf "%9.3f ms %-16s%-20s%s\n" "$rtTime" "$thisIp" "$thisIpMac" "$thisIpName" | tee -a "$tempFile" fi
There’s a bit going on in this if
block:
- We only run the below commands if there was an ICMP reply from the
ping
output - Since we got an ICMP reply, we check the ARP table via the
arp -a
command and filter for the current IP address’ MAC address and name. Havingjc
parse thearp-a
output into JSON allows us to use a simplejq
query to accomplish this. - Notice the use of the
--arg
option injq
that allows us to use the$thisIp
value in the query. - Notice the
jq -r '.name // empty'
section. This tellsjq
to output an empty string if it sees anull
value forname
. - We use the
printf
command with string format specifications to print our output in nice, even columns. - The
tee
command copies what is printed to the screen and appends it to a temporary file that we will use later.
} & done wait
The &
at the end of the Bash command grouping tells Bash to run all of the commands enclosed in {}
brackets in the background, so we get parallel processing. The wait
command tells bash to pause until all of the background processes complete.
Grab the end time
After all of the background ping
and arp
processes return, we can grab the end time by parsing the date
command with jc
and returning the iso
and epoch
values:
endTime=$(jc date) endTimeIso=$(jq -r '.iso' <<<"$endTime") endTimeEpoch=$(jq -r '.epoch' <<<"$endTime") totalTime=$((endTimeEpoch-startTimeEpoch))
Then we subtract the epoch values to get the total run time.
Grab the number of alive hosts
We can run the temporary file through wc
to get the number of lines. The wc
parser in jc
makes it easy to pull the number of lines with a quick jq
query. Then we delete the temporary file.
totalAlive=$(jc wc "$tempFile" | jq '.[0].lines') rm "$tempFile"
Print the summary message
Finally, we print a summary message with the total run-time, subnet information, number of alive hosts, and the human-readable end time:
echo "Scanned $network/$cidrMask subnet in $totalTime seconds." echo "$totalAlive alive hosts found." echo "End Time: $endTimeIso"
Conclusion
There you go – that was a pretty fun exercise demonstrating how you can rapidly develop a prototype in Bash using the output of existing commands on the system without needing to manually parse them. By using jq
to query the JSON output from the commands and jc
the script becomes very easy to understand. It’s nearly self documenting!
Let me know if you have built any cool scripts or programs with jc
and jq
!