I thought it would be fun to show the power of using JSON for rapid development of Bash scripts. When you don’t need to parse unstructured command output and common string types manually you are freed to focus on the code logic which can really reduce development and troubleshooting cycles.
To that end, I created a toy CLI application that scans the local subnet to see which hosts respond to ICMP requests. This application took less than a half-hour to write (plus some fine-tuning) and I think it demonstrates some cool concepts utilizing the JSON command output of several commands, including ifconfig, ping, arp, date, and wc. We use jc to convert the command output to JSON and jq to grab the values we want from that output.
We will also be utilizing the ip-address parser that comes with jc. It acts like a subnet calculator but provides all of the values in a nice JSON schema for easy access with jq.
Yes, this is not as efficient as pinging the broadcast address and checking the local ARP table, but this is arguably more fun!
Here is the Bash script of around 80 lines. Because there is no low-level command output parsing via grep, cut, sed, awk, etc. it is pretty simple to understand, but we’ll go through some of the interesting parts in more detail.
Here is some sample output when using the script:
% ./scansubnet.sh
Please enter the interface name as command argument.
Options:
lo0
en0
% ./scansubnet.sh wrong-interface
ifconfig: interface wrong-interface does not exist
% ./scansubnet.sh lo0
Subnet is too large (16777214 IPs). Exiting.
% ./scansubnet.sh en0
My IP: 192.168.1.221/24
Sending ICMP requests to 254 IPs: 192.168.1.1 - 192.168.1.254
Start Time: 2022-08-29T07:05:40
13.623 ms 192.168.1.249 f0:ef:86:f6:21:84 camera1.local
350.634 ms 192.168.1.72 f0:18:98:3:f8:39 laptop1.local
10.645 ms 192.168.1.243 fc:ae:34:a1:35:82
561.997 ms 192.168.1.188 18:3e:ef:d3:3f:82 laptop2.local
19.775 ms 192.168.1.254 fc:ae:34:a1:3a:80 router.local
<snip>
27.917 ms 192.168.1.197 cc:a7:c1:5e:c3:f1 camera2.local
28.582 ms 192.168.1.235 56:c8:36:64:2a:8d camera3.local
38.199 ms 192.168.1.246 d8:30:62:2e:a6:cf extender.local
44.617 ms 192.168.1.242 50:14:79:1e:42:3e vacuum.local
5.350 ms 192.168.1.88 c8:d0:83:cd:f4:2d tv.local
0.087 ms 192.168.1.221 a4:83:e7:2d:62:4e laptop3.local
Scanned 192.168.1.0/24 subnet in 27 seconds.
30 alive hosts found.
End Time: 2022-08-29T07:06:07
Note: For best results, use
jcversion 1.21.1 or higher in this script.
Notice there are some basic checks and features:
- If you don’t specify an interface it will suggest any interfaces on the system with a configured IPv4 address
- If you select an interface attached to too large of a subnet (say a 127.0.0.1/8 address of a loopback) it will exit
- It will show the subnet information, including the number of hosts it will scan and the range of IP addresses to be scanned
- It will log the start and end time and calculate how long it took to complete the scan
- It will run the pings in the background for parallel processing
- It will report the round-trip time, IP, MAC address, and name (if known) of the hosts that respond
Let’s take a closer look!
Displaying valid interface options
We could make the user figure out the valid interface names that can be used as the only command argument. Instead, we can use the ifconfig output passed through jc and a simple jq query to print valid options (that is, interfaces with an IPv4 address configured):
if [[ $1 == "" ]]; then
echo "Please enter the interface name as command argument."
echo
echo "Options:"
# Only show interfaces with an assigned IP address
jc ifconfig | jq -r '.[] | select(.ipv4_addr != null) | .name'
exit 1
fi
Once jc converts the ifconfig output to JSON we pipe it to jq so it will filter only items with an ipv4_addr field that is not set to null. Pretty straightforward and easy to read.
Note: We could have gotten the same result by using the
ipcommand with the JSON output option instead of parsing theifconfigoutput to JSON viajc, but this allows the script to be more cross-platform with macOS, BSD, etc.
Grab the selected interface IP and subnet mask
Once the user selects a valid interface, let’s grab the IP and Subnet:
interfaceInfo=$(jc ifconfig "$1") || exit 1 ip=$(jq -r '.[0].ipv4_addr' <<<"$interfaceInfo") mask=$(jq -r '.[0].ipv4_mask' <<<"$interfaceInfo")
Here you can see we parse the ifconfig output to JSON with jc in the first line. If the interface name from the user is invalid, ifconfig will give us a non-zero exit code and quit the program with a helpful message. Then we do two variable assignments to pull the IP address and subnet mask via jq queries.
Grab detailed subnet information for the IP/Mask
Next we take the Interface IP and Mask values and use the IP Address string parser in jc to gather more detailed subnet information we will use later in the script.
ipInfo=$(jc --ip-address <<<"$ip/$mask") network=$(jq -r '.network' <<<"$ipInfo") numHosts=$(jq -r '.hosts' <<<"$ipInfo") cidrMask=$(jq -r '.cidr_netmask' <<<"$ipInfo") firstHostIp=$(jq -r '.first_host' <<<"$ipInfo") lastHostIp=$(jq -r '.last_host' <<<"$ipInfo") firstHost=$(jq -r '.int.first_host' <<<"$ipInfo") lastHost=$(jq -r '.int.last_host' <<<"$ipInfo")
The IP Address parser in jc is nice because it acts like a subnet calculator and gives us a lot of data, including the subnet, number of hosts in the subnet, first host, and last host in different formats (decimal, hex, binary, and standard format). This will help us build a simple for loop that will do most of the work. All we need to do is pick the fields we want with jq. Here is an example of all of the information available with the jc IP Address string parser (IPv6 is also supported):
% echo 192.168.1.10/255.255.255.0 | jc --ip-address -p
{
"version": 4,
"max_prefix_length": 32,
"ip": "192.168.1.10",
"ip_compressed": "192.168.1.10",
"ip_exploded": "192.168.1.10",
"scope_id": null,
"ipv4_mapped": null,
"six_to_four": null,
"teredo_client": null,
"teredo_server": null,
"dns_ptr": "10.1.168.192.in-addr.arpa",
"network": "192.168.1.0",
"broadcast": "192.168.1.255",
"hostmask": "0.0.0.255",
"netmask": "255.255.255.0",
"cidr_netmask": 24,
"hosts": 254,
"first_host": "192.168.1.1",
"last_host": "192.168.1.254",
"is_multicast": false,
"is_private": true,
"is_global": false,
"is_link_local": false,
"is_loopback": false,
"is_reserved": false,
"is_unspecified": false,
"int": {
"ip": 3232235786,
"network": 3232235776,
"broadcast": 3232236031,
"first_host": 3232235777,
"last_host": 3232236030
},
"hex": {
"ip": "c0:a8:01:0a",
"network": "c0:a8:01:00",
"broadcast": "c0:a8:01:ff",
"hostmask": "00:00:00:ff",
"netmask": "ff:ff:ff:00",
"first_host": "c0:a8:01:01",
"last_host": "c0:a8:01:fe"
},
"bin": {
"ip": "11000000101010000000000100001010",
"network": "11000000101010000000000100000000",
"broadcast": "11000000101010000000000111111111",
"hostmask": "00000000000000000000000011111111",
"netmask": "11111111111111111111111100000000",
"first_host": "11000000101010000000000100000001",
"last_host": "11000000101010000000000111111110"
}
}
In addition to accepting a CIDR subnet mask, the jc IP Address string parser also accepts a dotted-quad subnet mask (that’s how ifconfig gives it to us) and provides us the CIDR notation for it in the cidr_netmask field. The IP Address string parser is fairly liberal in the IP formats it will accept.
We’ll see how having the first_host and last_host values in decimal makes for easy looping later.
Sanity check the subnet size
We can use the hosts value from the jc IP Address string parser to see if the subnet is a suitable size to scan. If the subnet supports any more than 1022 hosts (/22) then we don’t want to bother spinning up that many ping processes in the background for the scan. The following code does that sanity check for us:
if [[ $numHosts -gt 1022 ]]; then
echo "Subnet is too large ($numHosts IPs). Exiting."
exit 1
fi
Grab the start time in ISO and Unix Epoch format
Next we want to grab the start time. The date command parser in jc gives us the current time in ISO and Epoch formats that we can easily pull with jq. This allows us to display the time in a nice, standard human readable format and also have the date-time information in an easy-to-use format for calculating the duration later.
startTime=$(jc date) startTimeIso=$(jq -r '.iso' <<<"$startTime") startTimeEpoch=$(jq -r '.epoch' <<<"$startTime")
Here are all of the fields available when running the date command through jc:
% jc -p date
{
"year": 2022,
"month": "Aug",
"month_num": 8,
"day": 28,
"weekday": "Sun",
"weekday_num": 7,
"hour": 1,
"hour_24": 13,
"minute": 39,
"second": 20,
"period": "PM",
"timezone": "PDT",
"utc_offset": null,
"day_of_year": 240,
"week_of_year": 34,
"iso": "2022-08-28T13:39:20",
"epoch": 1661719160,
"epoch_utc": null,
"timezone_aware": false
}
Show the user what is going to happen
Next we use a series of simple echo commands to provide the subnet and time information back to the user before the scan:
echo "My IP: $ip/$cidrMask" echo "Sending ICMP requests to $numHosts IPs: $firstHostIp - $lastHostIp" echo "Start Time: $startTimeIso"
The main loop
Now comes the fun part – here is the main loop where we ping every host in the subnet and record the round-trip time, IP, MAC address, and name of each host that responds:
for (( ipDecimal=firstHost; ipDecimal<=lastHost; ipDecimal++ )); do
# Do each ping in the background for parallel processing
{
# grab the packets received and rtt values from the ping output
thisIp=$(jc --ip-address <<<"$ipDecimal" | jq -r '.ip')
pingResult=$(ping -c1 "$thisIp" | jc --ping)
packetsReceived=$(jq -r '.packets_received' <<<"$pingResult")
rtTime=$(jq -r '.round_trip_ms_max' <<<"$pingResult")
if [[ $packetsReceived -gt 0 ]]; then
# Grab the MAC address and name for each alive host from the arp command
thisIpArpInfo=$(arp -a | jc --arp | jq --arg myip "$thisIp" '.[] | select(.address == $myip)')
thisIpMac=$(jq -r '.hwaddress // empty' <<<"$thisIpArpInfo")
thisIpName=$(jq -r '.name // empty' <<<"$thisIpArpInfo")
printf "%9.3f ms %-16s%-20s%s\n" "$rtTime" "$thisIp" "$thisIpMac" "$thisIpName" | tee -a "$tempFile"
fi
} &
done
wait
Let’s break this down a little bit:
for (( ipDecimal=$firstHost; ipDecimal<=$lastHost; ipDecimal++ )); do
We use a C-style for loop which allows us to use those decimal versions of the first and last host IP addresses. I told you those decimal values would come in handy!
thisIp=$(jc --ip-address <<<"$ipDecimal" | jq -r '.ip')
pingResult=$(ping -c1 "$thisIp" | jc --ping)
packetsReceived=$(jq -r '.packets_received' <<<"$pingResult")
rtTime=$(jq -r '.round_trip_ms_max' <<<"$pingResult")
The decimal IP format is nice to loop over, but unfortunately the ping and arp commands do not seem to accept IP addresses in decimal format (at least not on all platforms). Not to worry – we simply send the decimal IP address to the jc IP Address string parser and it will tell us what the IP address is in standard dotted-quad notation.
Then we give ping that IP address and parse its output with jc. We only care about the packets_received and round_trip_ms_max fields, so we assign them to Bash variables.
Next, let’s take a look at the if block:
if [[ $packetsReceived -gt 0 ]]; then
# Grab the MAC address and name for each alive host from the arp command
thisIpArpInfo=$(arp -a | jc --arp | jq --arg myip "$thisIp" '.[] | select(.address == $myip)')
thisIpMac=$(jq -r '.hwaddress // empty' <<<"$thisIpArpInfo")
thisIpName=$(jq -r '.name // empty' <<<"$thisIpArpInfo")
printf "%9.3f ms %-16s%-20s%s\n" "$rtTime" "$thisIp" "$thisIpMac" "$thisIpName" | tee -a "$tempFile"
fi
There’s a bit going on in this if block:
- We only run the below commands if there was an ICMP reply from the
pingoutput - Since we got an ICMP reply, we check the ARP table via the
arp -acommand and filter for the current IP address’ MAC address and name. Havingjcparse thearp-aoutput into JSON allows us to use a simplejqquery to accomplish this. - Notice the use of the
--argoption injqthat allows us to use the$thisIpvalue in the query. - Notice the
jq -r '.name // empty'section. This tellsjqto output an empty string if it sees anullvalue forname. - We use the
printfcommand with string format specifications to print our output in nice, even columns. - The
teecommand copies what is printed to the screen and appends it to a temporary file that we will use later.
} & done wait
The & at the end of the Bash command grouping tells Bash to run all of the commands enclosed in {} brackets in the background, so we get parallel processing. The wait command tells bash to pause until all of the background processes complete.
Grab the end time
After all of the background ping and arp processes return, we can grab the end time by parsing the date command with jc and returning the iso and epoch values:
endTime=$(jc date) endTimeIso=$(jq -r '.iso' <<<"$endTime") endTimeEpoch=$(jq -r '.epoch' <<<"$endTime") totalTime=$((endTimeEpoch-startTimeEpoch))
Then we subtract the epoch values to get the total run time.
Grab the number of alive hosts
We can run the temporary file through wc to get the number of lines. The wc parser in jc makes it easy to pull the number of lines with a quick jq query. Then we delete the temporary file.
totalAlive=$(jc wc "$tempFile" | jq '.[0].lines') rm "$tempFile"
Print the summary message
Finally, we print a summary message with the total run-time, subnet information, number of alive hosts, and the human-readable end time:
echo "Scanned $network/$cidrMask subnet in $totalTime seconds." echo "$totalAlive alive hosts found." echo "End Time: $endTimeIso"
Conclusion
There you go – that was a pretty fun exercise demonstrating how you can rapidly develop a prototype in Bash using the output of existing commands on the system without needing to manually parse them. By using jq to query the JSON output from the commands and jc the script becomes very easy to understand. It’s nearly self documenting!
Let me know if you have built any cool scripts or programs with jc and jq!