Each: xargs for JSON, CSV, etc.

A better way of working with structured data on the command line

I wrote each to scratch a particular itch. I'm often working with JSON files or APIs which talk in JSON and using xargs to do something with that data. However xargs doesn't understand JSON, so I end up with horrible and fragile commands squeezing the data into the right shape, or I give up and write a shell script.

Here's a simple example:

[
	{
		"name": "Bart Simpson",
		"email": "bart@example.com"
	},
	{
		"name": "Homer Simpson",
		"email": "homer@example.com"
	}
]

Let's say we want to send an email to everyone in our list. We want to run something like the following command for each person:

echo 'Hello Bart, your pizza is ready.' | \
	mail -s 'PizzaBot update' bart@example.com

We could extract the emails from our JSON with grep and cut:

grep email < people.json | cut -d \" -f 4

We now have a list like:

bart@example.com
homer@example.com

So now we can pipe that into xargs and we're done, right?

grep email < people.json | \
	cut -d \" -f 4 | \
	xargs -n 1 mail -s 'PizzaBot update'

Except:

  1. This will fail for email addresses including quotes or whitespace
  2. How about the surprise person with "email" in their name?
  3. What if a future version of this file wasn't pretty printed, and had everything on one line?
  4. How do we construct the body of the email and supply it to mail on stdin?

If you're already familiar with the tool jq you've probably been grinding your teeth for a while already. It nicely solves problems 1-3:

jq '.[].email' < people.json | \
	xargs -n 1 mail -s 'PizzaBot update'

The 4th problem above is trickier and in this case, or if I had to supply multiple arguments for each person, I would usually give up and write a quick Python script.

This is really simple (and readable!) with each:

each -i people.json \
	--stdin 'Hello {{name}}, your pizza is ready.' \
	-- mail -s 'PizzaBot update' {{email}}

It will also happily read input from a pipe, so fits in well with other CLI tools like curl.

As a bonus it can also read CSV files so if you had a file like the following, exactly the same command would work:

name,email
Bart Simpson,bart@example.com
Homer Simpson,homer@example.com

You can find each on Github for more examples, installation instructions, and the full source code.