Reading output of a command into an array in Bash

Question

I need to read the output of a command in my script into an array. The command is, for example:

ps aux | grep | grep | x 

and it gives the output line by line like this:

10
20
30

I need to read the values from the command output into an array, and then I will do some work if the size of the array is less than three.

Answer 1

The other answers will break if output of command contains spaces (which is rather frequent) or glob characters like *, ?, [...].

To get the output of a command in an array, with one line per element, there are essentially 3 ways:

1. With Bash≥4 use mapfile—it's the most efficient:

   mapfile -t my_array < <( my_command )
   

2. Otherwise, a loop reading the output (slower, but safe):

   my_array=()
   while IFS= read -r line; do
       my_array+=( "$line" )
   done < <( my_command )
   

3. As suggested by Charles Duffy in the comments (thanks!), the following might perform better than the loop method in number 2:

   IFS=$'
   ' read -r -d '' -a my_array < <( my_command && printf '' )
   

   Please make sure you use exactly this form, i.e., make sure you have the following:

   • IFS=$' ' on the same line as the read statement: this will only set the environment variable IFS for the read statement only. So it won't affect the rest of your script at all. The purpose of this variable is to tell read to break the stream at the EOL character .
   • -r: this is important. It tells read to not interpret the backslashes as escape sequences.
   • -d '': please note the space between the -d option and its argument ''. If you don't leave a space here, the '' will never be seen, as it will disappear in the quote removal step when Bash parses the statement. This tells read to stop reading at the nil byte. Some people write it as -d $'', but it is not really necessary. -d '' is better.
   • -a my_array tells read to populate the array my_array while reading the stream.
   • You must use the printf '' statement after my_command, so that read returns 0; it's actually not a big deal if you don't (you'll just get an return code 1, which is okay if you don't use set -e – which you shouldn't anyway), but just bear that in mind. It's cleaner and more semantically correct. Note that this is different from printf '', which doesn't output anything. printf '' prints a null byte, needed by read to happily stop reading there (remember the -d '' option?).

---

If you can, i.e., if you're sure your code will run on Bash≥4, use the first method. And you can see it's shorter too.

If you want to use read, the loop (method 2) might have an advantage over method 3 if you want to do some processing as the lines are read: you have direct access to it (via the $line variable in the example I gave), and you also have access to the lines already read (via the array ${my_array[@]} in the example I gave).

Note that mapfile provides a way to have a callback eval'd on each line read, and in fact you can even tell it to only call this callback every N lines read; have a look at help mapfile and the options -C and -c therein. (My opinion about this is that it's a little bit clunky, but can be used sometimes if you only have simple things to do — I don't really understand why this was even implemented in the first place!).

---

Now I'm going to tell you why the following method:

my_array=( $( my_command) )

is broken when there are spaces:

$ # I'm using this command to test:
$ echo "one two"; echo "three four"
one two
three four
$ # Now I'm going to use the broken method:
$ my_array=( $( echo "one two"; echo "three four" ) )
$ declare -p my_array
declare -a my_array='([0]="one" [1]="two" [2]="three" [3]="four")'
$ # As you can see, the fields are not the lines
$
$ # Now look at the correct method:
$ mapfile -t my_array < <(echo "one two"; echo "three four")
$ declare -p my_array
declare -a my_array='([0]="one two" [1]="three four")'
$ # Good!

Then some people will then recommend using IFS=$' ' to fix it:

$ IFS=$'
'
$ my_array=( $(echo "one two"; echo "three four") )
$ declare -p my_array
declare -a my_array='([0]="one two" [1]="three four")'
$ # It works!

But now let's use another command, with globs:

$ echo "* one two"; echo "[three four]"
* one two
[three four]
$ IFS=$'
'
$ my_array=( $(echo "* one two"; echo "[three four]") )
$ declare -p my_array
declare -a my_array='([0]="* one two" [1]="t")'
$ # What?

That's because I have a file called t in the current directory… and this filename is matched by the glob [three four]… at this point some people would recommend using set -f to disable globbing: but look at it: you have to change IFS and use set -f to be able to fix a broken technique (and you're not even fixing it really)! when doing that we're really fighting against the shell, not working with the shell.

$ mapfile -t my_array < <( echo "* one two"; echo "[three four]")
$ declare -p my_array
declare -a my_array='([0]="* one two" [1]="[three four]")'

here we're working with the shell!