bash - Use Pipeline Viewer to show progress for uncompression of a list of processed files

Question

This is a follow-up question to this question.

I have a list of compressed *.bz files whose content I push through a pipe as follows:

result=$(find . -name '*bz2' -exec bzcat {} + 
        | tee >( some | other | pipeline ) 
        | grep -e "myString" 
        | wc -l)
echo "${result} occurrences found"

This will process the decompressed contents of the files with a sub-pipeline (some | other | pipeline) and at the same time count and return the occurrences of the string myString.

The pipeline works but now I want to display a progress bar using Pipeline Viewer. The progress should be shown as the number of already processed *.bz2 files.

Answer 1

If you care more about measuring the speed of find than the speed of the other components (which will eventually exhibit backpressure and slow the measured components down, but this may happen with more lag than you'd like without performance impeded by a smaller -n argument added to xargs):

result=$(find . -name '*bz2' -print0 
        | pv -0 --line-mode 
        | xargs -0 bzcat 
        | tee >( some | other | pipeline ) 
        | grep -e "myString" 
        | wc -l)
echo "${result} occurrences found"