regex - sed: return last occurrence match until end of file

Question

Using sed, how do I return the last occurance of a match until the End Of File? (FYI this has been simplified)

So far I've tried:

sed -n '/ Statistics |/,$p' logfile.log

Which returns all lines from the first match onwards (almost the entire file)

I've also tried:

$linenum=`tail -400 logfile.log | grep -n " Statistics |" | tail -1 | cut -d: -f1`
sed "$linenum,$!d" logfile.log

This works but won't work over an ssh connection in one command, really need it all to be in one pipeline.

Format of the log file is as follows:

(There are statistics headers with sub data written to the log file every minute, the purpose of this command is to return the most recent Statistics header together with any associated errors that occur after the header)

Statistics |
   Stuff
   More Stuff
   Even more Stuff
Statistics |
   Stuff
   More Stuff
Error: incorrect value
Statistics |
   Stuff
   More Stuff
   Even more Stuff
Statistics |
   Stuff
Error: error type one
Error: error type two

EOF

Return needs to be:

Statistics |
   Stuff
Error: error type one
Error: error type two

Answer 1

Your example script has a space before Statistics but your sample data doesn't seem to. This has a regex which assumes Statistics is at beginning of line; tweak if that's incorrect.

sed -n '/^Statistics |/h;/^Statistics |/!H;$!b;x;p'

When you see Statistics, replace the hold space with the current line (h). Otherwise, append to the hold space (H). If we are not at the end of file, stop here (b). At end of file, print out the hold space (x retrieve contents of hold space; p print).

In a sed script, commands are optionally prefixed by an "address". Most commonly this is a regex, but it can also be a line number. The address /^Statistics |/ selects all lines matching the regular expression; /^Statistics |/! selects lines not matching the regular expression; and $! matches all lines except the last line in the file. Commands with no explicit address are executed for all input lines.

Edit Explain the script in some more detail, and add the following.

Note that if you need to pass this to a remote host using ssh, you will need additional levels of quoting. One possible workaround if it gets too complex is to store this script on the remote host, and just ssh remotehost path/to/script. Another possible workaround is to change the addressing expressions so that they don't contain any exclamation marks (these are problematic on the command line e.g. in Bash).

sed -n '/^Statistics |/{h;b};H;${x;p}'

This is somewhat simpler, too!

A third possible workaround, if your ssh pipeline's stdin is not tied up for other things, is to pipe in the script from your local host.

echo '/^Statistics |/h;/^Statistics |/!H;$!b;x;p' |
ssh remotehost sed -n -f - file