regex - shell script: search and replace over multiple lines

Question

I'm looking for a way to search and replace over multiple lines through a shell script. This is what I'm trying to do:

source:
[stuff before]
<!--WIERD_SPECIAL_COMMENT_BEGIN-->
  [stuff here, possibly multiple lines.
<!--WIERD_SPECIAL_COMMENT_END-->
[stuff after]    

target:
[stuff before]
[new content]
[stuff after]

In short, I want to delete the comments and everything between them and replace with some new content. Basically, I want to do a simple sed command over multiple lines, and if possible just using some basic *nix tools, no additional scripting language.

Answer 1

If you only need to match complete lines then you can do this task with awk. Something like:

    awk -v NEWTEXT=foo 'BEGIN{n=0} /COMMENT_BEGIN/ {n=1} {if (n==0) {print $0}} /COMMENT_END/ {print NEWTEXT; n=0}' < myfile.txt

If the file is not so well formatted, with comments on the same line as text you want to keep or remove, then I would use perl, read the entire file into a single string, do a regular expression match and replace on that string, then write the new string to a new file. This is not so simple and you need to write a perl script to do the work. Something like:

#!/usr/bin/perl
$newtext = "foo
bar";
$/ = '';  # no input separator so whole file is read.
$s = <>;  # read whole file from stdin
$startPattern = quotemeta('<!--WIERD_SPECIAL_COMMENT_BEGIN-->');
$endPattern = quotemeta('<!--WIERD_SPECIAL_COMMENT_END-->');
$pattern = $startPattern . '.+' . $endPattern;
$s =~ s/$pattern/$newtext/sg;
print $s;