bash - sed - replace pattern with content of file, while the name of the file is the pattern itself

Question

Starting from the previous question, I have another one. If I make this work, I can just delete several lines of script :D

I want to transform this line:

sed -i -r -e "/$(basename "$token_file")/{r $token_file" -e "d}" "$out_dir_rug"/rug.frag

into this line:

sed -i -r -e "/(##_[_a-zA-Z0-9]+_##)/{r $out_dir_frags_rug/1" -e "d}" "$out_dir_rug"/rug.frag

The idea is the following. Originally (the first line), I searched for some patterns, and then replaced those patterns with their associated files. The names of the files are the patterns themselves.

Eample:

Pattern: ##_foo_##

File name: ##_foo_##

Content of file ##_foo_##:

first line of foo text
second line of foo text

so the text

bar
##_foo_##
bar

would become

bar
first line of foo text
second line of foo text
bar

---

In my second attempt, I used sed for both locating the patterns, and for the actual replacement.

The result is that the patterns are found, but replaced with pretty much nothing.

Is sed supposed to be able to do the replacement I want? If yes, how should I change my command?

---

Note: a file usually has several different patterns (I call them tokens), and the same pattern may appear more than one time.

So an input file might look like:

bar
bar
##_foo_##
bar
##_haa_##
bar
##_foo_##
and so on

---

I already tried to replace the / in the address with ,, to no useful result. Escaping the / in the path to / also does not help.

I verified that the path to the replacement files is good by adding the next line, just before the sed:

echo "$out_dir_frags_rug"

question from:https://stackoverflow.com/questions/65929556/sed-replace-pattern-with-content-of-file-while-the-name-of-the-file-is-the-pa

Answer 1

The names of the files are the patterns themselves.

If you need anything "dynamic", then sed is not enough for it. As sed can't do "eval" - can't reinterpret the content of pattern buffer or hold buffer as commands (that would be amazing!) - you can't use the line as part of the command.

You can use bash, untested, written here:

while IFS= read -r line; do
    if [[ "$line" =~ ^##_([_a-zA-Z0-9]+)_## ]]; then
       cat "${BASH_REMATCH[1]}"
    else
       printf "%s
" "$line"
    fi
done < inputfile

but that would be slow - bash is slow on reading lines. A similar design could be working in a POSIX shell with POSIX tools, by replacing [[ bash extension with some grep + sed or awk.

An awk would be waaaaaaay faster, something along, also untested:

awk '/^##_[_a-zA-Z0-9]+_##$/{
        gsub(/^##_/, "", $0);
        gsub(/_##$/, "", $0);
        file = $0
        while (getline tmp < (file)) print tmp;
        next
     }
     {print}
' inputfile

That said, for your specific problem, instead of reinventing the wheel and writing yet another templating and preprocessing tool, I would advise to concentrate on researching existing solutions. A simple cpp file with the following content can be preprocessed with C preprocessor:

bar
bar
#include "foo"
bar
#include "haa"
bar
#include "foo"
and so on

It's clear to anyone what it means and it has a very standarized format and you also get get all the #ifdef conditional expressions, macros and macro functions that you can use - but you can't start lines with #, dunno if that's important. For endless ultimate templating power, I could recommend m4 from the standard unix commands.