How to make text replacent between files with awk

Question

I am trying to use awk in order to make specific substitutions in a text file based on values taken from another file. More in detail and using simple examples, I have a raw file A.txt that must be processed:

000  y4fy      1  0h0l
000  rft5      1  yrt3
000  g34y      1  ht74
000  ll90      2  t964
000  ew3x      2  472e
000  jo7e      3  6rhf
000  f5gg      4  gs84
000  wp5y      5  6rru
000  3em1      6  c2cn
000  pti1      7  ldr3
000  4vhd      7  epp3
000  2kfb      8  twtv

and so on...

A second file B.txt contains a subset of values of $3 field of A.txt:

2
5
7

I am tryng to make specific replacements in the $4 field of A.txt based in values of B.txt.

Now, we can distinguish two different cases.

Case_1: the susbstitution text is a single string, let's say TTTT, and the output should be:

000  y4fy      1  0h0l
000  rft5      1  yrt3
000  g34y      1  ht74
000  ll90      2  TTTT
000  ew3x      2  TTTT
000  jo7e      3  6rhf
000  f5gg      4  gs84
000  wp5y      5  TTTT
000  3em1      6  c2cn
000  pti1      7  TTTT
000  4vhd      7  TTTT
000  2kfb      8  twtv

in which the format is preserved.

Case_2: the susbstitution text is associated to a specific value in B.txt, such as:

2   HHHH
5   AAAA
7   MMMM

and the output should be:

000  y4fy      1  0h0l
000  rft5      1  yrt3
000  g34y      1  ht74
000  ll90      2  HHHH
000  ew3x      2  HHHH
000  jo7e      3  6rhf
000  f5gg      4  gs84
000  wp5y      5  AAAA
000  3em1      6  c2cn
000  pti1      7  MMMM
000  4vhd      7  MMMM
000  2kfb      8  twtv

Basically, looks to me case_1 is a particular instance of case_2.

Because such process involves hundreds of very long files, I am trying to find a solution that use awk instead of the slower while read loop in bash.

After a couple of days, the best code I got for the case_1 is:

while read val; do awk -v x="$val" '{if ($1 == "0000" && $3 == x) $4 = "TTTT" ; else print $0}' A.txt; done < B.txt > res.txt

which does not work, of course. I wonder if someone could provide a solution for case_1 and, hopefully, for case_2 as well. Thanks.

question from:https://stackoverflow.com/questions/65831446/how-to-make-text-replacent-between-files-with-awk

Answer 1

You may use:

Case 1:

cat file1
2
5
7

cat file2
000  y4fy      1  0h0l
000  rft5      1  yrt3
000  g34y      1  ht74
000  ll90      2  t964
000  ew3x      2  472e
000  jo7e      3  6rhf
000  f5gg      4  gs84
000  wp5y      5  6rru
000  3em1      6  c2cn
000  pti1      7  ldr3
000  4vhd      7  epp3
000  2kfb      8  twtv

Then use awk as:

awk 'NR == FNR {map[$1] = $2; next} $3 in map {s = (map[$3] == "" ? "TTTT" : map[$3]); sub(/[^[:blank:]]+$/, s)} 1' file1 file2

000  y4fy  1  HHHH
000  rft5  1  HHHH
000  g34y  1  HHHH
000  ll90  2  TTTT
000  ew3x  2  TTTT
000  jo7e  3  AAAA
000  f5gg  4  gs84
000  wp5y  5  TTTT
000  3em1  6  c2cn
000  pti1  7  TTTT
000  4vhd  7  TTTT
000  2kfb  8  MMMM

---

Case 2:

cat file1

2   HHHH
5   AAAA
7   MMMM

Then run it as:

awk 'NR == FNR {map[$1] = $2; next} $3 in map {s = (map[$3] == "" ? "TTTT" : map[$3]); sub(/[^[:blank:]]+$/, s)} 1' file1 file2

000  y4fy  1  0h0l
000  rft5  1  yrt3
000  g34y  1  ht74
000  ll90  2  HHHH
000  ew3x  2  HHHH
000  jo7e  3  6rhf
000  f5gg  4  gs84
000  wp5y  5  AAAA
000  3em1  6  c2cn
000  pti1  7  MMMM
000  4vhd  7  MMMM
000  2kfb  8  twtv

To make it more readable:

awk 'NR == FNR {  # while processing first file
   map[$1] = $2   # store $2 in an associative array with key as $1
   next
}
$3 in map {       # if $3 is found in associative array
   # set $4=TTTT when value is empty otherwise use value stored in array
   s = (map[$3] == "" ? "TTTT" : map[$3])
   sub(/[^[:blank:]]+$/, s)
} 1' file1 fil22