bash - how to use cat | awk | xargs sed to replace a char

Question

I wrote a little bash script to parse Apache Access log to count POST|GET request.

My script works fine but I have a little graphical issue when I want to remove "[" char from the date field return by awk command.

Here is my script:

clear
ls /var/log/httpd | egrep *access_log$ > temp.txt

while read line
do
        linecount=$(cat /var/log/httpd/"$line" | wc -l)
        #echo -e "$line"
        #echo -e "$linecount"
        if [ $linecount -gt 0 ]
        then
                echo -e "==========================================="
                echo  -e "$line"
                echo  -e "Date de debut du log :"
                cat  /var/log/httpd/"$line"  | awk -v ligne=1 'NR == ligne, FS=":" {print $4}' | xargs  -0 sed -i 's/[//g'
                echo  -e "Date de fin du log :"
                cat  /var/log/httpd/"$line"  | awk 'END {print $4}'
                echo  -e "Nombre de requêtes sur la période  :"
                egrep -i 'post|get' /var/log/httpd/"$line" | wc -l
        fi
        linecount=0
done < temp.txt

rm -f temp.txt

An example of standard output of this code looks like this :

===========================================
xxx.xxx.xxx-ssl_access_log
Date de debut du log :
sed: impossible de lire [01/Jan/2021:07:34:59
: Aucun fichier ou dossier de ce type
Date de fin du log :
[22/Jan/2021:07:44:44
Nombre de requêtes sur la période  :
22

Why can't sed use the string piped by awk?

How can I correct it ?

Below an example of log imput file :

54.36.148.55 - - [29/Dec/2020:18:05:38 +0100] "GET /robots.txt HTTP/1.1" 404 159
54.36.149.92 - - [29/Dec/2020:18:05:38 +0100] "GET / HTTP/1.1" 200 2394
54.36.148.185 - - [30/Dec/2020:17:51:06 +0100] "GET / HTTP/1.1" 200 2394
54.36.149.77 - - [31/Dec/2020:17:19:18 +0100] "GET /robots.txt HTTP/1.1" 404 159
54.36.148.97 - - [31/Dec/2020:17:19:19 +0100] "GET / HTTP/1.1" 200 2394
54.36.149.61 - - [01/Jan/2021:14:45:59 +0100] "GET / HTTP/1.1" 200 2394
54.36.148.151 - - [02/Jan/2021:16:26:22 +0100] "GET /robots.txt HTTP/1.1" 404 159
54.36.148.71 - - [02/Jan/2021:16:26:24 +0100] "GET / HTTP/1.1" 200 2394
54.36.148.108 - - [03/Jan/2021:15:21:28 +0100] "GET / HTTP/1.1" 200 2394
208.100.26.249 - - [03/Jan/2021:23:15:13 +0100] "GET / HTTP/1.1" 200 2394
54.36.149.95 - - [04/Jan/2021:15:28:31 +0100] "GET /robots.txt HTTP/1.1" 404 159
54.36.148.202 - - [04/Jan/2021:15:28:32 +0100] "GET / HTTP/1.1" 200 2394
54.36.149.24 - - [05/Jan/2021:14:44:52 +0100] "GET / HTTP/1.1" 200 2394
54.36.148.184 - - [06/Jan/2021:15:00:55 +0100] "GET /robots.txt HTTP/1.1" 404 159
54.36.149.54 - - [06/Jan/2021:15:00:55 +0100] "GET / HTTP/1.1" 200 2394
54.36.148.185 - - [07/Jan/2021:14:03:13 +0100] "GET / HTTP/1.1" 200 2394
51.158.103.247 - - [08/Jan/2021:12:31:33 +0100] "GET / HTTP/1.1" 200 2394
54.36.148.17 - - [08/Jan/2021:14:10:18 +0100] "GET /robots.txt HTTP/1.1" 404 159
54.36.148.185 - - [08/Jan/2021:14:10:19 +0100] "GET / HTTP/1.1" 200 2394
54.36.148.101 - - [09/Jan/2021:14:17:39 +0100] "GET /robots.txt HTTP/1.1" 404 159
54.36.148.94 - - [09/Jan/2021:14:17:40 +0100] "GET / HTTP/1.1" 200 2394
54.36.148.103 - - [10/Jan/2021:15:21:24 +0100] "GET /robots.txt HTTP/1.1" 404 159
54.36.148.68 - - [10/Jan/2021:15:21:24 +0100] "GET / HTTP/1.1" 200 2394
54.36.148.208 - - [11/Jan/2021:18:15:40 +0100] "GET /robots.txt HTTP/1.1" 404 159
54.36.149.78 - - [11/Jan/2021:18:15:41 +0100] "GET / HTTP/1.1" 200 2394
54.36.148.64 - - [12/Jan/2021:20:37:08 +0100] "GET /robots.txt HTTP/1.1" 404 159
54.36.149.38 - - [12/Jan/2021:20:37:09 +0100] "GET / HTTP/1.1" 200 2394
54.36.149.66 - - [13/Jan/2021:20:40:09 +0100] "GET /robots.txt HTTP/1.1" 404 159
54.36.148.203 - - [13/Jan/2021:20:40:10 +0100] "GET / HTTP/1.1" 200 2394
51.158.127.119 - - [14/Jan/2021:11:41:05 +0100] "GET / HTTP/1.1" 200 2394
51.15.251.143 - - [14/Jan/2021:11:52:04 +0100] "GET / HTTP/1.1" 200 2394
54.36.149.76 - - [14/Jan/2021:20:05:36 +0100] "GET / HTTP/1.1" 200 2394
208.100.26.243 - - [18/Jan/2021:10:20:00 +0100] "GET / HTTP/1.1" 200 2394
208.100.26.248 - - [25/Jan/2021:04:10:37 +0100] "GET / HTTP/1.1" 200 2394

question from:https://stackoverflow.com/questions/65933155/how-to-use-cat-awk-xargs-sed-to-replace-a-char

Answer 1

Using awk as a "complete" solution

awk 'FNR==1 { 
              gsub("[[]","",$4);
              sdat=$4                                              # When the file record number (FNR) is 1, remove [ from the 4th space separated field with gsub and set sdat to this field
            } 
    ENDFILE { 
              gsub("[[]","",$4);
              fdat=$4;                                              # When we reach the end of each file, remove [ gain from the 4th field and set fdat to this field
              print "==========================================="
              print FILENAME                                        # Print the filename using awk's FILENAME variable
              print "Date de debut du log :"                        # Print the data required
              print sdat
              print "Date de fin du log :"
              print fdat
              print "Nombre de requêtes sur la période  :"
              print FNR                                             # Print the total number of records in the file (file number record)
            } ' /var/log/httpd/*access_log