fluentd - Use fluent-plugin-grok-parser with splunk-hec image

Question

I am trying to create an image that has grok-parser installed based on the fluentd-hec image.

This is the Dockerfile i'm using:

FROM splunk/fluentd-hec:1.2.4
USER root
RUN gem install fluent-plugin-grok-parser
RUN chown -R fluent:fluent /usr/local/share/gems/gems/fluent-plugin-grok-parser-*
USER fluent

This is the output from that build

Step 1/5 : FROM splunk/fluentd-hec:1.2.4
 ---> ac49b85acc6a
Step 2/5 : USER root
 ---> Running in 4ee81880e92a
Removing intermediate container 4ee81880e92a
 ---> e3748059e604
Step 3/5 : RUN gem install fluent-plugin-grok-parser
 ---> Running in 2a1debb084ec
Successfully installed bundler-2.2.5
Building native extensions. This could take a while...
Successfully installed msgpack-1.3.3
Building native extensions. This could take a while...
Successfully installed yajl-ruby-1.4.1
Building native extensions. This could take a while...
Successfully installed cool.io-1.7.0
Successfully installed sigdump-0.2.4
Successfully installed serverengine-2.2.2
Building native extensions. This could take a while...
Successfully installed http_parser.rb-0.6.0
Successfully installed concurrent-ruby-1.1.7
Successfully installed tzinfo-2.0.4
Successfully installed tzinfo-data-1.2020.6
Building native extensions. This could take a while...
Successfully installed strptime-0.2.5
Successfully installed fluentd-1.12.0
Successfully installed fluent-plugin-grok-parser-2.6.2
13 gems installed
Removing intermediate container 2a1debb084ec
 ---> c5155932810c
Step 4/5 : RUN chown -R fluent:fluent /usr/local/share/gems/gems/fluent-plugin-grok-parser-*
 ---> Running in 1c2550dcac74
Removing intermediate container 1c2550dcac74
 ---> 7e216a676427
Step 5/5 : USER fluent
 ---> Running in 5ee31ea2e78a
Removing intermediate container 5ee31ea2e78a
 ---> ea8bdee73ee5
Successfully built ea8bdee73ee5

the snippet of the configmap is:

      @id snow
      @type tail
      @label @SPLUNK
      tag tail.snow.*
      path /opt/snow/data/*.log
      pos_file /var/log/splunk-snow.log.pos
      path_key source
      <parse>
        @type grok
        grok_failure_key grokfailure
        <grok>
          pattern %{TIMESTAMP_ISO8601:time};%{SPACE}%{GREEDYDATA:log}
        </grok>
      </parse>
    </source>

when I deploy a daemonset using the new image I get the error config error file="/fluentd/etc/fluent.conf" error_class=Fluent::ConfigError error="Unknown parser plugin 'grok'. Run 'gem search -rd fluent-plugin' to find plugins" I have tried this in EKS v1.18 as well as docker desktop and they both have the same issue.

Is there anything else I need to add to the dockerfile so that I use extra plugins?

I'd appreciate any help at on this!

Answer 1

I have rewritten the config so grok isn't used so this question isn't needed any more