apache kafka - Compact display of consumer groups with respect to a certain topic

Question

I tried:

# kafka-consumer-groups.sh --bootstrap-server localhost:9092 --command-config client.properties --describe --all-groups --topic topic1

and got:

Option "[topic]" can't be used with option "[describe]"

So I came up with a simple patch:

# cat kafka-consumer-status.sh
#/bin/bash
if [[ -z $1 ]]; then
    echo "Missing parameter: topic"
    exit 1
fi
kafka-consumer-groups.sh --bootstrap-server localhost:9092 --command-config client.properties --describe 
--all-groups 2>/dev/null |grep $1|awk '{print $1}'|uniq |xargs -n 1 kafka-consumer-groups.sh --bootstrap- 
server localhost:9092 --command-config client.properties --describe --group |grep -E "GROUP|$1"
#

Was this actually necessary, did I overlook some option that is already available with apache kafka CLI shell commands?

question from:https://stackoverflow.com/questions/65829908/compact-display-of-consumer-groups-with-respect-to-a-certain-topic

Answer 1

Finding all groups, then getting the topic from those groups is the only way to get that information.

The standard use case is to find lag by an application (group.id). There's little user benefit to finding the lag of all applications (groups) consuming from one topic, and this is more of an administrative feature, thus be better suited to use Burrow or other lag collection utilities that can aggregate such metrics

If the use case is to find all client addresses consuming from that topic, then there's no good data lineage product that I'm aware of outside Cloudera SMM tool