I have been using the kafka-connect image confluentinc/cp-kafka-connect for some time now. According to the Confluent documentation, this docker image comes with pre-installed connector plugins including Elastic.
I have previously been using version 5.4.1-ccs
which works great, I can add Elastic sink connnector configs and they work just fine. However I have tried to update confluentinc/cp-kafka-connect
to latest v6.0.1
and now I get the error.
ConnectException: Failed to find any class that implements Connector and which name matches ElasticsearchSinkConnector
I have read a bunch of documentation on Confluent website but it's a bit sporadic. I understand that the issue is the plugins are not mounted due to either them being removed from the new docker image or a wrong path (unsure which one it is).
How can I fix this issue? (note: I have written my own java plugin also so both need to work)
Here is my docker-compose
file currently (again, this works with version 5.4.1-ccs
)
kafka-connect-node-1:
image: confluentinc/cp-kafka-connect:5.4.1 #using old version because of breaking change
hostname: kafka-connect-node-1
ports:
- '8083:8083'
environment:
CONNECT_BOOTSTRAP_SERVERS: [MY_SERVER]
CONNECT_REST_PORT: 8083
CONNECT_GROUP_ID: compose-connect-group
CONNECT_CONFIG_STORAGE_TOPIC: connect-configs
CONNECT_OFFSET_STORAGE_TOPIC: connect-offsets
CONNECT_STATUS_STORAGE_TOPIC: connect-status
CONNECT_KEY_CONVERTER: io.confluent.connect.avro.AvroConverter
CONNECT_KEY_CONVERTER_SCHEMA_REGISTRY_URL: 'http://kafka-schema-registry:8084'
CONNECT_VALUE_CONVERTER: io.confluent.connect.avro.AvroConverter
CONNECT_VALUE_CONVERTER_SCHEMA_REGISTRY_URL: 'http://kafka-schema-registry:8084'
CONNECT_INTERNAL_KEY_CONVERTER: 'org.apache.kafka.connect.json.JsonConverter'
CONNECT_INTERNAL_VALUE_CONVERTER: 'org.apache.kafka.connect.json.JsonConverter'
CONNECT_REST_ADVERTISED_HOST_NAME: 'kafka-connect-node-1'
CONNECT_LOG4J_ROOT_LOGLEVEL: 'INFO'
CONNECT_LOG4J_LOGGERS: 'org.apache.kafka.connect.runtime.rest=WARN,org.reflections=ERROR'
CONNECT_PLUGIN_PATH: '/usr/share/java,/etc/kafka-connect/jars'
CONNECT_ZOOKEEPER_CONNECT: [MY_ZOOKEEPER]
volumes:
- /efs/connector:/etc/kafka-connect/jars/ # mounting my custom JAR
depends_on:
- kafka-schema-registry
- kafka-rest-proxy
question from:
https://stackoverflow.com/questions/65901864/kafka-connect-docker-image-failed-to-find-any-class-that-implements-connector 与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…