Spring boot Kafka - Confusion over Avro object serialisation and use cases

Question

I think I'm not grasping some basic concepts of Kafka here, so I'm hoping Stack maybe able to the help.

I've been trying to learn Kafka with Spring boot by following this GIT repo here:

I understand how to without avro take a Java class from one Microservice, send it to Kafka and consume / serialise it on another Microservice...however I hate that idea. As it means I must have an identical class on the other Microservice in terms of package location / name etc

So overall I've two questions here I guess.

1. I want to understand how I can share message across my spring boot microservices and map them to classes without copying said classes from one service to the other

2. I want to be able to consume from my Spring Kafka listeners messages created from another language say C#

Where I'm currently at is, I have the avro example from the repo above up and running along with my local kafka and Schema registry instance.

However if I create a duplicate class and call it UserTest (For example) and have it identical to the User class consumed here I get stacktraces like the following:

Caused by: org.springframework.messaging.converter.MessageConversionException: Cannot convert from [io.confluent.developer.User] to [io.confluent.developer.kafkaworkshop.streams.User] for GenericMessage [payload={"name": "vik", "age": 33}, headers={kafka_offset=6, kafka_consumer=org.apache.kafka.clients.consumer.KafkaConsumer@54200a0e, kafka_timestampType=CREATE_TIME, kafka_receivedMessageKey=vik, kafka_receivedPartitionId=1, kafka_receivedTopic=users12, kafka_receivedTimestamp=1611278455840, kafka_groupId=simple-consumer}]

Am I missing something exceptionally basic here? I thought that once the message was send in Avro format that it could be consume and mapped to another object which had the same fields...that way if the object was created in c#, the spring service would be able to interpret it no?

If anyone can help me that would be great....

Thanks!

question from:https://stackoverflow.com/questions/65838223/spring-boot-kafka-confusion-over-avro-object-serialisation-and-use-cases

Answer 1

Waitting for answers