Kafka Consumer not polling reliably after upgrade to Java 15, with Producer seeking and consuming most recent record first in same JVM

Question

I upgraded my Apache Kafka app to Java 15 and now my app does not poll reliably.

The app has Producer and Consumer in the same JVM with a single partition. The Producer seeks to to most recent record and consumes it OK to read app data. Then the producer sends a new record to the topic. The Consumer no longer reliably polls the record. Occasionally it polls OK which is puzzling. When I upgraded to Java 15 in production, the same situation happened. I am using Kafka 2.12 version 2.7.0.

The consumer is implemented in the standard fashion..

      ConsumerRecords<String, byte[]> consumerRecords = kafkaConsumer.poll(Duration.ofSeconds(1)); // timeout
      for (ConsumerRecord<String, byte[]> consumerRecord : consumerRecords) {
        if (LOGGER.isDebugEnabled()) {
          LOGGER.debug("received consumerRecord " + consumerRecord);
        }

question from:https://stackoverflow.com/questions/65601678/kafka-consumer-not-polling-reliably-after-upgrade-to-java-15-with-producer-seek

Answer 1

Waitting for answers