java - How to collect All the offsetranges in a List in Spark? So that I can pass that list to another method

Question

JavaInputDStream<ConsumerRecord<String, String>> stream = KafkaUtils.createDirectStream(streamingContext, "","" );
Set<TopicInfo> topicInfos = new HashSet<>();            
   stream.foreachRDD(rdd -> {
            OffsetRange[] offsetRanges = ((HasOffsetRanges) rdd.rdd()).offsetRanges();
            topicInfos.addAll(Arrays.stream(offsetRanges).map(offsetRange -> new TopicInfo(offsetRange.topic(),offsetRange.partition(),offsetRange.untilOffset())).collect(Collectors.toSet()));});

1. I want to pass the Set to another method so that I can store the offset info along with some other things.
2. I am getting the Empty set(topicInfos) always.

question from:https://stackoverflow.com/questions/65920715/how-to-collect-all-the-offsetranges-in-a-list-in-spark-so-that-i-can-pass-that

Answer 1

Waitting for answers