Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
161 views
in Technique[技术] by (71.8m points)

Is it possible to change Kafka Topic config without data loss?

I'm trying to change Kafka Topic config, can I preserve my data and offset data with config change? I'm wondering if message and offset info is removed when I change partition num 1 to 3 (or I change compression type gzip to lz4) Or recreate after remove?

question from:https://stackoverflow.com/questions/65952456/is-it-possible-to-change-kafka-topic-config-without-data-loss

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)

Increasing partitions doesn't modify any data in the original partition set, so it'd be unlikely to lose any.

I'm not sure if topic compression can be modified after the fact, but if it were, data would need re-written in the segment files over all the brokers, which opens the possibility for corruption. The safer approach would be to consume the original compressed topic and write it to a new compression type

"Recreate after remove" a topic? - of course you're losing the original data. If you mean any other config, outside of retention settings or cleanup policy, nothing comes to mind


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...