There is an issue with Spark History Server that appeared when we upgraded from AWS EMR 5.25 to 6.1.0.
SHS works ok until a first EMR job fails. From that moment on it only launches but complains that it can't find any completed applications. And it doesn't show any incomplete applications either.
It seems it's incapable of parsing logs after a job failed and thus thinking that there are no completed applications.
It worked in 5.28.0 it does not work anymore in 6.1.0
AWS guys, can you fix it? Or does this problem appear when not run on EMR too?
Or is there some workaround to make it work?
question from:
https://stackoverflow.com/questions/65882511/spark-history-server-not-working-in-emr-6-1-0-after-first-job-failure 与恶龙缠斗过久,自身亦成为恶龙；凝视深渊过久,深渊将回以凝视…
