问题

共享访问模式集群用户在查看ApachesparkUI任务日志时偶发HTTP500错误也适用于管理员

ErrorCaused by:java.lang.Exception: Log viewing is disabled on this cluster      at org.apache.spark.deploy.worker.ui.LogPage.render(LogPage.scala:65)      at org.apache.spark.ui.WebUI$$anonfun$3.apply(WebUI.scala:100)      at org.apache.spark.ui.WebUI$$anonfun$3.apply(WebUI.scala:100)      at org.apache.spark.ui.JettyUtils$$anon$3.doGet(JettyUtils.scala:90)      at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)      at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)      at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:848)      at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:584)      at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)      at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)      at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)      at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)      at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:493)      at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)      at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)      at org.eclipse.jetty.server.Server.handle(Server.java:534)      at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)      at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)

因果

exceptex受控spark.databricks.ui.logViewingEnabledspark属性值定时虚伪日志浏览禁用当spark日志集群禁用时,SparkUI生成错误当您试图查看日志时

上头spark.databricks.ui.logViewingEnabled属性默认真实性,但有时还有其他spark配置spark.databricks.acl.dfAclsEnabled)可改变值并设置虚伪.

求解

集成spark.databricks.ui.logViewingEnabled至真实性内集群spark配置高山市AWS系统|休眠|GCP)

spark.databricks.ui.logViewingEnabled true

恢复默认配置