I have Azure Databricks workspace with cluster configured to run Standard 6.4 runtime (Apache Spark 2.4.5, Scala 2.11). Cluster uses shared metastore (Azure MySQL). I'm trying to figure out possible way to troubleshoot sporadically slow execution of jobs/queires - I have a test SELECT query which normally runs within 2-3 minutes but couple of times a day it takes 15 minutes. What would be the best way to troubleshoot this?
Asked
Active
Viewed 695 times
