Why it matters
Picking the wrong compute layer can waste money and slow down the team.
A clear guide to when you need Spark compute and when a SQL warehouse is the better tool for the job.
Clusters are general-purpose compute for Spark-style workloads. SQL warehouses are tuned for SQL queries and analytics serving.
Picking the wrong compute layer can waste money and slow down the team.
Use a cluster for ETL transformations and a SQL warehouse for dashboard queries.
Using one compute option for every workload because it is familiar.
Read: Delta Lake Basics