What is Databricks?

Databricks is a data platform built around the lakehouse: the cheap, open storage of a data lake combined with the reliability and speed you expect from a data warehouse. Storage, processing, catalogue, models and dashboards live in one environment.

The practical summary: one place where raw data arrives, is made reliable and can be queried - without copying it into a second system first.

The lakehouse in layers

Bronze, silver, gold: the same data, rising in reliability.

Raw data arrives as it is (bronze), is cleaned and joined (silver), and is then fixed in the tables the organisation steers on (gold). Delta Lake adds transactions and versioning, so you can look back at how a table stood last month - the difference between explaining a number and defending it.

Unity Catalog is the layer around it: one place for definitions, lineage and access rights. That is exactly the meaning layer from data architecture, and the reason we deploy the platform in many environments.

The parts you actually use

Databricks is one environment with a handful of clearly separable parts. You do not need all of them, but it helps to know what each is for.

• Delta Lake. The table format: open, with transactions, schema management and time travel to earlier versions.
• Unity Catalog. The register: definitions, owners, field-level lineage and role-based rights, across all workspaces.
• Jobs and pipelines. The execution: scheduled or event-driven processing, with declarative pipelines for the bronze-silver-gold steps.
• SQL warehouses. The querying: ordinary SQL for reporting and dashboards, on the same storage.
• MLflow and model serving. The model work: tracking experiments, versioning models and serving them as endpoints.
• Notebooks and clusters. The workbench for engineering and analysis, with compute that switches on and off.

An important detail for boards and procurement: the data sits in an open format in your own storage environment. Other tools can reach it, and a later move is a migration of processing, not of your whole archive.

Where it fits

Not a ranking but a profile: what the platform is strong at and where something else is more sensible.

Databricks is strong with large volumes, mixed workloads and teams that both analyse and build models. A classic data warehouse remains fine when demand is stable and tabular. Microsoft Fabric sits closer to organisations fully inside the Microsoft landscape that intend to stay there. Snowflake is often chosen when SQL and reporting are the centre of gravity and there is little in-house engineering.

The choice rarely hinges on functionality. It hinges on where your team is already at home, how many domains have to deliver at once, and how strict oversight of access must be.

The questions that actually decide it

The choice lands on organisation and oversight, not on a feature list.

Ask these four questions before choosing a platform, and write the answers down. They weigh more than any comparison table: how many domains deliver at once, where your team is already at home, how strict access oversight is, and what must still be possible in three years without a migration.

What it costs, broadly

Databricks charges for compute per consumption unit, on top of your cloud provider's storage and network costs. That means two things in practice. First, cost is steerable: clusters that shut down, smaller warehouses and well-partitioned tables save money directly. Second, cost is visible per workload, which makes it possible to spot an expensive question before it becomes a habit.

What makes it expensive is almost never the platform itself. It is pipelines nobody uses any more, the same transformation in three places, and tables refreshing every fifteen minutes for a decision made monthly.

What it does not solve

Databricks does not make your definitions unambiguous. It does not assign ownership, does not write data contracts and does not decide what "delivered on time" means. A platform can enforce what you agreed; it cannot make the agreement for you.

That is not criticism of the tool, but it is the reason a platform migration rarely solves the problem it was started for. Start with four versions of revenue and you will have four versions in a new environment.

What a first setup looks like

A workable start is small and complete, not large and half-finished:

1. One catalogue structure. Workspaces, rights and naming thought through once, before tables arrive.
2. Two sources, bronze to gold. For one question the business asks every month.
3. Definitions included. Every gold field has a definition in words and a named owner.
4. Rights per role. Configured in the catalogue, not in the dashboard.
5. Cost and freshness measured. So the conversation about the next step is about numbers.

After that, expanding is a repetition of the same pattern, which is exactly the intention.

How it fits with us

We always deploy Databricks together with the governed layer: ownership per domain, recorded meaning and traceability back to the source. What that setup looks like is on the engine. If you want to organise ownership per domain, read what is a data mesh.