Build or buy a data platform? How to make that call

Build or buy is rarely a technical discussion. It is a discussion about where you want your knowledge to sit, and who will still be working on it in two years.

Do not start with the comparison

Comparing brands before you have written down the question gets you a demo, not a decision.

The first question is not which product, but which decision has to get better. A platform nobody can name a purpose for becomes a project without an end. Then come volume and freshness: a few thousand rows once a night is a different design from a few million continuously.

The third question is the most honest one: who runs this in two years? If that is one internal person, a platform you assemble from parts is a risk parked on a single pair of shoulders.

What you give up and what you get

Building pays off where your work is unique. Buying pays off where your work fits the standard.

Building fits your process exactly and keeps knowledge inside, but maintenance is your job every week. Buying starts fast and the vendor maintains it, but you bend your process to the standard and part of your knowledge sits elsewhere.

In practice almost nobody chooses purely. Most organisations buy the lower layers, storage and processing, and build the layer that holds their own meaning. That is sensible: compute is a commodity, your definition of margin is not.

Three costs usually left out

• Operations. A platform nobody maintains loses trust within a year. Count hours, not licences.
• Knowledge leaving. Anything that lives only in a builder's head is an open invoice.
• The exit. Can you take your data and definitions with you if you stop? See ownership is proven at the exit.

A workable middle road

Buy the infrastructure, build the meaning. Record definitions and owners in a layer your vendor does not own, so switching platform is not switching meaning. See what is a data platform for those layers, and what is Databricks for what a purchased base layer looks like.

In short

Build what makes you different, buy what everyone does the same way, and only decide on technology once the owner of operations is known.