Not Every Data Question Deserves a Full Data Product. And that’s okay

When we started building the Data Product Portal, we had a clear conviction: data product thinking is the right approach for getting value out of data in organisations. A data product has an owner. It has a lifecycle. It has defined consumers and clear output ports. It is a commitment to quality, reliability, and trust. That conviction has not changed.

But after watching users interact with data, we noticed something. A significant portion of data access requests were not about building anything. They were questions. A finance analyst checking if January revenue figures match a spreadsheet before an executive meeting. A data scientist browsing tables to find features with predictive power before committing to a model. A business analyst replying to an urgent C-level question.

They are curiosity moments. And we were making them too hard.

Why the full process gets in the way

In most governed data environments, getting access to data requires going through a formal process. In Portal, that meant creating a Data Product: picking a name, a domain, configuring output ports, going through a lifecycle. All of that exists for good reasons when you are building something for your organisation. But when you just need to run one query to answer a Slack message, that overhead becomes a barrier.

The consequences are predictable. People avoid the governed system. They find workarounds. They ask a data engineer to send them a CSV. They pull data through a personal connection they set up months ago that nobody knows about. Data owners loses visibility into what data is being accessed, by whom, and why. Governance breaks down because the easy path runs around it. People are not careless. The governed route was simply slower than the workaround.

Most data governance programmes run into the same problem: the stricter the process, the more people tend to go around it.

Governance that fits the use case

Our answer was to right-size governance: match the level of process to the level of intent. Not every interaction with data needs the same level of process. What it does need, always, is accountability: a name, a reason, an owner, a record. The question is how much friction sits between the user and the data, and whether that friction is proportional to the intent.

Explorations are a new node type in Portal designed specifically for consumption. Someone who wants to access data to answer a question picks “I want to explore this data” at checkout, gives the Exploration a name and a brief business justification, selects a domain, and is in. The access is recorded, the business justification is logged, and the domain owner has full visibility into what is happening.

What an Exploration looks like in practice

Say a marketing analyst receives a message from their lead: how many customers churned in May? They head to the Marketplace, find the relevant output port, add it to their cart, and at checkout they choose to start an Exploration instead of creating a full Data Product. They name it “May Churn Check”, write a one-line justification, and land in a focused workspace with a guide on how to start querying.

Creating Exploration in Data Product Portal

There are no Output Ports tabs, no Settings, no Usage analytics. This is not a publishing environment. It is a consumption environment. The analyst runs their query, gets their number, and moves on. The whole process takes under two minutes. See the full flow in this short demo video:

Nothing about that interaction was ungoverned. The access request was logged. The business justification was recorded. The domain owner knows an Exploration was created in their domain. But the analyst did not have to become a data product owner to answer a question.

Exploration page in the Data Product Portal

From an Exploration to a full data product

What happens when that one-time question turns into a recurring need? The analyst realises the churn analysis is something the whole team wants to see every month. That is when an Exploration becomes the starting point for a full product.

We are building a promotion path that lets users convert an Exploration into a Standard Data Product, carrying the name, domain, and input ports as a starting point. The work done in the sandbox does not get thrown away. It becomes the foundation for something more permanent. This creates a natural pipeline from curiosity to production: explore first, commit when the value is proven.

What this means for data owners and admins

If you manage data domains, Explorations give you more signal. You see who is accessing data in your domain, what they are looking for, and whether there are patterns worth turning into products. A cluster of Explorations all pointing at the same output port is a strong signal that a formal product might be worth building there.

The bigger principle

Good data governance makes the right path the easy path. For teams building shared, reusable data assets, the full Data Product lifecycle is the right path. For individuals answering a question or testing a hypothesis, Explorations are the right path. The Portal now supports both, and the governance layer runs through both.

Want to try it yourself? The Data Product Portal is open source and free to get started.