Over the past few years, organizations have invested heavily in gathering information. They implemented CRM, ERP, e-commerce platforms, analytics tools, customer service systems, and internal applications. Each new solution promised greater visibility into operations, but also created an additional data source, new definitions, and further integrations that needed to be maintained.
The result is a frequent contradiction. The company has millions of records, but departments continue to argue about which figure is correct. Marketing reports one number of customers, sales presents another, and finance uses a third definition. The same metric changes depending on the system consulted, the time of data collection, or the person who prepared the report.
This fragmentation was problematic when data was primarily used in reports. With the advent of artificial intelligence and enterprise agents, the risk increases. An automated system cannot stop a meeting to ask which definition to use. It processes the available information and produces a response, recommendation, or action.
Therefore, the challenge is no longer simply about storing data. It's about transforming it into reliable assets that can be discovered, understood, and used consistently. That is the purpose of... data products.
A data product is a set of information designed to meet a specific user need. It can take the form of a table, an API, an analytical model, an event stream, a dashboard, or a combination of several components. The difference compared to a traditional dataset is that it is managed as a product with assigned responsibilities, users, documentation, quality criteria, and a defined lifecycle.
Google Cloud describes the data mesh approach as an architectural and organizational framework that treats data as products developed by the teams that best understand each business domain. These teams must adhere to common governance standards so that information can be discovered and consumed consistently across the organization.
Let's consider a product called "360 Customer." It shouldn't just be a table with names, emails, and purchases. It needs to define what constitutes an active customer, which systems provide information, how often it's updated, what level of quality it offers, and who is responsible when an error occurs.
The product does more than just deliver records. It delivers a reliable ability to make decisions about customers.
In many business systems, data appears as a byproduct of operations. A CRM records opportunities because it needs to manage sales. An ERP stores invoices because it must support financial processes. A support platform retains conversations to manage cases. The information exists, but it wasn't necessarily designed to be reused by other departments.
When an analytical team needs to combine these sources, an interpretation process begins. They must discover what each field represents, which records are incomplete, what rules the system uses, and how one entity relates to another. A significant part of the effort is not spent analyzing, but understanding.
This model fosters a reliance on informal knowledge. One person knows that a certain column shouldn't be used after a certain year. Another remembers that some records are duplicated. A third knows the correct query to exclude internal tests. When these people change roles, the organization loses some of the meaning behind its own data.
A product-focused approach requires making that knowledge visible. Information must be documented, tested, and maintained so that other consumers can use it without constantly relying on informal conversations.
When data is treated as technical waste, its quality often becomes nobody's responsibility. The source team claims they only manage the application. The analytics team receives incomplete information and tries to correct it. Business units use different reports, and problems only surface at the end of the process.
A data product needs clear ownership. The responsible team must understand the business domain, who consumes the information, and maintain a reasonable level of service. This doesn't mean a single person should resolve all technical and functional issues. It means there is a clear accountability for the product's reliability.
The data mesh model proposes precisely a domain-oriented ownership structure. Sales can be responsible for products related to opportunities and potential customers. Finance can manage information about billing, revenue, and payments. Human Resources can manage products related to talent and organizational structure.
Decentralization doesn't eliminate governance. It changes its form. Teams gain responsibility for their data, but operate under common standards that enable interoperability, security, and control.
One of the most important differences between storing data and building a product is understanding who it's for. A product without defined users risks becoming just another collection of information that no one knows how to use.
Consumers can be analysts, applications, executives, predictive models, or AI agents. Each requires different features. An analyst might need historical detail and flexibility. A transactional system needs fast responses and stable contracts. An AI agent needs understandable context, clear permissions, and sufficiently reliable data to support a decision.
Designing from the consumer's perspective requires asking specific questions. What problem are they trying to solve? What level of updates do they need? What is the impact of an incorrect answer? Which fields are essential? Which definitions need to be explained?
Google Cloud recommends that data mesh architectures allow for different consumption patterns and that users can evaluate whether a product is suitable based on its purpose, quality, and access conditions.
The value of a product is not in the number of columns it contains. It is in its ability to facilitate a specific decision or process.
A software product is not considered finished simply because it works once. It needs documentation, support, metrics, versioning, and mechanisms to manage changes. Data products require a similar discipline.
They must be discoverable so users know they exist. They need a clear description, visible ownership, and usage examples. They must also be interoperable, meaning they use standards that allow them to be combined with other products without excessive modifications.
Reliability is equally important. Consumers need to know the frequency of updates, completeness, availability, and any limitations. If a source stops receiving information for two days, the product should communicate this before a report or agent uses incomplete data.
Martin Fowler points out that designing data products involves automating governance and ensuring that independent teams follow common practices that enable interoperability at scale.
The data product should not be evaluated solely on its technical existence. It should be evaluated on its usefulness, reliability, and user experience.
One of the biggest problems in data architecture occurs when a system changes without notifying its users. A team might modify a column name, change a data type, or remove a value that seemed unnecessary. The original application continues to function, but the reports, models, and automations that relied on that structure begin to malfunction.
A data contract formally defines what a consumer can expect. It can include schema, required fields, types, update frequency, quality rules, owner, and compatibility policies. Its purpose is to prevent dependencies from remaining hidden.
Virgin Media O2 explained how it used data contracts as a quality and assurance layer to ensure that published datasets were reliable, documented, and ready for consumption.
The contract doesn't need to be a handwritten document that no one updates. It can be expressed in machine-readable formats and automatically validated within pipelines. If a change violates an agreed-upon condition, the system can halt publication or alert the relevant parties before it affects consumers.
Trust no longer depends solely on conversations. It becomes a verifiable property.
Every organization needs to evolve. New fields emerge, processes are modified, and business definitions change. The goal of a data contract is not to permanently freeze the structure, but to manage change responsibly.
When a modification is compatible, it can be implemented without affecting consumers. When it breaks the existing contract, it should be addressed through a new release, a transition period, or advance notice. This gives consumer teams time to adapt their processes.
This approach reduces silent failures. Without contracts, a column can continue to exist but change its meaning. The pipeline keeps running, and the problem appears in the wrong report. With semantic and quality rules, the organization can detect when information no longer meets the expected conditions.
Version control also improves traceability. A team can know which model a particular version of a product used and reconstruct why a result changed.
Stability does not mean the absence of change. It means that change occurs without unexpectedly disrupting the ecosystem.
Many organizations discover quality problems when an executive questions a figure or a customer receives incorrect information. At that point, an investigation begins that may involve multiple systems, consultations, and teams. The company corrects the incident, but doesn't always modify the process that allowed it to occur.
Data products require continuous monitoring. This can include validations for completeness, uniqueness, consistency, freshness, and compliance with domain rules. A customer table should not accept duplicate identifiers without explanation. A billing product needs to detect unexpected negative values. An inventory flow should alert when it stops updating.
Martin Fowler proposes using fitness functions or fitness functions to automate governance rules and continuously check if products meet organizational standards.
Quality is not a final activity performed by a separate team. It must be integrated into the development and deployment process. The goal is to prevent faulty data from becoming a reliable dependency for other systems.
Generative models can produce convincing answers even when the available information is incomplete or contradictory. This makes the quality of the context especially important. An agent connected to fragmented sources doesn't gain intelligence simply by using an advanced model. It gains a faster way to combine inconsistencies.
Microsoft points out that agents synthesize information rather than creating the business facts they need. For this reason, their accuracy depends on the quality, accessibility, and governance of the underlying sources. Fragmented or uncontrolled information can lead to misleading responses and security risks.
A data product provides defined context. A sales agent can consult an active opportunities product instead of connecting directly to multiple undocumented tables. A finance assistant can use a validated revenue product with known rules and responsible parties.
This reduces ambiguity and facilitates evaluation. The company can analyze whether the agent used the correct product, which version they consulted, and what limitations the information had.
AI does not eliminate the need for data architecture. It makes it much more urgent.
Connecting a model to all business databases might seem like a quick way to build a powerful assistant. However, this approach creates security, semantic, and maintenance problems. The agent encounters undocumented fields, sensitive data, and structures designed for internal needs that shouldn't be used directly.
The product layer allows you to control what information is made available for consumption. Each product can expose only the necessary fields, apply privacy policies, and offer definitions suitable for both humans and machines.
This separation also protects operational systems. Instead of running unpredictable queries on critical applications, agents consume controlled interfaces, replicas, or services designed for analytics and AI.
Modern architectures for enterprise agents recommend secure and governed access to structured and unstructured sources, especially when systems need to operate across different platforms or clouds.
The goal is not to artificially limit AI. It's to offer it an information surface that can be used with confidence.
One of the most common projects involves creating a “single source of truth.” The organization gathers information from various systems onto a central platform and expects to automatically resolve inconsistencies. However, consolidating data does not eliminate differences in meaning.
Sales might consider any company with an open opportunity a customer. Finance might only recognize organizations that have already received an invoice. Support might work with end users even if they don't have a direct contractual relationship. None of these definitions are necessarily incorrect. They simply reflect different contexts.
A data product should declare which definition it uses and for which decisions it is appropriate. Instead of pretending there is a single, universal view, the architecture can offer related but semantically clear products.
This discipline is especially important for AI. A question like “How many customers do we have?” needs context. Without an explicit definition, the agent might select a technically available number and present it with a confidence that the company shouldn't grant it.
Semantic quality matters just as much as structural quality. A field can be complete and yet completely useless if no one knows what it means.
The concept of a data mesh has gained popularity, but it has also been oversimplified. Some organizations interpret it as simply providing each department with its own platform or set of tables. The result can be even greater fragmentation.
Data mesh combines four principles: domain-oriented ownership, data as a product, self-service infrastructure, and federated governance. Decentralization only works when teams share standards, platforms, and interoperability mechanisms.
Federated governance allows decisions to be made closer to the domain without sacrificing business consistency. One team may have a better understanding of the sales logic, but it must still adhere to common rules regarding security, identification, documentation, and access.
Self-service infrastructure reduces the cost of complying with these rules. If each domain has to build its own pipelines, catalogs, and controls, the strategy will be prohibitively expensive. A common platform transforms best practices into reusable capabilities.
Data mesh is not a specific technology. It is an organizational and architectural model for scaling responsibility without losing control.
A traditional catalog might display technical names, locations, and diagrams. This helps in finding information, but it doesn't necessarily indicate whether it should be used. Two products can contain similar data but offer different levels of quality or updates.
A product-oriented catalog needs to include purpose, owner, consumers, quality, lineage, access policies, and examples. It should also indicate whether the product is active, in development, or nearing retirement.
Discoverability reduces duplication. When teams don't know what information exists, they create new extracts and parallel versions of the same data. Each copy introduces additional costs, different rules, and new opportunities for inconsistency.
For AI agents, the catalog can become a semantic layer that allows them to select appropriate sources. However, this capability requires reliable and governed metadata.
An organization doesn't gain value from cataloging thousands of tables that no one understands. It gains value when a user or system can quickly find the correct source and understand the conditions under which it should be used.
When each product needs to go through extensive manual processes, teams look for shortcuts. They create temporary datasets that end up becoming permanent, share files outside of official platforms, or develop integrations that no one records.
Governance should be integrated into tools and pipelines. Sensitive classifications can be applied automatically. Contracts can be validated during delivery. Access policies can be configured based on domains and responsibilities.
Google Cloud defines data governance as an approach that covers the entire lifecycle, from acquisition and use to secure disposal, and emphasizes that the value of information depends on it being trustworthy, discoverable, and governed.
Automation allows the right choice to also be the simplest. Teams shouldn't have to complete dozens of steps to publish a basic product if the platform can apply standards by default.
Effective governance doesn't prevent people from using data. It makes it possible to use data at scale without sacrificing security or trust.
Even a well-designed product can degrade. A source might stop sending records, an integration might start duplicating them, or an update might modify a critical distribution. The pipeline may continue to run technically, even though the information is no longer valid for its intended purpose.
Data observability analyzes behavior, volume, freshness, distribution, and lineage. It allows for the identification of anomalies before they affect reports, models, or agents.
It also makes it easier to assess impact. If a product experiences an incident, the product lineage helps identify which panels, applications, and models depend on it. Without this visibility, the organization discovers affected consumers gradually and reactively.
For AI systems, this capability is fundamental. An agent can continue responding even if its data source has stopped updating. The architecture needs to communicate the product's status and, when the risk warrants it, temporarily disable its use.
Trust shouldn't be binary. Consumers need to know the current state of information and decide whether it can be used for a recommendation, automation, or a critical decision.
Organizations often focus on creating new data sources, but few eliminate those that should no longer be used. Over time, duplicate products, abandoned versions, and datasets whose owners have moved to other departments appear.
Every product needs a defined lifecycle. There should be a design, release, operation, evolution, and retirement stage. When a version is no longer recommended, consumers should be provided with a migration path.
Removing the platform prevents it from becoming a catalog of conflicting options. It also reduces storage, support, and governance costs.
Before removing a product, the company needs to understand its dependencies. This is another reason why lineage and contracts are important. A seemingly inactive dataset might be fueling a monthly automation or a process that no one remembers.
Product management requires making decisions about continuity. Maintaining everything indefinitely is not a data strategy. It's technological accumulation.
The first step shouldn't be to reorganize all teams or immediately migrate to a new platform. It's best to identify a domain with clear trust, duplication, or access issues. Customers, sales, inventory, or billing are often good candidates because they generate cross-functional value.
Next, a specific use case should be selected. Instead of creating an overly broad product, the company can design one that addresses a specific need. This makes it easier to define users, expected quality, and metrics.
Next, ownership, documentation, a contract, and automated controls are established. The product is published in a catalog, and its usage is monitored. This feedback allows for adjustments to the design before extending the model to other domains.
Initial success is not measured by the amount of centralized data. It is measured by the reduction in time spent finding information, the decrease in errors, and the reuse of the product in different processes.
The strategy should grow from proven value, not from a theoretical transformation that is too large to execute.
At The Cloud Group, we help organizations transform fragmented information into a reliable architecture for analytics, automation, and artificial intelligence. Our work begins by understanding the processes, source systems, and decisions the company needs to improve.
We design data products, integrations, contracts, cloud platforms, and governance models that enable the consistent use of information from CRM, ERP, internal applications, and external sources. The goal is not simply to move data to a new platform; it's to preserve its meaning, quality, and traceability.
We also help prepare sources for AI agents and generative applications, establishing clearly defined access layers, security controls, and business contexts. An agent shouldn't have to navigate through conflicting systems hoping to find the truth. It needs reliable products built for its function.
Artificial intelligence can accelerate decisions and automate processes, but it cannot indefinitely compensate for a fragmented data architecture.
Because before building an AI-driven company, it is necessary to build a company capable of trusting its own information.
It is a set of data designed and managed to address a specific need of its consumers. It includes information, documentation, responsible parties, quality rules, access policies, and a defined lifecycle.
A dataset is a collection of information. A data product adds purpose, ownership, quality standards, documentation, support, and consumption conditions. Its goal is to generate recurring and reliable value.
It is a formal agreement between those who produce and consume information. It defines schemas, fields, types, update frequency, quality rules, and compatibility conditions. It can be automatically validated within pipelines.
The products provide AI systems with reliable, documented, and governed sources of information. This reduces the likelihood of models and agents using incorrect, outdated, or out-of-context information.
It is an architectural and organizational approach that distributes data responsibility across business domains. It relies on decentralized ownership, data as a product, self-service platforms, and federated governance.
No. Data mesh typically makes more sense in organizations with multiple domains, teams, and scalability challenges. Smaller companies can apply data product principles without undergoing a complete organizational transformation.
There must be a visible owner within the domain who understands its business significance. Data engineers, analysts, security specialists, and platform managers may also be involved.
Through dimensions such as completeness, accuracy, consistency, uniqueness, freshness, and availability. The metrics should relate to the real needs of your consumers.
The change must be evaluated against your contract. If it is incompatible, it may require a new version, advance notice, and a transition period for consumers to adapt their processes.
Español Español 
English
Français
العربية
Português
Deutsch
Italiano
