Mike Evans, Chief Innovation Officer at Amplifi discusses how data mesh has the power to transform your data into a vital business tool – essential for informed decision-making and driving innovation.

For organizations aspiring to unlock and leverage the value of their data, the concept of data mesh has piqued significant interest. At first glance, data mesh might appear to be a re-hashing of approaches we’ve seen before, and perhaps previously discouraged, approaches. However, when combined effectively, it offers a transformative strategy for integrating data into business operations. This approach promises to deliver scalability and agility in data and analytics, without the reliance emerging and immature technologies that data fabric (arguably) requires. As such, it can provide a route to increased scalability ahead of full adoption of the data fabric approach.

Zhamak Dehghani’s “Data Mesh: Delivering Data-Driven Value at Scale” (2022) is not merely about enhancing and optimizing existing processes; it’s about fundamentally redefining how businesses operate through data. Yet amidst the excitement, there are sceptics. Some may feel the enthusiasm is exaggerated, or that it’s been talked about to death. However, beneath the initial hype lies a robust framework founded on four key principles. Individually, these principles might not be novel, but together, they present a persuasive blueprint for a data strategy in certain organizations.

So, what do we mean when we say data mesh? There are four key principles:

• Domain Centric Ownership
• Federated Governance
• Data as a product
• Self-service platform

Whether you feel you need one more than the others or would like to apply all four, let’s explore each principle in a bit more depth.

Domain Centric Ownership

This principle encourages aligning the data architecture with the organizational structure by advocating for domain teams to fully own their data. Those with the closest understanding of the data’s context should manage it. By only worrying about the needs of your specific context, it’s simpler than trying to solve the rest of the organization’s needs as well.

The challenge here is to enable agility whilst avoiding the creation of disparate data silos that obscure the broader perspective. To prevent that, you’ll need to implement some governance.

Federated Governance

Federated governance is about striking the right balance between structure and flexibility. It sets up a framework of core principles and policies that guide the entire organization, while also empowering individual teams to tailor their own detailed practices. Think of it as a shared vision where the broad strokes are defined, leaving room for each team to fill in the details with their unique colors and textures.

It’s important to have a model that has a thin slice of central data governance, focused almost exclusively on ensuring that the mesh itself can function successfully, combined with distributed governance within the domains. The platform that underpins the mesh should enforce the principles and policies defined by this governance model, across the organization.

This method ensures that your organization operates as a united entity, without the constraints of a centralized, top-down governance model that can often dampen innovation and slow down decision-making.

Data as a product

Adopting a product-centric view of data in a data mesh environment means we handle it with the same care and attention that (for example) a retailer would with a real-world product – by having a clear description, exact specifications, origins, measurements, etc., the customer (data user) will know exactly what the product (data) can be used for and what to expect from it. Data becomes a living asset, continually enhanced to remain valuable and relevant, retired when it is superseded or obsolete.

This approach transforms data into a vital business tool, essential for informed decision-making and driving innovation. Those responsible for the data become its champions, focused on its integrity and alignment with the organization’s strategic goals. Short and sweet, it’s about making data work harder and smarter for everyone involved.

Self-service platform

Despite the ambiguous name, the self-serve data infrastructure principle is perhaps the most transformative, as it enables the other principles to be implemented seamlessly. It is not prescriptive about the technologies employed; instead, it advocates for a platform that equips domain teams with the necessary tools and systems to independently develop, execute, and maintain data products. This domain-agnostic platform is the cornerstone upon which the data mesh is built, ensuring that the infrastructure supports the data products and services required by the business.

From ideation to implementation

While these principles provide a structural framework, the true essence of data mesh is found in the cultural shift it necessitates. It’s a reimagining of the role of data within the organization, fostering a mindset where data is treated with the same strategic consideration as other core business functions. This cultural evolution doesn’t happen overnight; it requires dedication to change management, persistent advocacy, and a willingness to redefine the norms of data stewardship.

The full benefits of data mesh might not be revealed if you’re working in smaller teams or individual departments because it thrives on scale. Without a substantial uptake, you may find the available data is limited, which constrains the mesh’s effectiveness. It’s crucial, however, to start with a manageable approach that still offers tangible benefits.

What does a good data mesh look like?

We’ve seen some organizations excel in defining what quality data products look like, crafting guidelines that outline what these products should achieve and how they should be governed. The challenge lies in ensuring these guidelines don’t appear too daunting at first glance. It’s about emphasizing the essential steps and making it clear that additional quality elements can enhance a data product, without making them prerequisites from the start.

Remember we want to encourage participation in the mesh, without recreating the age-old problem of ungoverned silos. It should be straightforward to setup and register data products in the first instance but putting increased effort into managing the quality of data products should reap benefits. A crucial factor in the success of data mesh is ensuring there is appropriate incentive for data product owners to delight their customers across the organization.

The goal is to foster adoption by minimizing barriers to entry and simplifying the initial steps. It’s okay for early data products to be less than perfect; what’s important is establishing the process and understanding the data mesh framework. Over time, as more teams engage and contribute, the quality and value of the data products are expected to improve, driven by the evolving needs and feedback within the organization. It’s a progressive journey, not a pursuit of immediate perfection.

Do you need data mesh?

The main argument for data mesh is for scalability across complex organizations. With businesses generating so much data, across so many domains, it’s becoming more difficult to centralize all aspects of that data’s management and analysis: central IT/data teams can become a bottleneck. It is unlikely to be the right approach for every organization. Factors such as risk appetite and organizational culture will determine whether data mesh is the right approach. Further, hybrid approaches, which adopt the principles of mesh but not across all business or data domains, are already being adopted successfully by some organizations.

ata mesh empowers people across the organization to access and get value from data. When implemented well and underpinned by an effective data catalogue, it can help reduce the time that analysts and data scientists spend searching for, wrangling and interpreting data.

The future of data mesh

We believe that the data mesh model holds great promise for certain organizations, though not universally for all. It’s particularly potent when it meshes—no pun intended—with the principles of data fabric. These two concepts are distinct yet perfectly compatible, each enhancing the other’s strengths.

Balancing mesh and fabric

While data mesh may be amongst many current buzzwords, it’s crucial to consider it within the broader context of data management strategies. Data fabric, with its promise of a more interconnected and seamless data integration, holds considerable value for the future. Organizations should evaluate their readiness and strategic objectives to discern whether data mesh, data fabric, or a hybrid approach best suits their journey towards a data-centric future.

Data mesh’s big challenge

Data mesh is still in its infancy as an approach, and while a lot of organizations want to introduce aspects of it, many aren’t ready to introduce all four pillars. Moving away from centralized data management could lead to chaos if the organizational culture, governance and operating models are not right.

The secret sauce for a successful data mesh approach lies both in the cultural and organizational implementation but also in embracing advanced technologies and leveraging the metadata created by the participants in the mesh. Without this tech backbone, moving from a data mesh to a data fabric will be overwhelming and complicated. Those who tap into active metadata and let it guide the evolution of their data mesh will likely outpace competitors who don’t.

So, here’s a nugget of advice: if you’re considering data mesh, don’t overlook the technology and principles underpinning data fabric. Investing in these will not just serve you now, but also set you up for long-term success.

Let’s talk Data Mesh

Need help defining your strategy? Already started but need support delivering it? Amplifi is actively engaged in multiple Data Mesh engagements with our customers and happy to help support you on your journey. Want to know why to choose Amplifi as your Data Mesh consultancy? Read more on our Data Mesh services page here or get in touch with our data experts today.