Next Generation Data Platforms

Our expert for questions

Andreas Odenkirchen
Director and Data & Analytics Expert at PwC Germany
Tel: +49 151 1553-5019
Email

Using data even more effectively within your company based on modern data platform concepts like data mesh and data fabric

What leads our customers to buy a certain product? Which products are subject to the risk of supply shortages? How can I access the relevant data from other departments? And what exactly does a particular KPI mean?

Many companies are facing an increasing number of questions throughout all their departments regarding the handling of data. The correct usage and usability of data has long been a crucial factor in converting your strategic and economic vision into tangible success. Many challenges are arising because data sources are becoming more heterogeneous and data volumes are growing exponentially. This increases the entry barrier for non-specialized employees who have to process and analyse this data. Data quality fluctuates with each additional source, and the technologies required get even more complex.

Video

next-gen-data-platform 885x498

Thus, centralized data platforms such as data warehouses and data lakes which were established within the company are increasingly being complemented – or even replaced – by decentralized and more automated approaches. In this way, new concepts like data mesh and data fabric entail a paradigm shift for the handling of data in enterprises. However, there is not one blueprint applicable to every company. On the way to your next-generation data platform, we support you on your strategy, concept, realization, and upskilling to create a customized model tailored to your needs.

“There is no uniform solution for data management – each organization has to find its own degree of centralization and automation. However, modern approaches such as data mesh and data fabric can play a key role in making data usage faster and better within your company.”

Andreas Odenkirchen,Director and Data & Analytics Expert

Increasing Relevance and New Challenges

Enterprise data management is not a new discipline – but its significance has grown exponentially in recent years in line with rapid digitalization in order to make both data collected within the company and relevant external data as usable as possible. It is also crucial for ensuring compliance with regulatory requirements and avoiding costly penalties.

The systematic handling of data is becoming increasingly important for a variety of reasons:

B2C customers want new and personalized products and services, while corporate customers are also demanding greater flexibility and transparency.
Technological developments such as the IoT are increasing data volumes. At the same time, technologies such as artificial intelligence rely on ever more data.
Regulations are posing stricter requirements for data processing, and political and public awareness are growing.

This is set against a series of challenges arising from realization:

Effective data usage is difficult to achieve due to data silos, quality deficiencies, and ambiguous governance.
The shortage of qualified staff remains a problem, exemplified by the rarity of data architects, engineers, and analysts. There is often also a lack of understanding within the domains for the correct acquisition, classification, and usage of data.
Complex IT landscapes undergoing transformation impede uniform standards and interfaces.

Do you have any questions?

Contact Our Data Platform Experts

Model Overview: The Ideal Platform for Your Data

Data Warehouse
Data Lakes and Data Lakehouses
Data Mesh: Decentralized and specialized
Data Fabric: The Efficient Generation of Knowledge

Data Warehouse

A comprehensive enterprise data warehouse (EDW) is the original and firmly established approach for enterprise data management for analytical purposes. According to our Data Mesh Study 2022, approximately 84 percent of German companies rely on a central datawarehouse, in which they manage and combine structured data from various sources – e.g. ERP, CRM, MES – in a single monolithic system. Although this makes the everyday use of data easier, it also reaches its limits quickly, depending on the application. The issue is that due to its central administration and rigid data transformation paths, this approach is inflexible and sluggish for new processes, sources, and applications. In addition, its data variability has technical limits, and the costs associated with large highly available data sets are rising fast.

Data Lakes and Data Lakehouses

The data lake approach arose from the challenges with DWHs. Around 78 percent of all companies collect raw data – e.g. relating to operations, projects, and customers – in a data lake. It is not necessary to first homogenize this data, which makes its use and analysis much more flexible. If needed, this data is then moved to specific DWHs with their own guidelines – data lakehouses. In this way, any data can be used and analysed using big data methods and (cloud) technologies. The system is scalable, the storage and backup of large sets is affordable and ready to be processed using artificial intelligence – while data management remains central and the system is monolithic. Because of this, data lakes can quickly turn into an organization's bottleneck. Departments cannot access them directly, while conversely data engineers lack domain-specific data expertise. The quality of data can also suffer from the high degree of freedom and less direct responsibility.

Data Mesh: Decentralized and specialized

83 percent of all surveyed companies stated that they want to use the newly emerging data management approach of data mesh in the future, while 15 percent are already doing so. This approach is based on decentralized data architecture which manages data in individual data domains – for example for the Marketing, Finance, and HR departments. Each domain has its own data landing zones and manages data products which are provided to other domains for analytical use. This bolsters the autonomy of the data producers and leads to the domains analysing their requirements for data itself more closely. This can ultimately boost data quality, create new applications, diminish data silos, and reveal new synergies.

Data Fabric: The Efficient Generation of Knowledge

Data fabric is another new approach for efficient data management which - from our perspective - complements the data mesh very well. The objective of this approach is to automate the connection of logically linked data in order not just to integrate domain knowledge in the data trove but also to facilitate end-to-end thinking with regard to the available inventory of data. Metadata relating to the available data points which illustrates their semantic significance, source, and the flow of data between systems is essential for this purpose. Manual data modelling can be supported by an AI-based knowledge graph enabling its users to penetrate the relationships between data points more easily while also recommending new linked analyses. This facilitates the efficient generation of new insights and additional added value from data.

The Data Platform for the Next Generation

Eight Components of Your Next Gen Data Platform

Although the decentralized approach of data mesh has the potential to reduce bottlenecks in the value-adding handling of data, data lakes and data warehouses also continue to be relevant for enterprises. We can help you find the data architecture that best addresses your future requirements and generates maximum business value from data.

In doing so, we focus on eight key components which we consider crucial for your next-generation data platforms:

In addition, you can count on us for the

Assessment of your current architecture and organisation including challenges and requirements for your next-generation data platform
Strategies for data mesh and data fabric, as well as hybrid models along all requirements and key skills (e.g. data domains, interdisciplinary data product teams, DataOps methods, data catalog, data marketplace, and data governance automation)
Technical implementation or modernization of your data platform on the basis of modern cloud technologies
Holistic support of the data transformation or gradual implementation and enablement of individual use cases, domains, and applications
Upskilling of your employees for leveraging data and analytics possibilities

Data Mesh & SAP - Successfully connecting corporate data with SAP BTP

Many companies face challenges such as monolithic, centrally organized data platforms, organizational bottlenecks, and critical gaps between experts and the information relevant to them. Data mesh concepts provide a solution to these challenges. SAP offers the suitable technological foundation for this new form of data management with the Business Technology Platform (SAP BTP).

Learn more

“From the assessment of your status quo to the implementation of a central, decentral, or hybrid model: Our experts can assist you along the path to your next-generation data platform to unleash the most business value from your data.”

Andreas Odenkirchen,Director and Data & Analytics Expert