Data Mesh Manager

Architecture

Data Mesh Manager is a Software as a Service (SaaS) or self-hosted application.

Users interact with the Data Mesh Manager website to discover data products, manage data contracts, request and approve access, and create and track governance policies. Data Mesh Manager sends notifications to users, e.g., when a new data usage agreement was requested.

Data Mesh Manager has an extensive API to integrate with other application and for data platform integration. As a key design principle, for security reasons, no connections are established into the customer's network. All connections are initiated by the customer and data is pushed to the Data Mesh Manager or polled for events.

Integrations

While Data Mesh Manager can be used fully standalone and all data can be updated manually, integrations with the data platform is recommended for enhanced user experience. Integrations exists for major data platforms, and custom integration can be implemented using the Java SDK.

  1. Push data product and data contract metadata to Data Mesh Manager, when data products are deployed as part as a CI/CD pipeline step.
  2. Integrate with Data Contract CLI or other data quality engines to test data contracts and send test results to Data Mesh Manager
  3. Synchronize data assets in the data platform , such as tables, files, and dashboards to Data Mesh Manager. This can also be integrated with a data catalog, if available. Prebuilt integrations exists for Databricks / Unity Catalog, Snowflake, and GCP.
  4. Trigger automations, such as granting permissions or notifications based on Events