Data Mesh Manager

Quality

Quality checks are supported in data contracts.

Quality checks in the data contract describe executable expectations of the data quality.

The Data Mesh Manager supports quality checks in different types.

SodaCL

Use this type when you use SodaCL.

Learn more: SodaCL.

Example of a SodaCL quality check in YAML

quality:
  type: SodaCL
  specification: |-
    checks for search_queries:
      - freshness(search_timestamp) < 1d
      - row_count > 100000
      - missing_count(search_query) = 0

Custom

Use a custom quality check if none of the above quality checks fit your needs. Be aware that the Data Mesh Manager will not be able to validate or parse the quality check, and therefore, cannot provide any automation.

Example of a custom quality check

quality:
  type: custom
  specification: >
    Any quality check
    in any format
    as a string