Skip to main content

Constraints Check

Guzzle will Perform Data Quality (DQ) validation on specific columns and tables, Logging of records and statistics failing the constraint checks and the validation rules applicable for structured data and can currently be specified as SQL.

Guzzle Support below datastores for Constraint Checks.

  1. JDBC
  2. Hive
  3. Delta

Guzzle provider below functionality and parameters for Constraint Checks.

PropertyDescriptionDefault ValueRequired
DatastoreYou can choose any of the available datastore from drop-down (as appropriate). If the connection is not available in drop-down then you have to create new.NoneYes
Table + FilterSpecify the table from where to read data. The table can contain schema and also database name. We can specify the Schema Name along with the Table Name. If it is not specified it is taken from the Datastore.

Also, when the Table option is selected, you can also specify the Filter. The filter can be used to select only a part of the table. We may basically enter a condition and only the part of the table that satisfies the condition will be seen in the Target Section.
SQLAlternatively Specify the SQL query which will be run as-is on the sourceNoneYes
Primary KeysSpecify primary key columns to be logged into constraint_check_detail table if constraint check fails for a record. You can specify up to 10 columns if it is a composite primary keyNoneYes
Grouping ColumnsSpecify grouping columns in source table if source data has to be aggregated before performing constraint check. All grouping column names and values are logged into constraint_check_summary, constraint_check_detail tables if constraint check fails for a record. You can specify up to 10 columns for group by to perform aggregation.NoneNo

Validation Rules#

You can specify constraint checks to be performed on the source table. Below are the properties in that we can mention constraint checks with some rules.

PropertyDescriptionDefault ValueRequired
NameSpecify a name for validation ruleNoneYes
SQLSpecify validation SQL which should evaluate to TRUE if the record passes the constraint check. Records which doesn't qualify the rule specified under this option are logged into constraint_check_summary, constraint_check_detail tablesNoneYes
Constraint DataSpecify list of columns for which additional data to be logged into constraint_check_summary, constraint_check_detail tablesNoneNo
Detail Check EnabledSpecify if you want to bring detailed information on reconciliation records, if it’s true then guzzle will calculate results for the mismatched records and store them in the guzzle repository table constraint_check_detail.FalseNo