Configuration file format for synq-recon database reconciliation tool
No Additional PropertiesShort machine identifier for the suite.
Must match regular expression:^[a-zA-Z0-9][a-zA-Z0-9_-]*$
Must be at least 1 characters long
Must be at most 255 characters long
Human-readable title for the suite.
Human-readable description.
Database connection definitions.
Each additional property must conform to the following schema
Type: objectConnection is a single database connection entry in a connections map.
No Additional PropertiesDisplay name for this connection. Defaults to the connection ID (map key).
When true, the connection is skipped during execution.
Maximum number of parallel queries. Range: 1-256. Defaults to 8.
Value must be greater or equal to 1 and lesser or equal to 256
PostgresConf contains PostgreSQL connection parameters.
No Additional PropertiesValue must be greater or equal to 1 and lesser or equal to 65535
Disable SSL certificate verification.
SnowflakeConf contains Snowflake connection parameters.
No Additional PropertiesSnowflake account identifier.
Virtual warehouse to use for queries.
Role to assume after connecting.
PEM-encoded private key content for key-pair authentication.
Path to a PEM-encoded private key file.
Passphrase to decrypt the private key.
Databases to include. If empty, all accessible databases are scraped.
No Additional ItemsUse GET_DDL() to retrieve DDL for tables and views.
Database containing the ACCOUNT_USAGE schema. Defaults to SNOWFLAKE.
Set to "externalbrowser" to use SSO browser-based authentication.
BigQueryConf contains BigQuery connection parameters.
No Additional PropertiesGCP project ID.
Region for BigQuery resources.
Inline JSON content of the service account key.
Path to the service account key JSON file.
RedshiftConf contains Amazon Redshift connection parameters.
No Additional PropertiesValue must be greater or equal to 1 and lesser or equal to 65535
Estimate table freshness from Redshift query logs instead of metadata.
MySQLConf contains MySQL connection parameters.
No Additional PropertiesValue must be greater or equal to 1 and lesser or equal to 65535
Disable SSL certificate verification.
Additional DSN parameters passed to the driver.
Each additional property must conform to the following schema
Type: stringClickhouseConf contains ClickHouse connection parameters.
No Additional PropertiesValue must be greater or equal to 1 and lesser or equal to 65535
Database to connect to. If empty, all databases are scraped.
Disable SSL certificate verification.
TrinoConf contains Trino / Starburst connection parameters.
No Additional PropertiesValue must be greater or equal to 1 and lesser or equal to 65535
Use a plain HTTP connection instead of HTTPS.
Catalogs to include. Required for most Trino deployments.
No Additional ItemsDatabricksConf contains Databricks connection parameters.
No Additional PropertiesDatabricks workspace URL.
Personal access token for authentication.
OAuth client ID (M2M authentication).
OAuth client secret (M2M authentication).
SQL warehouse ID to use for queries.
MSSQLConf contains Microsoft SQL Server / Azure SQL Database connection parameters.
No Additional PropertiesValue must be greater or equal to 1 and lesser or equal to 65535
Trust the server certificate without validation.
Encryption mode (e.g. "true", "false", "strict").
Federated authentication method (e.g. "ActiveDirectoryDefault").
Pre-acquired access token for Azure AD authentication.
Azure AD application client ID for service principal auth.
OracleConf contains Oracle Database connection parameters.
No Additional PropertiesValue must be greater or equal to 1 and lesser or equal to 65535
Oracle service name.
Enable SSL/TLS for the connection.
Verify the server's SSL certificate.
Path to Oracle Wallet directory for authentication.
Enable Oracle Diagnostics Pack features (AWR, ASH).
DuckDBConf contains DuckDB / MotherDuck connection parameters.
No Additional PropertiesFile path, ':memory:' for in-memory, or MotherDuck database name.
MotherDuck organization/account name (for cloud mode).
MotherDuck authentication token (required for cloud MotherDuck).
Reconciliation scenario definitions.
Each additional property must conform to the following schema
Type: objectReconciliation defines a single reconciliation scenario.
No Additional PropertiesHuman-readable title.
Human-readable description.
Source dataset.
No Additional PropertiesConnection name (must exist in connections map).
Base SQL query (mutually exclusive with 'table'). Supports {{variable_name}} interpolation.
Table reference (mutually exclusive with 'query'). Accepts "db.schema.table" string or structured object.
Table in dotted format: 'table', 'schema.table', or 'database.schema.table'
Structured table reference
Database/catalog name
Schema/dataset name
Table or view name
Must be at least 1 characters long
Explicit column list (only with 'table'). Mutually exclusive with 'exclude_columns'.
No Additional ItemsColumns to exclude (only with 'table', resolved at runtime). Mutually exclusive with 'columns'.
No Additional ItemsTime-travel timestamp. Wraps query with database-specific time-travel syntax.
Column used for bisection split points and aggregate grouping.
Reconciliation mode.
Hash algorithm for row checksums.
Automatically match columns case-insensitively.
Column name mapping between source and target.
Map source column names to target column names (object format)
Each additional property must conform to the following schema
Type: stringMap source column names to target column names (array format)
No Additional ItemsMust be at least 1 characters long
Must be at least 1 characters long
BisectionConfig controls the bisection drill-down behavior.
No Additional PropertiesNumber of segments per bisection level.
Value must be greater or equal to 2 and lesser or equal to 1024
Stop drilling when segment row count falls below this.
Value must be greater or equal to 1
Segmentation strategy.
Column for time-based partitioning (required when strategy is 'time').
Time truncation granularity.
ReportingConfig controls what level of detail is reported.
No Additional PropertiesValue must be greater or equal to 1
AggregateConfig defines aggregate comparison settings.
No Additional PropertiesMust contain a minimum of 1 items
MeasureConfig defines a column + aggregation function pair.
No Additional PropertiesMust be at least 1 characters long
Single aggregation function
Multiple aggregation functions
Must contain a minimum of 1 items
Column(s) defining the drill-down hierarchy.
Single group column
Hierarchical group columns for drill-down
Must contain a minimum of 1 items
ThresholdConfig defines tolerance for aggregate differences.
No Additional PropertiesMaximum absolute difference.
Value must be greater or equal to 0
Maximum percentage difference (0.1 = 10%).
Value must be greater or equal to 0
Percentage formula: source, target, or symmetric.
Per group_column threshold overrides.
Each additional property must conform to the following schema
Type: objectThresholdConfig defines tolerance for aggregate differences.
Same definition as thresholdsPer measure threshold overrides, keyed by FUNC(column).
Each additional property must conform to the following schema
Type: objectThresholdConfig defines tolerance for aggregate differences.
Same definition as thresholdsErrorHandlingConfig controls retry behavior for query failures.
No Additional PropertiesPer-query timeout as a Go duration string (e.g., '30s', '5m').
Maximum retry attempts for transient query failures.
Value must be greater or equal to 0 and lesser or equal to 10
Initial backoff delay before the first retry.
Backoff multiplier applied after each retry.
Value must be greater or equal to 1
WindowConfig defines time-windowed reconciliation settings.
No Additional PropertiesColumn being windowed.
How far back to look from the current time (e.g., '14d', '2h').
Windowing strategy: 'sliding' or 'fixed'.
Dynamic cutoff filter for sync reconciliation. Derives a watermark from data to exclude not-yet-synced rows.
No Additional PropertiesSimple shorthand: same column for both sides. Expands to source.column + target.column.
Medium shorthand: column to derive watermark from on source side.
Medium shorthand: column to derive watermark from on target side.
Advanced: full source-side watermark derivation config.
No Additional PropertiesColumn to derive the watermark from (e.g., createdat, syncedat).
Must be at least 1 characters long
Aggregate function for watermark derivation. Default: MAX.
How to combine watermarks from both sides. Default: MIN (safer).
Truncate the cutoff value to a time boundary (e.g., HOUR truncates 08:47 to 08:00).
Time offset applied after truncation. Negative values subtract (safety buffer). Example: "-30m".
Advanced: custom WHERE clause generation per side.
No Additional PropertiesColumn to filter on. Defaults to the derivation column.
Comparison operator. Default: "<=".
Per-reconciliation setup/teardown.
Each additional property must conform to the following schema
Type: objectEach additional property must conform to the following schema
Type: objectEach additional property must conform to the following schema
Type: objectEach additional property must conform to the following schema
Type: objectWhether teardown runs on failure. Inherits from suite level when not set.
When true, logs setup errors as warnings. Inherits from suite level when not set.
Template variables for query interpolation via {{variable_name}} syntax.
Each additional property must conform to the following schema
Type: stringSetup queries run once before all reconciliations begin.
Each additional property must conform to the following schema
Type: objectTeardown queries run once after all reconciliations complete.
Each additional property must conform to the following schema
Type: objectSQL file(s) to load and run as setup queries.
Each additional property must conform to the following schema
Type: objectSQL file(s) to load and run as teardown queries.
Each additional property must conform to the following schema
Type: objectWhether teardown runs even when reconciliations fail with infrastructure errors.
When true, logs setup query errors as warnings and continues instead of aborting.
When true, time reference detection (NOW(), CURRENT_DATE, etc.) returns an error instead of a warning.
SYNQ API credentials for sending audit logs.
No Additional PropertiesOAuth client ID for authenticating with the SYNQ platform.
OAuth client secret for authenticating with the SYNQ platform.
gRPC endpoint for the SYNQ API (e.g. "developer.synq.io:443").
gRPC endpoint for the SYNQ ingest API. Defaults to Endpoint if empty.
OAuth token URL. Derived from Endpoint if empty.