SYNQ DWH Agent Configuration

Type: object

Configuration file format for synq-dwh agent

root agent

Agent

Type: object

Agent configuration

root agent log_json
Type: boolean

root agent log_level

Log Level

Default: 0

root agent log_level anyOf item 0
Type: string
Must match regular expression: ^LOG_LEVEL_UNSPECIFIED$
root agent log_level anyOf item 1
Type: enum (of string)

Must be one of:

  • "LOG_LEVEL_TRACE"
  • "LOG_LEVEL_DEBUG"
  • "LOG_LEVEL_INFO"
  • "LOG_LEVEL_WARN"
  • "LOG_LEVEL_ERROR"
root agent log_level anyOf item 2
Type: integer

Value must be greater or equal to -2147483648 and lesser or equal to 2147483647

root agent log_report_caller
Type: boolean

root agent name
Type: string Default: ""

Name of the agent instance

root agent tags
Type: array of string

Tags to categorize and organize the agent

 No Additional Items

Each item of this array must be:

root agent tags tags items
Type: string

All properties whose name matches the following regular expression must respect the following conditions

Property name regular expression: ^(logJson)$
root agent ^(logJson)$
Type: boolean

All properties whose name matches the following regular expression must respect the following conditions

Property name regular expression: ^(logLevel)$
root agent Log Level

Log Level

Default: 0

root agent Log Level anyOf item 0
Type: string
Must match regular expression: ^LOG_LEVEL_UNSPECIFIED$
root agent Log Level anyOf item 1
Type: enum (of string)

Must be one of:

  • "LOG_LEVEL_TRACE"
  • "LOG_LEVEL_DEBUG"
  • "LOG_LEVEL_INFO"
  • "LOG_LEVEL_WARN"
  • "LOG_LEVEL_ERROR"
root agent Log Level anyOf item 2
Type: integer

Value must be greater or equal to -2147483648 and lesser or equal to 2147483647

All properties whose name matches the following regular expression must respect the following conditions

Property name regular expression: ^(logReportCaller)$
root agent ^(logReportCaller)$
Type: boolean

Additional Properties of any type are allowed.

root agent additionalProperties
Type: object

root connections
Type: object

Map of connection configurations

Each additional property must conform to the following schema

root connections Connection

Connection

Type: object

Connection represents a database connection configuration

root connections Connection bigquery

Big Query Conf

Type: object

BigQuery specific configuration

root connections Connection bigquery project_id
Type: string

GCP project ID

root connections Connection bigquery region
Type: string

Region for BigQuery resources

root connections Connection bigquery service_account_key
Type: string Default: ""

Service account key JSON

root connections Connection bigquery service_account_key_file
Type: string Default: ""

Location of service account key file

All properties whose name matches the following regular expression must respect the following conditions

Property name regular expression: ^(projectId)$
root connections Connection bigquery ^(projectId)$
Type: string

GCP project ID

All properties whose name matches the following regular expression must respect the following conditions

Property name regular expression: ^(serviceAccountKey)$
root connections Connection bigquery ^(serviceAccountKey)$
Type: string Default: ""

Service account key JSON

All properties whose name matches the following regular expression must respect the following conditions

Property name regular expression: ^(serviceAccountKeyFile)$
root connections Connection bigquery ^(serviceAccountKeyFile)$
Type: string Default: ""

Location of service account key file

Additional Properties of any type are allowed.

root connections Connection bigquery additionalProperties
Type: object

root connections Connection clickhouse

Clickhouse Conf

Type: object

root connections Connection clickhouse allow_insecure
Type: boolean Default: false

Whether to use disable SSL for connection

root connections Connection clickhouse database
Type: string Default: ""

Database name

root connections Connection clickhouse host
Type: string

Host address

Must be at least 1 characters long

root connections Connection clickhouse password
Type: string

Password for authentication

root connections Connection clickhouse port

Port number

root connections Connection clickhouse port anyOf item 0
Type: integer

Value must be strictly greater than 0 and lesser or equal to 65535

root connections Connection clickhouse port anyOf item 1
Type: string
Must match regular expression: ^-?[0-9]+$

root connections Connection clickhouse username
Type: string

Username for authentication

All properties whose name matches the following regular expression must respect the following conditions

Property name regular expression: ^(allowInsecure)$
root connections Connection clickhouse ^(allowInsecure)$
Type: boolean Default: false

Whether to use disable SSL for connection

Additional Properties of any type are allowed.

root connections Connection clickhouse additionalProperties
Type: object

root connections Connection databricks

Databricks Conf

Type: object

root connections Connection databricks fetch_table_tags
Type: boolean Default: false

root connections Connection databricks refresh_table_metrics
Type: boolean Default: false

root connections Connection databricks use_show_create_table
Type: boolean Default: false

root connections Connection databricks workspace_url
Type: string
Must match regular expression: ^(?:(?:[a-zA-Z][a-zA-Z\d+\-.]*):)?(?://(?:[A-Za-z0-9\-\.]+(?::\d+)?))?(/[^\?#]*)?(?:\?([^\#]*))?(?:\#(.*))?$

Must be at least 1 characters long

All properties whose name matches the following regular expression must respect the following conditions

Property name regular expression: ^(authClient)$
root connections Connection databricks ^(authClient)$
Type: string

All properties whose name matches the following regular expression must respect the following conditions

Property name regular expression: ^(authSecret)$
root connections Connection databricks ^(authSecret)$
Type: string

All properties whose name matches the following regular expression must respect the following conditions

Property name regular expression: ^(authToken)$
root connections Connection databricks ^(authToken)$
Type: string

All properties whose name matches the following regular expression must respect the following conditions

Property name regular expression: ^(fetchTableTags)$
root connections Connection databricks ^(fetchTableTags)$
Type: boolean Default: false

All properties whose name matches the following regular expression must respect the following conditions

Property name regular expression: ^(refreshTableMetrics)$
root connections Connection databricks ^(refreshTableMetrics)$
Type: boolean Default: false

All properties whose name matches the following regular expression must respect the following conditions

Property name regular expression: ^(refreshTableMetricsUseScan)$
root connections Connection databricks ^(refreshTableMetricsUseScan)$
Type: boolean Default: false

All properties whose name matches the following regular expression must respect the following conditions

Property name regular expression: ^(useShowCreateTable)$
root connections Connection databricks ^(useShowCreateTable)$
Type: boolean Default: false

All properties whose name matches the following regular expression must respect the following conditions

Property name regular expression: ^(workspaceUrl)$
root connections Connection databricks ^(workspaceUrl)$
Type: string
Must match regular expression: ^(?:(?:[a-zA-Z][a-zA-Z\d+\-.]*):)?(?://(?:[A-Za-z0-9\-\.]+(?::\d+)?))?(/[^\?#]*)?(?:\?([^\#]*))?(?:\#(.*))?$

Must be at least 1 characters long

Additional Properties of any type are allowed.

root connections Connection databricks additionalProperties
Type: object

root connections Connection disabled
Type: boolean Default: false

root connections Connection mysql

MySQL Conf

Type: object

MySQL specific configuration

root connections Connection mysql allow_insecure
Type: boolean Default: false

Whether to allow insecure connections

root connections Connection mysql database
Type: string

Database name

root connections Connection mysql host
Type: string

Host address

Must be at least 1 characters long

root connections Connection mysql params
Type: object

Additional connection parameters

Each additional property must conform to the following schema

root connections Connection mysql params additionalProperties
Type: string

root connections Connection mysql password
Type: string

Password for authentication

root connections Connection mysql port

Port number

root connections Connection mysql port anyOf item 0
Type: integer

Value must be strictly greater than 0 and lesser or equal to 65535

root connections Connection mysql port anyOf item 1
Type: string
Must match regular expression: ^-?[0-9]+$

root connections Connection mysql username
Type: string

Username for authentication

All properties whose name matches the following regular expression must respect the following conditions

Property name regular expression: ^(allowInsecure)$
root connections Connection mysql ^(allowInsecure)$
Type: boolean Default: false

Whether to allow insecure connections

Additional Properties of any type are allowed.

root connections Connection mysql additionalProperties
Type: object

root connections Connection name
Type: string

Name of the connection

root connections Connection parallelism
Default: 0

How many queries to DWH can be executed in parallel, defaults to 2

root connections Connection parallelism anyOf item 0
Type: integer

Value must be greater or equal to 1 and lesser or equal to 256

root connections Connection parallelism anyOf item 1
Type: string
Must match regular expression: ^-?[0-9]+$

root connections Connection postgres

Postgres Conf

Type: object

Postgres specific configuration

root connections Connection postgres allow_insecure
Type: boolean Default: false

Whether to allow insecure connections

root connections Connection postgres database
Type: string

Database name

root connections Connection postgres host
Type: string

Host address

Must be at least 1 characters long

root connections Connection postgres password
Type: string

Password for authentication

root connections Connection postgres port
Default: 0

Port number

root connections Connection postgres port anyOf item 0
Type: integer

Value must be strictly greater than 0 and lesser or equal to 65535

root connections Connection postgres port anyOf item 1
Type: string
Must match regular expression: ^-?[0-9]+$

root connections Connection postgres username
Type: string

Username for authentication

All properties whose name matches the following regular expression must respect the following conditions

Property name regular expression: ^(allowInsecure)$
root connections Connection postgres ^(allowInsecure)$
Type: boolean Default: false

Whether to allow insecure connections

Additional Properties of any type are allowed.

root connections Connection postgres additionalProperties
Type: object

root connections Connection redshift

Redshift Conf

Type: object

Redshift specific configuration

root connections Connection redshift database
Type: string

Database name

root connections Connection redshift freshness_from_query_logs
Type: boolean Default: false

Estimate table freshness based on query logs

root connections Connection redshift host
Type: string

Host address

Must be at least 1 characters long

root connections Connection redshift password
Type: string

Password for authentication

root connections Connection redshift port

Port number

root connections Connection redshift port anyOf item 0
Type: integer

Value must be strictly greater than 0 and lesser or equal to 65535

root connections Connection redshift port anyOf item 1
Type: string
Must match regular expression: ^-?[0-9]+$

root connections Connection redshift username
Type: string

Username for authentication

All properties whose name matches the following regular expression must respect the following conditions

Property name regular expression: ^(freshnessFromQueryLogs)$
root connections Connection redshift ^(freshnessFromQueryLogs)$
Type: boolean Default: false

Estimate table freshness based on query logs

Additional Properties of any type are allowed.

root connections Connection redshift additionalProperties
Type: object

root connections Connection snowflake

Snowflake Conf

Type: object

Snowflake specific configuration

root connections Connection snowflake account
Type: string

Snowflake account identifier

root connections Connection snowflake account_usage_db
Type: string

Name of the database where ACCOUNT_USAGE schema is present, fallbacks to SNOWFLAKE

root connections Connection snowflake auth_type
Type: string

Authentication type: empty (default, uses password or private_key), "externalbrowser" (SSO via browser)
When set to "externalbrowser", opens browser for SSO login and caches the token locally.

root connections Connection snowflake databases
Type: array of string

Database to connect to

 No Additional Items

Each item of this array must be:

root connections Connection snowflake databases databases items
Type: string

root connections Connection snowflake password
Type: string Default: ""

Password for authentication

root connections Connection snowflake private_key
Type: string Default: ""

Content of Private key used for Snowflake authentication

root connections Connection snowflake private_key_file
Type: string Default: ""

Location of the file containing Private key used for Snowflake authentication

root connections Connection snowflake private_key_passphrase
Type: string

Passphrase used to decode Private key

root connections Connection snowflake role
Type: string

Role to assume

root connections Connection snowflake use_get_ddl
Type: boolean Default: false

Use GET_DDL to determine queries used for table/view creation

root connections Connection snowflake username
Type: string

Username for authentication

root connections Connection snowflake warehouse
Type: string

Virtual warehouse to use

All properties whose name matches the following regular expression must respect the following conditions

Property name regular expression: ^(accountUsageDb)$
root connections Connection snowflake ^(accountUsageDb)$
Type: string

Name of the database where ACCOUNT_USAGE schema is present, fallbacks to SNOWFLAKE

All properties whose name matches the following regular expression must respect the following conditions

Property name regular expression: ^(authType)$
root connections Connection snowflake ^(authType)$
Type: string

Authentication type: empty (default, uses password or private_key), "externalbrowser" (SSO via browser)
When set to "externalbrowser", opens browser for SSO login and caches the token locally.

All properties whose name matches the following regular expression must respect the following conditions

Property name regular expression: ^(privateKey)$
root connections Connection snowflake ^(privateKey)$
Type: string Default: ""

Content of Private key used for Snowflake authentication

All properties whose name matches the following regular expression must respect the following conditions

Property name regular expression: ^(privateKeyFile)$
root connections Connection snowflake ^(privateKeyFile)$
Type: string Default: ""

Location of the file containing Private key used for Snowflake authentication

All properties whose name matches the following regular expression must respect the following conditions

Property name regular expression: ^(privateKeyPassphrase)$
root connections Connection snowflake ^(privateKeyPassphrase)$
Type: string

Passphrase used to decode Private key

All properties whose name matches the following regular expression must respect the following conditions

Property name regular expression: ^(useGetDdl)$
root connections Connection snowflake ^(useGetDdl)$
Type: boolean Default: false

Use GET_DDL to determine queries used for table/view creation

Additional Properties of any type are allowed.

root connections Connection snowflake additionalProperties
Type: object

root connections Connection trino

Trino Conf

Type: object

Trino specific configuration

root connections Connection trino catalogs
Type: array of string

To which catalogs to connect

 No Additional Items

Each item of this array must be:

root connections Connection trino catalogs catalogs items
Type: string

root connections Connection trino fetch_table_comments
Type: boolean Default: false

Fetch Trino table comments from system.metadata.table_comments

root connections Connection trino host
Type: string

Host address

Must match regular expression: ^[A-Za-z0-9][A-Za-z0-9-]{0,63}(\.[A-Za-z0-9-][A-Za-z0-9-]{0,63})*$

Must be at least 1 characters long

root connections Connection trino no_materialized_views
Type: boolean Default: false

Should it fetch system.metadata.materialized_views to get information about Trino MVs

root connections Connection trino no_show_create_table
Type: boolean Default: false

Use SHOW CREATE TABLE to get tables DDLs

root connections Connection trino no_show_create_view
Type: boolean Default: false

Use SHOW CREATE VIEW to get views DDLs

root connections Connection trino password
Type: string Default: ""

Password for authentication

root connections Connection trino port

Optional port to use, otherwise it will use the default port 8080

root connections Connection trino port anyOf item 0
Type: integer

Value must be strictly greater than 0 and lesser or equal to 65535

root connections Connection trino port anyOf item 1
Type: string
Must match regular expression: ^-?[0-9]+$

root connections Connection trino use_plaintext
Type: boolean

Use non-SSL connection to Trino. This should only be enabled if the Trino cluster does not support SSL
or if the connection is secured through other means (e.g., a VPN). Defaults to false (SSL enabled).

root connections Connection trino username
Type: string Default: ""

Username for authentication

All properties whose name matches the following regular expression must respect the following conditions

Property name regular expression: ^(fetchTableComments)$
root connections Connection trino ^(fetchTableComments)$
Type: boolean Default: false

Fetch Trino table comments from system.metadata.table_comments

All properties whose name matches the following regular expression must respect the following conditions

Property name regular expression: ^(noMaterializedViews)$
root connections Connection trino ^(noMaterializedViews)$
Type: boolean Default: false

Should it fetch system.metadata.materialized_views to get information about Trino MVs

All properties whose name matches the following regular expression must respect the following conditions

Property name regular expression: ^(noShowCreateTable)$
root connections Connection trino ^(noShowCreateTable)$
Type: boolean Default: false

Use SHOW CREATE TABLE to get tables DDLs

All properties whose name matches the following regular expression must respect the following conditions

Property name regular expression: ^(noShowCreateView)$
root connections Connection trino ^(noShowCreateView)$
Type: boolean Default: false

Use SHOW CREATE VIEW to get views DDLs

All properties whose name matches the following regular expression must respect the following conditions

Property name regular expression: ^(usePlaintext)$
root connections Connection trino ^(usePlaintext)$
Type: boolean

Use non-SSL connection to Trino. This should only be enabled if the Trino cluster does not support SSL
or if the connection is secured through other means (e.g., a VPN). Defaults to false (SSL enabled).

Additional Properties of any type are allowed.

root connections Connection trino additionalProperties
Type: object

Additional Properties of any type are allowed.

root connections Connection additionalProperties
Type: object

root synq

SYNQ

Type: object

SYNQ platform configuration

root synq client_id
Type: string

Client ID for OAuth authentication

Must be at least 1 characters long

root synq client_secret
Type: string

Client secret for OAuth authentication

Must be at least 1 characters long

root synq endpoint
Type: string

SYNQ API agent endpoint (host:port)

Must match regular expression: ^([A-Za-z0-9][A-Za-z0-9-]{0,63}(\.[A-Za-z0-9-][A-Za-z0-9-]{0,63})*|((25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)|\[(([0-9a-fA-F]{1,4}::?){1,7}([0-9a-fA-F]{1,4})|([0-9a-fA-F]{1,4}:){1,7}:|:((([0-9a-fA-F]{1,4}:){1,6})?[0-9a-fA-F]{1,4})?|::)\]):([1-9][0-9]{0,4}|[1-5][0-9]{4}|6[0-4][0-9]{3}|65[0-4][0-9]{2}|655[0-2][0-9]|6553[0-5])$

Must be at least 1 characters long

root synq ingest_endpoint
Type: string

SYNQ API ingest endpoint (host:port)

Must match regular expression: ^([A-Za-z0-9][A-Za-z0-9-]{0,63}(\.[A-Za-z0-9-][A-Za-z0-9-]{0,63})*|((25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)|\[(([0-9a-fA-F]{1,4}::?){1,7}([0-9a-fA-F]{1,4})|([0-9a-fA-F]{1,4}:){1,7}:|:((([0-9a-fA-F]{1,4}:){1,6})?[0-9a-fA-F]{1,4})?|::)\]):([1-9][0-9]{0,4}|[1-5][0-9]{4}|6[0-4][0-9]{3}|65[0-4][0-9]{2}|655[0-2][0-9]|6553[0-5])$

Must be at least 1 characters long

root synq oauth_url
Type: string

OAuth authentication URL

Must match regular expression: ^(?:(?:[a-zA-Z][a-zA-Z\d+\-.]*):)?(?://(?:[A-Za-z0-9\-\.]+(?::\d+)?))?(/[^\?#]*)?(?:\?([^\#]*))?(?:\#(.*))?$

Must be at least 1 characters long

All properties whose name matches the following regular expression must respect the following conditions

Property name regular expression: ^(clientId)$
root synq ^(clientId)$
Type: string

Client ID for OAuth authentication

Must be at least 1 characters long

All properties whose name matches the following regular expression must respect the following conditions

Property name regular expression: ^(clientSecret)$
root synq ^(clientSecret)$
Type: string

Client secret for OAuth authentication

Must be at least 1 characters long

All properties whose name matches the following regular expression must respect the following conditions

Property name regular expression: ^(ingestEndpoint)$
root synq ^(ingestEndpoint)$
Type: string

SYNQ API ingest endpoint (host:port)

Must match regular expression: ^([A-Za-z0-9][A-Za-z0-9-]{0,63}(\.[A-Za-z0-9-][A-Za-z0-9-]{0,63})*|((25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)|\[(([0-9a-fA-F]{1,4}::?){1,7}([0-9a-fA-F]{1,4})|([0-9a-fA-F]{1,4}:){1,7}:|:((([0-9a-fA-F]{1,4}:){1,6})?[0-9a-fA-F]{1,4})?|::)\]):([1-9][0-9]{0,4}|[1-5][0-9]{4}|6[0-4][0-9]{3}|65[0-4][0-9]{2}|655[0-2][0-9]|6553[0-5])$

Must be at least 1 characters long

All properties whose name matches the following regular expression must respect the following conditions

Property name regular expression: ^(oauthUrl)$
root synq ^(oauthUrl)$
Type: string

OAuth authentication URL

Must match regular expression: ^(?:(?:[a-zA-Z][a-zA-Z\d+\-.]*):)?(?://(?:[A-Za-z0-9\-\.]+(?::\d+)?))?(/[^\?#]*)?(?:\?([^\#]*))?(?:\#(.*))?$

Must be at least 1 characters long

Additional Properties of any type are allowed.

root synq additionalProperties
Type: object

Additional Properties of any type are allowed.

root additionalProperties
Type: object