Skip to content

Operators API Reference

BaseSQLOperator

Bases: BaseOperator

Base class for SQL operators.

Supports dynamic task mapping through the dynamic_params parameter. Parameters in dynamic_params are stored as instance attributes for use in map_index_template.

SQLQueryOperator

Bases: BaseSQLOperator

Operator for SQL queries that write to a destination table.

Supports extra kwargs for dynamic task naming and mapping.

Attributes

template_fields class-attribute instance-attribute

template_fields = ('sql', 'template_vars')

template_ext class-attribute instance-attribute

template_ext = ('.sql',)

template_fields_renderers class-attribute instance-attribute

template_fields_renderers = {'sql': 'sql'}

sql instance-attribute

sql = sql

template_vars instance-attribute

template_vars = template_vars or {}

source_conn instance-attribute

source_conn = source_conn

hook_manager instance-attribute

hook_manager = SQLHookManager()

Methods:

__getattr__

__getattr__(name: str) -> Any

render_template_fields

render_template_fields(context: Any, jinja_env: Any | None = None) -> None

SQLAppendOperator

Bases: BaseSQLOperator

Operator for SQL queries that append data to a destination table.

Supports extra kwargs for dynamic task naming and mapping.

Attributes

template_fields class-attribute instance-attribute

template_fields = ('sql', 'template_vars')

template_ext class-attribute instance-attribute

template_ext = ('.sql',)

template_fields_renderers class-attribute instance-attribute

template_fields_renderers = {'sql': 'sql'}

sql instance-attribute

sql = sql

template_vars instance-attribute

template_vars = template_vars or {}

source_conn instance-attribute

source_conn = source_conn

hook_manager instance-attribute

hook_manager = SQLHookManager()

Methods:

__getattr__

__getattr__(name: str) -> Any

render_template_fields

render_template_fields(context: Any, jinja_env: Any | None = None) -> None

SQLDataFrameOperator

Bases: BaseSQLOperator

Operator for SQL queries that return a pandas DataFrame.

Attributes

template_fields class-attribute instance-attribute

template_fields = ('sql', 'template_vars')

template_ext class-attribute instance-attribute

template_ext = ('.sql',)

template_fields_renderers class-attribute instance-attribute

template_fields_renderers = {'sql': 'sql'}

is_dry_run instance-attribute

is_dry_run = bool(kwargs.pop('dry_run'))

sql instance-attribute

sql = sql

template_vars instance-attribute

template_vars = template_vars or {}

source_conn instance-attribute

source_conn = source_conn

hook_manager instance-attribute

hook_manager = SQLHookManager()

Methods:

__init__

__init__(sql: str, source_conn: Optional[str] = None, template_vars: Optional[dict[str, Any]] = None, dynamic_params: Optional[dict] = None, **kwargs)

Initialize the base SQL operator.

Parameters:

Name Type Description Default

sql

str

SQL template or rendered SQL string associated with the operator.

required

source_conn

Optional[str]

Optional connection id of the source database.

None

dynamic_params

Optional[dict]

Optional mapping used for dynamic task naming and mapping.

None

**kwargs

Additional keyword arguments passed to BaseOperator.

{}
Notes

Child operators commonly accept additional flags such as dry_run_flag or pre_truncate. This constructor stores parameters on the instance for use by operators and templates.

__getattr__

__getattr__(name: str) -> Any

render_template_fields

render_template_fields(context: Any, jinja_env: Any | None = None) -> None

SQLReplaceOperator

Bases: BaseSQLOperator

Operator for SQL queries that replace the destination table content.

Supports extra kwargs for dynamic task naming and mapping.

Attributes

template_fields class-attribute instance-attribute

template_fields = ('sql', 'template_vars')

template_ext class-attribute instance-attribute

template_ext = ('.sql',)

template_fields_renderers class-attribute instance-attribute

template_fields_renderers = {'sql': 'sql'}

sql instance-attribute

sql = sql

template_vars instance-attribute

template_vars = template_vars or {}

source_conn instance-attribute

source_conn = source_conn

hook_manager instance-attribute

hook_manager = SQLHookManager()

Methods:

__getattr__

__getattr__(name: str) -> Any

render_template_fields

render_template_fields(context: Any, jinja_env: Any | None = None) -> None

SQLTruncateOperator

Bases: BaseSQLOperator

Operator for SQL queries that truncate the destination table and insert new data, preserving structure.

Supports extra kwargs for dynamic task naming and mapping.

Attributes

template_fields class-attribute instance-attribute

template_fields = ('sql', 'template_vars')

template_ext class-attribute instance-attribute

template_ext = ('.sql',)

template_fields_renderers class-attribute instance-attribute

template_fields_renderers = {'sql': 'sql'}

sql instance-attribute

sql = sql

template_vars instance-attribute

template_vars = template_vars or {}

source_conn instance-attribute

source_conn = source_conn

hook_manager instance-attribute

hook_manager = SQLHookManager()

Methods:

__getattr__

__getattr__(name: str) -> Any

render_template_fields

render_template_fields(context: Any, jinja_env: Any | None = None) -> None

SQLMergeOperator

Bases: BaseSQLOperator

Operator for SQL queries that merge/upsert into the destination table.

Supports extra kwargs for dynamic task naming and mapping.

Attributes

template_fields class-attribute instance-attribute

template_fields = ('sql', 'template_vars')

template_ext class-attribute instance-attribute

template_ext = ('.sql',)

template_fields_renderers class-attribute instance-attribute

template_fields_renderers = {'sql': 'sql'}

sql instance-attribute

sql = sql

template_vars instance-attribute

template_vars = template_vars or {}

source_conn instance-attribute

source_conn = source_conn

hook_manager instance-attribute

hook_manager = SQLHookManager()

Methods:

__getattr__

__getattr__(name: str) -> Any

render_template_fields

render_template_fields(context: Any, jinja_env: Any | None = None) -> None

DataFrameLoadOperator

Bases: BaseOperator

Operator for loading DataFrame data into a table.

Supports extra kwargs for dynamic task naming and mapping.

DataFrameMergeOperator

Bases: BaseOperator

Operator for merging DataFrame data into a table.

Supports extra kwargs for dynamic task naming and mapping.

SQLCheckOperator

Bases: SQLCheckOperator

Wrapper around Airflow's native SQLCheckOperator that follows airsql standards.

This operator performs data quality checks using SQL. The SQL should return a single row where each value is evaluated using Python bool casting. If any value is False, the check fails.

For dbt tests, the SQL should return: - 0 (or empty result) = test passes - Any other value = test fails