# Debezium

[Debezium](https://debezium.io/) is an open-source platform that enables Change Data Capture (CDC) for efficient real-time updates of locally accelerated datasets. Spice supports connecting to a Kafka topic managed by Debezium to keep datasets up-to-date with the source data.

## Configuration

### `from`

The `from` field takes the form of `debezium:kafka_topic` where `kafka_topic` is the name of the Kafka topic where Debezium is notifying consumers about any upstream changes. In the example above it would listen to the `my_kafka_topic_with_debezium_changes` topic.

### `name`

The dataset name. This will be used as the table name within Spice.

```yaml
datasets:
  - from: debezium:my_kafka_topic_with_debezium_changes
    name: cool_dataset
```

```sql
SELECT COUNT(*) FROM cool_dataset;
```

```plaintext
+----------+
| count(*) |
+----------+
| 6001215  |
+----------+
```

### `params`

| Parameter Name                  | Description                                                                                                     |
|----------------------------------|-----------------------------------------------------------------------------------------------------------------|
| `debezium_transport`             | Optional. The message broker transport to use. The default is `kafka`. Possible values: `kafka`.               |
| `debezium_message_format`        | Optional. The message format to use. The default is `json`. Possible values: `json`.                            |
| `kafka_bootstrap_servers`        | **Required**. A list of host/port pairs for establishing the initial Kafka cluster connection.                  |
| `kafka_security_protocol`        | Security protocol for Kafka connections. Default: `SASL_SSL`. Options: `PLAINTEXT`, `SSL`, `SASL_PLAINTEXT`, `SASL_SSL`. |
| `kafka_sasl_mechanism`          | SASL (Simple Authentication and Security Layer) authentication mechanism. Default: `SCRAM-SHA-512`.            |
| `kafka_sasl_username`           | SASL username.                                                                                                |
| `kafka_sasl_password`           | SASL password.                                                                                                |
| `kafka_ssl_ca_location`         | Path to the SSL/TLS CA certificate file for server verification.                                               |
| `kafka_enable_ssl_certificate_verification` | Enable SSL/TLS certificate verification. Default: `true`.                               |
| `kafka_ssl_endpoint_identification_algorithm` | SSL/TLS endpoint identification algorithm. Default: `https`. Options: `none`, `https`.                  |

### Acceleration Settings

Using the Debezium connector **requires** [acceleration](https://github.com/spicehq/docs/blob/trunk/building-blocks/data-accelerators/README.md) to be enabled.

| Parameter Name                   | Description                                                                                                     |
|----------------------------------|-----------------------------------------------------------------------------------------------------------------|
| `enabled`                        | Required. Must be set to `true` to enable acceleration.                                                        |
| `engine`                         | Required. The acceleration engine to use. Valid values: `duckdb`, `sqlite`, `postgres`.                        |
| `refresh_mode`                   | Optional. The refresh mode to use. If specified, this must be set to `changes`. Any other value is an error.  |
| `mode`                           | Optional. The persistence mode to use. Recommended to set this to `file` to persist the data across restarts.