DuckDB Data Accelerator | Spice.ai Cloud Documentation

To use DuckDB as Data Accelerator, specify duckdb as the engine for acceleration.

Configuration

Spice.ai currently only supports mode: memory for DuckDB accelerator.

Configuration params are provided in the acceleration section for a data store. Other common acceleration fields can be configured for DuckDB, see datasets.

LIMITATIONS

• The DuckDB accelerator does not support nested lists, or structs with nested structs/lists field types. For example:

  • Supported:
    • SELECT {'x': 1, 'y': 2, 'z': 3}
  • Unsupported:
    • SELECT [['duck', 'goose', 'heron'], ['frog', 'toad']]);
    • SELECT {'x': [1, 2, 3]}

• The DuckDB accelerator does not support enum, dictionary, or map field types. For example:

  • Unsupported:
    • SELECT MAP(['key1', 'key2', 'key3'], [10, 20, 30])

• The DuckDB accelerator does not support Decimal256 (76 digits), as it exceeds DuckDB's maximum Decimal width of 38 digits.

• Updating a dataset with DuckDB acceleration while the Spice Runtime is running (hot-reload) will cause the DuckDB accelerator query federation to disable until the Runtime is restarted.

MEMORY CONSIDERATIONS

When accelerating a dataset using mode: memory (the default), some or all of the dataset is loaded into memory. Ensure sufficient memory is available, including overhead for queries and the runtime, especially with concurrent queries.