@flowblade/source-duckdb

Install

yarn add @flowblade/source-duckdb @flowblade/core @duckdb/node-api

Note that at this time duckdb neo is still in alpha. To install use the latest tag ie: @duckdb/node-api@1.2.0-alpha.14

Create a duckdb instance

import { type DuckDBConnection, DuckDBInstance } from '@duckdb/node-api';
import { DuckdbDatasource } from '@flowblade/source-duckdb';

// Create a connection to a DuckDB instance
const createConnection = async (): Promise<DuckDBConnection> => {
  const instance = await DuckDBInstance.create(':memory:', {
    // Choose between READ_ONLY or READ_WRITE
    // Note that in READ_WRITE mode concurrency is limited to 1
    // See: https://duckdb.org/docs/connect/concurrency.html
    access_mode: 'READ_ONLY',
    max_memory: '64MB',
  });
  return await instance.connect();
};

const duckdb = await createConnection();

// Create a duckdb datasource
export const ds = new DuckdbDatasource({ connection: duckdb });

Query the database

import { DuckdbDatasource, sql } from '@flowblade/source-duckdb';

import { ds } from "./config.ts";

const params = {
    min: 10,
    max: 99,
    name: 'test',
    createdAt: new Date().toISOString(),
};

type Row = { id: number; name: 'test'; createdAt: Date };

const rawSql = sql<Row>`

      WITH products(productId, createdAt)
          AS MATERIALIZED (
               FROM RANGE(1,100) SELECT 
               range::INT,
               TIMESTAMPTZ '2025-01-01 12:30:00.123456789+01:00'
          )
      
      SELECT productId, 
             ${params.name} as name,
             createdAt
             
      FROM products 
      WHERE productId BETWEEN ${params.min}::INTEGER AND ${params.max}::INTEGER
      AND createdAt < ${params.createdAt}::TIMESTAMPTZ
    `;

const result = await ds.query(rawSql);

// Option 1: The QResult object contains the data, metadata and error
//  - data:  the result rows (TData or undefined if error)
//  - error: the error (QError or undefined if success)
//  - meta:  the metadata (always present)

const { data, meta, error } = result;

// Option 2: You operate over the result, ie: mapping the data

const { data } = result.map((row) => {
    return {
        id: row.productId,
        key: `key-${row.productId}`
    })

if (data) {
    console.log(data);
}

Compatibility

Level	CI	Description
Node	✅	CI for 18.x, 20.x & 22.x.
Cloudflare	✅	Ensured with @cloudflare/vitest-pool-workers (see wrangler.toml
Browserslist	✅	> 95% on 01/2025. Chrome 96+, Firefox 90+, Edge 19+, ios 15+, Safari 15+ and Opera 77+
Typescript	✅	TS 5.0 + / are-the-type-wrong checks on CI.
ES2022	✅	Dist files checked with es-check
Performance	✅	Monitored with codspeed.io

Contributors

Contributions are welcome. Have a look to the CONTRIBUTING document.

Sponsors

Sponsor, coffee, or star – All is spent for quality time with loved ones. Thanks ! 🙏❤️

Special thanks to

JetBrains	Embie.be

License

MIT © Sébastien Vanvelthem and contributors.