# Development

> Build DuckLocal from source with Rust and DuckDB, run the tests, bundle the macOS app, and cut a release for macOS, Linux and Windows.

Source: https://ducklocal.app/docs/development

**[中文](zh/development.md)**

## Build and run

```bash
cargo run                     # empty in-memory workspace
cargo run -- ./data/logs/     # open something straight away
cargo test --workspace        # run the test suite, Quick Look engine included
```

Because DuckDB compiles directly from vendored source code, the initial build takes noticeable time and requires several gigabytes of disk space. Dependencies in the dev profile compile at `opt-level = 3`, which prolongs that first build but keeps iterative development responsive—subsequent compiles only rebuild `ducklocal`.

**Compact build.** `cargo build --profile compact --locked` writes a smaller
binary to `target/compact/`. The `compact` profile inherits `release` (LTO,
stripped symbols, `panic = "abort"`) but compiles Rust for size
(`opt-level = "s"`), while `libduckdb-sys` keeps `opt-level = 3`, so DuckDB's
scan, filter and aggregate loops stay fast. Use it when download or disk size
matters more than frame and query speed — measure both on the target machine.
The release workflow and the packages it publishes use the normal `release`
profile.

**Toolchain.** There is no `rust-toolchain.toml` and no `rust-version` key, so
the effective floor is set by dependencies: `1.85.1`, required by `duckdb`.
CI uses plain stable.

## The pinned toolkit

The UI toolkit, `gpui-kit`, comes from the `longbridge/gpui-kit` repository,
pinned to a single git revision on purpose. Mixing two revisions puts two
copies of `gpui-base` in the build, and the build fails with trait-mismatch
errors that do not name the real cause.

```bash
./scripts/check-deps.sh
```

It checks the things Cargo will not: one revision across every dependency on
the repository, a `Cargo.lock` that agrees with it, and no crate in the build
twice — once from the repository and once from crates.io. It reads the two
manifests and nothing else, so it is offline and immediate. CI runs it before
the tests. To move to a newer toolkit, change every `rev` in `Cargo.toml`
together, run `cargo update`, and run the script.

Pinning to a git revision is also why the DuckLocal crate is not publishable
to crates.io: a git dependency is refused there.
Distribution is the release packages below.

## Bundle a macOS app

```bash
./scripts/bundle.sh
# produces target/release/DuckLocal.app
```

That assembles the bundle — binary at `Contents/MacOS/ducklocal`, plus
`assets/Info.plist`, `assets/AppIcon.icns` and the Quick Look extension — but
**does not sign it** for distribution. The result runs on your machine and
will be refused by Gatekeeper anywhere else.

### The Quick Look extension

Finder's Space-bar preview of Parquet, DuckDB, CSV, TSV and JSON Lines files
is an app extension, `DuckLocalPreview.appex`, in the app's
`Contents/PlugIns`. It has two halves:

- `quicklook/` is a workspace crate, `ducklocal-quicklook`: the preview
  engine in Rust, built as a static library. It opens the file with the same
  bundled DuckDB as the app (from the same lockfile, so DuckDB compiles once)
  and renders its columns and first 50 rows as one self-contained HTML page.
  An extension is its own sandboxed process and cannot call into the app.
- `quicklook/extension/` is the Swift shell around it: `PreviewProvider.swift`,
  the C header, the extension's `Info.plist` (the content types it previews)
  and its sandbox entitlements.

```bash
scripts/build-quicklook.sh <DuckLocal.app> [signing identity]
```

The script builds the engine (`cargo build --release -p ducklocal-quicklook`,
or for `RUST_TARGET` when set, as the release workflow does, so the app's
DuckDB build is reused), compiles the Swift shell against it with `swiftc`
for macOS 12, copies the `Info.plist` stamped with the app's version, and
signs the `.appex` with the extension's own entitlements — ad hoc without an
identity. `bundle.sh` calls it with no identity. macOS finds the extension
once the app has been launched or registered.

## Package, sign, and notarize

```bash
scripts/package-macos.sh <binary> <output.dmg> <version>
```

Run it from the repository root; it writes `DuckLocal.app` and `dmg-root` into
the working directory and the dmg next to them. The script refuses to run
without credentials — an unnotarized artifact is not something a user can open
on current macOS, so it fails fast rather than producing one by accident.

Required:

| Variable | Meaning |
| --- | --- |
| `NOTARY_KEY` | Path to an App Store Connect API key (`.p8`) |
| `NOTARY_KEY_ID` | That key's Key ID |
| `NOTARY_ISSUER` | The issuer UUID the key belongs to |
| a Developer ID Application identity | Auto-detected when exactly one is in the keychain, otherwise set `MACOS_SIGN_IDENTITY` |

The script performs these steps:

1. Builds the bundle, with the version stamped from the argument.
2. Builds the Quick Look extension with `build-quicklook.sh`, signed with the
   Developer ID and the extension's own sandbox entitlements — never
   `scripts/entitlements.plist`, because an app extension must be sandboxed
   and the app is not.
3. Signs, inside out and without `--deep`, the inner executable and then the
   app with `--options runtime --timestamp` and `scripts/entitlements.plist`,
   and verifies the result with `codesign --verify --deep --strict`.
4. Builds a UDZO dmg with an `/Applications` symlink.
5. **Signs the dmg itself**.
6. Submits it with `notarytool submit --wait --timeout 300m`.
7. Staples the ticket.
8. Finishes with `stapler validate` plus two `spctl --assess` checks.

Two things to know before using it:

- The five-hour timeout is a client-side wait, not a cancellation. If it
  expires, the submission keeps processing on Apple's side, and the ticket can
  be collected later — but **a ticket binds to the exact bytes submitted**, so
  do not rebuild; staple the file you submitted.
- Only the dmg is notarized, so the inner `.app` carries no stapled ticket. A
  first launch from a just-mounted image needs network to check in with Apple.

The `com.apple.security.cs.disable-library-validation` entitlement is required,
because DuckDB `dlopen`s its extensions — `httpfs` among them.

## CI and releases

| Workflow | Trigger | What it does |
| --- | --- | --- |
| `ci.yml` | Push to `main`, any pull request | On `macos-latest`: `scripts/check-deps.sh`, `cargo fmt --all --check`, `cargo clippy --locked --workspace --all-targets -- -D warnings`, then `cargo test --locked --workspace` (the app and the Quick Look crate). On `windows-latest`: `cargo test --locked` (the `perf_probe` tests skipped), then `scripts/check-windows.py` on the debug executable. On `ubuntu-22.04`: `cargo build --locked` |
| `release.yml` | Tag matching `v*`, or manual dispatch with a version | Three builds: `aarch64-apple-darwin`, signed and notarized with `package-macos.sh` into `ducklocal-macos-arm64.dmg`; Linux on Ubuntu 22.04 into `ducklocal-linux-x86_64.tar.gz`; Windows into `ducklocal-windows-x86_64.zip`, after `scripts/check-windows.py` verifies the release executable. The release job waits for all three — one failing fails the release — and creates the GitHub release with generated notes |
| `gh-pages.yml` | Push to `main` touching the redirect script or the workflow, or manual dispatch | Publishes redirects from the old `jetsquirrel.github.io/DuckLocal/` addresses to the new site on GitHub Pages |

The release build needs these repository secrets: `MACOS_CERTIFICATE`,
`MACOS_CERTIFICATE_PWD`, `NOTARY_KEY_BASE64`, `NOTARY_KEY_ID`, and
`NOTARY_ISSUER`.

## Platform support

| Platform | Package | Notes |
| --- | --- | --- |
| macOS 12+, Apple silicon | `.dmg`, signed and notarized, with the Quick Look extension | Tested in CI, Quick Look crate included |
| Linux x86_64 | `.tar.gz` of the binary | Built on Ubuntu 22.04, for a glibc old enough for most distributions, with the system libraries gpui-kit's own CI installs |
| Windows x86_64 | `.zip` of `ducklocal.exe` | Tested in CI. Not code-signed: there is no Windows certificate, so SmartScreen warns on first launch |

The tests run on macOS and Windows; Linux is compiled on every pull request
but not tested. Windows runs them because its paths — drive letters, `\`, no
`$HOME` — are where a macOS run cannot catch a mistake.
`scripts/check-windows.py` (standard-library Python, run on Windows) then
checks the executable without launching the GUI: the GUI subsystem, so no
console window opens; the embedded duck icon; and that `--version` and a
`query` print to redirected output. There is no Intel Mac build.

Shortcuts are bound with gpui's `secondary` modifier — ⌘ on macOS, Ctrl on
Windows and Linux. `cmd` would be the Windows key on Windows and Linux, which
leaves every shortcut unreachable.

## The documentation site

The website lives in its own repository,
[JetSquirrel/ducklocal-site](https://github.com/JetSquirrel/ducklocal-site),
and is served by Cloudflare Workers as static assets:

| Address | What | Source |
| --- | --- | --- |
| [ducklocal.app](https://ducklocal.app/) | The product page | `home/`, static HTML and CSS |
| [ducklocal.app/docs](https://ducklocal.app/docs/) | These guides, English and Chinese | `docs/`, a [VitePress](https://vitepress.dev/) site |

A change to the app that changes what the guides say needs a pull request to
both repositories. To work on the guides locally, the prerequisites are
Node.js 22 or newer (24 LTS recommended) and a clone of the site repository.
Then run:

```bash
npm --prefix docs ci             # install the locked dependencies
npm --prefix docs run dev        # http://localhost:5173/docs/
sh scripts/build.sh              # the whole site, product page and docs, in dist/site
```

The old GitHub Pages address, `jetsquirrel.github.io/DuckLocal/`, is kept as a
set of redirects to the matching new pages; this repository's `gh-pages.yml`
publishes them.

## Project layout

| Path | Contents |
| --- | --- |
| `src/main.rs` | Entry point, early CLI dispatch, GUI window setup |
| `src/cli.rs` | Headless arguments, DuckDB parser validation, isolated connections, JSON errors and their hints |
| `tests/cli.rs` | Real executable integration tests; scratch data under `target/cli-tests` |
| `skills/ducklocal/` | Official schema-first SQL agent skill and CLI reference |
| `src/sources.rs` | Resolving paths into files to attach and databases to open |
| `src/db.rs` | The DuckDB connection, attaching files, server info |
| `src/query.rs` | Query execution, result serialization, export, EXPLAIN and the EXPLAIN ANALYZE profile behind **Profile** |
| `src/script.rs` | Splitting the editor's text into the statements Run executes, dot commands included |
| `src/overview.rs` | The results' **Overview** tab: per-column statistics and distributions |
| `src/extensions.rs` | The sidebar's **Extensions** tab: listing, `INSTALL`, `LOAD` and `UPDATE EXTENSIONS` |
| `src/storage.rs` | Reading a `.duckdb` file's header: why a file will not open, and which DuckDB reads one that does |
| `src/schema_cli.rs` | `ducklocal schema`: the relations and columns to query, sized for a prompt |
| `src/profile.rs` | Column statistics behind `ducklocal schema --stats` |
| `src/remote.rs` | `ducklocal open`: the loopback channel to the running window, both ways |
| `src/schema.rs` | Catalog introspection for the sidebar |
| `src/spec/` | `.dash` dashboards: the parser, validation, sources, filters, review comments (`ducklocal comments`), `ducklocal lsp`, and the dashboard tab with its plots |
| `src/excel.rs` | Excel and ODS workbooks, loaded as one table per sheet |
| `src/setup.rs` | **Command line & AI…**: putting `ducklocal` on the PATH and installing the agent skill |
| `src/recents.rs` | The recently opened dashboards the sidebar lists |
| `src/history.rs` | The app's own database: history, registered files, attached databases, settings |
| `src/s3.rs` | SigV4 signing and S3 listing |
| `src/i18n.rs` | The string table and language selection |
| `src/state.rs` | Application state shared across views |
| `src/ui/` | Editor, results grid, charts and maps (with the OpenStreetMap tiles), sidebar, dialogs, title and status bars, the update check |
| `src/windows.rs` | Windows startup: the caller's console for CLI commands, none for a GUI launch |
| `quicklook/` | The macOS Quick Look preview: a Rust library a Swift extension links, built by `scripts/build-quicklook.sh` |
| `src/assets.rs` | Icons embedded beyond the component library’s defaults |
| `src/perf_probe.rs` | Test-only performance probe harness: wall time, allocation count, and bytes for paths that scale with data size |
| `assets/` | Icons (`.icns`, Windows `.ico` and resource script), `Info.plist` with the document types Finder opens in DuckLocal, the diagram the README and docs use |
| `scripts/` | `bundle.sh`, `build-quicklook.sh`, `package-macos.sh`, `check-deps.sh`, `check-windows.py`, `make-icons.py`, entitlements |

## Testing notes

The test suite is `cargo test --workspace`; plain `cargo test` covers the app
but not the `quicklook/` crate. Query execution tests exercise DuckDB through
an in-memory connection; the export tests cover both CSV and Parquet, the
latter by reading the written file back with `read_parquet`.