# S3 and httpfs

> Query Amazon S3 and S3-compatible storage from DuckLocal with DuckDB's httpfs: configure credentials, browse buckets and run SQL on remote files.

Source: https://ducklocal.app/docs/s3

**[中文](zh/s3.md)**

DuckLocal can read objects from S3 and S3-compatible storage through DuckDB's
`httpfs` extension. Configure it from the **S3** button in the title bar.

## Configure credentials

The dialog has four fields:

| Field | Default | Notes |
| --- | --- | --- |
| Endpoint | `s3.amazonaws.com` | `http://` is honored; otherwise HTTPS is used |
| Region | `us-east-1` | Sent as typed |
| Access Key ID | — | Required |
| Secret Access Key | — | Required, masked, with a reveal toggle |

Confirming runs the equivalent of:

```sql
INSTALL httpfs; LOAD httpfs;
CREATE OR REPLACE SECRET ducklocal_s3 (
  TYPE s3, PROVIDER config,
  KEY_ID '…', SECRET '…',
  REGION '…', ENDPOINT '…'
);
```

**Credentials are session-only.** They are not written to disk and are cleared when you open or switch a database, so you must reconfigure afterwards. There is no separate way to clear credentials other than reconfiguring.

## Browse

Once configured, an **S3** root appears in the sidebar. Its contents load only when expanded: the root lists buckets, and each bucket or prefix lists its objects. Buckets and prefixes are sorted, and objects show a human-readable size. Use the refresh button to reload a level or retry after an error.

Listing is capped at **1,000 entries per level**, even when the response says
more are available. There is no deeper paging, no prefix filter box, and no
search — a bucket with more than a thousand objects at one level will appear
complete when it is not.

## Query objects

Clicking an object whose key has a data-file extension fills the active editor
with:

```sql
SELECT *
FROM 's3://bucket/key'
LIMIT 100;
```

That is an ordinary query: the read goes through `httpfs`, so DuckDB's
`read_parquet` / `read_csv_auto` behavior applies, including pushdown —
filters and column selection are pushed into the file read. The query replaces
whatever the active tab held.

Only keys recognized as data files are clickable. The recognized extensions
are the same as for local files; see [Data sources](data-sources.md#supported-file-formats).

## Limitations

- **Browsing is read-only.** The browser has no upload, download-to-file,
  delete, copy, or bucket-creation action. Queries still download remote data.
- **No session tokens.** The `X-Amz-Security-Token` header is never sent, so
  temporary STS credentials will not work — use long-lived keys, or an
  endpoint that accepts key/secret alone.
- **Endpoint style is mostly automatic.** S3 URLs can address a bucket in two
  ways: virtual-hosted addressing or path style. Virtual-hosted addressing is
  used when the host contains `amazonaws.com`, path style otherwise. For
  self-hosted gateways this is usually what you want; for one that needs the
  other style, there is no override.
- A wrong region is retried once, using the region DuckDB reports back. If
  that fails too, the error is shown as-is.
- Network is required: `INSTALL httpfs` fetches the extension on first use.
  Browsing sends HTTP or HTTPS requests to the configured endpoint (`http://`
  selects HTTP; otherwise HTTPS is used). These requests send metadata such
  as bucket names and key prefixes, plus authentication information in signed
  headers; browsing does not upload local database files or query results.
  Queries on remote objects download data to your machine through `httpfs`.

## Troubleshooting

| Symptom | Likely cause |
| --- | --- |
| Browsing fails immediately after configuring | Region or endpoint wrong; check the message shown in the sidebar |
| Errors on a self-hosted endpoint | Wrong addressing style, or an endpoint that requires a session token |
| `httpfs` installation error | No network, or a proxy that blocks the extension download |
| Credentials seem forgotten | A database was opened or switched, which clears them by design |

When DuckLocal is packaged as a signed app, loading the `httpfs` extension
needs the `com.apple.security.cs.disable-library-validation` entitlement; the
shipped packaging script already includes it.