14 encrypted source adapters

Bring your data from where it already lives.

Warehouses, relational databases, object storage, and the HuggingFace Hub — browse, sample, and import in four steps, with credentials vaulted by construction. The same four calls work for every adapter, so the engineer who wires Snowflake on Monday wires Azure Blob on Wednesday in the same afternoon.

Every connector runs through an encrypted secret vault, refuses plain-text connections where the driver supports TLS, and sandboxes the test session to a single read-only probe. Import jobs land under your tenant artefact prefix — cross-tenant reads and writes are structurally impossible.

Connectors guide
Adapters
14
Families
4
Vault
Encrypted
Plain secrets
0
◆ The catalogue

Fourteen adapters across four families.

The same browse / test / import surface across every adapter — no per-source quirks to learn.

Warehouse

Snowflake
BigQuery
Databricks
Redshift
Hive

Relational

Postgres
MySQL
MariaDB
MSSQL
Oracle

Object storage

Amazon S3
Google GCS
Azure Blob

Dataset hub

HuggingFace
◆ Four steps

Four steps from creds to dataset.

Step 1

Create connector

Enter creds in the UI or POST the config — inline passwords are auto-hoisted to the encrypted vault.

Step 2

Test connection

We open a short-lived test session against the source and surface the exact error if it fails.

Step 3

Browse

A per-connector dispatcher returns up to 500 tables / objects, paged and filtered.

Step 4

Import to dataset

Run an import job — it lands in your tenant artefact prefix, ready for Synthesize / ADS / Mock.

connectors.http
# 1. Create a Postgres connector — passwords auto-hoist
POST /v1/client/connectors
{
  "name": "warehouse-prod",
  "type": "postgres",
  "config": {
    "host": "warehouse.acme.internal",
    "database": "analytics",
    "user": "radmah_ro",
    "password": "p@ssw0rd-will-be-vaulted"
  }
}
→ 201 Created   secret_ref=cs_…   no plain secret in run-state

# 2. Test it
POST /v1/client/connectors/{id}/test     → 200 OK

# 3. Browse tables
POST /v1/client/connectors/{id}/browse
{ "schema": "public", "limit": 500 }
→ [
   {"name": "customers", "rows_estimate": 482931},
   {"name": "orders",    "rows_estimate": 1923810},
   …
]

# 4. Import as a dataset
POST /v1/client/connectors/{id}/import
{ "source": "public.customers", "row_limit": 200000,
  "target_dataset_name": "customers-snapshot-2026-04" }
→ 202 Accepted    job_id=imp_…
◆ Same four calls — every adapter

One integration pattern. Fourteen data sources.

A connector is a four-call commitment: create it, test it, browse it, import from it. The same four calls work whether the source is a petabyte warehouse in Snowflake, a legacy Oracle database behind a VPN, an S3 bucket of Parquet, or a public dataset on HuggingFace.

  • Credentials never live in plain text — POST a password inline and it moves to an encrypted vault before the row commits; every test/browse/import resolves it through a tenant-scoped, audit-logged read, and the value is never returned by any API.
  • Imports land under your tenant's artefact prefix — cross-tenant reads and writes are structurally impossible, not a policy promise.
  • Sample-first mode (1,000 rows) lets the data team eyeball the schema before committing to a 200,000-row import that bills real credits.
◆ Who wires what

Four families, four jobs each team finally gets to delete.

A connector is the last mile of every data project. Done badly, it's the reason integrations slip. Done once, right, across every source — it's what lets the rest of the platform move at the speed the sales team promised.

Warehouses

Snowflake · BigQuery · Databricks · Redshift · Hive

A data-platform team already paid for a warehouse; the problem is getting a representative sample out of it into a training or partner environment without a six-week DataOps ticket. Point a connector at the right database, pick the schema, import the sample you want — it lands under your tenant artefact prefix, not someone's laptop download folder.

Technical · Role-scoped warehouse credentials vaulted on POST, read-only test mode, up to 500 tables per browse page, sample-first imports before the committed big run.

Relational

Postgres · MySQL · MariaDB · MSSQL · Oracle

Every enterprise has a legacy RDBMS that runs half the business. The integration is rarely the exciting part — it's firewall rules, VPN tickets, rotating service accounts, and explaining to security why the connection string has a password in it. Our relational family removes the last problem completely (the password is vaulted before the row commits) and behaves identically across all five dialects.

Technical · TLS-required drivers, a SELECT 1 test probe, schema browse, parameterised imports, per-call audit-logged vault resolution.

Object storage

Amazon S3 · Google GCS · Azure Blob

If your organisation standardised on a lake and a cloud, we don't ask you to change either. The object-storage family imports directly from bucket-and-prefix paths, with the same four-call contract and vaulted credentials. Parquet, CSV, JSON-Lines and TSV all import cleanly; schema inference is first-class.

Technical · IAM-role or access-key auth, multipart-download, per-object streaming, sample-first import before the committed full scan.

Dataset hub

HuggingFace

The fastest path from a published research dataset to a sealed, evidence-chained training run. Paste a dataset ID, pick a split, import — private datasets supported with a scoped token, public datasets work out of the box. The same tenant artefact prefix and evidence record apply.

Technical · Dataset-ID resolution, split-aware browse, token-vaulted imports, split-scoped sampling.

◆ The secret vault

The encrypted secret vault, in plain English.

Connector secrets are not optional to encrypt. They're encrypted by construction — the auto-hoist path enforces it even if the caller forgets.

Fernet at rest

Per-tenant key, AES-128-CBC + HMAC-SHA256. No password is ever stored in the connector row.

Auto-hoist on POST

If you accidentally POST a plain password it is moved to the vault before the row is committed.

Scoped use

The vault entry is read only by the connector worker, only when running an explicit test/browse/import.

Audit log

Every secret resolve recorded with caller, connector ID, and reason — never the value.

TLS-required by default

PG/MySQL/MSSQL/Oracle drivers refuse plain-text connections.

Read-only test mode

The test session uses a single SELECT 1 — never writes to the source.

Tenant artefact prefix

Imported rows land under the calling tenant's artefact path. No cross-tenant write possible.

Sample-first imports

Sample mode (1k rows) lets you eyeball the schema before kicking off a 200k-row import.

Wire one in an afternoon.

Adapter not on the list? We add new connectors on a 2-week SLA when there's a named customer behind the request — drop us a line and we'll scope it.

Connectors guideRequest a new adapter