Getting Started
1.
Introduction
2.
Installation
3.
Your First Pipeline
Concepts
4.
What is Datapipe?
5.
Tables and TableStores
6.
Pipeline Steps
7.
Incremental Processing
8.
Primary Keys and Transform Keys
How-to Guides
9.
Transform Files (1-to-1)
10.
Pull Data from External Sources
11.
Run Model Inference (Multi-Input Transforms)
12.
Expand One Row Into Many (1-to-N)
13.
Map Mismatched Primary Keys
14.
Filter Steps by Labels
15.
Use SQLite as Metadata Store
16.
Manage Schema Changes with Alembic
17.
Extend the CLI
18.
Write a Custom TableStore
Reference
19.
CLI Commands
20.
Pipeline / Catalog / DatapipeApp
21.
Table
22.
Steps
22.1.
BatchTransform
22.2.
BatchGenerate
22.3.
UpdateExternalTable
22.4.
DatatableTransform
23.
TableStore Backends
23.1.
Database
23.2.
Filedir
23.3.
Redis
23.4.
Elasticsearch
23.5.
Qdrant
23.6.
Milvus
24.
Types
25.
Executors
Explanation
26.
Compute Step Lifecycle
27.
Change Detection and Merging
28.
Meta-Table Schema
Migration
29.
v0.13 → v0.14
30.
v0.14 → v0.15
Light
Rust
Coal
Navy
Ayu
Datapipe
How to Expand One Row Into Many (1-to-N)
Work in progress.
This page has not been written yet.