Getting Started
1.
Introduction
2.
Installation
3.
Your First Pipeline
Concepts
4.
What is Datapipe?
5.
Tables and TableStores
6.
Pipeline Steps
7.
Incremental Processing
8.
Primary Keys and Transform Keys
How-to Guides
9.
Transform Files (1-to-1)
10.
Pull Data from External Sources
11.
Run Model Inference (Multi-Input Transforms)
12.
Expand One Row Into Many (1-to-N)
13.
Map Mismatched Primary Keys
14.
Filter Steps by Labels
15.
Use SQLite as Metadata Store
16.
Manage Schema Changes with Alembic
17.
Extend the CLI
18.
Report Run Progress
19.
Write a Custom TableStore
Reference
20.
CLI Commands
21.
Pipeline / Catalog / DatapipeApp
22.
Table
23.
Steps
23.1.
BatchTransform
23.2.
BatchGenerate
23.3.
UpdateExternalTable
23.4.
DatatableTransform
24.
TableStore Backends
24.1.
Database
24.2.
Filedir
24.3.
Redis
24.4.
Elasticsearch
24.5.
Qdrant
24.6.
Milvus
25.
Types
26.
Executors
27.
Run Callbacks
Explanation
28.
Compute Step Lifecycle
29.
Change Detection and Merging
30.
Meta-Table Schema
Migration
31.
v0.13 → v0.14
32.
v0.14 → v0.15
Light
Rust
Coal
Navy
Ayu
Datapipe
Meta-Table Schema
Work in progress.
This page has not been written yet.