hflow
Open-source SDK for data quality validation and management for robotics teams
About hflow
hflow is a data quality validation SDK for robotics teams, designed to check the quality of data needed to train AI models. Teams can keep their own Python processing code and connect it to existing workflows through small adapters; quality check results are captured as queryable measurements, while metadata, quality measurements, labels, versions, and artifact locations are recorded in a Parquet catalog. The project supports single-tenant deployment with Docker Compose and deployment of the generated DAG bundle to a team's own Airflow 3 environment; the quick start can also run an example pipeline in-process.
Best for
Best use cases
Validate the quality of robot training data
Connect existing Python processing code to a workflow
Query dataset quality measurements and metadata
Deploy with Docker Compose or Airflow 3
Features
Bring Your Own Python Code
Run transformation, quality check, labeling, and data enrichment logic as Python functions in your team's environment, with adapters for existing code.
Queryable Quality Evidence
Store quality check results as queryable measurements; different datasets can use different thresholds without reprocessing media.
Parquet Data Catalog
Store metadata, quality measurements, labels, version markers, and artifact locations in a Parquet catalog, queryable without loading recordings.
Flexible Deployment and Quick Start
Support single-tenant Docker Compose workspaces and teams' own Airflow 3 environments; quick-start examples can also run pipelines in-process.
Pros
- Keep your team's own processing code
- Reuse and query quality check results
- Manage data metadata with a Parquet catalog
- Quick start without Docker or Airflow
Cons
- No user accounts, RBAC, or multi-tenant control plane provided