Skip to main content
bg-image

Apache HudiTM brings

to data lakes
Get StartedRead Docs
hudi-logo

What is Hudi 

Apache Hudi is an open data lakehouse platform, built on a high-performance open table format to bring database functionality to your data lakes. Hudi reimagines slow old-school batch data processing with a powerful new incremental processing framework for low latency minute-level analytics.
Hudi Data LakeHudi Data Lake

Integrations 

Data Streaming
Apache Kafka
Apache Kafka
Apache Pulsar
Apache Pulsar
Databases
PostgreSQL
PostgreSQL
MySQL
MySQL
CDC
Debezium
Debezium
Apache Flink CDC
Apache Flink CDC
File Formats
Apache Parquet
Apache Parquet
Apache ORC
Apache ORC
Apache Avro
Apache Avro
CSV
CSV
JSON
JSON
Lake Storage
Apache Hadoop
Apache Hadoop
Amazon S3
Amazon S3
Google Cloud Storage
Google Cloud Storage
Azure Blob Storage
Azure Blob Storage
Alibaba Cloud
Alibaba Cloud
IBM Cloud
IBM Cloud
Oracle Cloud
Oracle Cloud
Tencent Cloud
Tencent Cloud
MinIO
MinIO
Data Catalogs
AWS Glue Data Catalog
AWS Glue Data Catalog
Google BigQuery
Google BigQuery
Apache Hive Metastore
Apache Hive Metastore
DataHub
DataHub
Apache XTable (Incubating) (For sync)
Apache XTable (Incubating) (For sync)
Data Warehouses
Amazon Redshift
Amazon Redshift
ClickHouse
ClickHouse
Interactive Analytics
Presto
Presto
Trino
Trino
Apache Hive
Apache Hive
AWS Athena
AWS Athena
Google BigQuery
Google BigQuery
Apache Doris
Apache Doris
StarRocks
StarRocks
Apache Impala
Apache Impala
Data Processing
Apache Spark
Apache Spark
Apache Flink
Apache Flink
Databricks
Databricks
AWS EMR
AWS EMR
Azure HDInsight
Azure HDInsight
Onehouse
Onehouse
Ray
Ray
Daft
Daft
Orchestration
dbt
dbt
Apache Airflow
Apache Airflow

Hudi Features 

Mutability support for all workload shapes & sizes

Quickly update & delete data with fast, pluggable indexing. This includes database CDC and high-scale streaming data, with best-in-class support for out-of-order records, bursty traffic & data deduplication.

Unlock 10x efficiency by incrementally processing new data

Replace old-school batch pipelines with incremental streaming on your data lake. Experience faster ingestion and lower processing times for your data pipelines.

ACID Transactional guarantees for your data lake

Atomic writes, with relational/streaming data consistency models, snapshot isolation and non-blocking concurrency controls tailored for longer-running lake transactions.

Analyze historical data with time travel

Query historical data with the ability to roll back to a table version; debug data versions to understand what changed over time; audit data changes by viewing the commit history.

Interoperable multi-cloud ecosystem support

Built on open data formats with extensive ecosystem support across cloud vendor ecosystem, with plug-and-play options for popular data sources & query engines.

Automatic table services for a high-performance lakehouse

Fully automated table services that continuously schedule & orchestrate clustering, compaction, cleaning, file sizing & indexing to ensure tables are always optimized.

Open Data Lakehouse platform to get you going faster

Effortlessly build your lakehouse with built-in tools for auto ingestion from services like Debezium and Kafka and auto catalog sync to major cloud engines & more.

Query acceleration through multi-modal indexes.

Experience faster write transactions on huge/wide tables & faster query performance with first-of-its kind multi-modal indexing subsystem.

Resilient Pipelines with schema evolution & enforcement

Easily change the current schema of a Hudi table to adapt to the data that is changing over time and ensure pipeline resilience by failing fast and avoiding data corruption.

Why Hudi 

The most innovative and completely open data lakehouse platform in the industry!

Trusted Platform

Battle tested and proven in production in some of the largest data lakes on the planet.

Open Source

Hudi is a thriving & growing community that is built with contributions from people around the globe.

High Performance

Hudi's storage format is purpose-built to continuously deliver performance as data scales.

Data streams

Take advantage of built-in CDC sources and tools for streaming ingestion.

Hudi Blogs 

View All

Join our Community 

Get technical help, influence the product roadmap & see what’s new with Hudi!

Youtube

Linkedin

GitHub

Slack

Mailing

X