Roadmap
Hudi community strives to deliver major releases every 3-4 months, while offering minor releases every other month! This page captures the forward-looking roadmap of ongoing & upcoming projects and when they are expected to land, broken down by areas on our stack.
H2 2022 Releases
Next major release : 0.13.0 (Mid Q4 2022)
| Release | Timeline |
|---|---|
| 0.12.1 | Early Q4 2022 |
| 0.13.0 | Mid Q4 2022 |
| 0.13.1 | TBD |
Transactional Database Layer
| Feature | Target Release | Tracking |
|---|---|---|
| Metaserver for all metadata | 0.13.0 | Strawman design, RFC-36, HUDI-3345 |
| Supervised table service planning and execution | 0.13.0 | RFC-43, HUDI-4147 |
| Support of Change Data Capture (CDC) with Hudi change logs | 0.13.0 | RFC-51, HUDI-3478 |
| Record-level index to speed up UUID-based upserts and deletes | 0.13.0 | RFC-08, HUDI-53 |
| Consistent hashing index for dynamic buckets | 0.13.0 | RFC-42, HUDI-3000 |
| Secondary index to improve query performance | 0.13.0 | RFC-52, HUDI-3907 |
| Reducing write amplification with Log Compaction in MOR | 0.13.0 | RFC-48, HUDI-3580 |
| Eager conflict detection for Optimistic Concurrency Control | 0.13.0 | RFC-56, HUDI-1575 |
| Efficient bootstrap and migration of existing non-Hudi dataset | 0.13.0 | HUDI-1265 |
| Eliminating physical partitioning with efficient logical partitioning and file pruning | 0.13.0 | HUDI-512 |
| Lock-Free message queue to improve writing efficiency | 0.13.0 | RFC-53, HUDI-3963 |
| Lock free concurrency control | 1.0.0 onward | HUDI-3187 |
| Non-blocking/Lock-free updates during clustering | 1.0.0 onward | HUDI-1042 |
| Time Travel updates, deletes | 1.0.0 onward | |
| General purpose support for multi-table transactions | 1.0.0 onward |
Programming APIs
| Feature | Target Release | Tracking |
|---|---|---|
| Redesign and optimization of record payload abstraction | 0.13.0 | RFC-46, HUDI-3217 |
| Optimized storage layout for cloud object stores | 0.13.0 | RFC-60, HUDI-3625 |
| Support of verification with multiple event_time fields | 0.13.0 | RFC-59, HUDI-4569 |
Execution Engine Integration
| Feature | Target Release | Tracking |
|---|---|---|
| Spark datasource V2 read | 0.13.0 | HUDI-4449 |
| Integrate column stats index with all query engines | 0.13.0 | RFC-58, HUDI-4552 |
| Upgrade to Spark 3 as the default profile | 0.13.0 | HUDI-3431 |
| Materialized Views with incremental updates using Flink | 1.0.0 onward | |
| SQL DML support for Presto/Trino connectors (could be accelerated based on community feedback) | 1.0.0 onward | |
| Explore other execution engines/runtimes (Ray, native Rust, Python) | 1.0.0 onward |
Platform Services
| Feature | Target Release | Tracking |
|---|---|---|
| Protobuf source for DeltaStreamer | 0.13.0 | RFC-57, HUDI-4399 |
| Hudi integration with Snowflake | 0.13.0 | RFC-41, HUDI-2832 |
| Improving Hudi CLI features and usability | 0.13.0 | HUDI-1388 |
| Support for reliable, event based ingestion from cloud stores - GCS, Azure and the others | 1.0.0 onward | HUDI-1896 |
| Mutable, Transactional caching for Hudi Tables (could be accelerated based on community feedback) | 1.0.0 onward | Strawman design |