Roadmap
Hudi community strives to deliver major releases every 3-4 months, while offering minor releases every other month! This page captures the forward-looking roadmap of ongoing & upcoming projects and when they are expected to land, broken down by areas on our stack.
Future Releases
Next major release : 0.14.0 (September 2023)
| Release | Timeline |
|---|---|
| 0.14.0 | Sep 2023 |
| 1.0.0 | Oct 2023 |
Transactional Database Layer
| Feature | Target Release | Tracking |
|---|---|---|
| Support for primary key-less table | 0.14.0 | HUDI-4699 |
| Efficient bootstrap and migration of existing non-Hudi dataset | 0.14.0 | HUDI-1265 |
| Record-level index to speed up UUID-based upserts and deletes | 0.14.0 | RFC-08, HUDI-53 |
| 1.x Storage format | 1.0.0 | HUDI-6242 |
| Writer performance improvements | 1.0.0 | HUDI-3249 |
| Non-blocking concurrency control | 1.0.0 | HUDI-3187, HUDI-1042, RFC-66 |
| Time Travel updates, deletes | 1.0.0 | |
| General purpose support for multi-table transactions | 1.0.0 | |
| A more effective HoodieMergeHandler for COW table with parquet | 1.0.0 | RFC-68 |
| Secondary indexes to improve query performance | 1.0.0 | RFC-52, HUDI-3907 |
| Index Function for Optimizing Query Performance | 1.0.0 | RFC-63, HUDI-512 |
| Logical partitioning via indexing | 1.0.0 | HUDI-512 |
| Streaming CDC/Incremental read improvement | 1.0.0 | HUDI-2749 |
| Supervised table service planning and execution | 1.1.0 | RFC-43, HUDI-4147 |
Programming APIs
| Feature | Target Release | Tracking |
|---|---|---|
| MOR snapshot reader | 0.14.0 | HUDI-6452 |
| APIs/Abstractions, Record mergers | 1.0.0 | HUDI-6243, HUDI-3217 |
| New Hudi Table Format APIs for Query Integrations | 1.0.0 | RFC-64, HUDI-4141 |
| Snapshot view management | 1.0.0 | RFC-61, HUDI-4677 |
| Optimized storage layout for cloud object stores | 1.0.0 | RFC-60, HUDI-3625 |
| Support of verification with multiple event_time fields | 1.0.0 | RFC-59, HUDI-4569 |
Execution Engine Integration
| Feature | Target Release | Tracking |
|---|---|---|
| Support for Spark 3.4.0 | 0.14.0 | HUDI-6198 |
| Integrate column stats index with all query engines | 0.14.0 | RFC-58, HUDI-4552 |
| Upgrade to Spark 3 as the default profile | 0.14.0 | HUDI-3431 |
| Performance for Spark-SQL writes | 0.14.0 | HUDI-6315, HUDI-6376 |
| Presto/Trino queries with new format | 1.0.0 | HUDI-3210, HUDI-4394, HUDI-4552 |
| Materialized Views with incremental updates using Flink | 1.0.0 | |
| Explore other execution engines/runtimes (Ray, native Rust, Python) | 1.0.0 | |
| Spark datasource V2 read | 1.1.0 | HUDI-4449 |
Platform Services
| Feature | Target Release | Tracking |
|---|---|---|
| Hudi Reverse streamer | 1.0.0 | RFC-70 |
| Partition TTL management | 1.1.0 | RFC-65 |
| Diagnostic Reporter | 1.1.0 | RFC-62 |
| Hudi integration with Snowflake | 1.1.0 | RFC-41, HUDI-2832 |
| Support for reliable, event based ingestion from cloud stores - GCS, Azure and the others | 1.1.0 | HUDI-1896 |
| Mutable, Transactional caching for Hudi Tables (could be accelerated based on community feedback) | 1.1.0 | Strawman design, HUDI-6489 |