From d8d9b3a8e56d29b774e756f6faa0950c98c2d554 Mon Sep 17 00:00:00 2001 From: Hongshun Wang Date: Thu, 18 Dec 2025 17:16:37 +0800 Subject: [PATCH 1/2] [docs] Add document about add column at last. --- website/docs/engine-flink/ddl.md | 12 ++++++++++++ .../docs/maintenance/operations/upgrade-notes-0.9.md | 10 ++++++++++ 2 files changed, 22 insertions(+) diff --git a/website/docs/engine-flink/ddl.md b/website/docs/engine-flink/ddl.md index a542db143c..5038069d0d 100644 --- a/website/docs/engine-flink/ddl.md +++ b/website/docs/engine-flink/ddl.md @@ -247,7 +247,19 @@ The following example illustrates reset the `table.datalake.enabled` option to i ```sql title="Flink SQL" ALTER TABLE my_table RESET ('table.datalake.enabled'); ``` +### Add Column At Last +Currently, Fluss only supports adding Nullable columns at the end of the table. +Below are examples demonstrating how to add columns: +```sql title="Flink SQL" +-- Add a single column at the end of the table +ALTER TABLE my_table ADD user_email STRING COMMENT 'User email address'; +-- Add multiple columns at the end of the table +ALTER TABLE MyTable ADD ( + user_email STRING COMMENT 'User email address', + order_quantity INT +); +``` ## Add Partition diff --git a/website/docs/maintenance/operations/upgrade-notes-0.9.md b/website/docs/maintenance/operations/upgrade-notes-0.9.md index b8b54df840..0ceeada316 100644 --- a/website/docs/maintenance/operations/upgrade-notes-0.9.md +++ b/website/docs/maintenance/operations/upgrade-notes-0.9.md @@ -7,6 +7,16 @@ sidebar_position: 4 These upgrade notes discuss important aspects, such as configuration, behavior, or dependencies, that changed between Fluss 0.8 and Fluss 0.9. Please read these notes carefully if you are planning to upgrade your Fluss version to 0.9. +## Add Column At Last. +If you're only upgrading to Fluss v0.9 without modifying table schemas, you can safely skip this section. + +However, If you plan to add columns to existing tables, follow these mandatory steps: +1. Upgrade all Fluss servers to v0.9 +Ensure complete server infrastructure is running v0.9 before attempting any schema modifications. +2. Restart and upgrade Fluss client to v0.9 +Old clients are incompatible with mixed schema versions and cannot properly read data with different schemas. This includes flink job, tier service and java client. + +**Skipping these steps when adding columns may result in schema inconsistency issues or data inaccessibility.** ## Deprecation / End of Support From b4a8715e9fe415a32317620d08ff97c4c79c79fa Mon Sep 17 00:00:00 2001 From: Jark Wu Date: Fri, 26 Dec 2025 21:18:22 +0800 Subject: [PATCH 2/2] improve docs --- website/docs/engine-flink/ddl.md | 43 +++++++++++++------ .../operations/upgrade-notes-0.9.md | 15 +++---- 2 files changed, 37 insertions(+), 21 deletions(-) diff --git a/website/docs/engine-flink/ddl.md b/website/docs/engine-flink/ddl.md index 5038069d0d..60e29641d4 100644 --- a/website/docs/engine-flink/ddl.md +++ b/website/docs/engine-flink/ddl.md @@ -221,6 +221,36 @@ DROP TABLE my_table; This will entirely remove all the data of the table in the Fluss cluster. ## Alter Table + +### Add Columns + +Fluss allows you to evolve a table's schema by adding new columns. This is a lightweight, metadata-only operation that offers the following benefits: + +- **Zero Data Rewrite**: Adding a column does not require rewriting or migrating existing data files. +- **Instant Execution**: The operation completes in milli-seconds, regardless of the table size. +- **Availability**: The table remains online and fully accessible throughout schema evolution, with no disruption to active clients. + +Currently, this feature has the following characteristics: + +- **Position**: New columns are always appended to the end of the existing column list. +- **Nullability**: Only nullable columns can be added to an existing table to ensure compatibility with existing data. +- **Type Support**: You can add columns of any data type, including complex types such as `ROW`, `MAP`, and `ARRAY`. +- **Nested Fields**: Currently, adding fields within an existing nested `ROW` is not supported. Such operations are categorized as "updating column types" and will be supported in future versions. + +You can add a single column or multiple columns using the `ALTER TABLE statement. + +```sql title="Flink SQL" +-- Add a single column at the end of the table +ALTER TABLE my_table ADD user_email STRING COMMENT 'User email address'; + +-- Add multiple columns at the end of the table +ALTER TABLE MyTable ADD ( + user_email STRING COMMENT 'User email address', + order_quantity INT +); +``` + + ### SET properties The SET statement allows users to configure one or more connector options including the [Storage Options](engine-flink/options.md#storage-options) for a specified table. If a particular option is already configured on the table, it will be overridden with the new value. @@ -247,19 +277,6 @@ The following example illustrates reset the `table.datalake.enabled` option to i ```sql title="Flink SQL" ALTER TABLE my_table RESET ('table.datalake.enabled'); ``` -### Add Column At Last -Currently, Fluss only supports adding Nullable columns at the end of the table. -Below are examples demonstrating how to add columns: -```sql title="Flink SQL" --- Add a single column at the end of the table -ALTER TABLE my_table ADD user_email STRING COMMENT 'User email address'; - --- Add multiple columns at the end of the table -ALTER TABLE MyTable ADD ( - user_email STRING COMMENT 'User email address', - order_quantity INT -); -``` ## Add Partition diff --git a/website/docs/maintenance/operations/upgrade-notes-0.9.md b/website/docs/maintenance/operations/upgrade-notes-0.9.md index 0ceeada316..029710b806 100644 --- a/website/docs/maintenance/operations/upgrade-notes-0.9.md +++ b/website/docs/maintenance/operations/upgrade-notes-0.9.md @@ -7,16 +7,15 @@ sidebar_position: 4 These upgrade notes discuss important aspects, such as configuration, behavior, or dependencies, that changed between Fluss 0.8 and Fluss 0.9. Please read these notes carefully if you are planning to upgrade your Fluss version to 0.9. -## Add Column At Last. -If you're only upgrading to Fluss v0.9 without modifying table schemas, you can safely skip this section. +## Adding Columns -However, If you plan to add columns to existing tables, follow these mandatory steps: -1. Upgrade all Fluss servers to v0.9 -Ensure complete server infrastructure is running v0.9 before attempting any schema modifications. -2. Restart and upgrade Fluss client to v0.9 -Old clients are incompatible with mixed schema versions and cannot properly read data with different schemas. This includes flink job, tier service and java client. +Fluss storage format was designed with schema evolution protocols from day one. Therefore, tables created in v0.8 or earlier can immediately benefit from the `ADD COLUMN` feature after upgrading to v0.9 without dropping and re-creating table. +However, old clients (pre-v0.9) do not recognize the new schema versioning protocol. If an old client attempts to read data from a table that has undergone schema evolution, it may encounter decoding errors or data inaccessibility. +Therefore, it is crucial to ensure that all Fluss servers and all clients interacting with the Fluss cluster are upgraded to v0.9 before performing any schema modifications. -**Skipping these steps when adding columns may result in schema inconsistency issues or data inaccessibility.** +:::warning +Attempting to add columns while older clients (v0.8 or below) are still actively reading from the table will lead to Schema Inconsistency and may crash your streaming pipelines. +::: ## Deprecation / End of Support