Pranaykarvi opened a new pull request, #9590:
URL: https://github.com/apache/gravitino/pull/9590
### What changes were proposed in this pull request?
This PR fixes an interoperability issue between the Gravitino Flink
connector and
the native Flink Hive client.
Specifically, it ensures that Hive tables created via the Gravitino Flink
connector
explicitly set a compatible ROW FORMAT SERDE. When `serde-lib` is not
provided by
the user, the connector now defaults it to
`org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe`.
The change is implemented in `HivePropertiesConverter` and is scoped
strictly to
Hive tables created through the Flink connector. Explicit user-provided SerDe
configurations are preserved.
### Why are the changes needed?
Previously, when creating Hive tables via the Gravitino Flink connector, no
SerDe
was explicitly set. As a result, the Hive catalog defaulted to
`org.apache.hive.hcatalog.data.JsonSerDe`.
However, the native Flink Hive client expects
`org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe`. This mismatch caused
Hive
tables created by the Gravitino Flink connector to be unreadable by the
native
Flink client.
This PR resolves the issue by explicitly setting a Flink-compatible default
SerDe
when none is provided, ensuring Flink ↔ Hive interoperability.
Fix: #9508
### Does this PR introduce _any_ user-facing change?
Yes, but in a backward-compatible way.
- Hive tables created via the Gravitino Flink connector will now default to
`LazySimpleSerDe` when `serde-lib` is not explicitly specified.
- Existing tables and tables with an explicitly configured SerDe are
unaffected.
- Other connectors (e.g., Spark, Trino) and non-Hive catalogs are not
impacted.
No user-facing APIs are changed, and no existing configuration keys are
removed.
### How was this patch tested?
The patch was tested with both unit and integration tests:
1. **Unit tests**
- Added tests for `HivePropertiesConverter` to verify:
- `serde-lib` is set to `LazySimpleSerDe` when missing
- Explicitly provided `serde-lib` values are preserved
2. **Integration test**
- Added an end-to-end test that creates a Hive table via the Gravitino
Flink
connector without specifying `serde-lib`
- Verified that the generated table metadata contains the expected SerDe
All tests run in CI and do not require a live Hive Metastore or Flink
runtime.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
