hudi-agent commented on code in PR #19855:
URL: https://github.com/apache/hudi/pull/19855#discussion_r3965858690
##########
hudi-common/src/main/java/org/apache/hudi/common/schema/HoodieSchemaUtils.java:
##########
@@ -520,6 +520,49 @@ public static HoodieSchema
generateProjectionSchema(HoodieSchema originalSchema,
return HoodieSchema.createRecord(originalSchema.getName(),
originalSchema.getNamespace().orElse(null),
originalSchema.getDoc().orElse(null), projectedFields);
}
+ /**
+ * Generate a reader schema off the provided writeSchema, to just project
out the provided columns.
+ *
+ * <p>This overload is intended for callers that already have a
name-to-field map,
+ * such as the realtime reader.</p>
+ *
+ * @param writeSchema the source schema
+ * @param schemaFieldsMap prebuilt case-insensitive field-name map
+ * @param fieldNames the list of field names to include in the
projection
+ * @return new HoodieSchema containing only the specified fields
+ */
+ public static HoodieSchema generateProjectionSchema(HoodieSchema writeSchema,
+ Map<String,
HoodieSchemaField> schemaFieldsMap,
+ List<String> fieldNames)
{
+ ValidationUtils.checkArgument(writeSchema != null, "Write schema cannot be
null");
+ ValidationUtils.checkArgument(schemaFieldsMap != null, "Schema fields map
cannot be null");
+ ValidationUtils.checkArgument(fieldNames != null, "Field names cannot be
null");
+
+ /*
+ * Avro & Presto field names seems to be case sensitive (support fields
differing only in case) whereas
+ * Hive/Impala/SparkSQL(default) are case-insensitive. Spark allows this
to be configurable using
+ * spark.sql.caseSensitive=true
+ *
+ * For a RT table setup with no delta-files (for a latest file-slice) ->
we translate parquet schema to Avro Here
+ * the field-name case is dependent on parquet schema. Hive (1.x/2.x/CDH)
translate column projections to
+ * lower-cases
+ *
+ */
+ List<HoodieSchemaField> projectedFields = new
ArrayList<>(fieldNames.size());
+ for (String fn : fieldNames) {
Review Comment:
🤖 nit: since this is being moved into a shared util, could you spell out
`fn` as `fieldName`? It reads better now that the loop is in a public API.
<sub><i>⚠️ AI-generated; verify before applying. React 👍/👎 to flag
quality.</i></sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
