[
https://issues.apache.org/jira/browse/GOBBLIN-1080?focusedWorklogId=403165&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-403165
]
ASF GitHub Bot logged work on GOBBLIN-1080:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 13/Mar/20 20:48
Start Date: 13/Mar/20 20:48
Worklog Time Spent: 10m
Work Description: autumnust commented on pull request #2925:
[GOBBLIN-1080] Add configuration to add schema creation time in converter
URL: https://github.com/apache/incubator-gobblin/pull/2925#discussion_r392464622
##########
File path:
gobblin-core-base/src/main/java/org/apache/gobblin/converter/filter/AvroProjectionConverter.java
##########
@@ -74,10 +74,15 @@ public AvroProjectionConverter init(WorkUnitState
workUnit) {
*/
@Override
public Schema convertSchema(Schema inputSchema, WorkUnitState workUnit)
throws SchemaConversionException {
+ Schema outputSchema = inputSchema;
if (this.fieldRemover.isPresent()) {
- return this.fieldRemover.get().removeFields(inputSchema);
+ outputSchema = this.fieldRemover.get().removeFields(inputSchema);
}
- return inputSchema;
+
if(workUnit.getPropAsBoolean(ConfigurationKeys.CONVERTER_AVRO_INCLUDE_SCHEMA_CREATION_TIME,
+
ConfigurationKeys.DEFAULT_CONVERTER_AVRO_INCLUDE_SCHEMA_CREATION_TIME)) {
Review comment:
what if a pipeline needs this but not using this converter? shall we
consider to add it into higher level ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 403165)
Time Spent: 1h 40m (was: 1.5h)
> Add configuration to preserve schema creation time in converter
> ---------------------------------------------------------------
>
> Key: GOBBLIN-1080
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1080
> Project: Apache Gobblin
> Issue Type: Task
> Reporter: Zihan Li
> Priority: Major
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
