vinothchandar commented on a change in pull request #1566:
URL: https://github.com/apache/incubator-hudi/pull/1566#discussion_r416319065
##########
File path:
hudi-utilities/src/main/java/org/apache/hudi/utilities/deltastreamer/DeltaSync.java
##########
@@ -162,18 +162,23 @@ public DeltaSync(HoodieDeltaStreamer.Config cfg,
SparkSession sparkSession, Sche
this.fs = fs;
this.onInitializingHoodieWriteClient = onInitializingHoodieWriteClient;
this.props = props;
- this.schemaProvider = schemaProvider;
refreshTimeline();
-
this.transformer =
UtilHelpers.createTransformer(cfg.transformerClassNames);
this.keyGenerator = DataSourceUtils.createKeyGenerator(props);
-
- this.formatAdapter = new SourceFormatAdapter(
- UtilHelpers.createSource(cfg.sourceClassName, props, jssc,
sparkSession, schemaProvider));
-
this.conf = conf;
+ refreshSchemaProvider(schemaProvider);
+ }
+ /**
+ * Very useful when DeltaStreamer is running in continuous mode.
+ * @param schemaProvider
+ * @throws IOException
+ */
+ public void refreshSchemaProvider(SchemaProvider schemaProvider) throws
IOException {
Review comment:
right.. embedded server re-use is important. let's scratch my idea.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
