[
https://issues.apache.org/jira/browse/SQOOP-1803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14363452#comment-14363452
]
Veena Basavaraj commented on SQOOP-1803:
----------------------------------------
#1 seems simple and also apt since we already are storing information in this
context. The current context is mutable is already "explicit" in what the
Initializer extends.
#2. I do not perceive having a new api is any more useful. Did you mean having
this new method "updateConfigurationObjects" on the "Mutablecontext". Can you
clarify that?. Not sure if this adds to more confusion since there is no way to
restrict not updating the config via the mutable context. So there can be
multiple ways to do it and the code needs to handle both the cases. I would go
for simplicity and just use the mutable context. Hence I did not lay out these
additional workflows and if we do add "updater", it could as well be designed
to be called as part of the parallel execution of mappers/ reducers or the
spark operators, and not just at the end of the workflow. So It is best to give
it more thought when a use case prevails. For now #1 seems to solve the use
case we have for jdbc
> JobManager and Execution Engine changes: Support for a injecting and pulling
> out configs and job output in connectors
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: SQOOP-1803
> URL: https://issues.apache.org/jira/browse/SQOOP-1803
> Project: Sqoop
> Issue Type: Sub-task
> Reporter: Veena Basavaraj
> Assignee: Veena Basavaraj
> Fix For: 1.99.6
>
>
> The details are in the design wiki, as the implementation happens more
> discussions can happen here.
> https://cwiki.apache.org/confluence/display/SQOOP/Delta+Fetch+And+Merge+Design#DeltaFetchAndMergeDesign-Howtogetoutputfromconnectortosqoop?
> The goal is to dynamically inject a IncrementalConfig instance into the
> FromJobConfiguration. The current MFromConfig and MToConfig can already hold
> a list of configs, and a strong sentiment was expressed to keep it as a list,
> why not for the first time actually make use of it and group the incremental
> related configs in one config object
> This task will prepare the FromJobConfiguration from the job config data,
> ExtractorContext with the relevant values from the prev job run
> This task will prepare the ToJobConfiguration from the job config data,
> LoaderContext with the relevant values from the prev job run if any
> We will use DistributedCache to get State information from the Extractor and
> Loader out and finally persist it into the sqoop repository depending on
> SQOOP-1804 once the outputcommitter commit is called
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
