[
https://issues.apache.org/jira/browse/SQOOP-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14257166#comment-14257166
]
Veena Basavaraj commented on SQOOP-1938:
----------------------------------------
1. I do understand when we would need custom writables. But not sure the
current use case in Sqoop needs a custom one, I have another ticket that I
raised regarding this, in the parent ticket SQOOP-1934, lets discuss it there.
This ticket is just document what is the current state of code.
#2. We cannot always call sqoop job as a map only job, if we give the
numLoaders to anything other than the default 0, the reducer is executed. This
is what I understood and confirmed with [~abec] as well.
#3. I would hope you can help document the reader/ writer code in the output
format executor. In a non reducer case, i,e when numLoaders is zero, I would
like to see some documentation on who is the reader / writer? why do we need
the synchronization code? It is not very obvious, so atleast a write up around
that will be good, I will review it
> DOC:update the sqoop MR engine implementation details
> -----------------------------------------------------
>
> Key: SQOOP-1938
> URL: https://issues.apache.org/jira/browse/SQOOP-1938
> Project: Sqoop
> Issue Type: Sub-task
> Reporter: Veena Basavaraj
>
> 1. Why we need SqoopWritable, what can be done in future?
> 2. Even though we call sqoop as a map only, is that how it always works? what
> happend when numLoaders is non zero
> {code}
> // Set number of reducers as number of configured loaders or suppress
> // reduce phase entirely if loaders are not set at all.
> if(request.getLoaders() != null) {
> job.setNumReduceTasks(request.getLoaders());
> } else {
> job.setNumReduceTasks(0);
> }
> {code}
> 3. Internals of SqoopNullOutputFormat and how SqoopWritable is used in it
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
