[
https://issues.apache.org/jira/browse/BEAM-4076?focusedWorklogId=116284&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-116284
]
ASF GitHub Bot logged work on BEAM-4076:
----------------------------------------
Author: ASF GitHub Bot
Created on: 27/Jun/18 03:31
Start Date: 27/Jun/18 03:31
Worklog Time Spent: 10m
Work Description: chamikaramj commented on a change in pull request
#5545: [BEAM-4076] Import Schema branch into master
URL: https://github.com/apache/beam/pull/5545#discussion_r198357373
##########
File path:
runners/core-java/src/main/java/org/apache/beam/runners/core/SimpleDoFnRunner.java
##########
@@ -101,19 +119,64 @@ public SimpleDoFnRunner(
TupleTag<OutputT> mainOutputTag,
List<TupleTag<?>> additionalOutputTags,
StepContext stepContext,
+ @Nullable Coder<InputT> inputCoder,
+ Map<TupleTag<?>, Coder<?>> outputCoders,
WindowingStrategy<?, ?> windowingStrategy) {
this.options = options;
this.fn = fn;
this.signature = DoFnSignatures.getSignature(fn.getClass());
this.observesWindow = signature.processElement().observesWindow() ||
!sideInputReader.isEmpty();
this.invoker = DoFnInvokers.invokerFor(fn);
this.sideInputReader = sideInputReader;
+ this.schemaCoder = (inputCoder != null && inputCoder instanceof
SchemaCoder)
+ ? (SchemaCoder<InputT>) inputCoder : null;
+ this.outputCoders = outputCoders;
+ if (outputCoders != null) {
+ Coder<OutputT> outputCoder = (Coder<OutputT>)
outputCoders.get(mainOutputTag);
+ mainOutputSchemaCoder = (outputCoder instanceof SchemaCoder)
+ ? (SchemaCoder<OutputT>) outputCoder : null;
+ } else {
+ mainOutputSchemaCoder = null;
+ }
this.outputManager = outputManager;
this.mainOutputTag = mainOutputTag;
this.outputTags =
Sets.newHashSet(FluentIterable.<TupleTag<?>>of(mainOutputTag).append(additionalOutputTags));
this.stepContext = stepContext;
+ // Currently we only support a single FieldAccess on a processElement. We
should decide
+ // whether to get rid of the FieldAccess ids, or find a use for multiple.
+ DoFnSignature doFnSignature = DoFnSignatures.getSignature(fn.getClass());
+ DoFnSignature.ProcessElementMethod processElementMethod =
doFnSignature.processElement();
+ RowParameter rowParameter = processElementMethod.getRowParameter();
+ FieldAccessDescriptor fieldAccessDescriptor = null;
+ if (rowParameter != null) {
+ checkArgument(schemaCoder != null,
+ "Cannot access object as a row if the input PCollection does not
have a schema ."
Review comment:
Does "no schema coder" ==> "no schema" or do we have to talk about not
having the schema coder defined in this exception message ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 116284)
> Schema followups
> ----------------
>
> Key: BEAM-4076
> URL: https://issues.apache.org/jira/browse/BEAM-4076
> Project: Beam
> Issue Type: Improvement
> Components: beam-model, dsl-sql, sdk-java-core
> Reporter: Kenneth Knowles
> Priority: Major
> Time Spent: 2h
> Remaining Estimate: 0h
>
> This umbrella bug contains subtasks with followups for Beam schemas, which
> were moved from SQL to the core Java SDK and made to be type-name-based
> rather than coder based.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
