[
https://issues.apache.org/jira/browse/BEAM-4461?focusedWorklogId=160098&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-160098
]
ASF GitHub Bot logged work on BEAM-4461:
----------------------------------------
Author: ASF GitHub Bot
Created on: 29/Oct/18 17:35
Start Date: 29/Oct/18 17:35
Worklog Time Spent: 10m
Work Description: reuvenlax commented on issue #6856: [BEAM-4461] Switch
SQL over to use the new Beam Group transform.
URL: https://github.com/apache/beam/pull/6856#issuecomment-434005429
Did this only fail on DataflowRunner?
On Mon, Oct 29, 2018 at 3:57 AM Łukasz Gajowy <[email protected]>
wrote:
> It seems that this commit causes failures in SQL Nexmark queries on
> Dataflow runner.
>
> Caused by: java.lang.IllegalArgumentException: Provided coders for type
arguments of class
org.apache.beam.sdk.transforms.CombineFnBase$AbstractGlobalCombineFn contain
incompatibilities: Cannot encode elements of type class java.lang.Object with
coder org.apache.beam.sdk.coders.RowCoder@7fa2f06
> at
org.apache.beam.sdk.coders.CoderRegistry.getDefaultCoders(CoderRegistry.java:438)
> at
org.apache.beam.sdk.coders.CoderRegistry.getDefaultCoders(CoderRegistry.java:368)
> at
org.apache.beam.sdk.coders.CoderRegistry.getCoder(CoderRegistry.java:318)
> at
org.apache.beam.sdk.transforms.CombineFnBase$AbstractGlobalCombineFn.getAccumulatorCoder(CombineFnBase.java:119)
> at
org.apache.beam.sdk.transforms.Combine$CombineFn.getAccumulatorCoder(Combine.java:330)
> at
org.apache.beam.sdk.transforms.CombineFns$ComposedCombineFn.getAccumulatorCoder(CombineFns.java:426)
> at
org.apache.beam.sdk.schemas.transforms.SchemaAggregateFn$Inner.getAccumulatorCoder(SchemaAggregateFn.java:296)
> at
org.apache.beam.sdk.util.AppliedCombineFn.withInputCoder(AppliedCombineFn.java:106)
> at
org.apache.beam.sdk.transforms.Combine$GroupedValues.getAppliedFn(Combine.java:2129)
> at
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$3.translateHelper(DataflowPipelineTranslator.java:772)
> at
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$3.translate(DataflowPipelineTranslator.java:755)
> at
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$3.translate(DataflowPipelineTranslator.java:752)
> at
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator.visitPrimitiveTransform(DataflowPipelineTranslator.java:440)
> at
org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:657)
> at
org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:649)
> at
org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:649)
> at
org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:649)
> at
org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:649)
> at
org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:649)
> at
org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:649)
> at
org.apache.beam.sdk.runners.TransformHierarchy$Node.access$600(TransformHierarchy.java:311)
> at
org.apache.beam.sdk.runners.TransformHierarchy.visit(TransformHierarchy.java:245)
> at org.apache.beam.sdk.Pipeline.traverseTopologically(Pipeline.java:458)
> at
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator.translate(DataflowPipelineTranslator.java:379)
> at
org.apache.beam.runners.dataflow.DataflowPipelineTranslator.translate(DataflowPipelineTranslator.java:169)
> at
org.apache.beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:733)
> at
org.apache.beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:178)
> at org.apache.beam.sdk.Pipeline.run(Pipeline.java:313)
> at org.apache.beam.sdk.Pipeline.run(Pipeline.java:299)
> at
org.apache.beam.sdk.nexmark.NexmarkLauncher.run(NexmarkLauncher.java:1155)
> at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:108)
> at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:96)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by:
org.apache.beam.sdk.coders.CoderRegistry$IncompatibleCoderException: Cannot
encode elements of type class java.lang.Object with coder
org.apache.beam.sdk.coders.RowCoder@7fa2f06 because the coded type class
org.apache.beam.sdk.values.Row is not assignable from class java.lang.Object
> at
org.apache.beam.sdk.coders.CoderRegistry.verifyCompatible(CoderRegistry.java:529)
> at
org.apache.beam.sdk.coders.CoderRegistry.getDefaultCoders(CoderRegistry.java:435)
> ... 37 more
>
> See here: example logs
>
<https://builds.apache.org/view/A-D/view/Beam/job/beam_PostCommit_Java_Nexmark_Dataflow/854/consoleText>
>
> @reuvenlax <https://github.com/reuvenlax> I proposed a revert here: #6866
> <https://github.com/apache/beam/pull/6866>
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/beam/pull/6856#issuecomment-433868590>, or mute
> the thread
>
<https://github.com/notifications/unsubscribe-auth/AUGE1RBWyT_dr8aLVPoqRCwaXtAsvwpsks5upt8tgaJpZM4X9Ebh>
> .
>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 160098)
Time Spent: 14h (was: 13h 50m)
> Create a library of useful transforms that use schemas
> ------------------------------------------------------
>
> Key: BEAM-4461
> URL: https://issues.apache.org/jira/browse/BEAM-4461
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-java-core
> Reporter: Reuven Lax
> Assignee: Reuven Lax
> Priority: Major
> Time Spent: 14h
> Remaining Estimate: 0h
>
> e.g. JoinBy(fields). Project, Filter, etc.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
