[jira] [Work logged] (BEAM-4076) Schema followups

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4076?focusedWorklogId=424287=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-424287
 ]

ASF GitHub Bot logged work on BEAM-4076:


Author: ASF GitHub Bot
Created on: 17/Apr/20 16:51
Start Date: 17/Apr/20 16:51
Worklog Time Spent: 10m 
  Work Description: kennknowles commented on pull request #11041: 
[BEAM-4076] Use beam join api in sql
URL: https://github.com/apache/beam/pull/11041#discussion_r410347297
 
 

 ##
 File path: 
sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/transform/BeamJoinTransforms.java
 ##
 @@ -45,111 +41,35 @@
 /** Collections of {@code PTransform} and {@code DoFn} used to perform JOIN 
operation. */
 public class BeamJoinTransforms {
 
-  /** A {@code SimpleFunction} to extract join fields from the specified row. 
*/
-  public static class ExtractJoinFields extends SimpleFunction> {
-private final List joinColumns;
-private final Schema schema;
-private int leftRowColumnCount;
-
-public ExtractJoinFields(
-boolean isLeft,
-List> joinColumns,
-Schema schema,
-int leftRowColumnCount) {
-  this.joinColumns =
-  joinColumns.stream()
-  .map(pair -> SerializableRexNode.builder(isLeft ? pair.left : 
pair.right).build())
-  .collect(toList());
-  this.schema = schema;
-  this.leftRowColumnCount = leftRowColumnCount;
-}
-
-@Override
-public KV apply(Row input) {
-  Row row =
-  joinColumns.stream()
-  .map(v -> getValue(v, input, leftRowColumnCount))
-  .collect(toRow(schema));
-  return KV.of(row, input);
-}
-
-@SuppressWarnings("unused")
-private Schema.Field toField(Schema schema, Integer fieldIndex) {
-  Schema.Field original = schema.getField(fieldIndex);
-  return original.withName("c" + fieldIndex);
-}
-
-private Object getValue(
-SerializableRexNode serializableRexNode, Row input, int 
leftRowColumnCount) {
-  if (serializableRexNode instanceof SerializableRexInputRef) {
-return input.getValue(
-((SerializableRexInputRef) serializableRexNode).getIndex() - 
leftRowColumnCount);
-  } else { // It can only be SerializableFieldAccess.
-List indexes = ((SerializableRexFieldAccess) 
serializableRexNode).getIndexes();
-// retrieve row based on the first column reference.
-Row rowField = input.getValue(indexes.get(0) - leftRowColumnCount);
-for (int i = 1; i < indexes.size() - 1; i++) {
-  rowField = rowField.getRow(indexes.get(i));
-}
-return rowField.getValue(indexes.get(indexes.size() - 1));
-  }
-}
+  public static FieldAccessDescriptor getJoinColumns(
+  boolean isLeft,
+  List> joinColumns,
+  int leftRowColumnCount,
+  Schema schema) {
+List joinColumnsBuilt =
+joinColumns.stream()
+.map(pair -> SerializableRexNode.builder(isLeft ? pair.left : 
pair.right).build())
+.collect(toList());
+return FieldAccessDescriptor.union(
+joinColumnsBuilt.stream()
+.map(v -> getJoinColumn(v, leftRowColumnCount).resolve(schema))
+.collect(Collectors.toList()));
   }
 
-  /** A {@code DoFn} which implement the sideInput-JOIN. */
-  public static class SideInputJoinDoFn extends DoFn, Row> {
-private final PCollectionView>> sideInputView;
-private final JoinRelType joinType;
-private final Row rightNullRow;
-private final boolean swap;
-private final Schema schema;
-
-public SideInputJoinDoFn(
-JoinRelType joinType,
-Row rightNullRow,
-PCollectionView>> sideInputView,
-boolean swap,
-Schema schema) {
-  this.joinType = joinType;
-  this.rightNullRow = rightNullRow;
-  this.sideInputView = sideInputView;
-  this.swap = swap;
-  this.schema = schema;
-}
-
-@ProcessElement
-public void processElement(ProcessContext context) {
-  Row key = context.element().getKey();
-  Row leftRow = context.element().getValue();
-  Map> key2Rows = context.sideInput(sideInputView);
-  Iterable rightRowsIterable = key2Rows.get(key);
-
-  if (rightRowsIterable != null && rightRowsIterable.iterator().hasNext()) 
{
-for (Row aRightRowsIterable : rightRowsIterable) {
-  context.output(combineTwoRowsIntoOne(leftRow, aRightRowsIterable, 
swap, schema));
-}
-  } else {
-if (joinType == JoinRelType.LEFT) {
-  context.output(combineTwoRowsIntoOne(leftRow, rightNullRow, swap, 
schema));
-}
+  private static FieldAccessDescriptor getJoinColumn(
 
 Review comment:
   I noticed this, because in the current codebase on master we could inline 
and delete `SerializableRexNode` entirely. So I went looking for how 

[jira] [Work logged] (BEAM-4076) Schema followups

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4076?focusedWorklogId=424286=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-424286
 ]

ASF GitHub Bot logged work on BEAM-4076:


Author: ASF GitHub Bot
Created on: 17/Apr/20 16:50
Start Date: 17/Apr/20 16:50
Worklog Time Spent: 10m 
  Work Description: kennknowles commented on pull request #11041: 
[BEAM-4076] Use beam join api in sql
URL: https://github.com/apache/beam/pull/11041#discussion_r410346652
 
 

 ##
 File path: 
sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/transform/BeamJoinTransforms.java
 ##
 @@ -45,111 +41,35 @@
 /** Collections of {@code PTransform} and {@code DoFn} used to perform JOIN 
operation. */
 public class BeamJoinTransforms {
 
-  /** A {@code SimpleFunction} to extract join fields from the specified row. 
*/
-  public static class ExtractJoinFields extends SimpleFunction> {
-private final List joinColumns;
-private final Schema schema;
-private int leftRowColumnCount;
-
-public ExtractJoinFields(
-boolean isLeft,
-List> joinColumns,
-Schema schema,
-int leftRowColumnCount) {
-  this.joinColumns =
-  joinColumns.stream()
-  .map(pair -> SerializableRexNode.builder(isLeft ? pair.left : 
pair.right).build())
-  .collect(toList());
-  this.schema = schema;
-  this.leftRowColumnCount = leftRowColumnCount;
-}
-
-@Override
-public KV apply(Row input) {
-  Row row =
-  joinColumns.stream()
-  .map(v -> getValue(v, input, leftRowColumnCount))
-  .collect(toRow(schema));
-  return KV.of(row, input);
-}
-
-@SuppressWarnings("unused")
-private Schema.Field toField(Schema schema, Integer fieldIndex) {
-  Schema.Field original = schema.getField(fieldIndex);
-  return original.withName("c" + fieldIndex);
-}
-
-private Object getValue(
-SerializableRexNode serializableRexNode, Row input, int 
leftRowColumnCount) {
-  if (serializableRexNode instanceof SerializableRexInputRef) {
-return input.getValue(
-((SerializableRexInputRef) serializableRexNode).getIndex() - 
leftRowColumnCount);
-  } else { // It can only be SerializableFieldAccess.
-List indexes = ((SerializableRexFieldAccess) 
serializableRexNode).getIndexes();
-// retrieve row based on the first column reference.
-Row rowField = input.getValue(indexes.get(0) - leftRowColumnCount);
-for (int i = 1; i < indexes.size() - 1; i++) {
-  rowField = rowField.getRow(indexes.get(i));
-}
-return rowField.getValue(indexes.get(indexes.size() - 1));
-  }
-}
+  public static FieldAccessDescriptor getJoinColumns(
+  boolean isLeft,
+  List> joinColumns,
+  int leftRowColumnCount,
+  Schema schema) {
+List joinColumnsBuilt =
+joinColumns.stream()
+.map(pair -> SerializableRexNode.builder(isLeft ? pair.left : 
pair.right).build())
+.collect(toList());
+return FieldAccessDescriptor.union(
+joinColumnsBuilt.stream()
+.map(v -> getJoinColumn(v, leftRowColumnCount).resolve(schema))
+.collect(Collectors.toList()));
   }
 
-  /** A {@code DoFn} which implement the sideInput-JOIN. */
-  public static class SideInputJoinDoFn extends DoFn, Row> {
-private final PCollectionView>> sideInputView;
-private final JoinRelType joinType;
-private final Row rightNullRow;
-private final boolean swap;
-private final Schema schema;
-
-public SideInputJoinDoFn(
-JoinRelType joinType,
-Row rightNullRow,
-PCollectionView>> sideInputView,
-boolean swap,
-Schema schema) {
-  this.joinType = joinType;
-  this.rightNullRow = rightNullRow;
-  this.sideInputView = sideInputView;
-  this.swap = swap;
-  this.schema = schema;
-}
-
-@ProcessElement
-public void processElement(ProcessContext context) {
-  Row key = context.element().getKey();
-  Row leftRow = context.element().getValue();
-  Map> key2Rows = context.sideInput(sideInputView);
-  Iterable rightRowsIterable = key2Rows.get(key);
-
-  if (rightRowsIterable != null && rightRowsIterable.iterator().hasNext()) 
{
-for (Row aRightRowsIterable : rightRowsIterable) {
-  context.output(combineTwoRowsIntoOne(leftRow, aRightRowsIterable, 
swap, schema));
-}
-  } else {
-if (joinType == JoinRelType.LEFT) {
-  context.output(combineTwoRowsIntoOne(leftRow, rightNullRow, swap, 
schema));
-}
+  private static FieldAccessDescriptor getJoinColumn(
 
 Review comment:
   This actually hardcodes a bad assumption a bit further than it was before: 
that the join is only on columns. We want to move in the other 

[jira] [Work logged] (BEAM-4076) Schema followups

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4076?focusedWorklogId=407713=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-407713
 ]

ASF GitHub Bot logged work on BEAM-4076:


Author: ASF GitHub Bot
Created on: 23/Mar/20 05:43
Start Date: 23/Mar/20 05:43
Worklog Time Spent: 10m 
  Work Description: reuvenlax commented on pull request #11041: [BEAM-4076] 
Use beam join api in sql
URL: https://github.com/apache/beam/pull/11041
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 407713)
Time Spent: 22h 50m  (was: 22h 40m)

> Schema followups
> 
>
> Key: BEAM-4076
> URL: https://issues.apache.org/jira/browse/BEAM-4076
> Project: Beam
>  Issue Type: Improvement
>  Components: beam-model, dsl-sql, sdk-java-core
>Reporter: Kenneth Knowles
>Priority: Major
>  Time Spent: 22h 50m
>  Remaining Estimate: 0h
>
> This umbrella bug contains subtasks with followups for Beam schemas, which 
> were moved from SQL to the core Java SDK and made to be type-name-based 
> rather than coder based.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-4076) Schema followups

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4076?focusedWorklogId=407711=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-407711
 ]

ASF GitHub Bot logged work on BEAM-4076:


Author: ASF GitHub Bot
Created on: 23/Mar/20 05:25
Start Date: 23/Mar/20 05:25
Worklog Time Spent: 10m 
  Work Description: reuvenlax commented on issue #11041: [BEAM-4076] Use 
beam join api in sql
URL: https://github.com/apache/beam/pull/11041#issuecomment-602392753
 
 
   Run SQL Postcommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 407711)
Time Spent: 22h 40m  (was: 22.5h)

> Schema followups
> 
>
> Key: BEAM-4076
> URL: https://issues.apache.org/jira/browse/BEAM-4076
> Project: Beam
>  Issue Type: Improvement
>  Components: beam-model, dsl-sql, sdk-java-core
>Reporter: Kenneth Knowles
>Priority: Major
>  Time Spent: 22h 40m
>  Remaining Estimate: 0h
>
> This umbrella bug contains subtasks with followups for Beam schemas, which 
> were moved from SQL to the core Java SDK and made to be type-name-based 
> rather than coder based.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-4076) Schema followups

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4076?focusedWorklogId=407708=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-407708
 ]

ASF GitHub Bot logged work on BEAM-4076:


Author: ASF GitHub Bot
Created on: 23/Mar/20 04:16
Start Date: 23/Mar/20 04:16
Worklog Time Spent: 10m 
  Work Description: reuvenlax commented on issue #11041: [BEAM-4076] Use 
beam join api in sql
URL: https://github.com/apache/beam/pull/11041#issuecomment-602375762
 
 
   @apilloud Added a richer set of unit tests and a JIRA. Will merge once green.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 407708)
Time Spent: 22.5h  (was: 22h 20m)

> Schema followups
> 
>
> Key: BEAM-4076
> URL: https://issues.apache.org/jira/browse/BEAM-4076
> Project: Beam
>  Issue Type: Improvement
>  Components: beam-model, dsl-sql, sdk-java-core
>Reporter: Kenneth Knowles
>Priority: Major
>  Time Spent: 22.5h
>  Remaining Estimate: 0h
>
> This umbrella bug contains subtasks with followups for Beam schemas, which 
> were moved from SQL to the core Java SDK and made to be type-name-based 
> rather than coder based.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-4076) Schema followups

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4076?focusedWorklogId=202851=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-202851
 ]

ASF GitHub Bot logged work on BEAM-4076:


Author: ASF GitHub Bot
Created on: 22/Feb/19 20:14
Start Date: 22/Feb/19 20:14
Worklog Time Spent: 10m 
  Work Description: ryan-williams commented on pull request #7928: Revert 
"Merge pull request #7635: [BEAM-4076] Generalize schema input…
URL: https://github.com/apache/beam/pull/7928
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 202851)
Time Spent: 22h 20m  (was: 22h 10m)

> Schema followups
> 
>
> Key: BEAM-4076
> URL: https://issues.apache.org/jira/browse/BEAM-4076
> Project: Beam
>  Issue Type: Improvement
>  Components: beam-model, dsl-sql, sdk-java-core
>Reporter: Kenneth Knowles
>Priority: Major
>  Time Spent: 22h 20m
>  Remaining Estimate: 0h
>
> This umbrella bug contains subtasks with followups for Beam schemas, which 
> were moved from SQL to the core Java SDK and made to be type-name-based 
> rather than coder based.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4076) Schema followups

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4076?focusedWorklogId=202850=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-202850
 ]

ASF GitHub Bot logged work on BEAM-4076:


Author: ASF GitHub Bot
Created on: 22/Feb/19 20:14
Start Date: 22/Feb/19 20:14
Worklog Time Spent: 10m 
  Work Description: ryan-williams commented on issue #7928: Revert "Merge 
pull request #7635: [BEAM-4076] Generalize schema input…
URL: https://github.com/apache/beam/pull/7928#issuecomment-466531842
 
 
   Superceded by #7930 
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 202850)
Time Spent: 22h 10m  (was: 22h)

> Schema followups
> 
>
> Key: BEAM-4076
> URL: https://issues.apache.org/jira/browse/BEAM-4076
> Project: Beam
>  Issue Type: Improvement
>  Components: beam-model, dsl-sql, sdk-java-core
>Reporter: Kenneth Knowles
>Priority: Major
>  Time Spent: 22h 10m
>  Remaining Estimate: 0h
>
> This umbrella bug contains subtasks with followups for Beam schemas, which 
> were moved from SQL to the core Java SDK and made to be type-name-based 
> rather than coder based.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4076) Schema followups

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4076?focusedWorklogId=202830=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-202830
 ]

ASF GitHub Bot logged work on BEAM-4076:


Author: ASF GitHub Bot
Created on: 22/Feb/19 19:47
Start Date: 22/Feb/19 19:47
Worklog Time Spent: 10m 
  Work Description: ryan-williams commented on pull request #7928: Revert 
"Merge pull request #7635: [BEAM-4076] Generalize schema input…
URL: https://github.com/apache/beam/pull/7928
 
 
   …s to ParDo"
   
   This reverts commit 38daf8c45b14a94665939b562ab947dc72ad8f8f.
   
   Compilation failure was masked by flaky test and merged, I believe.
   
   R: @reuvenlax @kennknowles 
   
   Post-Commit Tests Status (on master branch)
   

   
   Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
   --- | --- | --- | --- | --- | --- | --- | ---
   Go | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/)
 | --- | --- | --- | --- | --- | ---
   Java | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)[![Build
 
Status](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)[![Build
 
Status](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)
   Python | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/)
 | --- | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)
  [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/lastCompletedBuild/)
 | --- | --- | ---
   
   See [.test-infra/jenkins/README](../.test-infra/jenkins/README.md) for 
trigger phrase, status and link of all Jenkins jobs.
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 202830)
Time Spent: 22h  (was: 21h 50m)

> Schema followups
> 
>
> Key: BEAM-4076
> URL: https://issues.apache.org/jira/browse/BEAM-4076
> Project: Beam
>  Issue Type: Improvement
>  Components: beam-model, dsl-sql, sdk-java-core
>Reporter: Kenneth Knowles
>Priority: Major
>  Time Spent: 22h
>  Remaining Estimate: 0h
>
> This umbrella bug 

[jira] [Work logged] (BEAM-4076) Schema followups

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4076?focusedWorklogId=202392=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-202392
 ]

ASF GitHub Bot logged work on BEAM-4076:


Author: ASF GitHub Bot
Created on: 22/Feb/19 05:55
Start Date: 22/Feb/19 05:55
Worklog Time Spent: 10m 
  Work Description: reuvenlax commented on pull request #7635: [BEAM-4076] 
Generalize schema inputs to ParDo
URL: https://github.com/apache/beam/pull/7635
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 202392)
Time Spent: 21h 50m  (was: 21h 40m)

> Schema followups
> 
>
> Key: BEAM-4076
> URL: https://issues.apache.org/jira/browse/BEAM-4076
> Project: Beam
>  Issue Type: Improvement
>  Components: beam-model, dsl-sql, sdk-java-core
>Reporter: Kenneth Knowles
>Priority: Major
>  Time Spent: 21h 50m
>  Remaining Estimate: 0h
>
> This umbrella bug contains subtasks with followups for Beam schemas, which 
> were moved from SQL to the core Java SDK and made to be type-name-based 
> rather than coder based.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4076) Schema followups

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4076?focusedWorklogId=202321=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-202321
 ]

ASF GitHub Bot logged work on BEAM-4076:


Author: ASF GitHub Bot
Created on: 22/Feb/19 01:16
Start Date: 22/Feb/19 01:16
Worklog Time Spent: 10m 
  Work Description: reuvenlax commented on issue #7635: [BEAM-4076] 
Generalize schema inputs to ParDo
URL: https://github.com/apache/beam/pull/7635#issuecomment-466234184
 
 
   @kennknowles any more comments?
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 202321)
Time Spent: 21h 40m  (was: 21.5h)

> Schema followups
> 
>
> Key: BEAM-4076
> URL: https://issues.apache.org/jira/browse/BEAM-4076
> Project: Beam
>  Issue Type: Improvement
>  Components: beam-model, dsl-sql, sdk-java-core
>Reporter: Kenneth Knowles
>Priority: Major
>  Time Spent: 21h 40m
>  Remaining Estimate: 0h
>
> This umbrella bug contains subtasks with followups for Beam schemas, which 
> were moved from SQL to the core Java SDK and made to be type-name-based 
> rather than coder based.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4076) Schema followups

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4076?focusedWorklogId=202016=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-202016
 ]

ASF GitHub Bot logged work on BEAM-4076:


Author: ASF GitHub Bot
Created on: 21/Feb/19 15:57
Start Date: 21/Feb/19 15:57
Worklog Time Spent: 10m 
  Work Description: reuvenlax commented on issue #7635: [BEAM-4076] 
Generalize schema inputs to ParDo
URL: https://github.com/apache/beam/pull/7635#issuecomment-466054020
 
 
   Run Java PreCommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 202016)
Time Spent: 21.5h  (was: 21h 20m)

> Schema followups
> 
>
> Key: BEAM-4076
> URL: https://issues.apache.org/jira/browse/BEAM-4076
> Project: Beam
>  Issue Type: Improvement
>  Components: beam-model, dsl-sql, sdk-java-core
>Reporter: Kenneth Knowles
>Priority: Major
>  Time Spent: 21.5h
>  Remaining Estimate: 0h
>
> This umbrella bug contains subtasks with followups for Beam schemas, which 
> were moved from SQL to the core Java SDK and made to be type-name-based 
> rather than coder based.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4076) Schema followups

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4076?focusedWorklogId=201789=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-201789
 ]

ASF GitHub Bot logged work on BEAM-4076:


Author: ASF GitHub Bot
Created on: 21/Feb/19 08:16
Start Date: 21/Feb/19 08:16
Worklog Time Spent: 10m 
  Work Description: reuvenlax commented on issue #7635: [BEAM-4076] 
Generalize schema inputs to ParDo
URL: https://github.com/apache/beam/pull/7635#issuecomment-465903909
 
 
   Run Java PreCommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 201789)
Time Spent: 21h 20m  (was: 21h 10m)

> Schema followups
> 
>
> Key: BEAM-4076
> URL: https://issues.apache.org/jira/browse/BEAM-4076
> Project: Beam
>  Issue Type: Improvement
>  Components: beam-model, dsl-sql, sdk-java-core
>Reporter: Kenneth Knowles
>Priority: Major
>  Time Spent: 21h 20m
>  Remaining Estimate: 0h
>
> This umbrella bug contains subtasks with followups for Beam schemas, which 
> were moved from SQL to the core Java SDK and made to be type-name-based 
> rather than coder based.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4076) Schema followups

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4076?focusedWorklogId=201713=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-201713
 ]

ASF GitHub Bot logged work on BEAM-4076:


Author: ASF GitHub Bot
Created on: 21/Feb/19 01:22
Start Date: 21/Feb/19 01:22
Worklog Time Spent: 10m 
  Work Description: reuvenlax commented on issue #7635: [BEAM-4076] 
Generalize schema inputs to ParDo
URL: https://github.com/apache/beam/pull/7635#issuecomment-465823091
 
 
   Run Python PreCommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 201713)
Time Spent: 21h 10m  (was: 21h)

> Schema followups
> 
>
> Key: BEAM-4076
> URL: https://issues.apache.org/jira/browse/BEAM-4076
> Project: Beam
>  Issue Type: Improvement
>  Components: beam-model, dsl-sql, sdk-java-core
>Reporter: Kenneth Knowles
>Priority: Major
>  Time Spent: 21h 10m
>  Remaining Estimate: 0h
>
> This umbrella bug contains subtasks with followups for Beam schemas, which 
> were moved from SQL to the core Java SDK and made to be type-name-based 
> rather than coder based.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4076) Schema followups

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4076?focusedWorklogId=201711=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-201711
 ]

ASF GitHub Bot logged work on BEAM-4076:


Author: ASF GitHub Bot
Created on: 21/Feb/19 01:22
Start Date: 21/Feb/19 01:22
Worklog Time Spent: 10m 
  Work Description: reuvenlax commented on issue #7635: [BEAM-4076] 
Generalize schema inputs to ParDo
URL: https://github.com/apache/beam/pull/7635#issuecomment-465823016
 
 
   Run Java PreCommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 201711)
Time Spent: 20h 50m  (was: 20h 40m)

> Schema followups
> 
>
> Key: BEAM-4076
> URL: https://issues.apache.org/jira/browse/BEAM-4076
> Project: Beam
>  Issue Type: Improvement
>  Components: beam-model, dsl-sql, sdk-java-core
>Reporter: Kenneth Knowles
>Priority: Major
>  Time Spent: 20h 50m
>  Remaining Estimate: 0h
>
> This umbrella bug contains subtasks with followups for Beam schemas, which 
> were moved from SQL to the core Java SDK and made to be type-name-based 
> rather than coder based.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4076) Schema followups

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4076?focusedWorklogId=201712=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-201712
 ]

ASF GitHub Bot logged work on BEAM-4076:


Author: ASF GitHub Bot
Created on: 21/Feb/19 01:22
Start Date: 21/Feb/19 01:22
Worklog Time Spent: 10m 
  Work Description: reuvenlax commented on issue #7635: [BEAM-4076] 
Generalize schema inputs to ParDo
URL: https://github.com/apache/beam/pull/7635#issuecomment-465823046
 
 
   Run JavaPortabilityApi PreCommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 201712)
Time Spent: 21h  (was: 20h 50m)

> Schema followups
> 
>
> Key: BEAM-4076
> URL: https://issues.apache.org/jira/browse/BEAM-4076
> Project: Beam
>  Issue Type: Improvement
>  Components: beam-model, dsl-sql, sdk-java-core
>Reporter: Kenneth Knowles
>Priority: Major
>  Time Spent: 21h
>  Remaining Estimate: 0h
>
> This umbrella bug contains subtasks with followups for Beam schemas, which 
> were moved from SQL to the core Java SDK and made to be type-name-based 
> rather than coder based.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4076) Schema followups

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4076?focusedWorklogId=201538=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-201538
 ]

ASF GitHub Bot logged work on BEAM-4076:


Author: ASF GitHub Bot
Created on: 20/Feb/19 19:12
Start Date: 20/Feb/19 19:12
Worklog Time Spent: 10m 
  Work Description: reuvenlax commented on pull request #7635: [BEAM-4076] 
Generalize schema inputs to ParDo
URL: https://github.com/apache/beam/pull/7635#discussion_r258634834
 
 

 ##
 File path: 
sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/transforms/Cast.java
 ##
 @@ -300,7 +300,7 @@ public void verifyCompatibility(Schema inputSchema) {
 
   @ProcessElement
   public void process(
-  @FieldAccess("filterFields") Row input, 
OutputReceiver r) {
+  @FieldAccess("filterFields") @Element Row input, 
OutputReceiver r) {
 
 Review comment:
   With this PR, elements always need Element annotations. This restriction is 
removed in the next PR.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 201538)
Time Spent: 20.5h  (was: 20h 20m)

> Schema followups
> 
>
> Key: BEAM-4076
> URL: https://issues.apache.org/jira/browse/BEAM-4076
> Project: Beam
>  Issue Type: Improvement
>  Components: beam-model, dsl-sql, sdk-java-core
>Reporter: Kenneth Knowles
>Priority: Major
>  Time Spent: 20.5h
>  Remaining Estimate: 0h
>
> This umbrella bug contains subtasks with followups for Beam schemas, which 
> were moved from SQL to the core Java SDK and made to be type-name-based 
> rather than coder based.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4076) Schema followups

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4076?focusedWorklogId=201537=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-201537
 ]

ASF GitHub Bot logged work on BEAM-4076:


Author: ASF GitHub Bot
Created on: 20/Feb/19 19:12
Start Date: 20/Feb/19 19:12
Worklog Time Spent: 10m 
  Work Description: reuvenlax commented on pull request #7635: [BEAM-4076] 
Generalize schema inputs to ParDo
URL: https://github.com/apache/beam/pull/7635#discussion_r258634692
 
 

 ##
 File path: 
runners/apex/src/main/java/org/apache/beam/runners/apex/translation/ParDoTranslator.java
 ##
 @@ -77,6 +80,13 @@ public void translate(ParDo.MultiOutput 
transform, TranslationC
 PCollection input = context.getInput();
 List> sideInputs = transform.getSideInputs();
 
+DoFnSchemaInformation doFnSchemaInformation;
+try {
+  doFnSchemaInformation = 
ParDoTranslation.getSchemaInformation(context.getCurrentTransform());
+} catch (IOException e) {
 
 Review comment:
   Done
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 201537)
Time Spent: 20h 20m  (was: 20h 10m)

> Schema followups
> 
>
> Key: BEAM-4076
> URL: https://issues.apache.org/jira/browse/BEAM-4076
> Project: Beam
>  Issue Type: Improvement
>  Components: beam-model, dsl-sql, sdk-java-core
>Reporter: Kenneth Knowles
>Priority: Major
>  Time Spent: 20h 20m
>  Remaining Estimate: 0h
>
> This umbrella bug contains subtasks with followups for Beam schemas, which 
> were moved from SQL to the core Java SDK and made to be type-name-based 
> rather than coder based.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4076) Schema followups

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4076?focusedWorklogId=201539=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-201539
 ]

ASF GitHub Bot logged work on BEAM-4076:


Author: ASF GitHub Bot
Created on: 20/Feb/19 19:12
Start Date: 20/Feb/19 19:12
Worklog Time Spent: 10m 
  Work Description: reuvenlax commented on issue #7635: [BEAM-4076] 
Generalize schema inputs to ParDo
URL: https://github.com/apache/beam/pull/7635#issuecomment-465714735
 
 
   @kennknowles comments addressed.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 201539)
Time Spent: 20h 40m  (was: 20.5h)

> Schema followups
> 
>
> Key: BEAM-4076
> URL: https://issues.apache.org/jira/browse/BEAM-4076
> Project: Beam
>  Issue Type: Improvement
>  Components: beam-model, dsl-sql, sdk-java-core
>Reporter: Kenneth Knowles
>Priority: Major
>  Time Spent: 20h 40m
>  Remaining Estimate: 0h
>
> This umbrella bug contains subtasks with followups for Beam schemas, which 
> were moved from SQL to the core Java SDK and made to be type-name-based 
> rather than coder based.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4076) Schema followups

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4076?focusedWorklogId=201536=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-201536
 ]

ASF GitHub Bot logged work on BEAM-4076:


Author: ASF GitHub Bot
Created on: 20/Feb/19 19:12
Start Date: 20/Feb/19 19:12
Worklog Time Spent: 10m 
  Work Description: reuvenlax commented on pull request #7635: [BEAM-4076] 
Generalize schema inputs to ParDo
URL: https://github.com/apache/beam/pull/7635#discussion_r258634638
 
 

 ##
 File path: 
runners/core-construction-java/src/main/java/org/apache/beam/runners/core/construction/ParDoTranslation.java
 ##
 @@ -161,9 +158,40 @@ public boolean canTranslate(PTransform pTransform) {
   }
 
   public static ParDoPayload translateParDo(
-  final ParDo.MultiOutput parDo, Pipeline pipeline, SdkComponents 
components)
-  throws IOException {
+  AppliedPTransform appliedPTransform, SdkComponents components) 
throws IOException {
 
 Review comment:
   Done
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 201536)
Time Spent: 20h 10m  (was: 20h)

> Schema followups
> 
>
> Key: BEAM-4076
> URL: https://issues.apache.org/jira/browse/BEAM-4076
> Project: Beam
>  Issue Type: Improvement
>  Components: beam-model, dsl-sql, sdk-java-core
>Reporter: Kenneth Knowles
>Priority: Major
>  Time Spent: 20h 10m
>  Remaining Estimate: 0h
>
> This umbrella bug contains subtasks with followups for Beam schemas, which 
> were moved from SQL to the core Java SDK and made to be type-name-based 
> rather than coder based.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4076) Schema followups

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4076?focusedWorklogId=201517=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-201517
 ]

ASF GitHub Bot logged work on BEAM-4076:


Author: ASF GitHub Bot
Created on: 20/Feb/19 18:49
Start Date: 20/Feb/19 18:49
Worklog Time Spent: 10m 
  Work Description: reuvenlax commented on pull request #7635: [BEAM-4076] 
Generalize schema inputs to ParDo
URL: https://github.com/apache/beam/pull/7635#discussion_r258625424
 
 

 ##
 File path: 
sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/transforms/Cast.java
 ##
 @@ -300,7 +300,7 @@ public void verifyCompatibility(Schema inputSchema) {
 
   @ProcessElement
   public void process(
-  @FieldAccess("filterFields") Row input, 
OutputReceiver r) {
+  @FieldAccess("filterFields") @Element Row input, 
OutputReceiver r) {
 
 Review comment:
   After this PR, FieldAccess no longer replaces Element. However the next PR 
will remove the need for Element (requires more refactoring)
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 201517)
Time Spent: 20h  (was: 19h 50m)

> Schema followups
> 
>
> Key: BEAM-4076
> URL: https://issues.apache.org/jira/browse/BEAM-4076
> Project: Beam
>  Issue Type: Improvement
>  Components: beam-model, dsl-sql, sdk-java-core
>Reporter: Kenneth Knowles
>Priority: Major
>  Time Spent: 20h
>  Remaining Estimate: 0h
>
> This umbrella bug contains subtasks with followups for Beam schemas, which 
> were moved from SQL to the core Java SDK and made to be type-name-based 
> rather than coder based.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4076) Schema followups

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4076?focusedWorklogId=201447=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-201447
 ]

ASF GitHub Bot logged work on BEAM-4076:


Author: ASF GitHub Bot
Created on: 20/Feb/19 17:05
Start Date: 20/Feb/19 17:05
Worklog Time Spent: 10m 
  Work Description: kennknowles commented on pull request #7635: 
[BEAM-4076] Generalize schema inputs to ParDo
URL: https://github.com/apache/beam/pull/7635#discussion_r258582744
 
 

 ##
 File path: 
runners/apex/src/main/java/org/apache/beam/runners/apex/translation/ParDoTranslator.java
 ##
 @@ -77,6 +80,13 @@ public void translate(ParDo.MultiOutput 
transform, TranslationC
 PCollection input = context.getInput();
 List> sideInputs = transform.getSideInputs();
 
+DoFnSchemaInformation doFnSchemaInformation;
+try {
+  doFnSchemaInformation = 
ParDoTranslation.getSchemaInformation(context.getCurrentTransform());
+} catch (IOException e) {
 
 Review comment:
   This exception handling is duplicated all over. Should it be moved into 
`getSchemaInformation`? When does the IOException get raised? Is there any 
value in forcing callers to handle it? If you wouldn't make 
`getSchemaInformation` return a `oneof` then it shouldn't throw a checked 
exception.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 201447)
Time Spent: 19h 40m  (was: 19.5h)

> Schema followups
> 
>
> Key: BEAM-4076
> URL: https://issues.apache.org/jira/browse/BEAM-4076
> Project: Beam
>  Issue Type: Improvement
>  Components: beam-model, dsl-sql, sdk-java-core
>Reporter: Kenneth Knowles
>Priority: Major
>  Time Spent: 19h 40m
>  Remaining Estimate: 0h
>
> This umbrella bug contains subtasks with followups for Beam schemas, which 
> were moved from SQL to the core Java SDK and made to be type-name-based 
> rather than coder based.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4076) Schema followups

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4076?focusedWorklogId=201446=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-201446
 ]

ASF GitHub Bot logged work on BEAM-4076:


Author: ASF GitHub Bot
Created on: 20/Feb/19 17:05
Start Date: 20/Feb/19 17:05
Worklog Time Spent: 10m 
  Work Description: kennknowles commented on pull request #7635: 
[BEAM-4076] Generalize schema inputs to ParDo
URL: https://github.com/apache/beam/pull/7635#discussion_r258583402
 
 

 ##
 File path: 
runners/core-construction-java/src/main/java/org/apache/beam/runners/core/construction/ParDoTranslation.java
 ##
 @@ -161,9 +158,40 @@ public boolean canTranslate(PTransform pTransform) {
   }
 
   public static ParDoPayload translateParDo(
-  final ParDo.MultiOutput parDo, Pipeline pipeline, SdkComponents 
components)
-  throws IOException {
+  AppliedPTransform appliedPTransform, SdkComponents components) 
throws IOException {
 
 Review comment:
   `AppliedPTransform, ? ,?>` or some such? Avoids the 
cast below. Might force a cast at the caller, and that makes sense because the 
caller is the one who is making the decision to pass the param.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 201446)
Time Spent: 19.5h  (was: 19h 20m)

> Schema followups
> 
>
> Key: BEAM-4076
> URL: https://issues.apache.org/jira/browse/BEAM-4076
> Project: Beam
>  Issue Type: Improvement
>  Components: beam-model, dsl-sql, sdk-java-core
>Reporter: Kenneth Knowles
>Priority: Major
>  Time Spent: 19.5h
>  Remaining Estimate: 0h
>
> This umbrella bug contains subtasks with followups for Beam schemas, which 
> were moved from SQL to the core Java SDK and made to be type-name-based 
> rather than coder based.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4076) Schema followups

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4076?focusedWorklogId=201448=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-201448
 ]

ASF GitHub Bot logged work on BEAM-4076:


Author: ASF GitHub Bot
Created on: 20/Feb/19 17:05
Start Date: 20/Feb/19 17:05
Worklog Time Spent: 10m 
  Work Description: kennknowles commented on pull request #7635: 
[BEAM-4076] Generalize schema inputs to ParDo
URL: https://github.com/apache/beam/pull/7635#discussion_r258583491
 
 

 ##
 File path: 
sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/transforms/Cast.java
 ##
 @@ -300,7 +300,7 @@ public void verifyCompatibility(Schema inputSchema) {
 
   @ProcessElement
   public void process(
-  @FieldAccess("filterFields") Row input, 
OutputReceiver r) {
+  @FieldAccess("filterFields") @Element Row input, 
OutputReceiver r) {
 
 Review comment:
   Why both annotations?
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 201448)
Time Spent: 19h 50m  (was: 19h 40m)

> Schema followups
> 
>
> Key: BEAM-4076
> URL: https://issues.apache.org/jira/browse/BEAM-4076
> Project: Beam
>  Issue Type: Improvement
>  Components: beam-model, dsl-sql, sdk-java-core
>Reporter: Kenneth Knowles
>Priority: Major
>  Time Spent: 19h 50m
>  Remaining Estimate: 0h
>
> This umbrella bug contains subtasks with followups for Beam schemas, which 
> were moved from SQL to the core Java SDK and made to be type-name-based 
> rather than coder based.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4076) Schema followups

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4076?focusedWorklogId=201445=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-201445
 ]

ASF GitHub Bot logged work on BEAM-4076:


Author: ASF GitHub Bot
Created on: 20/Feb/19 17:02
Start Date: 20/Feb/19 17:02
Worklog Time Spent: 10m 
  Work Description: kennknowles commented on issue #7635: [BEAM-4076] 
Generalize schema inputs to ParDo
URL: https://github.com/apache/beam/pull/7635#issuecomment-46530
 
 
   Looks like GitHub has changed in such a way that I cannot post with 
reviewable.io, though previously I could.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 201445)
Time Spent: 19h 20m  (was: 19h 10m)

> Schema followups
> 
>
> Key: BEAM-4076
> URL: https://issues.apache.org/jira/browse/BEAM-4076
> Project: Beam
>  Issue Type: Improvement
>  Components: beam-model, dsl-sql, sdk-java-core
>Reporter: Kenneth Knowles
>Priority: Major
>  Time Spent: 19h 20m
>  Remaining Estimate: 0h
>
> This umbrella bug contains subtasks with followups for Beam schemas, which 
> were moved from SQL to the core Java SDK and made to be type-name-based 
> rather than coder based.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4076) Schema followups

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4076?focusedWorklogId=201104=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-201104
 ]

ASF GitHub Bot logged work on BEAM-4076:


Author: ASF GitHub Bot
Created on: 20/Feb/19 05:28
Start Date: 20/Feb/19 05:28
Worklog Time Spent: 10m 
  Work Description: reuvenlax commented on issue #7635: [BEAM-4076] 
Generalize schema inputs to ParDo
URL: https://github.com/apache/beam/pull/7635#issuecomment-465430283
 
 
   Thanks! I'll also point out that the code in ParDo.java was refactored and
   simplified in the PR that follows this one. Let me know if you'd prefer me
   to backport that refactoring to this PR.
   
   On Tue, Feb 19, 2019 at 9:04 PM Kenn Knowles 
   wrote:
   
   > FYi I'm using reviewable.io to check through the files and I'm halfway
   > there. Most are pretty trivial plumbing so I want to be able to focus on
   > the real changes.
   >
   > —
   > You are receiving this because you authored the thread.
   > Reply to this email directly, view it on GitHub
   > , or mute
   > the thread
   > 

   > .
   >
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 201104)
Time Spent: 19h 10m  (was: 19h)

> Schema followups
> 
>
> Key: BEAM-4076
> URL: https://issues.apache.org/jira/browse/BEAM-4076
> Project: Beam
>  Issue Type: Improvement
>  Components: beam-model, dsl-sql, sdk-java-core
>Reporter: Kenneth Knowles
>Priority: Major
>  Time Spent: 19h 10m
>  Remaining Estimate: 0h
>
> This umbrella bug contains subtasks with followups for Beam schemas, which 
> were moved from SQL to the core Java SDK and made to be type-name-based 
> rather than coder based.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4076) Schema followups

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4076?focusedWorklogId=201089=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-201089
 ]

ASF GitHub Bot logged work on BEAM-4076:


Author: ASF GitHub Bot
Created on: 20/Feb/19 05:04
Start Date: 20/Feb/19 05:04
Worklog Time Spent: 10m 
  Work Description: kennknowles commented on issue #7635: [BEAM-4076] 
Generalize schema inputs to ParDo
URL: https://github.com/apache/beam/pull/7635#issuecomment-465425645
 
 
   FYi I'm using reviewable.io to check through the files and I'm halfway 
there. Most are pretty trivial plumbing so I want to be able to focus on the 
real changes.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 201089)
Time Spent: 19h  (was: 18h 50m)

> Schema followups
> 
>
> Key: BEAM-4076
> URL: https://issues.apache.org/jira/browse/BEAM-4076
> Project: Beam
>  Issue Type: Improvement
>  Components: beam-model, dsl-sql, sdk-java-core
>Reporter: Kenneth Knowles
>Priority: Major
>  Time Spent: 19h
>  Remaining Estimate: 0h
>
> This umbrella bug contains subtasks with followups for Beam schemas, which 
> were moved from SQL to the core Java SDK and made to be type-name-based 
> rather than coder based.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4076) Schema followups

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4076?focusedWorklogId=198194=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-198194
 ]

ASF GitHub Bot logged work on BEAM-4076:


Author: ASF GitHub Bot
Created on: 13/Feb/19 16:25
Start Date: 13/Feb/19 16:25
Worklog Time Spent: 10m 
  Work Description: reuvenlax commented on issue #7635: [BEAM-4076] 
Generalize schema inputs to ParDo
URL: https://github.com/apache/beam/pull/7635#issuecomment-463265598
 
 
   Run Java PreCommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 198194)
Time Spent: 18h 40m  (was: 18.5h)

> Schema followups
> 
>
> Key: BEAM-4076
> URL: https://issues.apache.org/jira/browse/BEAM-4076
> Project: Beam
>  Issue Type: Improvement
>  Components: beam-model, dsl-sql, sdk-java-core
>Reporter: Kenneth Knowles
>Priority: Major
>  Time Spent: 18h 40m
>  Remaining Estimate: 0h
>
> This umbrella bug contains subtasks with followups for Beam schemas, which 
> were moved from SQL to the core Java SDK and made to be type-name-based 
> rather than coder based.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4076) Schema followups

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4076?focusedWorklogId=198195=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-198195
 ]

ASF GitHub Bot logged work on BEAM-4076:


Author: ASF GitHub Bot
Created on: 13/Feb/19 16:25
Start Date: 13/Feb/19 16:25
Worklog Time Spent: 10m 
  Work Description: reuvenlax commented on issue #7635: [BEAM-4076] 
Generalize schema inputs to ParDo
URL: https://github.com/apache/beam/pull/7635#issuecomment-463265698
 
 
   @kennknowles any comments on this PR?
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 198195)
Time Spent: 18h 50m  (was: 18h 40m)

> Schema followups
> 
>
> Key: BEAM-4076
> URL: https://issues.apache.org/jira/browse/BEAM-4076
> Project: Beam
>  Issue Type: Improvement
>  Components: beam-model, dsl-sql, sdk-java-core
>Reporter: Kenneth Knowles
>Priority: Major
>  Time Spent: 18h 50m
>  Remaining Estimate: 0h
>
> This umbrella bug contains subtasks with followups for Beam schemas, which 
> were moved from SQL to the core Java SDK and made to be type-name-based 
> rather than coder based.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4076) Schema followups

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4076?focusedWorklogId=196742=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-196742
 ]

ASF GitHub Bot logged work on BEAM-4076:


Author: ASF GitHub Bot
Created on: 10/Feb/19 23:30
Start Date: 10/Feb/19 23:30
Worklog Time Spent: 10m 
  Work Description: reuvenlax commented on issue #7635: [BEAM-4076] 
Generalize schema inputs to ParDo
URL: https://github.com/apache/beam/pull/7635#issuecomment-462191852
 
 
   Run Java PreCommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 196742)
Time Spent: 18.5h  (was: 18h 20m)

> Schema followups
> 
>
> Key: BEAM-4076
> URL: https://issues.apache.org/jira/browse/BEAM-4076
> Project: Beam
>  Issue Type: Improvement
>  Components: beam-model, dsl-sql, sdk-java-core
>Reporter: Kenneth Knowles
>Priority: Major
>  Time Spent: 18.5h
>  Remaining Estimate: 0h
>
> This umbrella bug contains subtasks with followups for Beam schemas, which 
> were moved from SQL to the core Java SDK and made to be type-name-based 
> rather than coder based.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4076) Schema followups

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4076?focusedWorklogId=194132=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-194132
 ]

ASF GitHub Bot logged work on BEAM-4076:


Author: ASF GitHub Bot
Created on: 04/Feb/19 17:38
Start Date: 04/Feb/19 17:38
Worklog Time Spent: 10m 
  Work Description: reuvenlax commented on issue #7635: [BEAM-4076] 
Generalize schema inputs to ParDo
URL: https://github.com/apache/beam/pull/7635#issuecomment-460340328
 
 
   @kennknowles The next PR contains some cleanup refactoring of the 
DoFnSchemaInformation (in ParDo.java in this PR). Let me know if that's fine, 
or if you prefer that I backport that refactoring to this PR.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 194132)
Time Spent: 18h 20m  (was: 18h 10m)

> Schema followups
> 
>
> Key: BEAM-4076
> URL: https://issues.apache.org/jira/browse/BEAM-4076
> Project: Beam
>  Issue Type: Improvement
>  Components: beam-model, dsl-sql, sdk-java-core
>Reporter: Kenneth Knowles
>Priority: Major
>  Time Spent: 18h 20m
>  Remaining Estimate: 0h
>
> This umbrella bug contains subtasks with followups for Beam schemas, which 
> were moved from SQL to the core Java SDK and made to be type-name-based 
> rather than coder based.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4076) Schema followups

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4076?focusedWorklogId=193614=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-193614
 ]

ASF GitHub Bot logged work on BEAM-4076:


Author: ASF GitHub Bot
Created on: 02/Feb/19 03:53
Start Date: 02/Feb/19 03:53
Worklog Time Spent: 10m 
  Work Description: reuvenlax commented on issue #7635: [BEAM-4076] 
Generalize schema inputs to ParDo
URL: https://github.com/apache/beam/pull/7635#issuecomment-459932551
 
 
   Run Dataflow PortabilityApi ValidatesRunner
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 193614)
Time Spent: 18h 10m  (was: 18h)

> Schema followups
> 
>
> Key: BEAM-4076
> URL: https://issues.apache.org/jira/browse/BEAM-4076
> Project: Beam
>  Issue Type: Improvement
>  Components: beam-model, dsl-sql, sdk-java-core
>Reporter: Kenneth Knowles
>Priority: Major
>  Time Spent: 18h 10m
>  Remaining Estimate: 0h
>
> This umbrella bug contains subtasks with followups for Beam schemas, which 
> were moved from SQL to the core Java SDK and made to be type-name-based 
> rather than coder based.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4076) Schema followups

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4076?focusedWorklogId=185552=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-185552
 ]

ASF GitHub Bot logged work on BEAM-4076:


Author: ASF GitHub Bot
Created on: 16/Jan/19 02:53
Start Date: 16/Jan/19 02:53
Worklog Time Spent: 10m 
  Work Description: reuvenlax commented on pull request #7500: [BEAM-4076] 
Add antlr4 to beam
URL: https://github.com/apache/beam/pull/7500
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 185552)
Time Spent: 18h  (was: 17h 50m)

> Schema followups
> 
>
> Key: BEAM-4076
> URL: https://issues.apache.org/jira/browse/BEAM-4076
> Project: Beam
>  Issue Type: Improvement
>  Components: beam-model, dsl-sql, sdk-java-core
>Reporter: Kenneth Knowles
>Priority: Major
>  Time Spent: 18h
>  Remaining Estimate: 0h
>
> This umbrella bug contains subtasks with followups for Beam schemas, which 
> were moved from SQL to the core Java SDK and made to be type-name-based 
> rather than coder based.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4076) Schema followups

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4076?focusedWorklogId=185443=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-185443
 ]

ASF GitHub Bot logged work on BEAM-4076:


Author: ASF GitHub Bot
Created on: 15/Jan/19 21:54
Start Date: 15/Jan/19 21:54
Worklog Time Spent: 10m 
  Work Description: reuvenlax commented on issue #7500: [BEAM-4076] Add 
antlr4 to beam
URL: https://github.com/apache/beam/pull/7500#issuecomment-454566402
 
 
   Run Python PreCommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 185443)
Time Spent: 17h 50m  (was: 17h 40m)

> Schema followups
> 
>
> Key: BEAM-4076
> URL: https://issues.apache.org/jira/browse/BEAM-4076
> Project: Beam
>  Issue Type: Improvement
>  Components: beam-model, dsl-sql, sdk-java-core
>Reporter: Kenneth Knowles
>Priority: Major
>  Time Spent: 17h 50m
>  Remaining Estimate: 0h
>
> This umbrella bug contains subtasks with followups for Beam schemas, which 
> were moved from SQL to the core Java SDK and made to be type-name-based 
> rather than coder based.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4076) Schema followups

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4076?focusedWorklogId=185438=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-185438
 ]

ASF GitHub Bot logged work on BEAM-4076:


Author: ASF GitHub Bot
Created on: 15/Jan/19 21:37
Start Date: 15/Jan/19 21:37
Worklog Time Spent: 10m 
  Work Description: reuvenlax commented on issue #7500: [BEAM-4076] Add 
antlr4 to beam
URL: https://github.com/apache/beam/pull/7500#issuecomment-454561075
 
 
   Run Java PreCommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 185438)
Time Spent: 17h 40m  (was: 17.5h)

> Schema followups
> 
>
> Key: BEAM-4076
> URL: https://issues.apache.org/jira/browse/BEAM-4076
> Project: Beam
>  Issue Type: Improvement
>  Components: beam-model, dsl-sql, sdk-java-core
>Reporter: Kenneth Knowles
>Priority: Major
>  Time Spent: 17h 40m
>  Remaining Estimate: 0h
>
> This umbrella bug contains subtasks with followups for Beam schemas, which 
> were moved from SQL to the core Java SDK and made to be type-name-based 
> rather than coder based.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4076) Schema followups

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4076?focusedWorklogId=185430=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-185430
 ]

ASF GitHub Bot logged work on BEAM-4076:


Author: ASF GitHub Bot
Created on: 15/Jan/19 21:13
Start Date: 15/Jan/19 21:13
Worklog Time Spent: 10m 
  Work Description: reuvenlax commented on issue #7500: [BEAM-4076] Add 
antlr4 to beam
URL: https://github.com/apache/beam/pull/7500#issuecomment-454553531
 
 
   Run Website PreCommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 185430)
Time Spent: 17.5h  (was: 17h 20m)

> Schema followups
> 
>
> Key: BEAM-4076
> URL: https://issues.apache.org/jira/browse/BEAM-4076
> Project: Beam
>  Issue Type: Improvement
>  Components: beam-model, dsl-sql, sdk-java-core
>Reporter: Kenneth Knowles
>Priority: Major
>  Time Spent: 17.5h
>  Remaining Estimate: 0h
>
> This umbrella bug contains subtasks with followups for Beam schemas, which 
> were moved from SQL to the core Java SDK and made to be type-name-based 
> rather than coder based.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4076) Schema followups

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4076?focusedWorklogId=185426=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-185426
 ]

ASF GitHub Bot logged work on BEAM-4076:


Author: ASF GitHub Bot
Created on: 15/Jan/19 20:46
Start Date: 15/Jan/19 20:46
Worklog Time Spent: 10m 
  Work Description: reuvenlax commented on issue #7500: [BEAM-4076] Add 
antlr4 to beam
URL: https://github.com/apache/beam/pull/7500#issuecomment-454544825
 
 
   Run Java PreCommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 185426)
Time Spent: 17h 20m  (was: 17h 10m)

> Schema followups
> 
>
> Key: BEAM-4076
> URL: https://issues.apache.org/jira/browse/BEAM-4076
> Project: Beam
>  Issue Type: Improvement
>  Components: beam-model, dsl-sql, sdk-java-core
>Reporter: Kenneth Knowles
>Priority: Major
>  Time Spent: 17h 20m
>  Remaining Estimate: 0h
>
> This umbrella bug contains subtasks with followups for Beam schemas, which 
> were moved from SQL to the core Java SDK and made to be type-name-based 
> rather than coder based.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4076) Schema followups

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4076?focusedWorklogId=185425=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-185425
 ]

ASF GitHub Bot logged work on BEAM-4076:


Author: ASF GitHub Bot
Created on: 15/Jan/19 20:45
Start Date: 15/Jan/19 20:45
Worklog Time Spent: 10m 
  Work Description: reuvenlax commented on issue #7500: [BEAM-4076] Add 
antlr4 to beam
URL: https://github.com/apache/beam/pull/7500#issuecomment-454544666
 
 
   Run Java PreCommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 185425)
Time Spent: 17h 10m  (was: 17h)

> Schema followups
> 
>
> Key: BEAM-4076
> URL: https://issues.apache.org/jira/browse/BEAM-4076
> Project: Beam
>  Issue Type: Improvement
>  Components: beam-model, dsl-sql, sdk-java-core
>Reporter: Kenneth Knowles
>Priority: Major
>  Time Spent: 17h 10m
>  Remaining Estimate: 0h
>
> This umbrella bug contains subtasks with followups for Beam schemas, which 
> were moved from SQL to the core Java SDK and made to be type-name-based 
> rather than coder based.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4076) Schema followups

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4076?focusedWorklogId=185039=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-185039
 ]

ASF GitHub Bot logged work on BEAM-4076:


Author: ASF GitHub Bot
Created on: 15/Jan/19 00:08
Start Date: 15/Jan/19 00:08
Worklog Time Spent: 10m 
  Work Description: kennknowles commented on pull request #7500: 
[BEAM-4076] Add antlr4 to beam
URL: https://github.com/apache/beam/pull/7500#discussion_r247714307
 
 

 ##
 File path: sdks/java/core/build.gradle
 ##
 @@ -51,9 +58,11 @@ test {
 }
 
 dependencies {
+  antlr "org.antlr:antlr4:4.7"
 
 Review comment:
   This should be `library.java.antlr`
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 185039)
Time Spent: 16h 50m  (was: 16h 40m)

> Schema followups
> 
>
> Key: BEAM-4076
> URL: https://issues.apache.org/jira/browse/BEAM-4076
> Project: Beam
>  Issue Type: Improvement
>  Components: beam-model, dsl-sql, sdk-java-core
>Reporter: Kenneth Knowles
>Priority: Major
>  Time Spent: 16h 50m
>  Remaining Estimate: 0h
>
> This umbrella bug contains subtasks with followups for Beam schemas, which 
> were moved from SQL to the core Java SDK and made to be type-name-based 
> rather than coder based.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4076) Schema followups

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4076?focusedWorklogId=185040=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-185040
 ]

ASF GitHub Bot logged work on BEAM-4076:


Author: ASF GitHub Bot
Created on: 15/Jan/19 00:08
Start Date: 15/Jan/19 00:08
Worklog Time Spent: 10m 
  Work Description: kennknowles commented on pull request #7500: 
[BEAM-4076] Add antlr4 to beam
URL: https://github.com/apache/beam/pull/7500#discussion_r247714253
 
 

 ##
 File path: sdks/java/core/build.gradle
 ##
 @@ -51,9 +58,11 @@ test {
 }
 
 dependencies {
+  antlr "org.antlr:antlr4:4.7"
   // Required to load constants from the model, e.g. max timestamp for global 
window
   shadow project(path: ":beam-model-pipeline", configuration: "shadow")
   shadow library.java.vendored_guava_20_0
+  compile library.java.antlr
 
 Review comment:
   This should be `antlr_runtime`.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 185040)
Time Spent: 17h  (was: 16h 50m)

> Schema followups
> 
>
> Key: BEAM-4076
> URL: https://issues.apache.org/jira/browse/BEAM-4076
> Project: Beam
>  Issue Type: Improvement
>  Components: beam-model, dsl-sql, sdk-java-core
>Reporter: Kenneth Knowles
>Priority: Major
>  Time Spent: 17h
>  Remaining Estimate: 0h
>
> This umbrella bug contains subtasks with followups for Beam schemas, which 
> were moved from SQL to the core Java SDK and made to be type-name-based 
> rather than coder based.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4076) Schema followups

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4076?focusedWorklogId=184969=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-184969
 ]

ASF GitHub Bot logged work on BEAM-4076:


Author: ASF GitHub Bot
Created on: 14/Jan/19 22:32
Start Date: 14/Jan/19 22:32
Worklog Time Spent: 10m 
  Work Description: reuvenlax commented on pull request #7500: [BEAM-4076] 
Add antlr4 to beam
URL: https://github.com/apache/beam/pull/7500
 
 
   Just adds the dependency and the shading.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 184969)
Time Spent: 16.5h  (was: 16h 20m)

> Schema followups
> 
>
> Key: BEAM-4076
> URL: https://issues.apache.org/jira/browse/BEAM-4076
> Project: Beam
>  Issue Type: Improvement
>  Components: beam-model, dsl-sql, sdk-java-core
>Reporter: Kenneth Knowles
>Priority: Major
>  Time Spent: 16.5h
>  Remaining Estimate: 0h
>
> This umbrella bug contains subtasks with followups for Beam schemas, which 
> were moved from SQL to the core Java SDK and made to be type-name-based 
> rather than coder based.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)