[jira] [Comment Edited] (BEAM-3489) Expose the message id of received messages within PubsubMessage
[ https://issues.apache.org/jira/browse/BEAM-3489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17120343#comment-17120343 ] Thinh Ha edited comment on BEAM-3489 at 5/31/20, 12:23 PM: --- Hi [~reuvenlax], thanks for your feedback. I've added a new PR based on your comment: [https://github.com/apache/beam/pull/11873] Let me know if there is a test I should be running to check. was (Author: thinhha): Hi [~reuvenlax], thanks for your feedback. Could you advise what needs to change? In this PR the parseFn is always non-null, e.g. [https://github.com/apache/beam/commit/f1d02e9c6a741088479c9221af4c0250d5678b1f#diff-cfdca1a326496dabb9a075878204894fR471] [https://github.com/apache/beam/commit/f1d02e9c6a741088479c9221af4c0250d5678b1f#diff-cfdca1a326496dabb9a075878204894fR499] which should trigger this branch of code that returns the PubsubMessage [https://github.com/apache/beam/blob/master/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/PubsubReader.java#L120] I can think of changing setNeedsAttributes to true in [https://github.com/apache/beam/commit/f1d02e9c6a741088479c9221af4c0250d5678b1f#diff-cfdca1a326496dabb9a075878204894fR472] However, it appears the test against this did not fail? [https://github.com/apache/beam/commit/f1d02e9c6a741088479c9221af4c0250d5678b1f#diff-598fd709155261a38592b0612939551aR73] > Expose the message id of received messages within PubsubMessage > --- > > Key: BEAM-3489 > URL: https://issues.apache.org/jira/browse/BEAM-3489 > Project: Beam > Issue Type: New Feature > Components: io-java-gcp >Reporter: Luke Cwik >Assignee: Thinh Ha >Priority: P3 > Labels: newbie, starter > Time Spent: 8h 50m > Remaining Estimate: 0h > > This task is about passing forward the message id from the pubsub proto to > the java PubsubMessage. > Add a message id field to PubsubMessage. > Update the coder for PubsubMessage to encode the message id. > Update the translation from the Pubsub proto message to the Dataflow message: > https://github.com/apache/beam/blob/2e275264b21db45787833502e5e42907b05e28b8/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PubsubUnboundedSource.java#L976 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (BEAM-3489) Expose the message id of received messages within PubsubMessage
[ https://issues.apache.org/jira/browse/BEAM-3489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17120343#comment-17120343 ] Thinh Ha edited comment on BEAM-3489 at 5/30/20, 6:47 PM: -- Hi [~reuvenlax], thanks for your feedback. Could you advise what needs to change? In this PR the parseFn is always non-null, e.g. [https://github.com/apache/beam/commit/f1d02e9c6a741088479c9221af4c0250d5678b1f#diff-cfdca1a326496dabb9a075878204894fR471] [https://github.com/apache/beam/commit/f1d02e9c6a741088479c9221af4c0250d5678b1f#diff-cfdca1a326496dabb9a075878204894fR499] which should trigger this branch of code that returns the PubsubMessage [https://github.com/apache/beam/blob/master/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/PubsubReader.java#L120] I can think of changing setNeedsAttributes to true in [https://github.com/apache/beam/commit/f1d02e9c6a741088479c9221af4c0250d5678b1f#diff-cfdca1a326496dabb9a075878204894fR472] However, it appears the test against this did not fail? [https://github.com/apache/beam/commit/f1d02e9c6a741088479c9221af4c0250d5678b1f#diff-598fd709155261a38592b0612939551aR73] was (Author: thinhha): Hi [~reuvenlax], thanks for your feedback. Could you advise what needs to change? In this PR the parseFn is always non-null, e.g. [https://github.com/apache/beam/commit/f1d02e9c6a741088479c9221af4c0250d5678b1f#diff-a72606daf95658d0bb7c365b0d08416cR120] which should trigger this branch of code that returns the PubsubMessage [https://github.com/apache/beam/blob/master/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/PubsubReader.java#L120] I can think of changing setNeedsAttributes to true in [https://github.com/apache/beam/commit/f1d02e9c6a741088479c9221af4c0250d5678b1f#diff-cfdca1a326496dabb9a075878204894fR472] However, it appears the test against this did not fail? [https://github.com/apache/beam/commit/f1d02e9c6a741088479c9221af4c0250d5678b1f#diff-598fd709155261a38592b0612939551aR73] > Expose the message id of received messages within PubsubMessage > --- > > Key: BEAM-3489 > URL: https://issues.apache.org/jira/browse/BEAM-3489 > Project: Beam > Issue Type: New Feature > Components: io-java-gcp >Reporter: Luke Cwik >Assignee: Thinh Ha >Priority: P3 > Labels: newbie, starter > Time Spent: 8h 20m > Remaining Estimate: 0h > > This task is about passing forward the message id from the pubsub proto to > the java PubsubMessage. > Add a message id field to PubsubMessage. > Update the coder for PubsubMessage to encode the message id. > Update the translation from the Pubsub proto message to the Dataflow message: > https://github.com/apache/beam/blob/2e275264b21db45787833502e5e42907b05e28b8/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PubsubUnboundedSource.java#L976 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (BEAM-3489) Expose the message id of received messages within PubsubMessage
[ https://issues.apache.org/jira/browse/BEAM-3489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17120343#comment-17120343 ] Thinh Ha edited comment on BEAM-3489 at 5/30/20, 6:45 PM: -- Hi [~reuvenlax], thanks for your feedback. Could you advise what needs to change? In this PR the parseFn is always non-null, e.g. [https://github.com/apache/beam/commit/f1d02e9c6a741088479c9221af4c0250d5678b1f#diff-a72606daf95658d0bb7c365b0d08416cR120] which should trigger this branch of code that returns the PubsubMessage [https://github.com/apache/beam/blob/master/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/PubsubReader.java#L120] I can think of changing setNeedsAttributes to true in [https://github.com/apache/beam/commit/f1d02e9c6a741088479c9221af4c0250d5678b1f#diff-cfdca1a326496dabb9a075878204894fR472] However, it appears the test against this did not fail? [https://github.com/apache/beam/commit/f1d02e9c6a741088479c9221af4c0250d5678b1f#diff-598fd709155261a38592b0612939551aR73] was (Author: thinhha): Hi [~reuvenlax], thanks for your feedback. Could you advise what needs to change? Looking at > Expose the message id of received messages within PubsubMessage > --- > > Key: BEAM-3489 > URL: https://issues.apache.org/jira/browse/BEAM-3489 > Project: Beam > Issue Type: New Feature > Components: io-java-gcp >Reporter: Luke Cwik >Assignee: Thinh Ha >Priority: P3 > Labels: newbie, starter > Time Spent: 8h 20m > Remaining Estimate: 0h > > This task is about passing forward the message id from the pubsub proto to > the java PubsubMessage. > Add a message id field to PubsubMessage. > Update the coder for PubsubMessage to encode the message id. > Update the translation from the Pubsub proto message to the Dataflow message: > https://github.com/apache/beam/blob/2e275264b21db45787833502e5e42907b05e28b8/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PubsubUnboundedSource.java#L976 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (BEAM-3489) Expose the message id of received messages within PubsubMessage
[ https://issues.apache.org/jira/browse/BEAM-3489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17120343#comment-17120343 ] Thinh Ha commented on BEAM-3489: Hi [~reuvenlax], thanks for your feedback. Could you advise what needs to change? Looking at > Expose the message id of received messages within PubsubMessage > --- > > Key: BEAM-3489 > URL: https://issues.apache.org/jira/browse/BEAM-3489 > Project: Beam > Issue Type: New Feature > Components: io-java-gcp >Reporter: Luke Cwik >Assignee: Thinh Ha >Priority: P3 > Labels: newbie, starter > Time Spent: 8h 20m > Remaining Estimate: 0h > > This task is about passing forward the message id from the pubsub proto to > the java PubsubMessage. > Add a message id field to PubsubMessage. > Update the coder for PubsubMessage to encode the message id. > Update the translation from the Pubsub proto message to the Dataflow message: > https://github.com/apache/beam/blob/2e275264b21db45787833502e5e42907b05e28b8/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PubsubUnboundedSource.java#L976 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (BEAM-3489) Expose the message id of received messages within PubsubMessage
[ https://issues.apache.org/jira/browse/BEAM-3489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thinh Ha reassigned BEAM-3489: -- Assignee: Thinh Ha (was: Ahmed El.Hussaini) > Expose the message id of received messages within PubsubMessage > --- > > Key: BEAM-3489 > URL: https://issues.apache.org/jira/browse/BEAM-3489 > Project: Beam > Issue Type: New Feature > Components: io-java-gcp >Reporter: Luke Cwik >Assignee: Thinh Ha >Priority: Minor > Labels: newbie, starter > > This task is about passing forward the message id from the pubsub proto to > the java PubsubMessage. > Add a message id field to PubsubMessage. > Update the coder for PubsubMessage to encode the message id. > Update the translation from the Pubsub proto message to the Dataflow message: > https://github.com/apache/beam/blob/2e275264b21db45787833502e5e42907b05e28b8/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PubsubUnboundedSource.java#L976 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (BEAM-3489) Expose the message id of received messages within PubsubMessage
[ https://issues.apache.org/jira/browse/BEAM-3489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thinh Ha updated BEAM-3489: --- Attachment: (was: index.html) > Expose the message id of received messages within PubsubMessage > --- > > Key: BEAM-3489 > URL: https://issues.apache.org/jira/browse/BEAM-3489 > Project: Beam > Issue Type: New Feature > Components: io-java-gcp >Reporter: Luke Cwik >Assignee: Ahmed El.Hussaini >Priority: Minor > Labels: newbie, starter > > This task is about passing forward the message id from the pubsub proto to > the java PubsubMessage. > Add a message id field to PubsubMessage. > Update the coder for PubsubMessage to encode the message id. > Update the translation from the Pubsub proto message to the Dataflow message: > https://github.com/apache/beam/blob/2e275264b21db45787833502e5e42907b05e28b8/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PubsubUnboundedSource.java#L976 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (BEAM-3489) Expose the message id of received messages within PubsubMessage
[ https://issues.apache.org/jira/browse/BEAM-3489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thinh Ha updated BEAM-3489: --- Attachment: index.html > Expose the message id of received messages within PubsubMessage > --- > > Key: BEAM-3489 > URL: https://issues.apache.org/jira/browse/BEAM-3489 > Project: Beam > Issue Type: New Feature > Components: io-java-gcp >Reporter: Luke Cwik >Assignee: Ahmed El.Hussaini >Priority: Minor > Labels: newbie, starter > Attachments: index.html > > > This task is about passing forward the message id from the pubsub proto to > the java PubsubMessage. > Add a message id field to PubsubMessage. > Update the coder for PubsubMessage to encode the message id. > Update the translation from the Pubsub proto message to the Dataflow message: > https://github.com/apache/beam/blob/2e275264b21db45787833502e5e42907b05e28b8/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PubsubUnboundedSource.java#L976 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
