[jira] [Commented] (BEAM-8924) Beam Dependency Update Request: org.apache.tika
[ https://issues.apache.org/jira/browse/BEAM-8924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17062335#comment-17062335 ] Colm O hEigeartaigh commented on BEAM-8924: --- This issue can be resolved as "fixed". I don't have the permissions to do so. > Beam Dependency Update Request: org.apache.tika > --- > > Key: BEAM-8924 > URL: https://issues.apache.org/jira/browse/BEAM-8924 > Project: Beam > Issue Type: Bug > Components: dependencies >Reporter: Beam JIRA Bot >Assignee: Colm O hEigeartaigh >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > - 2019-12-09 12:20:19.972138 > - > Please consider upgrading the dependency org.apache.tika. > The current version is None. The latest version is None > cc: > Please refer to [Beam Dependency Guide > |https://beam.apache.org/contribute/dependencies/]for more information. > Do Not Modify The Description Above. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8173) Filesystems.matchSingleFileSpec throws away the actual failure exception
[ https://issues.apache.org/jira/browse/BEAM-8173?focusedWorklogId=405969&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-405969 ] ASF GitHub Bot logged work on BEAM-8173: Author: ASF GitHub Bot Created on: 19/Mar/20 07:14 Start Date: 19/Mar/20 07:14 Worklog Time Spent: 10m Work Description: mwalenia commented on issue #11137: [BEAM-8173] Fix shadowing the actual failure exception in Filesystems… URL: https://github.com/apache/beam/pull/11137#issuecomment-601024107 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 405969) Time Spent: 3h (was: 2h 50m) > Filesystems.matchSingleFileSpec throws away the actual failure exception > > > Key: BEAM-8173 > URL: https://issues.apache.org/jira/browse/BEAM-8173 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Kenneth Knowles >Assignee: Piotr Szuberski >Priority: Major > Labels: easyfix, starter > Time Spent: 3h > Remaining Estimate: 0h > > At [1] the result of a non-OK match causes an exception to be thrown. But the > exception does not include the actual cause of the failure, so it cannot be > efficiently debugged. It appears that the design of MatchResult is that it > should call metadata() without bothering to check status, so that the > underlying exception can be re-raised and caught and put in the chain of > causes, as it should be. > [1] > https://github.com/apache/beam/blob/c2f0d282337f3ae0196a7717712396a5a41fdde1/sdks/java/core/src/main/java/org/apache/beam/sdk/io/FileSystems.java#L190 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9035) BIP-1: Typed options for Row Schema and Fields
[ https://issues.apache.org/jira/browse/BEAM-9035?focusedWorklogId=405988&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-405988 ] ASF GitHub Bot logged work on BEAM-9035: Author: ASF GitHub Bot Created on: 19/Mar/20 08:00 Start Date: 19/Mar/20 08:00 Worklog Time Spent: 10m Work Description: alexvanboxel commented on issue #10413: [BEAM-9035] Typed options for Row Schema and Field URL: https://github.com/apache/beam/pull/10413#issuecomment-601039811 > @alexvanboxel I'm still not 100% sure that I agree that options are not schemas, and I don't love making SchemaVerification public. However I think it will be easier to iterate on those issues in isolation once the basic support is in, so I'm approving for now. Thanks, I'm always open for the schema discussion. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 405988) Time Spent: 8.5h (was: 8h 20m) > BIP-1: Typed options for Row Schema and Fields > -- > > Key: BEAM-9035 > URL: https://issues.apache.org/jira/browse/BEAM-9035 > Project: Beam > Issue Type: Sub-task > Components: sdk-java-core >Reporter: Alex Van Boxel >Assignee: Alex Van Boxel >Priority: Major > Fix For: 2.19.0 > > Time Spent: 8.5h > Remaining Estimate: 0h > > This is the first issue of a multipart commit: this ticket implements the > basic infrastructure of options on row and field. > Full explanation: > Introduce the concept of Options in Beam Schema’s to add extra context to > fields and schema. In contracts to metadata, options would be added to > fields, logical types and rows. In the options schema convertors can add > options/annotations/decorators that were in the original schema, this context > can be used in the rest of the pipeline for specific transformations or > augment the end schema in the target output. > Examples of options are: > * informational: like the source of the data, ... > * drive decisions further in the pipeline: flatten a row into another, > rename a field, ... > * influence something in the output: like cluster index, primary key, ... > * logical type information > And option is a key/typed value combination. The advantages of having the > value types is: > * Having strongly typed options would give a *portable way of Logical Types* > to have structured information that could be shared over different languages. > * This could keep the type intact when mapping from a formats that have > strongly typed options (example: Protobuf). > This is part of a multi ticket implementation. The following tickets are > related: > # Typed options for Row Schema and Fields > # Convert Proto Options to Beam Schema options > # Convert Avro extra information for Beam string options > # Replace meta data with Logical Type options > # Extract meta data in Calcite SQL to Beam options > # Extract meta data in Zeta SQL to Beam options > # Add java example of using option in a transform > This feature is discussed with Reuven Lax, Brian Hulette -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9279) Make HBase.ReadAll based on Reads instead of HBaseQuery
[
https://issues.apache.org/jira/browse/BEAM-9279?focusedWorklogId=405989&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-405989
]
ASF GitHub Bot logged work on BEAM-9279:
Author: ASF GitHub Bot
Created on: 19/Mar/20 08:00
Start Date: 19/Mar/20 08:00
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #10815: [BEAM-9279]
Make HBase.ReadAll based on Reads instead of HBaseQuery
URL: https://github.com/apache/beam/pull/10815#discussion_r394846218
##
File path:
sdks/java/io/hbase/src/main/java/org/apache/beam/sdk/io/hbase/HBaseIO.java
##
@@ -173,33 +182,33 @@ public static Read read() {
/** Reads from the HBase instance indicated by the* given configuration. */
public Read withConfiguration(Configuration configuration) {
checkArgument(configuration != null, "configuration can not be null");
- return new Read(new SerializableConfiguration(configuration), tableId,
serializableScan);
+ return new Read(new Configuration(configuration), tableId, scan);
}
/** Reads from the specified table. */
public Read withTableId(String tableId) {
checkArgument(tableId != null, "tableIdcan not be null");
- return new Read(serializableConfiguration, tableId, serializableScan);
+ return new Read(configuration, tableId, scan);
}
/** Filters the rows read from HBase using the given* scan. */
public Read withScan(Scan scan) {
checkArgument(scan != null, "scancan not be null");
Review comment:
good one, fixing it
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 405989)
Time Spent: 2h (was: 1h 50m)
> Make HBase.ReadAll based on Reads instead of HBaseQuery
> ---
>
> Key: BEAM-9279
> URL: https://issues.apache.org/jira/browse/BEAM-9279
> Project: Beam
> Issue Type: Improvement
> Components: io-java-hbase
>Reporter: Ismaël Mejía
>Assignee: Ismaël Mejía
>Priority: Minor
> Time Spent: 2h
> Remaining Estimate: 0h
>
> HBaseIO support for SplittableDoFn introduced a new request type HBaseQuery,
> however the attributes defined in that class are already available in
> HBase.Read. Allowing users to define pipelines based on HBaseIO.Read allows
> to create pipelines that can read from multiple clusters because the
> Configuration now is part of the request object.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9035) BIP-1: Typed options for Row Schema and Fields
[ https://issues.apache.org/jira/browse/BEAM-9035?focusedWorklogId=405990&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-405990 ] ASF GitHub Bot logged work on BEAM-9035: Author: ASF GitHub Bot Created on: 19/Mar/20 08:00 Start Date: 19/Mar/20 08:00 Worklog Time Spent: 10m Work Description: alexvanboxel commented on pull request #10413: [BEAM-9035] Typed options for Row Schema and Field URL: https://github.com/apache/beam/pull/10413 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 405990) Time Spent: 8h 40m (was: 8.5h) > BIP-1: Typed options for Row Schema and Fields > -- > > Key: BEAM-9035 > URL: https://issues.apache.org/jira/browse/BEAM-9035 > Project: Beam > Issue Type: Sub-task > Components: sdk-java-core >Reporter: Alex Van Boxel >Assignee: Alex Van Boxel >Priority: Major > Fix For: 2.19.0 > > Time Spent: 8h 40m > Remaining Estimate: 0h > > This is the first issue of a multipart commit: this ticket implements the > basic infrastructure of options on row and field. > Full explanation: > Introduce the concept of Options in Beam Schema’s to add extra context to > fields and schema. In contracts to metadata, options would be added to > fields, logical types and rows. In the options schema convertors can add > options/annotations/decorators that were in the original schema, this context > can be used in the rest of the pipeline for specific transformations or > augment the end schema in the target output. > Examples of options are: > * informational: like the source of the data, ... > * drive decisions further in the pipeline: flatten a row into another, > rename a field, ... > * influence something in the output: like cluster index, primary key, ... > * logical type information > And option is a key/typed value combination. The advantages of having the > value types is: > * Having strongly typed options would give a *portable way of Logical Types* > to have structured information that could be shared over different languages. > * This could keep the type intact when mapping from a formats that have > strongly typed options (example: Protobuf). > This is part of a multi ticket implementation. The following tickets are > related: > # Typed options for Row Schema and Fields > # Convert Proto Options to Beam Schema options > # Convert Avro extra information for Beam string options > # Replace meta data with Logical Type options > # Extract meta data in Calcite SQL to Beam options > # Extract meta data in Zeta SQL to Beam options > # Add java example of using option in a transform > This feature is discussed with Reuven Lax, Brian Hulette -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9279) Make HBase.ReadAll based on Reads instead of HBaseQuery
[
https://issues.apache.org/jira/browse/BEAM-9279?focusedWorklogId=405991&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-405991
]
ASF GitHub Bot logged work on BEAM-9279:
Author: ASF GitHub Bot
Created on: 19/Mar/20 08:01
Start Date: 19/Mar/20 08:01
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #10815: [BEAM-9279]
Make HBase.ReadAll based on Reads instead of HBaseQuery
URL: https://github.com/apache/beam/pull/10815#discussion_r394846361
##
File path:
sdks/java/io/hbase/src/main/java/org/apache/beam/sdk/io/hbase/HBaseIO.java
##
@@ -173,33 +182,33 @@ public static Read read() {
/** Reads from the HBase instance indicated by the* given configuration. */
public Read withConfiguration(Configuration configuration) {
checkArgument(configuration != null, "configuration can not be null");
- return new Read(new SerializableConfiguration(configuration), tableId,
serializableScan);
+ return new Read(new Configuration(configuration), tableId, scan);
}
/** Reads from the specified table. */
public Read withTableId(String tableId) {
checkArgument(tableId != null, "tableIdcan not be null");
Review comment:
fixing it, actually they were more ocurrencdes of this 'can' mistake I fixed
them all now.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 405991)
Time Spent: 2h 10m (was: 2h)
> Make HBase.ReadAll based on Reads instead of HBaseQuery
> ---
>
> Key: BEAM-9279
> URL: https://issues.apache.org/jira/browse/BEAM-9279
> Project: Beam
> Issue Type: Improvement
> Components: io-java-hbase
>Reporter: Ismaël Mejía
>Assignee: Ismaël Mejía
>Priority: Minor
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> HBaseIO support for SplittableDoFn introduced a new request type HBaseQuery,
> however the attributes defined in that class are already available in
> HBase.Read. Allowing users to define pipelines based on HBaseIO.Read allows
> to create pipelines that can read from multiple clusters because the
> Configuration now is part of the request object.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (BEAM-9554) Improve connection reuse on HBaseIO.ReadAll
[ https://issues.apache.org/jira/browse/BEAM-9554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-9554: --- Status: Open (was: Triage Needed) > Improve connection reuse on HBaseIO.ReadAll > --- > > Key: BEAM-9554 > URL: https://issues.apache.org/jira/browse/BEAM-9554 > Project: Beam > Issue Type: Improvement > Components: io-java-hbase >Reporter: Ismaël Mejía >Assignee: Ismaël Mejía >Priority: Minor > > The recent refactor of HBase.ReadAll in BEAM-9279 creates new connections in > the @ProcessElement method (once per element), in the case that a pipeline is > used on streaming mode this could be costly so we should find a way to cache > and reuse connections to avoid both slow start of reads and saturating the > clusters. > Notice that this is an ongoing issue for DoFn based IOs that manifested first > on Writes for JdbcIO BEAM-7230 and was recently discussed too in the context > of the CassandraIO refactor: > https://github.com/apache/beam/pull/10546#issuecomment-580619044 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (BEAM-9554) Improve connection reuse on HBaseIO.ReadAll
Ismaël Mejía created BEAM-9554: -- Summary: Improve connection reuse on HBaseIO.ReadAll Key: BEAM-9554 URL: https://issues.apache.org/jira/browse/BEAM-9554 Project: Beam Issue Type: Improvement Components: io-java-hbase Reporter: Ismaël Mejía Assignee: Ismaël Mejía The recent refactor of HBase.ReadAll in BEAM-9279 creates new connections in the @ProcessElement method (once per element), in the case that a pipeline is used on streaming mode this could be costly so we should find a way to cache and reuse connections to avoid both slow start of reads and saturating the clusters. Notice that this is an ongoing issue for DoFn based IOs that manifested first on Writes for JdbcIO BEAM-7230 and was recently discussed too in the context of the CassandraIO refactor: https://github.com/apache/beam/pull/10546#issuecomment-580619044 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (BEAM-9554) Improve connection reuse on HBaseIO.ReadAll
[ https://issues.apache.org/jira/browse/BEAM-9554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía reassigned BEAM-9554: -- Assignee: (was: Ismaël Mejía) > Improve connection reuse on HBaseIO.ReadAll > --- > > Key: BEAM-9554 > URL: https://issues.apache.org/jira/browse/BEAM-9554 > Project: Beam > Issue Type: Improvement > Components: io-java-hbase >Reporter: Ismaël Mejía >Priority: Minor > > The recent refactor of HBase.ReadAll in BEAM-9279 creates new connections in > the @ProcessElement method (once per element), in the case that a pipeline is > used on streaming mode this could be costly so we should find a way to cache > and reuse connections to avoid both slow start of reads and saturating the > clusters. > Notice that this is an ongoing issue for DoFn based IOs that manifested first > on Writes for JdbcIO BEAM-7230 and was recently discussed too in the context > of the CassandraIO refactor: > https://github.com/apache/beam/pull/10546#issuecomment-580619044 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8972) Add a Jenkins job running Combine load test on Java with Flink in Portability mode
[ https://issues.apache.org/jira/browse/BEAM-8972?focusedWorklogId=405998&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-405998 ] ASF GitHub Bot logged work on BEAM-8972: Author: ASF GitHub Bot Created on: 19/Mar/20 08:22 Start Date: 19/Mar/20 08:22 Worklog Time Spent: 10m Work Description: mwalenia commented on issue #10386: [BEAM-8972] Add Jenkins job with Combine test for portable Java URL: https://github.com/apache/beam/pull/10386#issuecomment-601048332 run seed job This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 405998) Time Spent: 20h 10m (was: 20h) > Add a Jenkins job running Combine load test on Java with Flink in Portability > mode > -- > > Key: BEAM-8972 > URL: https://issues.apache.org/jira/browse/BEAM-8972 > Project: Beam > Issue Type: Improvement > Components: testing >Reporter: Michał Walenia >Assignee: Michał Walenia >Priority: Minor > Time Spent: 20h 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9279) Make HBase.ReadAll based on Reads instead of HBaseQuery
[
https://issues.apache.org/jira/browse/BEAM-9279?focusedWorklogId=406000&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406000
]
ASF GitHub Bot logged work on BEAM-9279:
Author: ASF GitHub Bot
Created on: 19/Mar/20 08:28
Start Date: 19/Mar/20 08:28
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #10815: [BEAM-9279]
Make HBase.ReadAll based on Reads instead of HBaseQuery
URL: https://github.com/apache/beam/pull/10815#discussion_r394859324
##
File path:
sdks/java/io/hbase/src/main/java/org/apache/beam/sdk/io/hbase/HBaseReadSplittableDoFn.java
##
@@ -32,65 +31,50 @@
import org.apache.hadoop.hbase.client.ConnectionFactory;
import org.apache.hadoop.hbase.client.Result;
import org.apache.hadoop.hbase.client.ResultScanner;
-import org.apache.hadoop.hbase.client.Scan;
import org.apache.hadoop.hbase.client.Table;
/** A SplittableDoFn to read from HBase. */
@BoundedPerElement
-class HBaseReadSplittableDoFn extends DoFn {
- private final SerializableConfiguration serializableConfiguration;
-
- private transient Connection connection;
-
- HBaseReadSplittableDoFn(SerializableConfiguration serializableConfiguration)
{
-this.serializableConfiguration = serializableConfiguration;
- }
-
- @Setup
- public void setup() throws Exception {
-connection =
ConnectionFactory.createConnection(serializableConfiguration.get());
- }
-
- private static Scan newScanInRange(Scan scan, ByteKeyRange range) throws
IOException {
-return new Scan(scan)
-.setStartRow(range.getStartKey().getBytes())
-.setStopRow(range.getEndKey().getBytes());
- }
+class HBaseReadSplittableDoFn extends DoFn {
+ HBaseReadSplittableDoFn() {}
@ProcessElement
- public void processElement(ProcessContext c,
RestrictionTracker tracker)
+ public void processElement(
+ @Element Read read,
+ OutputReceiver out,
+ RestrictionTracker tracker)
throws Exception {
-final HBaseQuery query = c.element();
-TableName tableName = TableName.valueOf(query.getTableId());
+Connection connection =
ConnectionFactory.createConnection(read.getConfiguration());
Review comment:
Better filled https://issues.apache.org/jira/browse/BEAM-9554 to track this.
No I did not test performance because this is a quite particular case as I
mention, for users doing 1 to n queries when n is big this time would not be
considerable, the real issue can manifest mostly in pipelines with streaming
where we would like to do reads per window with multiple windows kind of
similar to what we found for JdbcIO writes (but this case is waaay more common).
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 406000)
Time Spent: 2h 20m (was: 2h 10m)
> Make HBase.ReadAll based on Reads instead of HBaseQuery
> ---
>
> Key: BEAM-9279
> URL: https://issues.apache.org/jira/browse/BEAM-9279
> Project: Beam
> Issue Type: Improvement
> Components: io-java-hbase
>Reporter: Ismaël Mejía
>Assignee: Ismaël Mejía
>Priority: Minor
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> HBaseIO support for SplittableDoFn introduced a new request type HBaseQuery,
> however the attributes defined in that class are already available in
> HBase.Read. Allowing users to define pipelines based on HBaseIO.Read allows
> to create pipelines that can read from multiple clusters because the
> Configuration now is part of the request object.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9279) Make HBase.ReadAll based on Reads instead of HBaseQuery
[
https://issues.apache.org/jira/browse/BEAM-9279?focusedWorklogId=406002&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406002
]
ASF GitHub Bot logged work on BEAM-9279:
Author: ASF GitHub Bot
Created on: 19/Mar/20 08:31
Start Date: 19/Mar/20 08:31
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #10815: [BEAM-9279]
Make HBase.ReadAll based on Reads instead of HBaseQuery
URL: https://github.com/apache/beam/pull/10815#discussion_r394860657
##
File path:
sdks/java/io/hbase/src/main/java/org/apache/beam/sdk/io/hbase/HBaseIO.java
##
@@ -240,63 +245,109 @@ private Read(
@Override
public void populateDisplayData(DisplayData.Builder builder) {
super.populateDisplayData(builder);
- builder.add(DisplayData.item("configuration",
serializableConfiguration.get().toString()));
+ builder.add(DisplayData.item("configuration", configuration.toString()));
builder.add(DisplayData.item("tableId", tableId));
- builder.addIfNotNull(DisplayData.item("scan",
serializableScan.get().toString()));
+ builder.addIfNotNull(DisplayData.item("scan", scan.toString()));
}
public Configuration getConfiguration() {
- return serializableConfiguration.get();
+ return configuration;
}
public String getTableId() {
return tableId;
}
public Scan getScan() {
- return serializableScan.get();
+ return scan;
}
/** Returns the range of keys that will be read from the table. */
public ByteKeyRange getKeyRange() {
- byte[] startRow = serializableScan.get().getStartRow();
- byte[] stopRow = serializableScan.get().getStopRow();
+ byte[] startRow = scan.getStartRow();
+ byte[] stopRow = scan.getStopRow();
return ByteKeyRange.of(ByteKey.copyFrom(startRow),
ByteKey.copyFrom(stopRow));
}
-private final SerializableConfiguration serializableConfiguration;
+@Override
+public boolean equals(Object o) {
+ if (this == o) {
+return true;
+ }
+ if (o == null || getClass() != o.getClass()) {
+return false;
+ }
+ Read read = (Read) o;
+ return configuration.toString().equals(read.configuration.toString())
+ && Objects.equals(tableId, read.tableId)
+ && scan.toString().equals(read.scan.toString());
+}
+
+@Override
+public int hashCode() {
+ return Objects.hash(configuration, tableId, scan);
+}
+
+private Object writeReplace() {
Review comment:
good idea, doing it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 406002)
Time Spent: 2.5h (was: 2h 20m)
> Make HBase.ReadAll based on Reads instead of HBaseQuery
> ---
>
> Key: BEAM-9279
> URL: https://issues.apache.org/jira/browse/BEAM-9279
> Project: Beam
> Issue Type: Improvement
> Components: io-java-hbase
>Reporter: Ismaël Mejía
>Assignee: Ismaël Mejía
>Priority: Minor
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> HBaseIO support for SplittableDoFn introduced a new request type HBaseQuery,
> however the attributes defined in that class are already available in
> HBase.Read. Allowing users to define pipelines based on HBaseIO.Read allows
> to create pipelines that can read from multiple clusters because the
> Configuration now is part of the request object.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8972) Add a Jenkins job running Combine load test on Java with Flink in Portability mode
[ https://issues.apache.org/jira/browse/BEAM-8972?focusedWorklogId=406004&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406004 ] ASF GitHub Bot logged work on BEAM-8972: Author: ASF GitHub Bot Created on: 19/Mar/20 08:45 Start Date: 19/Mar/20 08:45 Worklog Time Spent: 10m Work Description: mwalenia commented on issue #10386: [BEAM-8972] Add Jenkins job with Combine test for portable Java URL: https://github.com/apache/beam/pull/10386#issuecomment-601057353 Run Load Tests Java Combine Portable Flink Streaming This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 406004) Time Spent: 20h 20m (was: 20h 10m) > Add a Jenkins job running Combine load test on Java with Flink in Portability > mode > -- > > Key: BEAM-8972 > URL: https://issues.apache.org/jira/browse/BEAM-8972 > Project: Beam > Issue Type: Improvement > Components: testing >Reporter: Michał Walenia >Assignee: Michał Walenia >Priority: Minor > Time Spent: 20h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8078) streaming_wordcount_debugging.py is missing a test
[ https://issues.apache.org/jira/browse/BEAM-8078?focusedWorklogId=406006&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406006 ] ASF GitHub Bot logged work on BEAM-8078: Author: ASF GitHub Bot Created on: 19/Mar/20 08:52 Start Date: 19/Mar/20 08:52 Worklog Time Spent: 10m Work Description: Tesio commented on issue #10914: [BEAM-8078] streaming_wordcount_debugging.py is missing a test URL: https://github.com/apache/beam/pull/10914#issuecomment-601060116 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 406006) Time Spent: 3h (was: 2h 50m) > streaming_wordcount_debugging.py is missing a test > -- > > Key: BEAM-8078 > URL: https://issues.apache.org/jira/browse/BEAM-8078 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Udi Meiri >Assignee: Aleksey Vysotin >Priority: Minor > Labels: beginner, easy, newbie, starter > Time Spent: 3h > Remaining Estimate: 0h > > It's example code and should have a basic_test (like the other wordcount > variants in [1]) to at least verify that it runs in the latest Beam release. > [1] > https://github.com/apache/beam/tree/master/sdks/python/apache_beam/examples -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9279) Make HBase.ReadAll based on Reads instead of HBaseQuery
[
https://issues.apache.org/jira/browse/BEAM-9279?focusedWorklogId=406009&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406009
]
ASF GitHub Bot logged work on BEAM-9279:
Author: ASF GitHub Bot
Created on: 19/Mar/20 09:02
Start Date: 19/Mar/20 09:02
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #10815: [BEAM-9279]
Make HBase.ReadAll based on Reads instead of HBaseQuery
URL: https://github.com/apache/beam/pull/10815#discussion_r394877026
##
File path:
sdks/java/io/hbase/src/main/java/org/apache/beam/sdk/io/hbase/HBaseIO.java
##
@@ -240,63 +245,109 @@ private Read(
@Override
public void populateDisplayData(DisplayData.Builder builder) {
super.populateDisplayData(builder);
- builder.add(DisplayData.item("configuration",
serializableConfiguration.get().toString()));
+ builder.add(DisplayData.item("configuration", configuration.toString()));
builder.add(DisplayData.item("tableId", tableId));
- builder.addIfNotNull(DisplayData.item("scan",
serializableScan.get().toString()));
+ builder.addIfNotNull(DisplayData.item("scan", scan.toString()));
}
public Configuration getConfiguration() {
- return serializableConfiguration.get();
+ return configuration;
}
public String getTableId() {
return tableId;
}
public Scan getScan() {
- return serializableScan.get();
+ return scan;
}
/** Returns the range of keys that will be read from the table. */
public ByteKeyRange getKeyRange() {
- byte[] startRow = serializableScan.get().getStartRow();
- byte[] stopRow = serializableScan.get().getStopRow();
+ byte[] startRow = scan.getStartRow();
+ byte[] stopRow = scan.getStopRow();
return ByteKeyRange.of(ByteKey.copyFrom(startRow),
ByteKey.copyFrom(stopRow));
}
-private final SerializableConfiguration serializableConfiguration;
+@Override
+public boolean equals(Object o) {
+ if (this == o) {
+return true;
+ }
+ if (o == null || getClass() != o.getClass()) {
+return false;
+ }
+ Read read = (Read) o;
+ return configuration.toString().equals(read.configuration.toString())
+ && Objects.equals(tableId, read.tableId)
+ && scan.toString().equals(read.scan.toString());
+}
+
+@Override
+public int hashCode() {
+ return Objects.hash(configuration, tableId, scan);
+}
+
+private Object writeReplace() {
+ return new SerializationProxy(this);
+}
+
+private static class SerializationProxy implements Serializable {
+ public SerializationProxy() {}
+
+ public SerializationProxy(Read read) {
+configuration = read.configuration;
+tableId = read.tableId;
+scan = read.scan;
+ }
+
+ private void writeObject(ObjectOutputStream out) throws IOException {
+SerializableCoder.of(SerializableConfiguration.class)
+.encode(new SerializableConfiguration(this.configuration), out);
+StringUtf8Coder.of().encode(this.tableId, out);
+ProtobufUtil.toScan(this.scan).writeDelimitedTo(out);
+ }
+
+ private void readObject(ObjectInputStream in) throws IOException {
+this.configuration =
SerializableCoder.of(SerializableConfiguration.class).decode(in).get();
+this.tableId = StringUtf8Coder.of().decode(in);
+this.scan =
ProtobufUtil.toScan(ClientProtos.Scan.parseDelimitedFrom(in));
+ }
+
+ Object readResolve() {
+return
HBaseIO.read().withConfiguration(configuration).withTableId(tableId).withScan(scan);
+ }
+
+ private Configuration configuration;
+ private String tableId;
+ private Scan scan;
+}
+
+@SuppressFBWarnings("SE_BAD_FIELD")
+private final Configuration configuration;
+
private final String tableId;
-private final SerializableScan serializableScan;
+
+@SuppressFBWarnings("SE_BAD_FIELD")
+private final Scan scan;
}
/**
* A {@link PTransform} that works like {@link #read}, but executes read
operations coming from a
- * {@link PCollection} of {@link HBaseQuery}.
+ * {@link PCollection} of {@link Read}.
*/
public static ReadAll readAll() {
-return new ReadAll(null);
+return new ReadAll();
}
/** Implementation of {@link #readAll}. */
- public static class ReadAll extends PTransform,
PCollection> {
-
-private ReadAll(SerializableConfiguration serializableConfiguration) {
- this.serializableConfiguration = serializableConfiguration;
-}
-
-/** Reads from the HBase instance indicated by the* given configuration. */
-public ReadAll withConfiguration(Configuration configuration) {
- checkArgument(configuration != null, "configuration can not be null");
- return new Rea
[jira] [Work logged] (BEAM-9553) portableWordCount test using incorrect job server
[ https://issues.apache.org/jira/browse/BEAM-9553?focusedWorklogId=406029&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406029 ] ASF GitHub Bot logged work on BEAM-9553: Author: ASF GitHub Bot Created on: 19/Mar/20 09:34 Start Date: 19/Mar/20 09:34 Worklog Time Spent: 10m Work Description: mxm commented on pull request #11170: [BEAM-9553] Use latest Flink job server image as default. URL: https://github.com/apache/beam/pull/11170 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 406029) Time Spent: 20m (was: 10m) > portableWordCount test using incorrect job server > - > > Key: BEAM-9553 > URL: https://issues.apache.org/jira/browse/BEAM-9553 > Project: Beam > Issue Type: Bug > Components: runner-flink >Reporter: Kyle Weaver >Assignee: Kyle Weaver >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > portableWordCount test is building Flink job server 1.10 container [1], but > is using 1.9 container in the test [2]. > [1] > https://github.com/lukecwik/incubator-beam/blob/9f4c5f66bb3979aeeebd648f76cbfaca301a1753/sdks/python/test-suites/portable/py2/build.gradle#L32 > [2] > https://github.com/lukecwik/incubator-beam/blob/9f4c5f66bb3979aeeebd648f76cbfaca301a1753/sdks/python/apache_beam/runners/portability/job_server.py#L185 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (BEAM-9541) Single source of truth for supported Flink versions
[ https://issues.apache.org/jira/browse/BEAM-9541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels updated BEAM-9541: - Status: Open (was: Triage Needed) > Single source of truth for supported Flink versions > --- > > Key: BEAM-9541 > URL: https://issues.apache.org/jira/browse/BEAM-9541 > Project: Beam > Issue Type: Improvement > Components: runner-flink >Reporter: Kyle Weaver >Priority: Major > > Currently, there is a large number of hard-coded references to the supported > Flink versions (either the newest version, or a list of all supported > versions). These should be condensed to a single source of truth to make > upgrades easier and more robust. > Previously, we awked the list of Flink subdirectories, but this is not > reliable because old, unsupported Flink versions can leave build directories > behind. It would be better to read versions from a handwritten file. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (BEAM-9553) portableWordCount test using incorrect job server
[ https://issues.apache.org/jira/browse/BEAM-9553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maximilian Michels closed BEAM-9553. Fix Version/s: 2.21.0 Resolution: Fixed > portableWordCount test using incorrect job server > - > > Key: BEAM-9553 > URL: https://issues.apache.org/jira/browse/BEAM-9553 > Project: Beam > Issue Type: Bug > Components: runner-flink >Reporter: Kyle Weaver >Assignee: Kyle Weaver >Priority: Major > Fix For: 2.21.0 > > Time Spent: 20m > Remaining Estimate: 0h > > portableWordCount test is building Flink job server 1.10 container [1], but > is using 1.9 container in the test [2]. > [1] > https://github.com/lukecwik/incubator-beam/blob/9f4c5f66bb3979aeeebd648f76cbfaca301a1753/sdks/python/test-suites/portable/py2/build.gradle#L32 > [2] > https://github.com/lukecwik/incubator-beam/blob/9f4c5f66bb3979aeeebd648f76cbfaca301a1753/sdks/python/apache_beam/runners/portability/job_server.py#L185 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9279) Make HBase.ReadAll based on Reads instead of HBaseQuery
[ https://issues.apache.org/jira/browse/BEAM-9279?focusedWorklogId=406035&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406035 ] ASF GitHub Bot logged work on BEAM-9279: Author: ASF GitHub Bot Created on: 19/Mar/20 09:37 Start Date: 19/Mar/20 09:37 Worklog Time Spent: 10m Work Description: iemejia commented on issue #10815: [BEAM-9279] Make HBase.ReadAll based on Reads instead of HBaseQuery URL: https://github.com/apache/beam/pull/10815#issuecomment-601080539 Changes done, I rebased since the PR was a bit old. It should be good to go now. PTAL @aromanenko-dev ! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 406035) Time Spent: 2h 50m (was: 2h 40m) > Make HBase.ReadAll based on Reads instead of HBaseQuery > --- > > Key: BEAM-9279 > URL: https://issues.apache.org/jira/browse/BEAM-9279 > Project: Beam > Issue Type: Improvement > Components: io-java-hbase >Reporter: Ismaël Mejía >Assignee: Ismaël Mejía >Priority: Minor > Time Spent: 2h 50m > Remaining Estimate: 0h > > HBaseIO support for SplittableDoFn introduced a new request type HBaseQuery, > however the attributes defined in that class are already available in > HBase.Read. Allowing users to define pipelines based on HBaseIO.Read allows > to create pipelines that can read from multiple clusters because the > Configuration now is part of the request object. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9279) Make HBase.ReadAll based on Reads instead of HBaseQuery
[
https://issues.apache.org/jira/browse/BEAM-9279?focusedWorklogId=406042&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406042
]
ASF GitHub Bot logged work on BEAM-9279:
Author: ASF GitHub Bot
Created on: 19/Mar/20 09:54
Start Date: 19/Mar/20 09:54
Worklog Time Spent: 10m
Work Description: aromanenko-dev commented on pull request #10815:
[BEAM-9279] Make HBase.ReadAll based on Reads instead of HBaseQuery
URL: https://github.com/apache/beam/pull/10815#discussion_r394907044
##
File path:
sdks/java/io/hbase/src/main/java/org/apache/beam/sdk/io/hbase/HBaseIO.java
##
@@ -173,33 +182,33 @@ public static Read read() {
/** Reads from the HBase instance indicated by the* given configuration. */
public Read withConfiguration(Configuration configuration) {
checkArgument(configuration != null, "configuration can not be null");
- return new Read(new SerializableConfiguration(configuration), tableId,
serializableScan);
+ return new Read(new Configuration(configuration), tableId, scan);
}
/** Reads from the specified table. */
public Read withTableId(String tableId) {
checkArgument(tableId != null, "tableIdcan not be null");
Review comment:
thanks
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 406042)
Time Spent: 3h (was: 2h 50m)
> Make HBase.ReadAll based on Reads instead of HBaseQuery
> ---
>
> Key: BEAM-9279
> URL: https://issues.apache.org/jira/browse/BEAM-9279
> Project: Beam
> Issue Type: Improvement
> Components: io-java-hbase
>Reporter: Ismaël Mejía
>Assignee: Ismaël Mejía
>Priority: Minor
> Time Spent: 3h
> Remaining Estimate: 0h
>
> HBaseIO support for SplittableDoFn introduced a new request type HBaseQuery,
> however the attributes defined in that class are already available in
> HBase.Read. Allowing users to define pipelines based on HBaseIO.Read allows
> to create pipelines that can read from multiple clusters because the
> Configuration now is part of the request object.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8972) Add a Jenkins job running Combine load test on Java with Flink in Portability mode
[ https://issues.apache.org/jira/browse/BEAM-8972?focusedWorklogId=406052&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406052 ] ASF GitHub Bot logged work on BEAM-8972: Author: ASF GitHub Bot Created on: 19/Mar/20 10:06 Start Date: 19/Mar/20 10:06 Worklog Time Spent: 10m Work Description: mwalenia commented on issue #10386: [BEAM-8972] Add Jenkins job with Combine test for portable Java URL: https://github.com/apache/beam/pull/10386#issuecomment-601093845 run seed job This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 406052) Time Spent: 20.5h (was: 20h 20m) > Add a Jenkins job running Combine load test on Java with Flink in Portability > mode > -- > > Key: BEAM-8972 > URL: https://issues.apache.org/jira/browse/BEAM-8972 > Project: Beam > Issue Type: Improvement > Components: testing >Reporter: Michał Walenia >Assignee: Michał Walenia >Priority: Minor > Time Spent: 20.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8972) Add a Jenkins job running Combine load test on Java with Flink in Portability mode
[ https://issues.apache.org/jira/browse/BEAM-8972?focusedWorklogId=406056&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406056 ] ASF GitHub Bot logged work on BEAM-8972: Author: ASF GitHub Bot Created on: 19/Mar/20 10:09 Start Date: 19/Mar/20 10:09 Worklog Time Spent: 10m Work Description: mwalenia commented on issue #10386: [BEAM-8972] Add Jenkins job with Combine test for portable Java URL: https://github.com/apache/beam/pull/10386#issuecomment-601094876 Run Load Tests Java Combine Portable Flink Streaming This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 406056) Time Spent: 20h 40m (was: 20.5h) > Add a Jenkins job running Combine load test on Java with Flink in Portability > mode > -- > > Key: BEAM-8972 > URL: https://issues.apache.org/jira/browse/BEAM-8972 > Project: Beam > Issue Type: Improvement > Components: testing >Reporter: Michał Walenia >Assignee: Michał Walenia >Priority: Minor > Time Spent: 20h 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9279) Make HBase.ReadAll based on Reads instead of HBaseQuery
[ https://issues.apache.org/jira/browse/BEAM-9279?focusedWorklogId=406057&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406057 ] ASF GitHub Bot logged work on BEAM-9279: Author: ASF GitHub Bot Created on: 19/Mar/20 10:11 Start Date: 19/Mar/20 10:11 Worklog Time Spent: 10m Work Description: aromanenko-dev commented on issue #10815: [BEAM-9279] Make HBase.ReadAll based on Reads instead of HBaseQuery URL: https://github.com/apache/beam/pull/10815#issuecomment-601095915 Thanks! It LGTM, just let's wait for all tests passed before merging. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 406057) Time Spent: 3h 10m (was: 3h) > Make HBase.ReadAll based on Reads instead of HBaseQuery > --- > > Key: BEAM-9279 > URL: https://issues.apache.org/jira/browse/BEAM-9279 > Project: Beam > Issue Type: Improvement > Components: io-java-hbase >Reporter: Ismaël Mejía >Assignee: Ismaël Mejía >Priority: Minor > Time Spent: 3h 10m > Remaining Estimate: 0h > > HBaseIO support for SplittableDoFn introduced a new request type HBaseQuery, > however the attributes defined in that class are already available in > HBase.Read. Allowing users to define pipelines based on HBaseIO.Read allows > to create pipelines that can read from multiple clusters because the > Configuration now is part of the request object. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9470) :sdks:java:io:kinesis:test is flaky
[ https://issues.apache.org/jira/browse/BEAM-9470?focusedWorklogId=406059&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406059 ] ASF GitHub Bot logged work on BEAM-9470: Author: ASF GitHub Bot Created on: 19/Mar/20 10:14 Start Date: 19/Mar/20 10:14 Worklog Time Spent: 10m Work Description: aromanenko-dev commented on issue #11090: [BEAM-9470] :sdks:java:io:kinesis:test is flaky URL: https://github.com/apache/beam/pull/11090#issuecomment-601097324 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 406059) Time Spent: 9h (was: 8h 50m) > :sdks:java:io:kinesis:test is flaky > --- > > Key: BEAM-9470 > URL: https://issues.apache.org/jira/browse/BEAM-9470 > Project: Beam > Issue Type: Test > Components: io-java-kinesis >Reporter: Etienne Chauchot >Assignee: Jonothan Farr >Priority: Major > Time Spent: 9h > Remaining Estimate: 0h > > [https://scans.gradle.com/s/b4jmmu72ku5jc/console-log?task=:sdks:java:io:kinesis:test] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9470) :sdks:java:io:kinesis:test is flaky
[ https://issues.apache.org/jira/browse/BEAM-9470?focusedWorklogId=406062&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406062 ] ASF GitHub Bot logged work on BEAM-9470: Author: ASF GitHub Bot Created on: 19/Mar/20 10:24 Start Date: 19/Mar/20 10:24 Worklog Time Spent: 10m Work Description: aromanenko-dev commented on issue #11090: [BEAM-9470] :sdks:java:io:kinesis:test is flaky URL: https://github.com/apache/beam/pull/11090#issuecomment-601102227 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 406062) Time Spent: 9h 10m (was: 9h) > :sdks:java:io:kinesis:test is flaky > --- > > Key: BEAM-9470 > URL: https://issues.apache.org/jira/browse/BEAM-9470 > Project: Beam > Issue Type: Test > Components: io-java-kinesis >Reporter: Etienne Chauchot >Assignee: Jonothan Farr >Priority: Major > Time Spent: 9h 10m > Remaining Estimate: 0h > > [https://scans.gradle.com/s/b4jmmu72ku5jc/console-log?task=:sdks:java:io:kinesis:test] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9470) :sdks:java:io:kinesis:test is flaky
[ https://issues.apache.org/jira/browse/BEAM-9470?focusedWorklogId=406064&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406064 ] ASF GitHub Bot logged work on BEAM-9470: Author: ASF GitHub Bot Created on: 19/Mar/20 10:24 Start Date: 19/Mar/20 10:24 Worklog Time Spent: 10m Work Description: aromanenko-dev commented on issue #11090: [BEAM-9470] :sdks:java:io:kinesis:test is flaky URL: https://github.com/apache/beam/pull/11090#issuecomment-601097324 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 406064) Time Spent: 9h 20m (was: 9h 10m) > :sdks:java:io:kinesis:test is flaky > --- > > Key: BEAM-9470 > URL: https://issues.apache.org/jira/browse/BEAM-9470 > Project: Beam > Issue Type: Test > Components: io-java-kinesis >Reporter: Etienne Chauchot >Assignee: Jonothan Farr >Priority: Major > Time Spent: 9h 20m > Remaining Estimate: 0h > > [https://scans.gradle.com/s/b4jmmu72ku5jc/console-log?task=:sdks:java:io:kinesis:test] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8972) Add a Jenkins job running Combine load test on Java with Flink in Portability mode
[ https://issues.apache.org/jira/browse/BEAM-8972?focusedWorklogId=406068&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406068 ] ASF GitHub Bot logged work on BEAM-8972: Author: ASF GitHub Bot Created on: 19/Mar/20 10:35 Start Date: 19/Mar/20 10:35 Worklog Time Spent: 10m Work Description: mwalenia commented on issue #10386: [BEAM-8972] Add Jenkins job with Combine test for portable Java URL: https://github.com/apache/beam/pull/10386#issuecomment-601107028 Run Load Tests Java Combine Portable Flink Streaming This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 406068) Time Spent: 20h 50m (was: 20h 40m) > Add a Jenkins job running Combine load test on Java with Flink in Portability > mode > -- > > Key: BEAM-8972 > URL: https://issues.apache.org/jira/browse/BEAM-8972 > Project: Beam > Issue Type: Improvement > Components: testing >Reporter: Michał Walenia >Assignee: Michał Walenia >Priority: Minor > Time Spent: 20h 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8972) Add a Jenkins job running Combine load test on Java with Flink in Portability mode
[ https://issues.apache.org/jira/browse/BEAM-8972?focusedWorklogId=406071&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406071 ] ASF GitHub Bot logged work on BEAM-8972: Author: ASF GitHub Bot Created on: 19/Mar/20 10:43 Start Date: 19/Mar/20 10:43 Worklog Time Spent: 10m Work Description: mwalenia commented on issue #10386: [BEAM-8972] Add Jenkins job with Combine test for portable Java URL: https://github.com/apache/beam/pull/10386#issuecomment-601110114 run seed job This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 406071) Time Spent: 21h (was: 20h 50m) > Add a Jenkins job running Combine load test on Java with Flink in Portability > mode > -- > > Key: BEAM-8972 > URL: https://issues.apache.org/jira/browse/BEAM-8972 > Project: Beam > Issue Type: Improvement > Components: testing >Reporter: Michał Walenia >Assignee: Michał Walenia >Priority: Minor > Time Spent: 21h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8972) Add a Jenkins job running Combine load test on Java with Flink in Portability mode
[ https://issues.apache.org/jira/browse/BEAM-8972?focusedWorklogId=406074&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406074 ] ASF GitHub Bot logged work on BEAM-8972: Author: ASF GitHub Bot Created on: 19/Mar/20 10:45 Start Date: 19/Mar/20 10:45 Worklog Time Spent: 10m Work Description: mwalenia commented on issue #10386: [BEAM-8972] Add Jenkins job with Combine test for portable Java URL: https://github.com/apache/beam/pull/10386#issuecomment-60079 Run Load Tests Java Combine Portable Flink Streaming This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 406074) Time Spent: 21h 10m (was: 21h) > Add a Jenkins job running Combine load test on Java with Flink in Portability > mode > -- > > Key: BEAM-8972 > URL: https://issues.apache.org/jira/browse/BEAM-8972 > Project: Beam > Issue Type: Improvement > Components: testing >Reporter: Michał Walenia >Assignee: Michał Walenia >Priority: Minor > Time Spent: 21h 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9279) Make HBase.ReadAll based on Reads instead of HBaseQuery
[ https://issues.apache.org/jira/browse/BEAM-9279?focusedWorklogId=406079&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406079 ] ASF GitHub Bot logged work on BEAM-9279: Author: ASF GitHub Bot Created on: 19/Mar/20 10:55 Start Date: 19/Mar/20 10:55 Worklog Time Spent: 10m Work Description: aromanenko-dev commented on pull request #10815: [BEAM-9279] Make HBase.ReadAll based on Reads instead of HBaseQuery URL: https://github.com/apache/beam/pull/10815 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 406079) Time Spent: 3h 20m (was: 3h 10m) > Make HBase.ReadAll based on Reads instead of HBaseQuery > --- > > Key: BEAM-9279 > URL: https://issues.apache.org/jira/browse/BEAM-9279 > Project: Beam > Issue Type: Improvement > Components: io-java-hbase >Reporter: Ismaël Mejía >Assignee: Ismaël Mejía >Priority: Minor > Time Spent: 3h 20m > Remaining Estimate: 0h > > HBaseIO support for SplittableDoFn introduced a new request type HBaseQuery, > however the attributes defined in that class are already available in > HBase.Read. Allowing users to define pipelines based on HBaseIO.Read allows > to create pipelines that can read from multiple clusters because the > Configuration now is part of the request object. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (BEAM-9279) Make HBase.ReadAll based on Reads instead of HBaseQuery
[ https://issues.apache.org/jira/browse/BEAM-9279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Romanenko resolved BEAM-9279. Fix Version/s: 2.21.0 Resolution: Fixed > Make HBase.ReadAll based on Reads instead of HBaseQuery > --- > > Key: BEAM-9279 > URL: https://issues.apache.org/jira/browse/BEAM-9279 > Project: Beam > Issue Type: Improvement > Components: io-java-hbase >Reporter: Ismaël Mejía >Assignee: Ismaël Mejía >Priority: Minor > Fix For: 2.21.0 > > Time Spent: 3h 20m > Remaining Estimate: 0h > > HBaseIO support for SplittableDoFn introduced a new request type HBaseQuery, > however the attributes defined in that class are already available in > HBase.Read. Allowing users to define pipelines based on HBaseIO.Read allows > to create pipelines that can read from multiple clusters because the > Configuration now is part of the request object. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9470) :sdks:java:io:kinesis:test is flaky
[ https://issues.apache.org/jira/browse/BEAM-9470?focusedWorklogId=406080&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406080 ] ASF GitHub Bot logged work on BEAM-9470: Author: ASF GitHub Bot Created on: 19/Mar/20 10:59 Start Date: 19/Mar/20 10:59 Worklog Time Spent: 10m Work Description: aromanenko-dev commented on issue #11090: [BEAM-9470] :sdks:java:io:kinesis:test is flaky URL: https://github.com/apache/beam/pull/11090#issuecomment-601117023 Run Java_Examples_Dataflow PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 406080) Time Spent: 9.5h (was: 9h 20m) > :sdks:java:io:kinesis:test is flaky > --- > > Key: BEAM-9470 > URL: https://issues.apache.org/jira/browse/BEAM-9470 > Project: Beam > Issue Type: Test > Components: io-java-kinesis >Reporter: Etienne Chauchot >Assignee: Jonothan Farr >Priority: Major > Time Spent: 9.5h > Remaining Estimate: 0h > > [https://scans.gradle.com/s/b4jmmu72ku5jc/console-log?task=:sdks:java:io:kinesis:test] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9279) Make HBase.ReadAll based on Reads instead of HBaseQuery
[ https://issues.apache.org/jira/browse/BEAM-9279?focusedWorklogId=406083&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406083 ] ASF GitHub Bot logged work on BEAM-9279: Author: ASF GitHub Bot Created on: 19/Mar/20 11:05 Start Date: 19/Mar/20 11:05 Worklog Time Spent: 10m Work Description: iemejia commented on issue #10815: [BEAM-9279] Make HBase.ReadAll based on Reads instead of HBaseQuery URL: https://github.com/apache/beam/pull/10815#issuecomment-601119412 Thanks Alexey! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 406083) Time Spent: 3.5h (was: 3h 20m) > Make HBase.ReadAll based on Reads instead of HBaseQuery > --- > > Key: BEAM-9279 > URL: https://issues.apache.org/jira/browse/BEAM-9279 > Project: Beam > Issue Type: Improvement > Components: io-java-hbase >Reporter: Ismaël Mejía >Assignee: Ismaël Mejía >Priority: Minor > Fix For: 2.21.0 > > Time Spent: 3.5h > Remaining Estimate: 0h > > HBaseIO support for SplittableDoFn introduced a new request type HBaseQuery, > however the attributes defined in that class are already available in > HBase.Read. Allowing users to define pipelines based on HBaseIO.Read allows > to create pipelines that can read from multiple clusters because the > Configuration now is part of the request object. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9470) :sdks:java:io:kinesis:test is flaky
[ https://issues.apache.org/jira/browse/BEAM-9470?focusedWorklogId=406085&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406085 ] ASF GitHub Bot logged work on BEAM-9470: Author: ASF GitHub Bot Created on: 19/Mar/20 11:09 Start Date: 19/Mar/20 11:09 Worklog Time Spent: 10m Work Description: aromanenko-dev commented on issue #11090: [BEAM-9470] :sdks:java:io:kinesis:test is flaky URL: https://github.com/apache/beam/pull/11090#issuecomment-601120924 Run Java_Examples_Dataflow PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 406085) Time Spent: 9h 40m (was: 9.5h) > :sdks:java:io:kinesis:test is flaky > --- > > Key: BEAM-9470 > URL: https://issues.apache.org/jira/browse/BEAM-9470 > Project: Beam > Issue Type: Test > Components: io-java-kinesis >Reporter: Etienne Chauchot >Assignee: Jonothan Farr >Priority: Major > Time Spent: 9h 40m > Remaining Estimate: 0h > > [https://scans.gradle.com/s/b4jmmu72ku5jc/console-log?task=:sdks:java:io:kinesis:test] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9470) :sdks:java:io:kinesis:test is flaky
[ https://issues.apache.org/jira/browse/BEAM-9470?focusedWorklogId=406086&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406086 ] ASF GitHub Bot logged work on BEAM-9470: Author: ASF GitHub Bot Created on: 19/Mar/20 11:09 Start Date: 19/Mar/20 11:09 Worklog Time Spent: 10m Work Description: aromanenko-dev commented on issue #11090: [BEAM-9470] :sdks:java:io:kinesis:test is flaky URL: https://github.com/apache/beam/pull/11090#issuecomment-601121044 Run Java_Examples_Dataflow PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 406086) Time Spent: 9h 50m (was: 9h 40m) > :sdks:java:io:kinesis:test is flaky > --- > > Key: BEAM-9470 > URL: https://issues.apache.org/jira/browse/BEAM-9470 > Project: Beam > Issue Type: Test > Components: io-java-kinesis >Reporter: Etienne Chauchot >Assignee: Jonothan Farr >Priority: Major > Time Spent: 9h 50m > Remaining Estimate: 0h > > [https://scans.gradle.com/s/b4jmmu72ku5jc/console-log?task=:sdks:java:io:kinesis:test] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9470) :sdks:java:io:kinesis:test is flaky
[ https://issues.apache.org/jira/browse/BEAM-9470?focusedWorklogId=406087&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406087 ] ASF GitHub Bot logged work on BEAM-9470: Author: ASF GitHub Bot Created on: 19/Mar/20 11:09 Start Date: 19/Mar/20 11:09 Worklog Time Spent: 10m Work Description: aromanenko-dev commented on issue #11090: [BEAM-9470] :sdks:java:io:kinesis:test is flaky URL: https://github.com/apache/beam/pull/11090#issuecomment-601117023 Run Java_Examples_Dataflow PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 406087) Time Spent: 10h (was: 9h 50m) > :sdks:java:io:kinesis:test is flaky > --- > > Key: BEAM-9470 > URL: https://issues.apache.org/jira/browse/BEAM-9470 > Project: Beam > Issue Type: Test > Components: io-java-kinesis >Reporter: Etienne Chauchot >Assignee: Jonothan Farr >Priority: Major > Time Spent: 10h > Remaining Estimate: 0h > > [https://scans.gradle.com/s/b4jmmu72ku5jc/console-log?task=:sdks:java:io:kinesis:test] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9470) :sdks:java:io:kinesis:test is flaky
[ https://issues.apache.org/jira/browse/BEAM-9470?focusedWorklogId=406088&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406088 ] ASF GitHub Bot logged work on BEAM-9470: Author: ASF GitHub Bot Created on: 19/Mar/20 11:09 Start Date: 19/Mar/20 11:09 Worklog Time Spent: 10m Work Description: aromanenko-dev commented on issue #11090: [BEAM-9470] :sdks:java:io:kinesis:test is flaky URL: https://github.com/apache/beam/pull/11090#issuecomment-601120924 Run Java_Examples_Dataflow PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 406088) Time Spent: 10h 10m (was: 10h) > :sdks:java:io:kinesis:test is flaky > --- > > Key: BEAM-9470 > URL: https://issues.apache.org/jira/browse/BEAM-9470 > Project: Beam > Issue Type: Test > Components: io-java-kinesis >Reporter: Etienne Chauchot >Assignee: Jonothan Farr >Priority: Major > Time Spent: 10h 10m > Remaining Estimate: 0h > > [https://scans.gradle.com/s/b4jmmu72ku5jc/console-log?task=:sdks:java:io:kinesis:test] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9470) :sdks:java:io:kinesis:test is flaky
[ https://issues.apache.org/jira/browse/BEAM-9470?focusedWorklogId=406089&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406089 ] ASF GitHub Bot logged work on BEAM-9470: Author: ASF GitHub Bot Created on: 19/Mar/20 11:10 Start Date: 19/Mar/20 11:10 Worklog Time Spent: 10m Work Description: aromanenko-dev commented on issue #11090: [BEAM-9470] :sdks:java:io:kinesis:test is flaky URL: https://github.com/apache/beam/pull/11090#issuecomment-601121044 Run Java_Examples_Dataflow PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 406089) Time Spent: 10h 20m (was: 10h 10m) > :sdks:java:io:kinesis:test is flaky > --- > > Key: BEAM-9470 > URL: https://issues.apache.org/jira/browse/BEAM-9470 > Project: Beam > Issue Type: Test > Components: io-java-kinesis >Reporter: Etienne Chauchot >Assignee: Jonothan Farr >Priority: Major > Time Spent: 10h 20m > Remaining Estimate: 0h > > [https://scans.gradle.com/s/b4jmmu72ku5jc/console-log?task=:sdks:java:io:kinesis:test] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9470) :sdks:java:io:kinesis:test is flaky
[ https://issues.apache.org/jira/browse/BEAM-9470?focusedWorklogId=406092&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406092 ] ASF GitHub Bot logged work on BEAM-9470: Author: ASF GitHub Bot Created on: 19/Mar/20 11:17 Start Date: 19/Mar/20 11:17 Worklog Time Spent: 10m Work Description: aromanenko-dev commented on issue #11090: [BEAM-9470] :sdks:java:io:kinesis:test is flaky URL: https://github.com/apache/beam/pull/11090#issuecomment-601123987 Run Java_Examples_Dataflow PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 406092) Time Spent: 10.5h (was: 10h 20m) > :sdks:java:io:kinesis:test is flaky > --- > > Key: BEAM-9470 > URL: https://issues.apache.org/jira/browse/BEAM-9470 > Project: Beam > Issue Type: Test > Components: io-java-kinesis >Reporter: Etienne Chauchot >Assignee: Jonothan Farr >Priority: Major > Time Spent: 10.5h > Remaining Estimate: 0h > > [https://scans.gradle.com/s/b4jmmu72ku5jc/console-log?task=:sdks:java:io:kinesis:test] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9470) :sdks:java:io:kinesis:test is flaky
[ https://issues.apache.org/jira/browse/BEAM-9470?focusedWorklogId=406093&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406093 ] ASF GitHub Bot logged work on BEAM-9470: Author: ASF GitHub Bot Created on: 19/Mar/20 11:18 Start Date: 19/Mar/20 11:18 Worklog Time Spent: 10m Work Description: aromanenko-dev commented on issue #11090: [BEAM-9470] :sdks:java:io:kinesis:test is flaky URL: https://github.com/apache/beam/pull/11090#issuecomment-601124492 Run Java_Examples_Dataflow PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 406093) Time Spent: 10h 40m (was: 10.5h) > :sdks:java:io:kinesis:test is flaky > --- > > Key: BEAM-9470 > URL: https://issues.apache.org/jira/browse/BEAM-9470 > Project: Beam > Issue Type: Test > Components: io-java-kinesis >Reporter: Etienne Chauchot >Assignee: Jonothan Farr >Priority: Major > Time Spent: 10h 40m > Remaining Estimate: 0h > > [https://scans.gradle.com/s/b4jmmu72ku5jc/console-log?task=:sdks:java:io:kinesis:test] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9470) :sdks:java:io:kinesis:test is flaky
[ https://issues.apache.org/jira/browse/BEAM-9470?focusedWorklogId=406096&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406096 ] ASF GitHub Bot logged work on BEAM-9470: Author: ASF GitHub Bot Created on: 19/Mar/20 11:19 Start Date: 19/Mar/20 11:19 Worklog Time Spent: 10m Work Description: aromanenko-dev commented on issue #11090: [BEAM-9470] :sdks:java:io:kinesis:test is flaky URL: https://github.com/apache/beam/pull/11090#issuecomment-601124492 Run Java_Examples_Dataflow PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 406096) Time Spent: 11h 10m (was: 11h) > :sdks:java:io:kinesis:test is flaky > --- > > Key: BEAM-9470 > URL: https://issues.apache.org/jira/browse/BEAM-9470 > Project: Beam > Issue Type: Test > Components: io-java-kinesis >Reporter: Etienne Chauchot >Assignee: Jonothan Farr >Priority: Major > Time Spent: 11h 10m > Remaining Estimate: 0h > > [https://scans.gradle.com/s/b4jmmu72ku5jc/console-log?task=:sdks:java:io:kinesis:test] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9470) :sdks:java:io:kinesis:test is flaky
[ https://issues.apache.org/jira/browse/BEAM-9470?focusedWorklogId=406095&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406095 ] ASF GitHub Bot logged work on BEAM-9470: Author: ASF GitHub Bot Created on: 19/Mar/20 11:19 Start Date: 19/Mar/20 11:19 Worklog Time Spent: 10m Work Description: aromanenko-dev commented on issue #11090: [BEAM-9470] :sdks:java:io:kinesis:test is flaky URL: https://github.com/apache/beam/pull/11090#issuecomment-601123987 Run Java_Examples_Dataflow PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 406095) Time Spent: 11h (was: 10h 50m) > :sdks:java:io:kinesis:test is flaky > --- > > Key: BEAM-9470 > URL: https://issues.apache.org/jira/browse/BEAM-9470 > Project: Beam > Issue Type: Test > Components: io-java-kinesis >Reporter: Etienne Chauchot >Assignee: Jonothan Farr >Priority: Major > Time Spent: 11h > Remaining Estimate: 0h > > [https://scans.gradle.com/s/b4jmmu72ku5jc/console-log?task=:sdks:java:io:kinesis:test] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9470) :sdks:java:io:kinesis:test is flaky
[ https://issues.apache.org/jira/browse/BEAM-9470?focusedWorklogId=406094&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406094 ] ASF GitHub Bot logged work on BEAM-9470: Author: ASF GitHub Bot Created on: 19/Mar/20 11:19 Start Date: 19/Mar/20 11:19 Worklog Time Spent: 10m Work Description: aromanenko-dev commented on issue #11090: [BEAM-9470] :sdks:java:io:kinesis:test is flaky URL: https://github.com/apache/beam/pull/11090#issuecomment-601124858 Run Java_Examples_Dataflow PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 406094) Time Spent: 10h 50m (was: 10h 40m) > :sdks:java:io:kinesis:test is flaky > --- > > Key: BEAM-9470 > URL: https://issues.apache.org/jira/browse/BEAM-9470 > Project: Beam > Issue Type: Test > Components: io-java-kinesis >Reporter: Etienne Chauchot >Assignee: Jonothan Farr >Priority: Major > Time Spent: 10h 50m > Remaining Estimate: 0h > > [https://scans.gradle.com/s/b4jmmu72ku5jc/console-log?task=:sdks:java:io:kinesis:test] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8972) Add a Jenkins job running Combine load test on Java with Flink in Portability mode
[ https://issues.apache.org/jira/browse/BEAM-8972?focusedWorklogId=406099&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406099 ] ASF GitHub Bot logged work on BEAM-8972: Author: ASF GitHub Bot Created on: 19/Mar/20 11:35 Start Date: 19/Mar/20 11:35 Worklog Time Spent: 10m Work Description: mwalenia commented on issue #10386: [BEAM-8972] Add Jenkins job with Combine test for portable Java URL: https://github.com/apache/beam/pull/10386#issuecomment-601130890 Run Load Tests Java Combine Portable Flink Batch This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 406099) Time Spent: 21h 20m (was: 21h 10m) > Add a Jenkins job running Combine load test on Java with Flink in Portability > mode > -- > > Key: BEAM-8972 > URL: https://issues.apache.org/jira/browse/BEAM-8972 > Project: Beam > Issue Type: Improvement > Components: testing >Reporter: Michał Walenia >Assignee: Michał Walenia >Priority: Minor > Time Spent: 21h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (BEAM-9555) Kotlin example incompatible with Gradle 6
Xuebing Li created BEAM-9555:
Summary: Kotlin example incompatible with Gradle 6
Key: BEAM-9555
URL: https://issues.apache.org/jira/browse/BEAM-9555
Project: Beam
Issue Type: Bug
Components: examples-java
Reporter: Xuebing Li
When compiling Kotlin example with Gradle 6, the following error appears. Note
the "What went wrong" section.
{noformat}
$ gradle build
Starting a Gradle Daemon, 1 incompatible and 1 stopped Daemons could not be
reused, use --status for details
> Task :buildSrc:spotlessGroovy FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':buildSrc:spotlessGroovy'.
> You must declare outputs or use `TaskOutputs.upToDateWhen()` when using the
> incremental task API
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/6.2.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 4s
{noformat}
The cause is likely outdated version of spotless. [This GitHub
issue|https://github.com/diffplug/spotless/issues/440] mentions the exact same
error.
P.S. I chose _examples-java_ as this Jira issue's component because
_examples-kotlin_ is not an option.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9044) BIP-1: Convert protobuf options to Schema options
[ https://issues.apache.org/jira/browse/BEAM-9044?focusedWorklogId=406128&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406128 ] ASF GitHub Bot logged work on BEAM-9044: Author: ASF GitHub Bot Created on: 19/Mar/20 12:31 Start Date: 19/Mar/20 12:31 Worklog Time Spent: 10m Work Description: alexvanboxel commented on issue #10529: [BEAM-9044] Protobuf options to Schema options URL: https://github.com/apache/beam/pull/10529#issuecomment-601151797 Run Java_Examples_Dataflow PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 406128) Time Spent: 20m (was: 10m) > BIP-1: Convert protobuf options to Schema options > - > > Key: BEAM-9044 > URL: https://issues.apache.org/jira/browse/BEAM-9044 > Project: Beam > Issue Type: Sub-task > Components: extensions-java-protobuf >Reporter: Alex Van Boxel >Assignee: Alex Van Boxel >Priority: Minor > Time Spent: 20m > Remaining Estimate: 0h > > Protobuf has a rich metadata system called options. This system is fully > typed and matches Beams Schema Option system. For now we can only convert the > following protobuf options: > * File Options -> _Beam doesn't have this concept_ > * Message Options -> *Beam Schema Options* > * Field Options -> *Beam Schema Options* > * Enum Options -> _This can only be done when logical type options are > available_ > * EnumValue Options -> _This can only be done when logical type options are > available_ > * Service Options -> _Beam doesn't have this concept_ > * Method Options -> _Beam doesn't have this concept_ -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (BEAM-2466) Add Kafka Streams runner
[ https://issues.apache.org/jira/browse/BEAM-2466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17062529#comment-17062529 ] Artur Burtsev commented on BEAM-2466: - Hi, any plans to release Kafka Streams runner? We are running our services on Kafka Streams for over a year now, as our service grows we are looking for more features. GCP is already available for us to use, also our infra teams are working on managed Flink setup. If Beam supports Kafka Streams it will simplify a lot our migration to GCP or Flink, that's the whole point of Beam I believe. Having Kafka Streams runner should be also beneficial for Beam in terms of acquiring new customers. Kafka Streams requires minimal operation efforts to run and main it, comparing to existing runners. Pretty powerful at the same time. Which makes it a perfect candidate to start a project. > Add Kafka Streams runner > > > Key: BEAM-2466 > URL: https://issues.apache.org/jira/browse/BEAM-2466 > Project: Beam > Issue Type: Wish > Components: runner-ideas >Reporter: Lorand Peter Kasler >Assignee: Kyle Winkelman >Priority: Minor > > Kafka Streams (https://kafka.apache.org/documentation/streams) has more and > more features that could make it a viable candidate for a streaming runner. > It uses DataFlow-like model -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9470) :sdks:java:io:kinesis:test is flaky
[ https://issues.apache.org/jira/browse/BEAM-9470?focusedWorklogId=406140&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406140 ] ASF GitHub Bot logged work on BEAM-9470: Author: ASF GitHub Bot Created on: 19/Mar/20 12:46 Start Date: 19/Mar/20 12:46 Worklog Time Spent: 10m Work Description: aromanenko-dev commented on pull request #11090: [BEAM-9470] :sdks:java:io:kinesis:test is flaky URL: https://github.com/apache/beam/pull/11090 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 406140) Time Spent: 11h 20m (was: 11h 10m) > :sdks:java:io:kinesis:test is flaky > --- > > Key: BEAM-9470 > URL: https://issues.apache.org/jira/browse/BEAM-9470 > Project: Beam > Issue Type: Test > Components: io-java-kinesis >Reporter: Etienne Chauchot >Assignee: Jonothan Farr >Priority: Major > Time Spent: 11h 20m > Remaining Estimate: 0h > > [https://scans.gradle.com/s/b4jmmu72ku5jc/console-log?task=:sdks:java:io:kinesis:test] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8972) Add a Jenkins job running Combine load test on Java with Flink in Portability mode
[ https://issues.apache.org/jira/browse/BEAM-8972?focusedWorklogId=406142&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406142 ] ASF GitHub Bot logged work on BEAM-8972: Author: ASF GitHub Bot Created on: 19/Mar/20 12:47 Start Date: 19/Mar/20 12:47 Worklog Time Spent: 10m Work Description: mwalenia commented on issue #10386: [BEAM-8972] Add Jenkins job with Combine test for portable Java URL: https://github.com/apache/beam/pull/10386#issuecomment-601158639 run seed job This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 406142) Time Spent: 21.5h (was: 21h 20m) > Add a Jenkins job running Combine load test on Java with Flink in Portability > mode > -- > > Key: BEAM-8972 > URL: https://issues.apache.org/jira/browse/BEAM-8972 > Project: Beam > Issue Type: Improvement > Components: testing >Reporter: Michał Walenia >Assignee: Michał Walenia >Priority: Minor > Time Spent: 21.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (BEAM-9470) :sdks:java:io:kinesis:test is flaky
[ https://issues.apache.org/jira/browse/BEAM-9470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Romanenko resolved BEAM-9470. Fix Version/s: 2.21.0 Resolution: Fixed > :sdks:java:io:kinesis:test is flaky > --- > > Key: BEAM-9470 > URL: https://issues.apache.org/jira/browse/BEAM-9470 > Project: Beam > Issue Type: Test > Components: io-java-kinesis >Reporter: Etienne Chauchot >Assignee: Jonothan Farr >Priority: Major > Fix For: 2.21.0 > > Time Spent: 11h 20m > Remaining Estimate: 0h > > [https://scans.gradle.com/s/b4jmmu72ku5jc/console-log?task=:sdks:java:io:kinesis:test] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (BEAM-9550) beam_PostCommit_Python_Chicago_Taxi_Flink OOM
[ https://issues.apache.org/jira/browse/BEAM-9550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17062534#comment-17062534 ] Kamil Wasilewski commented on BEAM-9550: Thanks. At the moment, not only Chicago Taxi is affected, but also every load test running on Flink runner (for example: [https://builds.apache.org/job/beam_LoadTests_Python_ParDo_Flink_Batch/175/)] This is happening since last Flink 1.10 update. I'll take a look at it. > beam_PostCommit_Python_Chicago_Taxi_Flink OOM > - > > Key: BEAM-9550 > URL: https://issues.apache.org/jira/browse/BEAM-9550 > Project: Beam > Issue Type: Bug > Components: runner-flink, test-failures >Reporter: Kyle Weaver >Assignee: Kamil Wasilewski >Priority: Major > Labels: currently-failing > > https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ > The following error has been occurring consistently for several days: > 07:57:26 ERROR:root:java.lang.OutOfMemoryError: Metaspace > 07:57:27 Traceback (most recent call last): > 07:57:27 File "tfdv_analyze_and_validate.py", line 227, in > 07:57:27 main() > 07:57:27 File "tfdv_analyze_and_validate.py", line 212, in main > 07:57:27 project=known_args.metric_reporting_project) > 07:57:27 File "tfdv_analyze_and_validate.py", line 132, in compute_stats > 07:57:27 result.wait_until_finish() > 07:57:27 File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Chicago_Taxi_Flink/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/apache_beam/runners/portability/portable_runner.py", > line 545, in wait_until_finish > 07:57:27 (self._job_id, self._state, self._last_error_message())) > 07:57:27 RuntimeError: Pipeline > chicago-taxi-tfdv-20200317-144954-eval_9742ac2b-26bf-4d1d-835e-572d4efacfcb > failed in state FAILED: java.lang.OutOfMemoryError: Metaspace -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8173) Filesystems.matchSingleFileSpec throws away the actual failure exception
[ https://issues.apache.org/jira/browse/BEAM-8173?focusedWorklogId=406149&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406149 ] ASF GitHub Bot logged work on BEAM-8173: Author: ASF GitHub Bot Created on: 19/Mar/20 12:55 Start Date: 19/Mar/20 12:55 Worklog Time Spent: 10m Work Description: piotr-szuberski commented on issue #11137: [BEAM-8173] Fix shadowing the actual failure exception in Filesystems… URL: https://github.com/apache/beam/pull/11137#issuecomment-601161707 @mwalenia at least tests passed This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 406149) Time Spent: 3h 10m (was: 3h) > Filesystems.matchSingleFileSpec throws away the actual failure exception > > > Key: BEAM-8173 > URL: https://issues.apache.org/jira/browse/BEAM-8173 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Kenneth Knowles >Assignee: Piotr Szuberski >Priority: Major > Labels: easyfix, starter > Time Spent: 3h 10m > Remaining Estimate: 0h > > At [1] the result of a non-OK match causes an exception to be thrown. But the > exception does not include the actual cause of the failure, so it cannot be > efficiently debugged. It appears that the design of MatchResult is that it > should call metadata() without bothering to check status, so that the > underlying exception can be re-raised and caught and put in the chain of > causes, as it should be. > [1] > https://github.com/apache/beam/blob/c2f0d282337f3ae0196a7717712396a5a41fdde1/sdks/java/core/src/main/java/org/apache/beam/sdk/io/FileSystems.java#L190 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8173) Filesystems.matchSingleFileSpec throws away the actual failure exception
[ https://issues.apache.org/jira/browse/BEAM-8173?focusedWorklogId=406156&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406156 ] ASF GitHub Bot logged work on BEAM-8173: Author: ASF GitHub Bot Created on: 19/Mar/20 13:09 Start Date: 19/Mar/20 13:09 Worklog Time Spent: 10m Work Description: piotr-szuberski commented on issue #11137: [BEAM-8173] Fix shadowing the actual failure exception in Filesystems… URL: https://github.com/apache/beam/pull/11137#issuecomment-601161707 @mwalenia at least the tests passed This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 406156) Time Spent: 3h 20m (was: 3h 10m) > Filesystems.matchSingleFileSpec throws away the actual failure exception > > > Key: BEAM-8173 > URL: https://issues.apache.org/jira/browse/BEAM-8173 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Kenneth Knowles >Assignee: Piotr Szuberski >Priority: Major > Labels: easyfix, starter > Time Spent: 3h 20m > Remaining Estimate: 0h > > At [1] the result of a non-OK match causes an exception to be thrown. But the > exception does not include the actual cause of the failure, so it cannot be > efficiently debugged. It appears that the design of MatchResult is that it > should call metadata() without bothering to check status, so that the > underlying exception can be re-raised and caught and put in the chain of > causes, as it should be. > [1] > https://github.com/apache/beam/blob/c2f0d282337f3ae0196a7717712396a5a41fdde1/sdks/java/core/src/main/java/org/apache/beam/sdk/io/FileSystems.java#L190 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (BEAM-9550) beam_PostCommit_Python_Chicago_Taxi_Flink OOM
[ https://issues.apache.org/jira/browse/BEAM-9550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamil Wasilewski updated BEAM-9550: --- Status: Open (was: Triage Needed) > beam_PostCommit_Python_Chicago_Taxi_Flink OOM > - > > Key: BEAM-9550 > URL: https://issues.apache.org/jira/browse/BEAM-9550 > Project: Beam > Issue Type: Bug > Components: runner-flink, test-failures >Reporter: Kyle Weaver >Assignee: Kamil Wasilewski >Priority: Major > Labels: currently-failing > > https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ > The following error has been occurring consistently for several days: > 07:57:26 ERROR:root:java.lang.OutOfMemoryError: Metaspace > 07:57:27 Traceback (most recent call last): > 07:57:27 File "tfdv_analyze_and_validate.py", line 227, in > 07:57:27 main() > 07:57:27 File "tfdv_analyze_and_validate.py", line 212, in main > 07:57:27 project=known_args.metric_reporting_project) > 07:57:27 File "tfdv_analyze_and_validate.py", line 132, in compute_stats > 07:57:27 result.wait_until_finish() > 07:57:27 File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Chicago_Taxi_Flink/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/apache_beam/runners/portability/portable_runner.py", > line 545, in wait_until_finish > 07:57:27 (self._job_id, self._state, self._last_error_message())) > 07:57:27 RuntimeError: Pipeline > chicago-taxi-tfdv-20200317-144954-eval_9742ac2b-26bf-4d1d-835e-572d4efacfcb > failed in state FAILED: java.lang.OutOfMemoryError: Metaspace -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (BEAM-9556) java.lang.IllegalAccessError with apache beam 2.19 and BigQuery
Stefano created BEAM-9556:
-
Summary: java.lang.IllegalAccessError with apache beam 2.19 and
BigQuery
Key: BEAM-9556
URL: https://issues.apache.org/jira/browse/BEAM-9556
Project: Beam
Issue Type: Bug
Components: beam-community
Affects Versions: 2.19.0
Environment: jdk8 kotlin 1.3.70
Reporter: Stefano
Assignee: Aizhamal Nurmamat kyzy
Hi
I was updating my dataflow code from Apache beam 2.18 to Apache beam 2.19
As soon as I upgrade the dependency in my build.grade and try to run the flow
with the direct runner to test, the code immediately fails with this exception:
{code:java}
Exception in thread "main" com.google.common.util.concurrent.ExecutionError:
java.lang.IllegalAccessError: tried to access field
io.opencensus.trace.unsafe.ContextUtils.CONTEXT_SPAN_KEY from class
io.grpc.internal.CensusTracingModule$TracingClientInterceptor
at
com.google.common.util.concurrent.Futures.wrapAndThrowUnchecked(Futures.java:1183
undefined)
at
com.google.common.util.concurrent.Futures.getUnchecked(Futures.java:1176
undefined)
at
com.google.api.gax.rpc.ApiExceptions.callAndTranslateApiException(ApiExceptions.java:53
undefined)
at com.google.api.gax.rpc.UnaryCallable.call(UnaryCallable.java:112
undefined)
at
com.google.cloud.bigquery.storage.v1beta1.BigQueryStorageClient.createReadSession(BigQueryStorageClient.java:237
undefined)
at
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$StorageClientImpl.createReadSession(BigQueryServicesImpl.java:1044
undefined)
at
org.apache.beam.sdk.io.gcp.bigquery.BigQueryStorageSourceBase.split(BigQueryStorageSourceBase.java:139
undefined)
at
org.apache.beam.sdk.io.gcp.bigquery.BigQueryStorageTableSource.split(BigQueryStorageTableSource.java:42
undefined)
at
org.apache.beam.runners.direct.BoundedReadEvaluatorFactory$InputProvider.getInitialInputs(BoundedReadEvaluatorFactory.java:214
undefined)
at
org.apache.beam.runners.direct.ReadEvaluatorFactory$InputProvider.getInitialInputs(ReadEvaluatorFactory.java:89
undefined)
at
org.apache.beam.runners.direct.RootProviderRegistry.getInitialInputs(RootProviderRegistry.java:76
undefined)
at
org.apache.beam.runners.direct.ExecutorServiceParallelExecutor.start(ExecutorServiceParallelExecutor.java:155
undefined)
at
org.apache.beamners.direct.DirectRunner.run.run(DirectRunner.java:208 undefined)
at org.apache.beamners.direct.DirectRunner.run.run(DirectRunner.java:67
undefined)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:315 undefined)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:301 undefined)
at reports.pipelines.LeadsEnricher.runPipeline(LeadsEnricher.kt:87
undefined)
at reports.PipelineRunnerKt.main(PipelineRunner.kt:13 undefined)
Caused by: java.lang.IllegalAccessError: tried to access field
io.opencensus.trace.unsafe.ContextUtils.CONTEXT_SPAN_KEY from class
io.grpc.internal.CensusTracingModule$TracingClientInterceptor
at
io.grpc.internal.CensusTracingModule$TracingClientInterceptor.interceptCall(CensusTracingModule.java:384
undefined)
at
io.grpc.ClientInterceptors$InterceptorChannel.newCall(ClientInterceptors.java:156
undefined)
at
io.grpc.internal.CensusStatsModule$StatsClientInterceptor.interceptCall(CensusStatsModule.java:685
undefined)
at
io.grpc.ClientInterceptors$InterceptorChannel.newCall(ClientInterceptors.java:156
undefined)
at
com.google.api.gax.grpc.GrpcHeaderInterceptor.interceptCall(GrpcHeaderInterceptor.java:81
undefined)
at
io.grpc.ClientInterceptors$InterceptorChannel.newCall(ClientInterceptors.java:156
undefined)
at
com.google.api.gax.grpc.GrpcMetadataHandlerInterceptor.interceptCall(GrpcMetadataHandlerInterceptor.java:55
undefined)
at
io.grpc.ClientInterceptors$InterceptorChannel.newCall(ClientInterceptors.java:156
undefined)
at
io.grpc.internal.ManagedChannelImpl.newCall(ManagedChannelImpl.java:766
undefined)
at
io.grpc.internal.ForwardingManagedChannel.newCall(ForwardingManagedChannel.java:63
undefined)
at com.google.api.gax.grpc.ChannelPool.newCall(ChannelPool.java:77
undefined)
at
com.google.api.gax.grpc.GrpcClientCalls.newCall(GrpcClientCalls.java:88
undefined)
at
com.google.api.gax.grpc.GrpcDirectCallable.futureCall(GrpcDirectCallable.java:58
undefined)
at
com.google.api.gax.grpc.GrpcUnaryRequestParamCallable.futureCall(GrpcUnaryRequestParamCallable.java:65
undefined)
at
com.google.api.gax.grpc.GrpcExceptionCallable.futureCall(GrpcExceptionCallable.java:64
undefined)
at com.google.api.gax.rpc.AttemptCallable.call(AttemptCallable.java:86
undefined)
at
com.google.api.gax.rpc.RetryingCallable.futureCall(RetryingCallable.jav
[jira] [Work logged] (BEAM-9421) AI Platform pipeline patterns
[ https://issues.apache.org/jira/browse/BEAM-9421?focusedWorklogId=406177&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406177 ] ASF GitHub Bot logged work on BEAM-9421: Author: ASF GitHub Bot Created on: 19/Mar/20 13:43 Start Date: 19/Mar/20 13:43 Worklog Time Spent: 10m Work Description: kamilwu commented on pull request #11075: [BEAM-9421] Website section that describes getting predictions using AI Platform Prediciton URL: https://github.com/apache/beam/pull/11075#discussion_r395033918 ## File path: website/src/documentation/patterns/ai-platform.md ## @@ -0,0 +1,87 @@ +--- +layout: section +title: "AI Platform integration patterns" +section_menu: section-menu/documentation.html +permalink: /documentation/patterns/ai-platform/ +--- + + +# AI Platform integration patterns + +This page describes common patterns in pipelines with Google AI Platform transforms. + + + Adapt for: + +Java SDK +Python SDK + + + +## Getting predictions + +This section shows how to use a cloud-hosted machine learning model to make predictions about new data using Google Cloud AI Platform Prediction within Beam's pipeline. Review comment: No, AI Platform is a service that manages cloud-hosted machine learning models. How about something like this? > This section shows how to use Google Cloud AI Platform Prediction to make predictions about new data from a cloud-hosted machine learning model. I'll also put an url link to an overview of AI Platform Prediction. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 406177) Time Spent: 5h 10m (was: 5h) > AI Platform pipeline patterns > - > > Key: BEAM-9421 > URL: https://issues.apache.org/jira/browse/BEAM-9421 > Project: Beam > Issue Type: Sub-task > Components: website >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Labels: pipeline-patterns > Time Spent: 5h 10m > Remaining Estimate: 0h > > New pipeline patterns should be contributed to the Beam's website in order to > demonstrate how newly implemented Google Cloud AI PTransforms can be used in > pipelines. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8972) Add a Jenkins job running Combine load test on Java with Flink in Portability mode
[ https://issues.apache.org/jira/browse/BEAM-8972?focusedWorklogId=406178&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406178 ] ASF GitHub Bot logged work on BEAM-8972: Author: ASF GitHub Bot Created on: 19/Mar/20 13:43 Start Date: 19/Mar/20 13:43 Worklog Time Spent: 10m Work Description: mwalenia commented on issue #10386: [BEAM-8972] Add Jenkins job with Combine test for portable Java URL: https://github.com/apache/beam/pull/10386#issuecomment-601185879 Run Load Tests Java Combine Portable Flink Batch This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 406178) Time Spent: 21h 40m (was: 21.5h) > Add a Jenkins job running Combine load test on Java with Flink in Portability > mode > -- > > Key: BEAM-8972 > URL: https://issues.apache.org/jira/browse/BEAM-8972 > Project: Beam > Issue Type: Improvement > Components: testing >Reporter: Michał Walenia >Assignee: Michał Walenia >Priority: Minor > Time Spent: 21h 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9421) AI Platform pipeline patterns
[ https://issues.apache.org/jira/browse/BEAM-9421?focusedWorklogId=406179&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406179 ] ASF GitHub Bot logged work on BEAM-9421: Author: ASF GitHub Bot Created on: 19/Mar/20 13:48 Start Date: 19/Mar/20 13:48 Worklog Time Spent: 10m Work Description: kamilwu commented on pull request #11075: [BEAM-9421] Website section that describes getting predictions using AI Platform Prediciton URL: https://github.com/apache/beam/pull/11075#discussion_r395037728 ## File path: website/src/documentation/patterns/ai-platform.md ## @@ -0,0 +1,87 @@ +--- +layout: section +title: "AI Platform integration patterns" +section_menu: section-menu/documentation.html +permalink: /documentation/patterns/ai-platform/ +--- + + +# AI Platform integration patterns + +This page describes common patterns in pipelines with Google AI Platform transforms. + + + Adapt for: + +Java SDK +Python SDK + + + +## Getting predictions + +This section shows how to use a cloud-hosted machine learning model to make predictions about new data using Google Cloud AI Platform Prediction within Beam's pipeline. + +[tfx_bsl](https://github.com/tensorflow/tfx-bsl) is a library that provides `RunInference` Beam's PTransform. `RunInference` is a PTransform able to perform two types of inference. One of them can use a service endpoint. When using a service endpoint, the transform takes a PCollection of type `tf.train.Example` and, for each element, sends a request to Google Cloud AI Platform Prediction service. The transform produces a PCollection of type `PredictLog` which contains predictions. Review comment: Fair enough, this is an another evidence that this mention causes confusion. I'll remote it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 406179) Time Spent: 5h 20m (was: 5h 10m) > AI Platform pipeline patterns > - > > Key: BEAM-9421 > URL: https://issues.apache.org/jira/browse/BEAM-9421 > Project: Beam > Issue Type: Sub-task > Components: website >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Labels: pipeline-patterns > Time Spent: 5h 20m > Remaining Estimate: 0h > > New pipeline patterns should be contributed to the Beam's website in order to > demonstrate how newly implemented Google Cloud AI PTransforms can be used in > pipelines. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8173) Filesystems.matchSingleFileSpec throws away the actual failure exception
[ https://issues.apache.org/jira/browse/BEAM-8173?focusedWorklogId=406185&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406185 ] ASF GitHub Bot logged work on BEAM-8173: Author: ASF GitHub Bot Created on: 19/Mar/20 14:04 Start Date: 19/Mar/20 14:04 Worklog Time Spent: 10m Work Description: piotr-szuberski commented on issue #11137: [BEAM-8173] Fix shadowing the actual failure exception in Filesystems… URL: https://github.com/apache/beam/pull/11137#issuecomment-601161707 @mwalenia the tests passed This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 406185) Time Spent: 3.5h (was: 3h 20m) > Filesystems.matchSingleFileSpec throws away the actual failure exception > > > Key: BEAM-8173 > URL: https://issues.apache.org/jira/browse/BEAM-8173 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Kenneth Knowles >Assignee: Piotr Szuberski >Priority: Major > Labels: easyfix, starter > Time Spent: 3.5h > Remaining Estimate: 0h > > At [1] the result of a non-OK match causes an exception to be thrown. But the > exception does not include the actual cause of the failure, so it cannot be > efficiently debugged. It appears that the design of MatchResult is that it > should call metadata() without bothering to check status, so that the > underlying exception can be re-raised and caught and put in the chain of > causes, as it should be. > [1] > https://github.com/apache/beam/blob/c2f0d282337f3ae0196a7717712396a5a41fdde1/sdks/java/core/src/main/java/org/apache/beam/sdk/io/FileSystems.java#L190 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9421) AI Platform pipeline patterns
[ https://issues.apache.org/jira/browse/BEAM-9421?focusedWorklogId=406187&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406187 ] ASF GitHub Bot logged work on BEAM-9421: Author: ASF GitHub Bot Created on: 19/Mar/20 14:04 Start Date: 19/Mar/20 14:04 Worklog Time Spent: 10m Work Description: kamilwu commented on pull request #11075: [BEAM-9421] Website section that describes getting predictions using AI Platform Prediciton URL: https://github.com/apache/beam/pull/11075#discussion_r395049430 ## File path: website/src/documentation/patterns/ai-platform.md ## @@ -0,0 +1,87 @@ +--- +layout: section +title: "AI Platform integration patterns" +section_menu: section-menu/documentation.html +permalink: /documentation/patterns/ai-platform/ +--- + + +# AI Platform integration patterns + +This page describes common patterns in pipelines with Google AI Platform transforms. + + + Adapt for: + +Java SDK +Python SDK + + + +## Getting predictions + +This section shows how to use a cloud-hosted machine learning model to make predictions about new data using Google Cloud AI Platform Prediction within Beam's pipeline. + +[tfx_bsl](https://github.com/tensorflow/tfx-bsl) is a library that provides `RunInference` Beam's PTransform. `RunInference` is a PTransform able to perform two types of inference. One of them can use a service endpoint. When using a service endpoint, the transform takes a PCollection of type `tf.train.Example` and, for each element, sends a request to Google Cloud AI Platform Prediction service. The transform produces a PCollection of type `PredictLog` which contains predictions. + +Before getting started, deploy a machine learning model to the cloud. The cloud service manages the infrastructure needed to handle prediction requests in both efficient and scalable way. Only Tensorflow models are supported. For more information, see [Exporting a SavedModel for prediction](https://cloud.google.com/ai-platform/prediction/docs/exporting-savedmodel-for-prediction). Review comment: Alright, but I'd rather keep this sentence: "only TensorFlow models are supported by the transform" anyway. AI Platform, unlike `RunInference`, doesn't only support TensorFlow models. It's important to put an emphasis on that. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 406187) Time Spent: 5.5h (was: 5h 20m) > AI Platform pipeline patterns > - > > Key: BEAM-9421 > URL: https://issues.apache.org/jira/browse/BEAM-9421 > Project: Beam > Issue Type: Sub-task > Components: website >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Labels: pipeline-patterns > Time Spent: 5.5h > Remaining Estimate: 0h > > New pipeline patterns should be contributed to the Beam's website in order to > demonstrate how newly implemented Google Cloud AI PTransforms can be used in > pipelines. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-1819) Key should be available in @OnTimer methods
[
https://issues.apache.org/jira/browse/BEAM-1819?focusedWorklogId=406222&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406222
]
ASF GitHub Bot logged work on BEAM-1819:
Author: ASF GitHub Bot
Created on: 19/Mar/20 14:05
Start Date: 19/Mar/20 14:05
Worklog Time Spent: 10m
Work Description: aromanenko-dev commented on issue #11154: [BEAM-1819]
Key should be available in @OnTimer methods
URL: https://github.com/apache/beam/pull/11154#issuecomment-601197814
retest this please
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 406222)
Time Spent: 1h 20m (was: 1h 10m)
> Key should be available in @OnTimer methods
> ---
>
> Key: BEAM-1819
> URL: https://issues.apache.org/jira/browse/BEAM-1819
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Thomas Groh
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Every timer firing has an associated key. This key should be available when
> the timer is delivered to a user's {{DoFn}}, so they don't have to store it
> in state.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-1819) Key should be available in @OnTimer methods
[
https://issues.apache.org/jira/browse/BEAM-1819?focusedWorklogId=406221&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406221
]
ASF GitHub Bot logged work on BEAM-1819:
Author: ASF GitHub Bot
Created on: 19/Mar/20 14:05
Start Date: 19/Mar/20 14:05
Worklog Time Spent: 10m
Work Description: aromanenko-dev commented on issue #11154: [BEAM-1819]
Key should be available in @OnTimer methods
URL: https://github.com/apache/beam/pull/11154#issuecomment-601197747
retest this please
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 406221)
Time Spent: 1h 10m (was: 1h)
> Key should be available in @OnTimer methods
> ---
>
> Key: BEAM-1819
> URL: https://issues.apache.org/jira/browse/BEAM-1819
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Thomas Groh
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> Every timer firing has an associated key. This key should be available when
> the timer is delivered to a user's {{DoFn}}, so they don't have to store it
> in state.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-1819) Key should be available in @OnTimer methods
[
https://issues.apache.org/jira/browse/BEAM-1819?focusedWorklogId=406189&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406189
]
ASF GitHub Bot logged work on BEAM-1819:
Author: ASF GitHub Bot
Created on: 19/Mar/20 14:05
Start Date: 19/Mar/20 14:05
Worklog Time Spent: 10m
Work Description: aromanenko-dev commented on issue #11154: [BEAM-1819]
Key should be available in @OnTimer methods
URL: https://github.com/apache/beam/pull/11154#issuecomment-601197519
retest this please
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 406189)
Time Spent: 1h (was: 50m)
> Key should be available in @OnTimer methods
> ---
>
> Key: BEAM-1819
> URL: https://issues.apache.org/jira/browse/BEAM-1819
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Thomas Groh
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Every timer firing has an associated key. This key should be available when
> the timer is delivered to a user's {{DoFn}}, so they don't have to store it
> in state.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-1819) Key should be available in @OnTimer methods
[
https://issues.apache.org/jira/browse/BEAM-1819?focusedWorklogId=406224&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406224
]
ASF GitHub Bot logged work on BEAM-1819:
Author: ASF GitHub Bot
Created on: 19/Mar/20 14:06
Start Date: 19/Mar/20 14:06
Worklog Time Spent: 10m
Work Description: aromanenko-dev commented on issue #11154: [BEAM-1819]
Key should be available in @OnTimer methods
URL: https://github.com/apache/beam/pull/11154#issuecomment-601197747
retest this please
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 406224)
Time Spent: 1h 40m (was: 1.5h)
> Key should be available in @OnTimer methods
> ---
>
> Key: BEAM-1819
> URL: https://issues.apache.org/jira/browse/BEAM-1819
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Thomas Groh
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> Every timer firing has an associated key. This key should be available when
> the timer is delivered to a user's {{DoFn}}, so they don't have to store it
> in state.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-1819) Key should be available in @OnTimer methods
[
https://issues.apache.org/jira/browse/BEAM-1819?focusedWorklogId=406225&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406225
]
ASF GitHub Bot logged work on BEAM-1819:
Author: ASF GitHub Bot
Created on: 19/Mar/20 14:06
Start Date: 19/Mar/20 14:06
Worklog Time Spent: 10m
Work Description: aromanenko-dev commented on issue #11154: [BEAM-1819]
Key should be available in @OnTimer methods
URL: https://github.com/apache/beam/pull/11154#issuecomment-601198377
retest this please
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 406225)
Time Spent: 1h 50m (was: 1h 40m)
> Key should be available in @OnTimer methods
> ---
>
> Key: BEAM-1819
> URL: https://issues.apache.org/jira/browse/BEAM-1819
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Thomas Groh
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Every timer firing has an associated key. This key should be available when
> the timer is delivered to a user's {{DoFn}}, so they don't have to store it
> in state.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-1819) Key should be available in @OnTimer methods
[
https://issues.apache.org/jira/browse/BEAM-1819?focusedWorklogId=406223&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406223
]
ASF GitHub Bot logged work on BEAM-1819:
Author: ASF GitHub Bot
Created on: 19/Mar/20 14:06
Start Date: 19/Mar/20 14:06
Worklog Time Spent: 10m
Work Description: aromanenko-dev commented on issue #11154: [BEAM-1819]
Key should be available in @OnTimer methods
URL: https://github.com/apache/beam/pull/11154#issuecomment-601197519
retest this please
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 406223)
Time Spent: 1.5h (was: 1h 20m)
> Key should be available in @OnTimer methods
> ---
>
> Key: BEAM-1819
> URL: https://issues.apache.org/jira/browse/BEAM-1819
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Thomas Groh
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Every timer firing has an associated key. This key should be available when
> the timer is delivered to a user's {{DoFn}}, so they don't have to store it
> in state.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-1819) Key should be available in @OnTimer methods
[
https://issues.apache.org/jira/browse/BEAM-1819?focusedWorklogId=406226&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406226
]
ASF GitHub Bot logged work on BEAM-1819:
Author: ASF GitHub Bot
Created on: 19/Mar/20 14:07
Start Date: 19/Mar/20 14:07
Worklog Time Spent: 10m
Work Description: aromanenko-dev commented on issue #11154: [BEAM-1819]
Key should be available in @OnTimer methods
URL: https://github.com/apache/beam/pull/11154#issuecomment-601198848
retest this please
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 406226)
Time Spent: 2h (was: 1h 50m)
> Key should be available in @OnTimer methods
> ---
>
> Key: BEAM-1819
> URL: https://issues.apache.org/jira/browse/BEAM-1819
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Thomas Groh
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 2h
> Remaining Estimate: 0h
>
> Every timer firing has an associated key. This key should be available when
> the timer is delivered to a user's {{DoFn}}, so they don't have to store it
> in state.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-1819) Key should be available in @OnTimer methods
[
https://issues.apache.org/jira/browse/BEAM-1819?focusedWorklogId=406229&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406229
]
ASF GitHub Bot logged work on BEAM-1819:
Author: ASF GitHub Bot
Created on: 19/Mar/20 14:08
Start Date: 19/Mar/20 14:08
Worklog Time Spent: 10m
Work Description: aromanenko-dev commented on issue #11154: [BEAM-1819]
Key should be available in @OnTimer methods
URL: https://github.com/apache/beam/pull/11154#issuecomment-601198377
retest this please
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 406229)
Time Spent: 2.5h (was: 2h 20m)
> Key should be available in @OnTimer methods
> ---
>
> Key: BEAM-1819
> URL: https://issues.apache.org/jira/browse/BEAM-1819
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Thomas Groh
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> Every timer firing has an associated key. This key should be available when
> the timer is delivered to a user's {{DoFn}}, so they don't have to store it
> in state.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-1819) Key should be available in @OnTimer methods
[
https://issues.apache.org/jira/browse/BEAM-1819?focusedWorklogId=406227&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406227
]
ASF GitHub Bot logged work on BEAM-1819:
Author: ASF GitHub Bot
Created on: 19/Mar/20 14:08
Start Date: 19/Mar/20 14:08
Worklog Time Spent: 10m
Work Description: aromanenko-dev commented on issue #11154: [BEAM-1819]
Key should be available in @OnTimer methods
URL: https://github.com/apache/beam/pull/11154#issuecomment-601199144
Run Java PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 406227)
Time Spent: 2h 10m (was: 2h)
> Key should be available in @OnTimer methods
> ---
>
> Key: BEAM-1819
> URL: https://issues.apache.org/jira/browse/BEAM-1819
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Thomas Groh
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Every timer firing has an associated key. This key should be available when
> the timer is delivered to a user's {{DoFn}}, so they don't have to store it
> in state.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-1819) Key should be available in @OnTimer methods
[
https://issues.apache.org/jira/browse/BEAM-1819?focusedWorklogId=406228&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406228
]
ASF GitHub Bot logged work on BEAM-1819:
Author: ASF GitHub Bot
Created on: 19/Mar/20 14:08
Start Date: 19/Mar/20 14:08
Worklog Time Spent: 10m
Work Description: aromanenko-dev commented on issue #11154: [BEAM-1819]
Key should be available in @OnTimer methods
URL: https://github.com/apache/beam/pull/11154#issuecomment-601197814
retest this please
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 406228)
Time Spent: 2h 20m (was: 2h 10m)
> Key should be available in @OnTimer methods
> ---
>
> Key: BEAM-1819
> URL: https://issues.apache.org/jira/browse/BEAM-1819
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Thomas Groh
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> Every timer firing has an associated key. This key should be available when
> the timer is delivered to a user's {{DoFn}}, so they don't have to store it
> in state.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-1819) Key should be available in @OnTimer methods
[
https://issues.apache.org/jira/browse/BEAM-1819?focusedWorklogId=406231&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406231
]
ASF GitHub Bot logged work on BEAM-1819:
Author: ASF GitHub Bot
Created on: 19/Mar/20 14:09
Start Date: 19/Mar/20 14:09
Worklog Time Spent: 10m
Work Description: aromanenko-dev commented on issue #11154: [BEAM-1819]
Key should be available in @OnTimer methods
URL: https://github.com/apache/beam/pull/11154#issuecomment-601199519
Run Java PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 406231)
Time Spent: 2h 40m (was: 2.5h)
> Key should be available in @OnTimer methods
> ---
>
> Key: BEAM-1819
> URL: https://issues.apache.org/jira/browse/BEAM-1819
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Thomas Groh
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> Every timer firing has an associated key. This key should be available when
> the timer is delivered to a user's {{DoFn}}, so they don't have to store it
> in state.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-1819) Key should be available in @OnTimer methods
[
https://issues.apache.org/jira/browse/BEAM-1819?focusedWorklogId=406236&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406236
]
ASF GitHub Bot logged work on BEAM-1819:
Author: ASF GitHub Bot
Created on: 19/Mar/20 14:09
Start Date: 19/Mar/20 14:09
Worklog Time Spent: 10m
Work Description: aromanenko-dev commented on issue #11154: [BEAM-1819]
Key should be available in @OnTimer methods
URL: https://github.com/apache/beam/pull/11154#issuecomment-601199613
Run Java PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 406236)
Time Spent: 3.5h (was: 3h 20m)
> Key should be available in @OnTimer methods
> ---
>
> Key: BEAM-1819
> URL: https://issues.apache.org/jira/browse/BEAM-1819
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Thomas Groh
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> Every timer firing has an associated key. This key should be available when
> the timer is delivered to a user's {{DoFn}}, so they don't have to store it
> in state.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-1819) Key should be available in @OnTimer methods
[
https://issues.apache.org/jira/browse/BEAM-1819?focusedWorklogId=406232&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406232
]
ASF GitHub Bot logged work on BEAM-1819:
Author: ASF GitHub Bot
Created on: 19/Mar/20 14:09
Start Date: 19/Mar/20 14:09
Worklog Time Spent: 10m
Work Description: aromanenko-dev commented on issue #11154: [BEAM-1819]
Key should be available in @OnTimer methods
URL: https://github.com/apache/beam/pull/11154#issuecomment-601199613
Run Java PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 406232)
Time Spent: 2h 50m (was: 2h 40m)
> Key should be available in @OnTimer methods
> ---
>
> Key: BEAM-1819
> URL: https://issues.apache.org/jira/browse/BEAM-1819
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Thomas Groh
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 2h 50m
> Remaining Estimate: 0h
>
> Every timer firing has an associated key. This key should be available when
> the timer is delivered to a user's {{DoFn}}, so they don't have to store it
> in state.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-1819) Key should be available in @OnTimer methods
[
https://issues.apache.org/jira/browse/BEAM-1819?focusedWorklogId=406234&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406234
]
ASF GitHub Bot logged work on BEAM-1819:
Author: ASF GitHub Bot
Created on: 19/Mar/20 14:09
Start Date: 19/Mar/20 14:09
Worklog Time Spent: 10m
Work Description: aromanenko-dev commented on issue #11154: [BEAM-1819]
Key should be available in @OnTimer methods
URL: https://github.com/apache/beam/pull/11154#issuecomment-601199144
Run Java PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 406234)
Time Spent: 3h 10m (was: 3h)
> Key should be available in @OnTimer methods
> ---
>
> Key: BEAM-1819
> URL: https://issues.apache.org/jira/browse/BEAM-1819
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Thomas Groh
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> Every timer firing has an associated key. This key should be available when
> the timer is delivered to a user's {{DoFn}}, so they don't have to store it
> in state.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-1819) Key should be available in @OnTimer methods
[
https://issues.apache.org/jira/browse/BEAM-1819?focusedWorklogId=406235&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406235
]
ASF GitHub Bot logged work on BEAM-1819:
Author: ASF GitHub Bot
Created on: 19/Mar/20 14:09
Start Date: 19/Mar/20 14:09
Worklog Time Spent: 10m
Work Description: aromanenko-dev commented on issue #11154: [BEAM-1819]
Key should be available in @OnTimer methods
URL: https://github.com/apache/beam/pull/11154#issuecomment-601199519
Run Java PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 406235)
Time Spent: 3h 20m (was: 3h 10m)
> Key should be available in @OnTimer methods
> ---
>
> Key: BEAM-1819
> URL: https://issues.apache.org/jira/browse/BEAM-1819
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Thomas Groh
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> Every timer firing has an associated key. This key should be available when
> the timer is delivered to a user's {{DoFn}}, so they don't have to store it
> in state.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-1819) Key should be available in @OnTimer methods
[
https://issues.apache.org/jira/browse/BEAM-1819?focusedWorklogId=406233&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406233
]
ASF GitHub Bot logged work on BEAM-1819:
Author: ASF GitHub Bot
Created on: 19/Mar/20 14:09
Start Date: 19/Mar/20 14:09
Worklog Time Spent: 10m
Work Description: aromanenko-dev commented on issue #11154: [BEAM-1819]
Key should be available in @OnTimer methods
URL: https://github.com/apache/beam/pull/11154#issuecomment-601199717
Run Java PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 406233)
Time Spent: 3h (was: 2h 50m)
> Key should be available in @OnTimer methods
> ---
>
> Key: BEAM-1819
> URL: https://issues.apache.org/jira/browse/BEAM-1819
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Thomas Groh
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 3h
> Remaining Estimate: 0h
>
> Every timer firing has an associated key. This key should be available when
> the timer is delivered to a user's {{DoFn}}, so they don't have to store it
> in state.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-1819) Key should be available in @OnTimer methods
[
https://issues.apache.org/jira/browse/BEAM-1819?focusedWorklogId=406238&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406238
]
ASF GitHub Bot logged work on BEAM-1819:
Author: ASF GitHub Bot
Created on: 19/Mar/20 14:10
Start Date: 19/Mar/20 14:10
Worklog Time Spent: 10m
Work Description: aromanenko-dev commented on issue #11154: [BEAM-1819]
Key should be available in @OnTimer methods
URL: https://github.com/apache/beam/pull/11154#issuecomment-601200225
Run Java PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 406238)
Time Spent: 3h 50m (was: 3h 40m)
> Key should be available in @OnTimer methods
> ---
>
> Key: BEAM-1819
> URL: https://issues.apache.org/jira/browse/BEAM-1819
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Thomas Groh
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 3h 50m
> Remaining Estimate: 0h
>
> Every timer firing has an associated key. This key should be available when
> the timer is delivered to a user's {{DoFn}}, so they don't have to store it
> in state.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-1819) Key should be available in @OnTimer methods
[
https://issues.apache.org/jira/browse/BEAM-1819?focusedWorklogId=406239&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406239
]
ASF GitHub Bot logged work on BEAM-1819:
Author: ASF GitHub Bot
Created on: 19/Mar/20 14:10
Start Date: 19/Mar/20 14:10
Worklog Time Spent: 10m
Work Description: aromanenko-dev commented on issue #11154: [BEAM-1819]
Key should be available in @OnTimer methods
URL: https://github.com/apache/beam/pull/11154#issuecomment-601199717
Run Java PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 406239)
Time Spent: 4h (was: 3h 50m)
> Key should be available in @OnTimer methods
> ---
>
> Key: BEAM-1819
> URL: https://issues.apache.org/jira/browse/BEAM-1819
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Thomas Groh
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 4h
> Remaining Estimate: 0h
>
> Every timer firing has an associated key. This key should be available when
> the timer is delivered to a user's {{DoFn}}, so they don't have to store it
> in state.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-1819) Key should be available in @OnTimer methods
[
https://issues.apache.org/jira/browse/BEAM-1819?focusedWorklogId=406240&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406240
]
ASF GitHub Bot logged work on BEAM-1819:
Author: ASF GitHub Bot
Created on: 19/Mar/20 14:10
Start Date: 19/Mar/20 14:10
Worklog Time Spent: 10m
Work Description: aromanenko-dev commented on issue #11154: [BEAM-1819]
Key should be available in @OnTimer methods
URL: https://github.com/apache/beam/pull/11154#issuecomment-601200096
Run Java PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 406240)
Time Spent: 4h 10m (was: 4h)
> Key should be available in @OnTimer methods
> ---
>
> Key: BEAM-1819
> URL: https://issues.apache.org/jira/browse/BEAM-1819
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Thomas Groh
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 4h 10m
> Remaining Estimate: 0h
>
> Every timer firing has an associated key. This key should be available when
> the timer is delivered to a user's {{DoFn}}, so they don't have to store it
> in state.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-1819) Key should be available in @OnTimer methods
[
https://issues.apache.org/jira/browse/BEAM-1819?focusedWorklogId=406237&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406237
]
ASF GitHub Bot logged work on BEAM-1819:
Author: ASF GitHub Bot
Created on: 19/Mar/20 14:10
Start Date: 19/Mar/20 14:10
Worklog Time Spent: 10m
Work Description: aromanenko-dev commented on issue #11154: [BEAM-1819]
Key should be available in @OnTimer methods
URL: https://github.com/apache/beam/pull/11154#issuecomment-601200096
Run Java PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 406237)
Time Spent: 3h 40m (was: 3.5h)
> Key should be available in @OnTimer methods
> ---
>
> Key: BEAM-1819
> URL: https://issues.apache.org/jira/browse/BEAM-1819
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Thomas Groh
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> Every timer firing has an associated key. This key should be available when
> the timer is delivered to a user's {{DoFn}}, so they don't have to store it
> in state.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (BEAM-9556) java.lang.IllegalAccessError with apache beam 2.19 and BigQuery
[
https://issues.apache.org/jira/browse/BEAM-9556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ismaël Mejía reassigned BEAM-9556:
--
Assignee: (was: Aizhamal Nurmamat kyzy)
> java.lang.IllegalAccessError with apache beam 2.19 and BigQuery
> ---
>
> Key: BEAM-9556
> URL: https://issues.apache.org/jira/browse/BEAM-9556
> Project: Beam
> Issue Type: Bug
> Components: beam-community
>Affects Versions: 2.19.0
> Environment: jdk8 kotlin 1.3.70
>Reporter: Stefano
>Priority: Major
>
> Hi
>
> I was updating my dataflow code from Apache beam 2.18 to Apache beam 2.19
> As soon as I upgrade the dependency in my build.grade and try to run the flow
> with the direct runner to test, the code immediately fails with this
> exception:
>
> {code:java}
> Exception in thread "main" com.google.common.util.concurrent.ExecutionError:
> java.lang.IllegalAccessError: tried to access field
> io.opencensus.trace.unsafe.ContextUtils.CONTEXT_SPAN_KEY from class
> io.grpc.internal.CensusTracingModule$TracingClientInterceptor
> at
> com.google.common.util.concurrent.Futures.wrapAndThrowUnchecked(Futures.java:1183
> undefined)
> at
> com.google.common.util.concurrent.Futures.getUnchecked(Futures.java:1176
> undefined)
> at
> com.google.api.gax.rpc.ApiExceptions.callAndTranslateApiException(ApiExceptions.java:53
> undefined)
> at com.google.api.gax.rpc.UnaryCallable.call(UnaryCallable.java:112
> undefined)
> at
> com.google.cloud.bigquery.storage.v1beta1.BigQueryStorageClient.createReadSession(BigQueryStorageClient.java:237
> undefined)
> at
> org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$StorageClientImpl.createReadSession(BigQueryServicesImpl.java:1044
> undefined)
> at
> org.apache.beam.sdk.io.gcp.bigquery.BigQueryStorageSourceBase.split(BigQueryStorageSourceBase.java:139
> undefined)
> at
> org.apache.beam.sdk.io.gcp.bigquery.BigQueryStorageTableSource.split(BigQueryStorageTableSource.java:42
> undefined)
> at
> org.apache.beam.runners.direct.BoundedReadEvaluatorFactory$InputProvider.getInitialInputs(BoundedReadEvaluatorFactory.java:214
> undefined)
> at
> org.apache.beam.runners.direct.ReadEvaluatorFactory$InputProvider.getInitialInputs(ReadEvaluatorFactory.java:89
> undefined)
> at
> org.apache.beam.runners.direct.RootProviderRegistry.getInitialInputs(RootProviderRegistry.java:76
> undefined)
> at
> org.apache.beam.runners.direct.ExecutorServiceParallelExecutor.start(ExecutorServiceParallelExecutor.java:155
> undefined)
> at
> org.apache.beamners.direct.DirectRunner.run.run(DirectRunner.java:208
> undefined)
> at org.apache.beamners.direct.DirectRunner.run.run(DirectRunner.java:67
> undefined)
> at org.apache.beam.sdk.Pipeline.run(Pipeline.java:315 undefined)
> at org.apache.beam.sdk.Pipeline.run(Pipeline.java:301 undefined)
> at reports.pipelines.LeadsEnricher.runPipeline(LeadsEnricher.kt:87
> undefined)
> at reports.PipelineRunnerKt.main(PipelineRunner.kt:13 undefined)
> Caused by: java.lang.IllegalAccessError: tried to access field
> io.opencensus.trace.unsafe.ContextUtils.CONTEXT_SPAN_KEY from class
> io.grpc.internal.CensusTracingModule$TracingClientInterceptor
> at
> io.grpc.internal.CensusTracingModule$TracingClientInterceptor.interceptCall(CensusTracingModule.java:384
> undefined)
> at
> io.grpc.ClientInterceptors$InterceptorChannel.newCall(ClientInterceptors.java:156
> undefined)
> at
> io.grpc.internal.CensusStatsModule$StatsClientInterceptor.interceptCall(CensusStatsModule.java:685
> undefined)
> at
> io.grpc.ClientInterceptors$InterceptorChannel.newCall(ClientInterceptors.java:156
> undefined)
> at
> com.google.api.gax.grpc.GrpcHeaderInterceptor.interceptCall(GrpcHeaderInterceptor.java:81
> undefined)
> at
> io.grpc.ClientInterceptors$InterceptorChannel.newCall(ClientInterceptors.java:156
> undefined)
> at
> com.google.api.gax.grpc.GrpcMetadataHandlerInterceptor.interceptCall(GrpcMetadataHandlerInterceptor.java:55
> undefined)
> at
> io.grpc.ClientInterceptors$InterceptorChannel.newCall(ClientInterceptors.java:156
> undefined)
> at
> io.grpc.internal.ManagedChannelImpl.newCall(ManagedChannelImpl.java:766
> undefined)
> at
> io.grpc.internal.ForwardingManagedChannel.newCall(ForwardingManagedChannel.java:63
> undefined)
> at com.google.api.gax.grpc.ChannelPool.newCall(ChannelPool.java:77
> undefined)
> at
> com.google.api.gax.grpc.GrpcClientCalls.newCall(GrpcClientCalls.java:88
> undefined)
> at
> com.google.api.gax.grpc.GrpcDirectCallable.futureCall(GrpcDirectCallable.java:58
> undefined)
> at
> com.google.api.gax.grpc.GrpcUnaryRequ
[jira] [Updated] (BEAM-9540) Rename beam:source:runner:0.1/beam:sink:runner:0.1 to beam:runner:source:v1/beam:runner:sink:v1
[ https://issues.apache.org/jira/browse/BEAM-9540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-9540: --- Status: Open (was: Triage Needed) > Rename beam:source:runner:0.1/beam:sink:runner:0.1 to > beam:runner:source:v1/beam:runner:sink:v1 > --- > > Key: BEAM-9540 > URL: https://issues.apache.org/jira/browse/BEAM-9540 > Project: Beam > Issue Type: Bug > Components: sdk-java-harness, sdk-py-harness >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Minor > Time Spent: 2.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (BEAM-9546) Support for batching a schema-aware PCollection and processing as a Dataframe
[ https://issues.apache.org/jira/browse/BEAM-9546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-9546: --- Status: Open (was: Triage Needed) > Support for batching a schema-aware PCollection and processing as a Dataframe > - > > Key: BEAM-9546 > URL: https://issues.apache.org/jira/browse/BEAM-9546 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Brian Hulette >Assignee: Brian Hulette >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (BEAM-9546) Support for batching a schema-aware PCollection and processing as a Dataframe
[ https://issues.apache.org/jira/browse/BEAM-9546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17062624#comment-17062624 ] Ismaël Mejía commented on BEAM-9546: Any particular reason for batching to be only for schema-aware PCollections? > Support for batching a schema-aware PCollection and processing as a Dataframe > - > > Key: BEAM-9546 > URL: https://issues.apache.org/jira/browse/BEAM-9546 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Brian Hulette >Assignee: Brian Hulette >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (BEAM-9547) Raise NotImplementedError for remaining pandas methods
[ https://issues.apache.org/jira/browse/BEAM-9547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-9547: --- Status: Open (was: Triage Needed) > Raise NotImplementedError for remaining pandas methods > -- > > Key: BEAM-9547 > URL: https://issues.apache.org/jira/browse/BEAM-9547 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Brian Hulette >Assignee: Brian Hulette >Priority: Major > > We should have an implementation for every DataFrame, Series, and GroupBy > method. Everything that's not actually implemented should get a default > implementation that raises NotImplementedError. > See https://github.com/apache/beam/pull/10757#discussion_r389132292 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (BEAM-9545) MVP: DataframeTransform
[ https://issues.apache.org/jira/browse/BEAM-9545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-9545: --- Status: Open (was: Triage Needed) > MVP: DataframeTransform > --- > > Key: BEAM-9545 > URL: https://issues.apache.org/jira/browse/BEAM-9545 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Brian Hulette >Assignee: Robert Bradshaw >Priority: Major > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (BEAM-9548) Bad error handling with errors from TestStreamService when using Interactive Beam
[ https://issues.apache.org/jira/browse/BEAM-9548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-9548: --- Status: Open (was: Triage Needed) > Bad error handling with errors from TestStreamService when using Interactive > Beam > - > > Key: BEAM-9548 > URL: https://issues.apache.org/jira/browse/BEAM-9548 > Project: Beam > Issue Type: Bug > Components: runner-py-interactive >Reporter: Sam Rohde >Assignee: Sam Rohde >Priority: Major > > The error handling when an error is generated on the GRPC server side is very > verbose and hides the problem. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (BEAM-9542) Where the BeamModulePlugin's force is needed?
[
https://issues.apache.org/jira/browse/BEAM-9542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ismaël Mejía updated BEAM-9542:
---
Status: Open (was: Triage Needed)
> Where the BeamModulePlugin's force is needed?
> -
>
> Key: BEAM-9542
> URL: https://issues.apache.org/jira/browse/BEAM-9542
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Followup of https://github.com/apache/beam/pull/11156#discussion_r394408735
> {noformat}
> > Task :sdks:java:core:compileTestJava FAILED
> /Users/suztomo/beam/sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/DeduplicateTest.java:21:
> error: cannot find symbol
> import static org.junit.Assert.assertThrows;
> ^
> symbol: static assertThrows
> location: class
> /Users/suztomo/beam/sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/splittabledofn/WatermarkEstimatorsTest.java:21:
> error: cannot find symbol
> import static org.junit.Assert.assertThrows;
> ^
> symbol: static assertThrows
> location: class
> Note: Some input files use or override a deprecated API.
> Note: Recompile with -Xlint:deprecation for details.
> 2 errors
> <-> 36% EXECUTING [19m 37s]
> {noformat}
> Memo for my Mac:
> {noformat}
> suztomo-macbookpro44% ./gradlew -p sdks/java check -x
> extensions:sql:zetasql:check -x harness:test -x io:jdbc:test -x
> io:kafka:test -x io:solr:test -x core:test
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-1819) Key should be available in @OnTimer methods
[
https://issues.apache.org/jira/browse/BEAM-1819?focusedWorklogId=406253&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406253
]
ASF GitHub Bot logged work on BEAM-1819:
Author: ASF GitHub Bot
Created on: 19/Mar/20 14:25
Start Date: 19/Mar/20 14:25
Worklog Time Spent: 10m
Work Description: iemejia commented on issue #11154: [BEAM-1819] Key
should be available in @OnTimer methods
URL: https://github.com/apache/beam/pull/11154#issuecomment-601208276
Run Apex ValidatesRunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 406253)
Time Spent: 4h 20m (was: 4h 10m)
> Key should be available in @OnTimer methods
> ---
>
> Key: BEAM-1819
> URL: https://issues.apache.org/jira/browse/BEAM-1819
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Thomas Groh
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 4h 20m
> Remaining Estimate: 0h
>
> Every timer firing has an associated key. This key should be available when
> the timer is delivered to a user's {{DoFn}}, so they don't have to store it
> in state.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-1819) Key should be available in @OnTimer methods
[
https://issues.apache.org/jira/browse/BEAM-1819?focusedWorklogId=406254&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406254
]
ASF GitHub Bot logged work on BEAM-1819:
Author: ASF GitHub Bot
Created on: 19/Mar/20 14:25
Start Date: 19/Mar/20 14:25
Worklog Time Spent: 10m
Work Description: iemejia commented on issue #11154: [BEAM-1819] Key
should be available in @OnTimer methods
URL: https://github.com/apache/beam/pull/11154#issuecomment-601208448
Run Dataflow ValidatesRunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 406254)
Time Spent: 4.5h (was: 4h 20m)
> Key should be available in @OnTimer methods
> ---
>
> Key: BEAM-1819
> URL: https://issues.apache.org/jira/browse/BEAM-1819
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Thomas Groh
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 4.5h
> Remaining Estimate: 0h
>
> Every timer firing has an associated key. This key should be available when
> the timer is delivered to a user's {{DoFn}}, so they don't have to store it
> in state.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (BEAM-9556) java.lang.IllegalAccessError with apache beam 2.19 and BigQuery
[
https://issues.apache.org/jira/browse/BEAM-9556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ismaël Mejía updated BEAM-9556:
---
Status: Open (was: Triage Needed)
> java.lang.IllegalAccessError with apache beam 2.19 and BigQuery
> ---
>
> Key: BEAM-9556
> URL: https://issues.apache.org/jira/browse/BEAM-9556
> Project: Beam
> Issue Type: Bug
> Components: beam-community
>Affects Versions: 2.19.0
> Environment: jdk8 kotlin 1.3.70
>Reporter: Stefano
>Priority: Major
>
> Hi
>
> I was updating my dataflow code from Apache beam 2.18 to Apache beam 2.19
> As soon as I upgrade the dependency in my build.grade and try to run the flow
> with the direct runner to test, the code immediately fails with this
> exception:
>
> {code:java}
> Exception in thread "main" com.google.common.util.concurrent.ExecutionError:
> java.lang.IllegalAccessError: tried to access field
> io.opencensus.trace.unsafe.ContextUtils.CONTEXT_SPAN_KEY from class
> io.grpc.internal.CensusTracingModule$TracingClientInterceptor
> at
> com.google.common.util.concurrent.Futures.wrapAndThrowUnchecked(Futures.java:1183
> undefined)
> at
> com.google.common.util.concurrent.Futures.getUnchecked(Futures.java:1176
> undefined)
> at
> com.google.api.gax.rpc.ApiExceptions.callAndTranslateApiException(ApiExceptions.java:53
> undefined)
> at com.google.api.gax.rpc.UnaryCallable.call(UnaryCallable.java:112
> undefined)
> at
> com.google.cloud.bigquery.storage.v1beta1.BigQueryStorageClient.createReadSession(BigQueryStorageClient.java:237
> undefined)
> at
> org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$StorageClientImpl.createReadSession(BigQueryServicesImpl.java:1044
> undefined)
> at
> org.apache.beam.sdk.io.gcp.bigquery.BigQueryStorageSourceBase.split(BigQueryStorageSourceBase.java:139
> undefined)
> at
> org.apache.beam.sdk.io.gcp.bigquery.BigQueryStorageTableSource.split(BigQueryStorageTableSource.java:42
> undefined)
> at
> org.apache.beam.runners.direct.BoundedReadEvaluatorFactory$InputProvider.getInitialInputs(BoundedReadEvaluatorFactory.java:214
> undefined)
> at
> org.apache.beam.runners.direct.ReadEvaluatorFactory$InputProvider.getInitialInputs(ReadEvaluatorFactory.java:89
> undefined)
> at
> org.apache.beam.runners.direct.RootProviderRegistry.getInitialInputs(RootProviderRegistry.java:76
> undefined)
> at
> org.apache.beam.runners.direct.ExecutorServiceParallelExecutor.start(ExecutorServiceParallelExecutor.java:155
> undefined)
> at
> org.apache.beamners.direct.DirectRunner.run.run(DirectRunner.java:208
> undefined)
> at org.apache.beamners.direct.DirectRunner.run.run(DirectRunner.java:67
> undefined)
> at org.apache.beam.sdk.Pipeline.run(Pipeline.java:315 undefined)
> at org.apache.beam.sdk.Pipeline.run(Pipeline.java:301 undefined)
> at reports.pipelines.LeadsEnricher.runPipeline(LeadsEnricher.kt:87
> undefined)
> at reports.PipelineRunnerKt.main(PipelineRunner.kt:13 undefined)
> Caused by: java.lang.IllegalAccessError: tried to access field
> io.opencensus.trace.unsafe.ContextUtils.CONTEXT_SPAN_KEY from class
> io.grpc.internal.CensusTracingModule$TracingClientInterceptor
> at
> io.grpc.internal.CensusTracingModule$TracingClientInterceptor.interceptCall(CensusTracingModule.java:384
> undefined)
> at
> io.grpc.ClientInterceptors$InterceptorChannel.newCall(ClientInterceptors.java:156
> undefined)
> at
> io.grpc.internal.CensusStatsModule$StatsClientInterceptor.interceptCall(CensusStatsModule.java:685
> undefined)
> at
> io.grpc.ClientInterceptors$InterceptorChannel.newCall(ClientInterceptors.java:156
> undefined)
> at
> com.google.api.gax.grpc.GrpcHeaderInterceptor.interceptCall(GrpcHeaderInterceptor.java:81
> undefined)
> at
> io.grpc.ClientInterceptors$InterceptorChannel.newCall(ClientInterceptors.java:156
> undefined)
> at
> com.google.api.gax.grpc.GrpcMetadataHandlerInterceptor.interceptCall(GrpcMetadataHandlerInterceptor.java:55
> undefined)
> at
> io.grpc.ClientInterceptors$InterceptorChannel.newCall(ClientInterceptors.java:156
> undefined)
> at
> io.grpc.internal.ManagedChannelImpl.newCall(ManagedChannelImpl.java:766
> undefined)
> at
> io.grpc.internal.ForwardingManagedChannel.newCall(ForwardingManagedChannel.java:63
> undefined)
> at com.google.api.gax.grpc.ChannelPool.newCall(ChannelPool.java:77
> undefined)
> at
> com.google.api.gax.grpc.GrpcClientCalls.newCall(GrpcClientCalls.java:88
> undefined)
> at
> com.google.api.gax.grpc.GrpcDirectCallable.futureCall(GrpcDirectCallable.java:58
> undefined)
> at
> com.google.api.gax.grpc.GrpcUnaryRequestParamCallable
[jira] [Updated] (BEAM-9552) TestPubsub ACK deadline is too short
[ https://issues.apache.org/jira/browse/BEAM-9552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-9552: --- Status: Open (was: Triage Needed) > TestPubsub ACK deadline is too short > > > Key: BEAM-9552 > URL: https://issues.apache.org/jira/browse/BEAM-9552 > Project: Beam > Issue Type: Improvement > Components: sdk-java-core, testing >Reporter: Brian Hulette >Assignee: Brian Hulette >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Increase to 60 to match PubsubUnboundedSource -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (BEAM-9556) java.lang.IllegalAccessError with apache beam 2.19 and BigQuery
[
https://issues.apache.org/jira/browse/BEAM-9556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ismaël Mejía resolved BEAM-9556.
Fix Version/s: Not applicable
Resolution: Fixed
Thanks for reporting this issue is already fixed and should be out soon as part
of 2.20.0
> java.lang.IllegalAccessError with apache beam 2.19 and BigQuery
> ---
>
> Key: BEAM-9556
> URL: https://issues.apache.org/jira/browse/BEAM-9556
> Project: Beam
> Issue Type: Bug
> Components: beam-community
>Affects Versions: 2.19.0
> Environment: jdk8 kotlin 1.3.70
>Reporter: Stefano
>Priority: Major
> Fix For: Not applicable
>
>
> Hi
>
> I was updating my dataflow code from Apache beam 2.18 to Apache beam 2.19
> As soon as I upgrade the dependency in my build.grade and try to run the flow
> with the direct runner to test, the code immediately fails with this
> exception:
>
> {code:java}
> Exception in thread "main" com.google.common.util.concurrent.ExecutionError:
> java.lang.IllegalAccessError: tried to access field
> io.opencensus.trace.unsafe.ContextUtils.CONTEXT_SPAN_KEY from class
> io.grpc.internal.CensusTracingModule$TracingClientInterceptor
> at
> com.google.common.util.concurrent.Futures.wrapAndThrowUnchecked(Futures.java:1183
> undefined)
> at
> com.google.common.util.concurrent.Futures.getUnchecked(Futures.java:1176
> undefined)
> at
> com.google.api.gax.rpc.ApiExceptions.callAndTranslateApiException(ApiExceptions.java:53
> undefined)
> at com.google.api.gax.rpc.UnaryCallable.call(UnaryCallable.java:112
> undefined)
> at
> com.google.cloud.bigquery.storage.v1beta1.BigQueryStorageClient.createReadSession(BigQueryStorageClient.java:237
> undefined)
> at
> org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$StorageClientImpl.createReadSession(BigQueryServicesImpl.java:1044
> undefined)
> at
> org.apache.beam.sdk.io.gcp.bigquery.BigQueryStorageSourceBase.split(BigQueryStorageSourceBase.java:139
> undefined)
> at
> org.apache.beam.sdk.io.gcp.bigquery.BigQueryStorageTableSource.split(BigQueryStorageTableSource.java:42
> undefined)
> at
> org.apache.beam.runners.direct.BoundedReadEvaluatorFactory$InputProvider.getInitialInputs(BoundedReadEvaluatorFactory.java:214
> undefined)
> at
> org.apache.beam.runners.direct.ReadEvaluatorFactory$InputProvider.getInitialInputs(ReadEvaluatorFactory.java:89
> undefined)
> at
> org.apache.beam.runners.direct.RootProviderRegistry.getInitialInputs(RootProviderRegistry.java:76
> undefined)
> at
> org.apache.beam.runners.direct.ExecutorServiceParallelExecutor.start(ExecutorServiceParallelExecutor.java:155
> undefined)
> at
> org.apache.beamners.direct.DirectRunner.run.run(DirectRunner.java:208
> undefined)
> at org.apache.beamners.direct.DirectRunner.run.run(DirectRunner.java:67
> undefined)
> at org.apache.beam.sdk.Pipeline.run(Pipeline.java:315 undefined)
> at org.apache.beam.sdk.Pipeline.run(Pipeline.java:301 undefined)
> at reports.pipelines.LeadsEnricher.runPipeline(LeadsEnricher.kt:87
> undefined)
> at reports.PipelineRunnerKt.main(PipelineRunner.kt:13 undefined)
> Caused by: java.lang.IllegalAccessError: tried to access field
> io.opencensus.trace.unsafe.ContextUtils.CONTEXT_SPAN_KEY from class
> io.grpc.internal.CensusTracingModule$TracingClientInterceptor
> at
> io.grpc.internal.CensusTracingModule$TracingClientInterceptor.interceptCall(CensusTracingModule.java:384
> undefined)
> at
> io.grpc.ClientInterceptors$InterceptorChannel.newCall(ClientInterceptors.java:156
> undefined)
> at
> io.grpc.internal.CensusStatsModule$StatsClientInterceptor.interceptCall(CensusStatsModule.java:685
> undefined)
> at
> io.grpc.ClientInterceptors$InterceptorChannel.newCall(ClientInterceptors.java:156
> undefined)
> at
> com.google.api.gax.grpc.GrpcHeaderInterceptor.interceptCall(GrpcHeaderInterceptor.java:81
> undefined)
> at
> io.grpc.ClientInterceptors$InterceptorChannel.newCall(ClientInterceptors.java:156
> undefined)
> at
> com.google.api.gax.grpc.GrpcMetadataHandlerInterceptor.interceptCall(GrpcMetadataHandlerInterceptor.java:55
> undefined)
> at
> io.grpc.ClientInterceptors$InterceptorChannel.newCall(ClientInterceptors.java:156
> undefined)
> at
> io.grpc.internal.ManagedChannelImpl.newCall(ManagedChannelImpl.java:766
> undefined)
> at
> io.grpc.internal.ForwardingManagedChannel.newCall(ForwardingManagedChannel.java:63
> undefined)
> at com.google.api.gax.grpc.ChannelPool.newCall(ChannelPool.java:77
> undefined)
> at
> com.google.api.gax.grpc.GrpcClientCalls.newCall(GrpcClientCalls.java:88
> undefined)
> at
> com.goog
[jira] [Work logged] (BEAM-9421) AI Platform pipeline patterns
[
https://issues.apache.org/jira/browse/BEAM-9421?focusedWorklogId=406258&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406258
]
ASF GitHub Bot logged work on BEAM-9421:
Author: ASF GitHub Bot
Created on: 19/Mar/20 14:34
Start Date: 19/Mar/20 14:34
Worklog Time Spent: 10m
Work Description: kamilwu commented on pull request #11075: [BEAM-9421]
Website section that describes getting predictions using AI Platform Prediciton
URL: https://github.com/apache/beam/pull/11075#discussion_r395071873
##
File path: website/src/documentation/patterns/overview.md
##
@@ -38,6 +38,9 @@ Pipeline patterns demonstrate common Beam use cases.
Pipeline patterns are based
**Custom window patterns** - Patterns for windowing functions
* [Using data to dynamically set session window gaps]({{ site.baseurl
}}/documentation/patterns/custom-windows/#using-data-to-dynamically-set-session-window-gaps)
+**AI Platform integration patterns** - Patterns for Google AI Platform
transforms
Review comment:
Looks good. One minor comment: there are more Google Cloud AI Platform
transforms. This pull request is just about Prediction, more patterns could be
added later. So I'd stick with `Patterns for Google Cloud AI Platform
transforms`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 406258)
Time Spent: 5h 40m (was: 5.5h)
> AI Platform pipeline patterns
> -
>
> Key: BEAM-9421
> URL: https://issues.apache.org/jira/browse/BEAM-9421
> Project: Beam
> Issue Type: Sub-task
> Components: website
>Reporter: Kamil Wasilewski
>Assignee: Kamil Wasilewski
>Priority: Major
> Labels: pipeline-patterns
> Time Spent: 5h 40m
> Remaining Estimate: 0h
>
> New pipeline patterns should be contributed to the Beam's website in order to
> demonstrate how newly implemented Google Cloud AI PTransforms can be used in
> pipelines.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (BEAM-9555) Kotlin example incompatible with Gradle 6
[
https://issues.apache.org/jira/browse/BEAM-9555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ismaël Mejía updated BEAM-9555:
---
Status: Open (was: Triage Needed)
> Kotlin example incompatible with Gradle 6
> -
>
> Key: BEAM-9555
> URL: https://issues.apache.org/jira/browse/BEAM-9555
> Project: Beam
> Issue Type: Bug
> Components: examples-java
>Reporter: Xuebing Li
>Priority: Major
>
> When compiling Kotlin example with Gradle 6, the following error appears.
> Note the "What went wrong" section.
> {noformat}
> $ gradle build
> Starting a Gradle Daemon, 1 incompatible and 1 stopped Daemons could not be
> reused, use --status for details
> > Task :buildSrc:spotlessGroovy FAILED
> FAILURE: Build failed with an exception.
> * What went wrong:
> Execution failed for task ':buildSrc:spotlessGroovy'.
> > You must declare outputs or use `TaskOutputs.upToDateWhen()` when using the
> > incremental task API
> * Try:
> Run with --stacktrace option to get the stack trace. Run with --info or
> --debug option to get more log output. Run with --scan to get full insights.
> * Get more help at https://help.gradle.org
> Deprecated Gradle features were used in this build, making it incompatible
> with Gradle 7.0.
> Use '--warning-mode all' to show the individual deprecation warnings.
> See
> https://docs.gradle.org/6.2.2/userguide/command_line_interface.html#sec:command_line_warnings
> BUILD FAILED in 4s
> {noformat}
> The cause is likely outdated version of spotless. [This GitHub
> issue|https://github.com/diffplug/spotless/issues/440] mentions the exact
> same error.
> P.S. I chose _examples-java_ as this Jira issue's component because
> _examples-kotlin_ is not an option.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Closed] (BEAM-9555) Kotlin example incompatible with Gradle 6
[
https://issues.apache.org/jira/browse/BEAM-9555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ismaël Mejía closed BEAM-9555.
--
> Kotlin example incompatible with Gradle 6
> -
>
> Key: BEAM-9555
> URL: https://issues.apache.org/jira/browse/BEAM-9555
> Project: Beam
> Issue Type: Bug
> Components: examples-java
>Reporter: Xuebing Li
>Priority: Major
> Fix For: Not applicable
>
>
> When compiling Kotlin example with Gradle 6, the following error appears.
> Note the "What went wrong" section.
> {noformat}
> $ gradle build
> Starting a Gradle Daemon, 1 incompatible and 1 stopped Daemons could not be
> reused, use --status for details
> > Task :buildSrc:spotlessGroovy FAILED
> FAILURE: Build failed with an exception.
> * What went wrong:
> Execution failed for task ':buildSrc:spotlessGroovy'.
> > You must declare outputs or use `TaskOutputs.upToDateWhen()` when using the
> > incremental task API
> * Try:
> Run with --stacktrace option to get the stack trace. Run with --info or
> --debug option to get more log output. Run with --scan to get full insights.
> * Get more help at https://help.gradle.org
> Deprecated Gradle features were used in this build, making it incompatible
> with Gradle 7.0.
> Use '--warning-mode all' to show the individual deprecation warnings.
> See
> https://docs.gradle.org/6.2.2/userguide/command_line_interface.html#sec:command_line_warnings
> BUILD FAILED in 4s
> {noformat}
> The cause is likely outdated version of spotless. [This GitHub
> issue|https://github.com/diffplug/spotless/issues/440] mentions the exact
> same error.
> P.S. I chose _examples-java_ as this Jira issue's component because
> _examples-kotlin_ is not an option.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (BEAM-9555) Kotlin example incompatible with Gradle 6
[
https://issues.apache.org/jira/browse/BEAM-9555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ismaël Mejía resolved BEAM-9555.
Fix Version/s: Not applicable
Resolution: Duplicate
Beam currently does not support gradle 6. WIP is part of BEAM-9456. Please
follow that ticket for more info or in case you want to contribute to it.
> Kotlin example incompatible with Gradle 6
> -
>
> Key: BEAM-9555
> URL: https://issues.apache.org/jira/browse/BEAM-9555
> Project: Beam
> Issue Type: Bug
> Components: examples-java
>Reporter: Xuebing Li
>Priority: Major
> Fix For: Not applicable
>
>
> When compiling Kotlin example with Gradle 6, the following error appears.
> Note the "What went wrong" section.
> {noformat}
> $ gradle build
> Starting a Gradle Daemon, 1 incompatible and 1 stopped Daemons could not be
> reused, use --status for details
> > Task :buildSrc:spotlessGroovy FAILED
> FAILURE: Build failed with an exception.
> * What went wrong:
> Execution failed for task ':buildSrc:spotlessGroovy'.
> > You must declare outputs or use `TaskOutputs.upToDateWhen()` when using the
> > incremental task API
> * Try:
> Run with --stacktrace option to get the stack trace. Run with --info or
> --debug option to get more log output. Run with --scan to get full insights.
> * Get more help at https://help.gradle.org
> Deprecated Gradle features were used in this build, making it incompatible
> with Gradle 7.0.
> Use '--warning-mode all' to show the individual deprecation warnings.
> See
> https://docs.gradle.org/6.2.2/userguide/command_line_interface.html#sec:command_line_warnings
> BUILD FAILED in 4s
> {noformat}
> The cause is likely outdated version of spotless. [This GitHub
> issue|https://github.com/diffplug/spotless/issues/440] mentions the exact
> same error.
> P.S. I chose _examples-java_ as this Jira issue's component because
> _examples-kotlin_ is not an option.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-1819) Key should be available in @OnTimer methods
[
https://issues.apache.org/jira/browse/BEAM-1819?focusedWorklogId=406273&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406273
]
ASF GitHub Bot logged work on BEAM-1819:
Author: ASF GitHub Bot
Created on: 19/Mar/20 14:43
Start Date: 19/Mar/20 14:43
Worklog Time Spent: 10m
Work Description: iemejia commented on issue #11154: [BEAM-1819] Key
should be available in @OnTimer methods
URL: https://github.com/apache/beam/pull/11154#issuecomment-601218474
Run Apex ValidatesRunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 406273)
Time Spent: 4h 40m (was: 4.5h)
> Key should be available in @OnTimer methods
> ---
>
> Key: BEAM-1819
> URL: https://issues.apache.org/jira/browse/BEAM-1819
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Thomas Groh
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 4h 40m
> Remaining Estimate: 0h
>
> Every timer firing has an associated key. This key should be available when
> the timer is delivered to a user's {{DoFn}}, so they don't have to store it
> in state.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-1819) Key should be available in @OnTimer methods
[
https://issues.apache.org/jira/browse/BEAM-1819?focusedWorklogId=406274&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406274
]
ASF GitHub Bot logged work on BEAM-1819:
Author: ASF GitHub Bot
Created on: 19/Mar/20 14:43
Start Date: 19/Mar/20 14:43
Worklog Time Spent: 10m
Work Description: iemejia commented on issue #11154: [BEAM-1819] Key
should be available in @OnTimer methods
URL: https://github.com/apache/beam/pull/11154#issuecomment-601218578
Run Dataflow ValidatesRunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 406274)
Time Spent: 4h 50m (was: 4h 40m)
> Key should be available in @OnTimer methods
> ---
>
> Key: BEAM-1819
> URL: https://issues.apache.org/jira/browse/BEAM-1819
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Thomas Groh
>Assignee: Shehzaad Nakhoda
>Priority: Major
> Time Spent: 4h 50m
> Remaining Estimate: 0h
>
> Every timer firing has an associated key. This key should be available when
> the timer is delivered to a user's {{DoFn}}, so they don't have to store it
> in state.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9085) Performance regression in np.random.RandomState() skews performance test results across Python 2/3 on Dataflow
[ https://issues.apache.org/jira/browse/BEAM-9085?focusedWorklogId=406275&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-406275 ] ASF GitHub Bot logged work on BEAM-9085: Author: ASF GitHub Bot Created on: 19/Mar/20 14:43 Start Date: 19/Mar/20 14:43 Worklog Time Spent: 10m Work Description: kamilwu commented on issue #11092: [BEAM-9085] Fix performance regression in SyntheticSource URL: https://github.com/apache/beam/pull/11092#issuecomment-601218678 I think branching the generation makes the most sense, since Py2 deprecation in Beam is closer than ever. I'll try to go back with a solution in the next week. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 406275) Time Spent: 6h 40m (was: 6.5h) > Performance regression in np.random.RandomState() skews performance test > results across Python 2/3 on Dataflow > -- > > Key: BEAM-9085 > URL: https://issues.apache.org/jira/browse/BEAM-9085 > Project: Beam > Issue Type: Bug > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Time Spent: 6h 40m > Remaining Estimate: 0h > > Tests show that the performance of core Beam operations in Python 3.x on > Dataflow can be a few time slower than in Python 2.7. We should investigate > what's the cause of the problem. > Currently, we have one ParDo test that is run both in Py3 and Py2 [1]. A > dashboard with runtime results can be found here [2]. > [1] sdks/python/apache_beam/testing/load_tests/pardo_test.py > [2] https://apache-beam-testing.appspot.com/explore?dashboard=5678187241537536 -- This message was sent by Atlassian Jira (v8.3.4#803005)
