[GitHub] [beam] KhaninArtur commented on pull request #16361: [BEAM-13543][Playground] Add logic of sending validation's output on the backend side

2022-01-07 Thread GitBox
KhaninArtur commented on pull request #16361: URL: https://github.com/apache/beam/pull/16361#issuecomment-1007302996 @aijamalnk @aaltay our team is on holiday until the 10th of January, that is why there is no recent activity in this PR. This and some other PRs are at the internal re

[GitHub] [beam] vitaly-ivanov opened a new pull request #16451: [BEAM-13609] Not all variable length types works properly for Oracle DB

2022-01-07 Thread GitBox
vitaly-ivanov opened a new pull request #16451: URL: https://github.com/apache/beam/pull/16451 For example RAW(16) fails while comparing field length because the Oracle JDBC driver returns precision = 0 but columnDisplaySize = 16. Looks like we can take into account columnDisplaySize

[GitHub] [beam] vitaly-ivanov commented on pull request #16451: [BEAM-13609] Not all variable length types works properly for Oracle DB

2022-01-07 Thread GitBox
vitaly-ivanov commented on pull request #16451: URL: https://github.com/apache/beam/pull/16451#issuecomment-1007431138 R: @timrobertson100 @pabloem -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [beam] suztomo commented on pull request #16441: [WIP][no need to review] Testing Google Cloud libraries dependencies

2022-01-07 Thread GitBox
suztomo commented on pull request #16441: URL: https://github.com/apache/beam/pull/16441#issuecomment-1007446146 Java_PVR_Flink_Docker (not my interest) failed and other test succeeded. -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

[GitHub] [beam] vitaly-ivanov commented on pull request #16451: [BEAM-13609] Not all variable length types works properly for Oracle DB

2022-01-07 Thread GitBox
vitaly-ivanov commented on pull request #16451: URL: https://github.com/apache/beam/pull/16451#issuecomment-1007446643 retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

[GitHub] [beam] RustedBones commented on pull request #16271: [BEAM-8388] Make sdk-java-core compatible with avro 1.8 & 1.9

2022-01-07 Thread GitBox
RustedBones commented on pull request #16271: URL: https://github.com/apache/beam/pull/16271#issuecomment-1007453667 There are no breaking changes to the `AvroCoder` directly. I'd like to add the possibility to create coders using datum factories. Users could customize (de)serialization

[GitHub] [beam] AlikRodriguez opened a new pull request #16452: [WIP] 13004 debezium test

2022-01-07 Thread GitBox
AlikRodriguez opened a new pull request #16452: URL: https://github.com/apache/beam/pull/16452 -- Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] [**Choose reviewer(s)**

[GitHub] [beam] aromanenko-dev opened a new pull request #16453: [adhoc] Fix BigTableIO description

2022-01-07 Thread GitBox
aromanenko-dev opened a new pull request #16453: URL: https://github.com/apache/beam/pull/16453 [adhoc] Fix BigTableIO description Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[GitHub] [beam] aromanenko-dev commented on pull request #16453: [adhoc] Fix BigTableIO description

2022-01-07 Thread GitBox
aromanenko-dev commented on pull request #16453: URL: https://github.com/apache/beam/pull/16453#issuecomment-1007561419 R: @chamikaramj ptal -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

[GitHub] [beam] chamikaramj commented on pull request #16453: [adhoc] Fix BigTableIO description

2022-01-07 Thread GitBox
chamikaramj commented on pull request #16453: URL: https://github.com/apache/beam/pull/16453#issuecomment-1007564804 LGTM. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

[GitHub] [beam] laraschmidt commented on pull request #15275: [BEAM-7386] Adding EventTimeEquiJoin

2022-01-07 Thread GitBox
laraschmidt commented on pull request #15275: URL: https://github.com/apache/beam/pull/15275#issuecomment-1007579994 The other one is in but it does not work for portable runner because making it work for protable runner breaks dataflow on portable runner. :) But the changes can actu

[GitHub] [beam] aromanenko-dev merged pull request #16453: [adhoc] Fix BigTableIO description

2022-01-07 Thread GitBox
aromanenko-dev merged pull request #16453: URL: https://github.com/apache/beam/pull/16453 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-u

[GitHub] [beam] aromanenko-dev commented on pull request #16454: [BEAM-4868] Bump com.amazonaws to 1.12.135

2022-01-07 Thread GitBox
aromanenko-dev commented on pull request #16454: URL: https://github.com/apache/beam/pull/16454#issuecomment-1007591803 @mosche -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

[GitHub] [beam] aromanenko-dev edited a comment on pull request #16454: [BEAM-4868] Bump com.amazonaws to 1.12.135

2022-01-07 Thread GitBox
aromanenko-dev edited a comment on pull request #16454: URL: https://github.com/apache/beam/pull/16454#issuecomment-1007591803 R: @mosche -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

[GitHub] [beam] aromanenko-dev opened a new pull request #16454: [BEAM-4868] Bump com.amazonaws to 1.12.135

2022-01-07 Thread GitBox
aromanenko-dev opened a new pull request #16454: URL: https://github.com/apache/beam/pull/16454 Bump com.amazonaws to 1.12.135 Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: -

[GitHub] [beam] lostluck opened a new pull request #16455: [BEAM-13574] Large Wordcount

2022-01-07 Thread GitBox
lostluck opened a new pull request #16455: URL: https://github.com/apache/beam/pull/16455 Adds a more complex example pipeline demonstrating a more production oriented worcount. * Uses the SDF based textio read * Sorts the output as long as keys fit in a single machine's memory

[GitHub] [beam] aromanenko-dev opened a new pull request #16456: [BEAM-8727] Bump software.amazon.awssdk to 2.17.106

2022-01-07 Thread GitBox
aromanenko-dev opened a new pull request #16456: URL: https://github.com/apache/beam/pull/16456 Bump software.amazon.awssdk to 2.17.106 Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[GitHub] [beam] aromanenko-dev commented on pull request #16456: [BEAM-8727] Bump software.amazon.awssdk to 2.17.106

2022-01-07 Thread GitBox
aromanenko-dev commented on pull request #16456: URL: https://github.com/apache/beam/pull/16456#issuecomment-1007597608 R: @mosche -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

[GitHub] [beam] aromanenko-dev commented on pull request #16456: [BEAM-8727] Bump software.amazon.awssdk to 2.17.106

2022-01-07 Thread GitBox
aromanenko-dev commented on pull request #16456: URL: https://github.com/apache/beam/pull/16456#issuecomment-1007598117 Run Java PostCommit -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[GitHub] [beam] aromanenko-dev commented on pull request #16454: [BEAM-4868] Bump com.amazonaws to 1.12.135

2022-01-07 Thread GitBox
aromanenko-dev commented on pull request #16454: URL: https://github.com/apache/beam/pull/16454#issuecomment-1007598248 Run Java PostCommit -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[GitHub] [beam] codecov[bot] edited a comment on pull request #16250: [BEAM-13386] Add RLock support for cloudpickle

2022-01-07 Thread GitBox
codecov[bot] edited a comment on pull request #16250: URL: https://github.com/apache/beam/pull/16250#issuecomment-995255415 # [Codecov](https://codecov.io/gh/apache/beam/pull/16250?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+A

[GitHub] [beam] codecov[bot] commented on pull request #16454: [BEAM-4868] Bump com.amazonaws to 1.12.135

2022-01-07 Thread GitBox
codecov[bot] commented on pull request #16454: URL: https://github.com/apache/beam/pull/16454#issuecomment-1007607190 # [Codecov](https://codecov.io/gh/apache/beam/pull/16454?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+

[GitHub] [beam] codecov[bot] edited a comment on pull request #16454: [BEAM-4868] Bump com.amazonaws to 1.12.135

2022-01-07 Thread GitBox
codecov[bot] edited a comment on pull request #16454: URL: https://github.com/apache/beam/pull/16454#issuecomment-1007607190 # [Codecov](https://codecov.io/gh/apache/beam/pull/16454?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+

[GitHub] [beam] reuvenlax commented on pull request #15275: [BEAM-7386] Adding EventTimeEquiJoin

2022-01-07 Thread GitBox
reuvenlax commented on pull request #15275: URL: https://github.com/apache/beam/pull/15275#issuecomment-1007610045 But you needed to make that function apply to timers as well, right? On Fri, Jan 7, 2022 at 9:10 AM laraschmidt ***@***.***> wrote: > The other one is in but it do

[GitHub] [beam] codecov[bot] edited a comment on pull request #16454: [BEAM-4868] Bump com.amazonaws to 1.12.135

2022-01-07 Thread GitBox
codecov[bot] edited a comment on pull request #16454: URL: https://github.com/apache/beam/pull/16454#issuecomment-1007607190 # [Codecov](https://codecov.io/gh/apache/beam/pull/16454?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+

[GitHub] [beam] codecov[bot] edited a comment on pull request #16454: [BEAM-4868] Bump com.amazonaws to 1.12.135

2022-01-07 Thread GitBox
codecov[bot] edited a comment on pull request #16454: URL: https://github.com/apache/beam/pull/16454#issuecomment-1007607190 # [Codecov](https://codecov.io/gh/apache/beam/pull/16454?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+

[GitHub] [beam] codecov[bot] edited a comment on pull request #16454: [BEAM-4868] Bump com.amazonaws to 1.12.135

2022-01-07 Thread GitBox
codecov[bot] edited a comment on pull request #16454: URL: https://github.com/apache/beam/pull/16454#issuecomment-1007607190 # [Codecov](https://codecov.io/gh/apache/beam/pull/16454?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+

[GitHub] [beam] codecov[bot] commented on pull request #16456: [BEAM-8727] Bump software.amazon.awssdk to 2.17.106

2022-01-07 Thread GitBox
codecov[bot] commented on pull request #16456: URL: https://github.com/apache/beam/pull/16456#issuecomment-1007619975 # [Codecov](https://codecov.io/gh/apache/beam/pull/16456?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+

[GitHub] [beam] codecov[bot] edited a comment on pull request #16456: [BEAM-8727] Bump software.amazon.awssdk to 2.17.106

2022-01-07 Thread GitBox
codecov[bot] edited a comment on pull request #16456: URL: https://github.com/apache/beam/pull/16456#issuecomment-1007619975 # [Codecov](https://codecov.io/gh/apache/beam/pull/16456?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+

[GitHub] [beam] codecov[bot] edited a comment on pull request #16250: [BEAM-13386] Add RLock support for cloudpickle

2022-01-07 Thread GitBox
codecov[bot] edited a comment on pull request #16250: URL: https://github.com/apache/beam/pull/16250#issuecomment-995255415 # [Codecov](https://codecov.io/gh/apache/beam/pull/16250?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+A

[GitHub] [beam] codecov[bot] edited a comment on pull request #16454: [BEAM-4868] Bump com.amazonaws to 1.12.135

2022-01-07 Thread GitBox
codecov[bot] edited a comment on pull request #16454: URL: https://github.com/apache/beam/pull/16454#issuecomment-1007607190 # [Codecov](https://codecov.io/gh/apache/beam/pull/16454?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+

[GitHub] [beam] codecov[bot] edited a comment on pull request #16456: [BEAM-8727] Bump software.amazon.awssdk to 2.17.106

2022-01-07 Thread GitBox
codecov[bot] edited a comment on pull request #16456: URL: https://github.com/apache/beam/pull/16456#issuecomment-1007619975 # [Codecov](https://codecov.io/gh/apache/beam/pull/16456?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+

[GitHub] [beam] codecov[bot] edited a comment on pull request #16456: [BEAM-8727] Bump software.amazon.awssdk to 2.17.106

2022-01-07 Thread GitBox
codecov[bot] edited a comment on pull request #16456: URL: https://github.com/apache/beam/pull/16456#issuecomment-1007619975 # [Codecov](https://codecov.io/gh/apache/beam/pull/16456?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+

[GitHub] [beam] codecov[bot] edited a comment on pull request #16456: [BEAM-8727] Bump software.amazon.awssdk to 2.17.106

2022-01-07 Thread GitBox
codecov[bot] edited a comment on pull request #16456: URL: https://github.com/apache/beam/pull/16456#issuecomment-1007619975 # [Codecov](https://codecov.io/gh/apache/beam/pull/16456?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+

[GitHub] [beam] codecov[bot] edited a comment on pull request #16454: [BEAM-4868] Bump com.amazonaws to 1.12.135

2022-01-07 Thread GitBox
codecov[bot] edited a comment on pull request #16454: URL: https://github.com/apache/beam/pull/16454#issuecomment-1007607190 # [Codecov](https://codecov.io/gh/apache/beam/pull/16454?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+

[GitHub] [beam] laraschmidt commented on pull request #15275: [BEAM-7386] Adding EventTimeEquiJoin

2022-01-07 Thread GitBox
laraschmidt commented on pull request #15275: URL: https://github.com/apache/beam/pull/15275#issuecomment-1007632856 That was for an earlier iteration. The solution we ended up going with only outputs from the actual doFn and only uses timers to hold the watermark. So we didn't end up actu

[GitHub] [beam] laraschmidt edited a comment on pull request #15275: [BEAM-7386] Adding EventTimeEquiJoin

2022-01-07 Thread GitBox
laraschmidt edited a comment on pull request #15275: URL: https://github.com/apache/beam/pull/15275#issuecomment-1007632856 That was for an earlier iteration. The solution we ended up going with only outputs from processElement and only uses timers to hold the watermark. So we didn't end u

[GitHub] [beam] reuvenlax commented on pull request #15275: [BEAM-7386] Adding EventTimeEquiJoin

2022-01-07 Thread GitBox
reuvenlax commented on pull request #15275: URL: https://github.com/apache/beam/pull/15275#issuecomment-1007633829 Right, but you need to be able to set a timer with a hold older than the current input element, right? On Fri, Jan 7, 2022 at 10:23 AM laraschmidt ***@***.***> wrot

[GitHub] [beam] codecov[bot] edited a comment on pull request #16250: [BEAM-13386] Add RLock support for cloudpickle

2022-01-07 Thread GitBox
codecov[bot] edited a comment on pull request #16250: URL: https://github.com/apache/beam/pull/16250#issuecomment-995255415 # [Codecov](https://codecov.io/gh/apache/beam/pull/16250?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+A

[GitHub] [beam] codecov[bot] edited a comment on pull request #16456: [BEAM-8727] Bump software.amazon.awssdk to 2.17.106

2022-01-07 Thread GitBox
codecov[bot] edited a comment on pull request #16456: URL: https://github.com/apache/beam/pull/16456#issuecomment-1007619975 # [Codecov](https://codecov.io/gh/apache/beam/pull/16456?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+

[GitHub] [beam] codecov[bot] edited a comment on pull request #16456: [BEAM-8727] Bump software.amazon.awssdk to 2.17.106

2022-01-07 Thread GitBox
codecov[bot] edited a comment on pull request #16456: URL: https://github.com/apache/beam/pull/16456#issuecomment-1007619975 # [Codecov](https://codecov.io/gh/apache/beam/pull/16456?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+

[GitHub] [beam] codecov[bot] edited a comment on pull request #16456: [BEAM-8727] Bump software.amazon.awssdk to 2.17.106

2022-01-07 Thread GitBox
codecov[bot] edited a comment on pull request #16456: URL: https://github.com/apache/beam/pull/16456#issuecomment-1007619975 # [Codecov](https://codecov.io/gh/apache/beam/pull/16456?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+

[GitHub] [beam] codecov[bot] edited a comment on pull request #16454: [BEAM-4868] Bump com.amazonaws to 1.12.135

2022-01-07 Thread GitBox
codecov[bot] edited a comment on pull request #16454: URL: https://github.com/apache/beam/pull/16454#issuecomment-1007607190 # [Codecov](https://codecov.io/gh/apache/beam/pull/16454?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+

[GitHub] [beam] codecov[bot] edited a comment on pull request #16456: [BEAM-8727] Bump software.amazon.awssdk to 2.17.106

2022-01-07 Thread GitBox
codecov[bot] edited a comment on pull request #16456: URL: https://github.com/apache/beam/pull/16456#issuecomment-1007619975 # [Codecov](https://codecov.io/gh/apache/beam/pull/16456?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+

[GitHub] [beam] laraschmidt commented on pull request #15275: [BEAM-7386] Adding EventTimeEquiJoin

2022-01-07 Thread GitBox
laraschmidt commented on pull request #15275: URL: https://github.com/apache/beam/pull/15275#issuecomment-1007641545 I'm trying to page this back in. But if I remember correctly, the first element holds the watermark back until all not-late second elements would have appeared. So we don't

[GitHub] [beam] codecov[bot] edited a comment on pull request #16250: [BEAM-13386] Add RLock support for cloudpickle

2022-01-07 Thread GitBox
codecov[bot] edited a comment on pull request #16250: URL: https://github.com/apache/beam/pull/16250#issuecomment-995255415 # [Codecov](https://codecov.io/gh/apache/beam/pull/16250?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+A

[GitHub] [beam] codecov[bot] edited a comment on pull request #16454: [BEAM-4868] Bump com.amazonaws to 1.12.135

2022-01-07 Thread GitBox
codecov[bot] edited a comment on pull request #16454: URL: https://github.com/apache/beam/pull/16454#issuecomment-1007607190 # [Codecov](https://codecov.io/gh/apache/beam/pull/16454?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+

[GitHub] [beam] codecov[bot] edited a comment on pull request #16456: [BEAM-8727] Bump software.amazon.awssdk to 2.17.106

2022-01-07 Thread GitBox
codecov[bot] edited a comment on pull request #16456: URL: https://github.com/apache/beam/pull/16456#issuecomment-1007619975 # [Codecov](https://codecov.io/gh/apache/beam/pull/16456?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+

[GitHub] [beam] y1chi commented on pull request #16439: [BEAM-13193] Aggregates fn api outbound data/timers of different endpoints

2022-01-07 Thread GitBox
y1chi commented on pull request #16439: URL: https://github.com/apache/beam/pull/16439#issuecomment-1007662186 Run Java PreCommit -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

[GitHub] [beam] KevinGG commented on a change in pull request #16449: [BEAM-13602] Prevented metrics gathering from failing bigtable io

2022-01-07 Thread GitBox
KevinGG commented on a change in pull request #16449: URL: https://github.com/apache/beam/pull/16449#discussion_r780473158 ## File path: sdks/python/apache_beam/internal/metrics/metric.py ## @@ -262,4 +262,8 @@ def bigtable_error_code_to_grpc_status_string(grpc_status_code):

[GitHub] [beam] KevinGG commented on a change in pull request #16449: [BEAM-13602] Prevented metrics gathering from failing bigtable io

2022-01-07 Thread GitBox
KevinGG commented on a change in pull request #16449: URL: https://github.com/apache/beam/pull/16449#discussion_r780476775 ## File path: sdks/python/apache_beam/io/gcp/bigtableio.py ## @@ -116,11 +116,13 @@ def __setstate__(self, options): self.service_call_metric = None

[GitHub] [beam] reuvenlax commented on pull request #15275: [BEAM-7386] Adding EventTimeEquiJoin

2022-01-07 Thread GitBox
reuvenlax commented on pull request #15275: URL: https://github.com/apache/beam/pull/15275#issuecomment-1007674641 I need to take a look at the code, since it's changed a lot. However elements don't arrive in timestamp order, so I'm not sure how you would avoid this. On Fri, Jan

[GitHub] [beam] codecov[bot] edited a comment on pull request #16449: [BEAM-13602] Prevented metrics gathering from failing bigtable io

2022-01-07 Thread GitBox
codecov[bot] edited a comment on pull request #16449: URL: https://github.com/apache/beam/pull/16449#issuecomment-1007058009 # [Codecov](https://codecov.io/gh/apache/beam/pull/16449?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+

[GitHub] [beam] codecov[bot] edited a comment on pull request #16449: [BEAM-13602] Prevented metrics gathering from failing bigtable io

2022-01-07 Thread GitBox
codecov[bot] edited a comment on pull request #16449: URL: https://github.com/apache/beam/pull/16449#issuecomment-1007058009 # [Codecov](https://codecov.io/gh/apache/beam/pull/16449?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+

[GitHub] [beam] aijamalnk commented on pull request #16361: [BEAM-13543][Playground] Add logic of sending validation's output on the backend side

2022-01-07 Thread GitBox
aijamalnk commented on pull request #16361: URL: https://github.com/apache/beam/pull/16361#issuecomment-1007696492 @KhaninArtur Thanks Artur :) and apologies for bothering during well earned holiday break. -- This is an automated message from the Apache Git Service. To respond to the mes

[GitHub] [beam] ibzib commented on pull request #16200: [BEAM-11808][BEAM-9879] Support aggregate functions with two arguments

2022-01-07 Thread GitBox
ibzib commented on pull request #16200: URL: https://github.com/apache/beam/pull/16200#issuecomment-1007702822 Apologies - I've been a bit busy this week but I'll give this another review next week. -- This is an automated message from the Apache Git Service. To respond to the message, p

[GitHub] [beam] lukecwik merged pull request #16447: [BEAM-13015] Remove dead code now that all instances have migrated to the state caching implementation.

2022-01-07 Thread GitBox
lukecwik merged pull request #16447: URL: https://github.com/apache/beam/pull/16447 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubsc

[GitHub] [beam] codecov[bot] edited a comment on pull request #16449: [BEAM-13602] Prevented metrics gathering from failing bigtable io

2022-01-07 Thread GitBox
codecov[bot] edited a comment on pull request #16449: URL: https://github.com/apache/beam/pull/16449#issuecomment-1007058009 # [Codecov](https://codecov.io/gh/apache/beam/pull/16449?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+

[GitHub] [beam] codecov[bot] commented on pull request #16433: [BEAM-13599] Fix overflow error in Python Datastore RampupThrottlingFn

2022-01-07 Thread GitBox
codecov[bot] commented on pull request #16433: URL: https://github.com/apache/beam/pull/16433#issuecomment-1007705298 # [Codecov](https://codecov.io/gh/apache/beam/pull/16433?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+

[GitHub] [beam] codecov[bot] edited a comment on pull request #16250: [BEAM-13386] Add RLock support for cloudpickle

2022-01-07 Thread GitBox
codecov[bot] edited a comment on pull request #16250: URL: https://github.com/apache/beam/pull/16250#issuecomment-995255415 # [Codecov](https://codecov.io/gh/apache/beam/pull/16250?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+A

[GitHub] [beam] danthev commented on pull request #16433: [BEAM-13599] Fix overflow error in Python Datastore RampupThrottlingFn

2022-01-07 Thread GitBox
danthev commented on pull request #16433: URL: https://github.com/apache/beam/pull/16433#issuecomment-1007705849 Thanks. I've added test code and amended it onto the commit. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [beam] codecov[bot] edited a comment on pull request #16449: [BEAM-13602] Prevented metrics gathering from failing bigtable io

2022-01-07 Thread GitBox
codecov[bot] edited a comment on pull request #16449: URL: https://github.com/apache/beam/pull/16449#issuecomment-1007058009 # [Codecov](https://codecov.io/gh/apache/beam/pull/16449?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+

[GitHub] [beam] codecov[bot] edited a comment on pull request #16433: [BEAM-13599] Fix overflow error in Python Datastore RampupThrottlingFn

2022-01-07 Thread GitBox
codecov[bot] edited a comment on pull request #16433: URL: https://github.com/apache/beam/pull/16433#issuecomment-1007705298 # [Codecov](https://codecov.io/gh/apache/beam/pull/16433?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+

[GitHub] [beam] codecov[bot] edited a comment on pull request #16154: [BEAM-12572] Run python examples on multiple runners

2022-01-07 Thread GitBox
codecov[bot] edited a comment on pull request #16154: URL: https://github.com/apache/beam/pull/16154#issuecomment-988168868 # [Codecov](https://codecov.io/gh/apache/beam/pull/16154?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+A

[GitHub] [beam] dannymartinm opened a new pull request #16457: [BEAM-12621] - Update Jenkins VMs to modern Ubuntu version

2022-01-07 Thread GitBox
in the latest Jenkins worker image (`jenkins-worker-boot-image-20220107`). * We can use the `python_installer.sh` tool to create pyenv environments in any Ubuntu distributions. Thank you for your contribution! Follow this checklist to help us

[GitHub] [beam] benWize commented on pull request #16154: [BEAM-12572] Run python examples on multiple runners

2022-01-07 Thread GitBox
benWize commented on pull request #16154: URL: https://github.com/apache/beam/pull/16154#issuecomment-1007717399 Run Python Examples_Direct -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[GitHub] [beam] benWize commented on pull request #16154: [BEAM-12572] Run python examples on multiple runners

2022-01-07 Thread GitBox
benWize commented on pull request #16154: URL: https://github.com/apache/beam/pull/16154#issuecomment-1007717561 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscr

[GitHub] [beam] ryanthompson591 commented on a change in pull request #16250: [BEAM-13386] Add RLock support for cloudpickle

2022-01-07 Thread GitBox
ryanthompson591 commented on a change in pull request #16250: URL: https://github.com/apache/beam/pull/16250#discussion_r780511727 ## File path: sdks/python/apache_beam/internal/cloudpickle_pickler.py ## @@ -99,6 +103,14 @@ def _create_absl_flags(): return flags.FLAGS +d

[GitHub] [beam] codecov[bot] edited a comment on pull request #16250: [BEAM-13386] Add RLock support for cloudpickle

2022-01-07 Thread GitBox
codecov[bot] edited a comment on pull request #16250: URL: https://github.com/apache/beam/pull/16250#issuecomment-995255415 # [Codecov](https://codecov.io/gh/apache/beam/pull/16250?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+A

[GitHub] [beam] codecov[bot] edited a comment on pull request #16433: [BEAM-13599] Fix overflow error in Python Datastore RampupThrottlingFn

2022-01-07 Thread GitBox
codecov[bot] edited a comment on pull request #16433: URL: https://github.com/apache/beam/pull/16433#issuecomment-1007705298 # [Codecov](https://codecov.io/gh/apache/beam/pull/16433?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+

[GitHub] [beam] lukecwik opened a new pull request #16458: [BEAM-13614] Add OnWindowExpiration as a requirement so that portable runners can reject such jobs.

2022-01-07 Thread GitBox
lukecwik opened a new pull request #16458: URL: https://github.com/apache/beam/pull/16458 Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] [**Choose reviewer(s)**](https

[GitHub] [beam] lukecwik commented on pull request #16458: [BEAM-13614] Add OnWindowExpiration as a requirement so that portable runners can reject such jobs.

2022-01-07 Thread GitBox
lukecwik commented on pull request #16458: URL: https://github.com/apache/beam/pull/16458#issuecomment-1007729456 R: @TheNeuralBit CC: @reuvenlax @robertwb -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [beam] codecov[bot] edited a comment on pull request #16154: [BEAM-12572] Run python examples on multiple runners

2022-01-07 Thread GitBox
codecov[bot] edited a comment on pull request #16154: URL: https://github.com/apache/beam/pull/16154#issuecomment-988168868 # [Codecov](https://codecov.io/gh/apache/beam/pull/16154?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+A

[GitHub] [beam] TheNeuralBit commented on a change in pull request #16256: [WIP][BEAM-12562] Dataframe pipe implementation

2022-01-07 Thread GitBox
TheNeuralBit commented on a change in pull request #16256: URL: https://github.com/apache/beam/pull/16256#discussion_r780528895 ## File path: sdks/python/apache_beam/dataframe/frames_test.py ## @@ -1245,6 +1245,39 @@ def test_idxmax(self): self._run_test(lambda s2: s2.idxm

[GitHub] [beam] codecov[bot] commented on pull request #16458: [BEAM-13614] Add OnWindowExpiration as a requirement so that portable runners can reject such jobs.

2022-01-07 Thread GitBox
codecov[bot] commented on pull request #16458: URL: https://github.com/apache/beam/pull/16458#issuecomment-1007741254 # [Codecov](https://codecov.io/gh/apache/beam/pull/16458?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+

[GitHub] [beam] KevinGG commented on pull request #16449: [BEAM-13602] Prevented metrics gathering from failing bigtable io

2022-01-07 Thread GitBox
KevinGG commented on pull request #16449: URL: https://github.com/apache/beam/pull/16449#issuecomment-1007742888 Additionally filed https://github.com/googleapis/python-bigtable/issues/485 and included a code snippet that verified the bug did exist and None types were returned in the analy

[GitHub] [beam] codecov[bot] edited a comment on pull request #16458: [BEAM-13614] Add OnWindowExpiration as a requirement so that portable runners can reject such jobs.

2022-01-07 Thread GitBox
codecov[bot] edited a comment on pull request #16458: URL: https://github.com/apache/beam/pull/16458#issuecomment-1007741254 # [Codecov](https://codecov.io/gh/apache/beam/pull/16458?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+

[GitHub] [beam] codecov[bot] edited a comment on pull request #15675: [BEAM-12313] - Initial changes to support Java 17 SDK Harness docker image

2022-01-07 Thread GitBox
codecov[bot] edited a comment on pull request #15675: URL: https://github.com/apache/beam/pull/15675#issuecomment-937364403 # [Codecov](https://codecov.io/gh/apache/beam/pull/15675?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+A

[GitHub] [beam] codecov[bot] edited a comment on pull request #16250: [BEAM-13386] Add RLock support for cloudpickle

2022-01-07 Thread GitBox
codecov[bot] edited a comment on pull request #16250: URL: https://github.com/apache/beam/pull/16250#issuecomment-995255415 # [Codecov](https://codecov.io/gh/apache/beam/pull/16250?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+A

[GitHub] [beam] codecov[bot] edited a comment on pull request #15675: [BEAM-12313] - Initial changes to support Java 17 SDK Harness docker image

2022-01-07 Thread GitBox
codecov[bot] edited a comment on pull request #15675: URL: https://github.com/apache/beam/pull/15675#issuecomment-937364403 # [Codecov](https://codecov.io/gh/apache/beam/pull/15675?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+A

[GitHub] [beam] codecov[bot] edited a comment on pull request #15675: [BEAM-12313] - Initial changes to support Java 17 SDK Harness docker image

2022-01-07 Thread GitBox
codecov[bot] edited a comment on pull request #15675: URL: https://github.com/apache/beam/pull/15675#issuecomment-937364403 # [Codecov](https://codecov.io/gh/apache/beam/pull/15675?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+A

[GitHub] [beam] codecov[bot] edited a comment on pull request #16458: [BEAM-13614] Add OnWindowExpiration as a requirement so that portable runners can reject such jobs.

2022-01-07 Thread GitBox
codecov[bot] edited a comment on pull request #16458: URL: https://github.com/apache/beam/pull/16458#issuecomment-1007741254 # [Codecov](https://codecov.io/gh/apache/beam/pull/16458?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+

[GitHub] [beam] codecov[bot] edited a comment on pull request #16154: [BEAM-12572] Run python examples on multiple runners

2022-01-07 Thread GitBox
codecov[bot] edited a comment on pull request #16154: URL: https://github.com/apache/beam/pull/16154#issuecomment-988168868 # [Codecov](https://codecov.io/gh/apache/beam/pull/16154?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+A

[GitHub] [beam] codecov[bot] edited a comment on pull request #15675: [BEAM-12313] - Initial changes to support Java 17 SDK Harness docker image

2022-01-07 Thread GitBox
codecov[bot] edited a comment on pull request #15675: URL: https://github.com/apache/beam/pull/15675#issuecomment-937364403 # [Codecov](https://codecov.io/gh/apache/beam/pull/15675?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+A

[GitHub] [beam] codecov[bot] edited a comment on pull request #16458: [BEAM-13614] Add OnWindowExpiration as a requirement so that portable runners can reject such jobs.

2022-01-07 Thread GitBox
codecov[bot] edited a comment on pull request #16458: URL: https://github.com/apache/beam/pull/16458#issuecomment-1007741254 # [Codecov](https://codecov.io/gh/apache/beam/pull/16458?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+

[GitHub] [beam] codecov[bot] edited a comment on pull request #15675: [BEAM-12313] - Initial changes to support Java 17 SDK Harness docker image

2022-01-07 Thread GitBox
codecov[bot] edited a comment on pull request #15675: URL: https://github.com/apache/beam/pull/15675#issuecomment-937364403 # [Codecov](https://codecov.io/gh/apache/beam/pull/15675?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+A

[GitHub] [beam] codecov[bot] edited a comment on pull request #16458: [BEAM-13614] Add OnWindowExpiration as a requirement so that portable runners can reject such jobs.

2022-01-07 Thread GitBox
codecov[bot] edited a comment on pull request #16458: URL: https://github.com/apache/beam/pull/16458#issuecomment-1007741254 # [Codecov](https://codecov.io/gh/apache/beam/pull/16458?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+

[GitHub] [beam] tvalentyn merged pull request #16250: [BEAM-13386] Add RLock support for cloudpickle

2022-01-07 Thread GitBox
tvalentyn merged pull request #16250: URL: https://github.com/apache/beam/pull/16250 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubs

[GitHub] [beam] codecov[bot] edited a comment on pull request #16154: [BEAM-12572] Run python examples on multiple runners

2022-01-07 Thread GitBox
codecov[bot] edited a comment on pull request #16154: URL: https://github.com/apache/beam/pull/16154#issuecomment-988168868 # [Codecov](https://codecov.io/gh/apache/beam/pull/16154?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+A

[GitHub] [beam] codecov[bot] edited a comment on pull request #15675: [BEAM-12313] - Initial changes to support Java 17 SDK Harness docker image

2022-01-07 Thread GitBox
codecov[bot] edited a comment on pull request #15675: URL: https://github.com/apache/beam/pull/15675#issuecomment-937364403 # [Codecov](https://codecov.io/gh/apache/beam/pull/15675?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+A

[GitHub] [beam] kileys commented on pull request #15549: [BEAM-11997] Changed RedisIO implementation to SDF

2022-01-07 Thread GitBox
kileys commented on pull request #15549: URL: https://github.com/apache/beam/pull/15549#issuecomment-1007782447 R: @pabloem Thanks Pablo! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] codecov[bot] edited a comment on pull request #16154: [BEAM-12572] Run python examples on multiple runners

2022-01-07 Thread GitBox
codecov[bot] edited a comment on pull request #16154: URL: https://github.com/apache/beam/pull/16154#issuecomment-988168868 # [Codecov](https://codecov.io/gh/apache/beam/pull/16154?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+A

[GitHub] [beam] lukecwik commented on pull request #16439: [BEAM-13193] Aggregates fn api outbound data/timers of different endpoints

2022-01-07 Thread GitBox
lukecwik commented on pull request #16439: URL: https://github.com/apache/beam/pull/16439#issuecomment-1007800054 Will get to this when I can, plat-mon has been busy. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

[GitHub] [beam] lukecwik commented on pull request #16458: [BEAM-13614] Add OnWindowExpiration as a requirement so that portable runners can reject such jobs.

2022-01-07 Thread GitBox
lukecwik commented on pull request #16458: URL: https://github.com/apache/beam/pull/16458#issuecomment-1007800704 Run Java PreCommit -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [beam] codecov[bot] edited a comment on pull request #16154: [BEAM-12572] Run python examples on multiple runners

2022-01-07 Thread GitBox
codecov[bot] edited a comment on pull request #16154: URL: https://github.com/apache/beam/pull/16154#issuecomment-988168868 # [Codecov](https://codecov.io/gh/apache/beam/pull/16154?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+A

[GitHub] [beam] lukecwik commented on pull request #15675: [BEAM-12313] - Initial changes to support Java 17 SDK Harness docker image

2022-01-07 Thread GitBox
lukecwik commented on pull request #15675: URL: https://github.com/apache/beam/pull/15675#issuecomment-1007801930 --add-opens should be done for all modules that exist on the classpath and only access to jamm should be provided. If jamm doesn't support java 9 modules yet then we can modify

[GitHub] [beam] ihji commented on pull request #16433: [BEAM-13599] Fix overflow error in Python Datastore RampupThrottlingFn

2022-01-07 Thread GitBox
ihji commented on pull request #16433: URL: https://github.com/apache/beam/pull/16433#issuecomment-1007820932 @danthev Thanks for adding tests. Can you please fix formatting errors? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Gi

[GitHub] [beam] codecov[bot] edited a comment on pull request #16433: [BEAM-13599] Fix overflow error in Python Datastore RampupThrottlingFn

2022-01-07 Thread GitBox
codecov[bot] edited a comment on pull request #16433: URL: https://github.com/apache/beam/pull/16433#issuecomment-1007705298 # [Codecov](https://codecov.io/gh/apache/beam/pull/16433?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+

[GitHub] [beam] codecov[bot] edited a comment on pull request #16433: [BEAM-13599] Fix overflow error in Python Datastore RampupThrottlingFn

2022-01-07 Thread GitBox
codecov[bot] edited a comment on pull request #16433: URL: https://github.com/apache/beam/pull/16433#issuecomment-1007705298 # [Codecov](https://codecov.io/gh/apache/beam/pull/16433?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+

[GitHub] [beam] laraschmidt commented on pull request #15275: [BEAM-7386] Adding EventTimeEquiJoin

2022-01-07 Thread GitBox
laraschmidt commented on pull request #15275: URL: https://github.com/apache/beam/pull/15275#issuecomment-1007822342 We use the same watermark hold for a bucket of elements (E.g. arriving between t and t+e). And we just hold it back til t+e. So it shouldn't matter that they come out of ord

  1   2   >