cloudpickle [1] and dill [2] are two Python packages that implement
extensions of Python's pickle protocol for arbitrary objects. Beam
currently uses dill, but I'm wondering if we could consider additionally or
alternatively use cloudpickle instead.
Overall, cloudpickle seems to be a more popular
Got you. We are definitely interested in java worker pool to support Samza
runner use case, and I think we could help implement on it if no one is
currently working on it.
Comparing with what python offers, what I see that are missing are:
1. Main class/method to start ExternalWorkerService
Our oldest open P1 issue is BEAM-6466 - "KafkaIO doesn't commit offsets
while being used as bounded source" [1]. I'm not sure this is an actual
issue since KafkaIO doesn't seem to officially support this use-case. The
relevant parameters indicate they are "mainly used for tests and demo
This is your daily summary of Beam's current flaky tests. These are P1 issues
because they have a major negative impact on the community and make it hard to
determine the quality of the software.
BEAM-12250: Java ValidatesRunner Postcommits timing out
This is your daily summary of Beam's current P1 issues, not including flaky
tests.
See https://beam.apache.org/contribute/jira-priorities/#p1-critical for the
meaning and expectations around P1 issues.
BEAM-12256: PubsubIO.readAvroGenericRecord creates SchemaCoder that fails
to decode
>
> Thanks for the info. In order to use supported remote file systems, does
> it mean it needs to be passed in as FILE_ARTIFACT_URN since neither
> *ArtifactRetrievalService#URL_ARTIFACT_URN = "beam:artifact:type:url:v1” *
> nor *ArtifactRetrievalService#STAGING_TO_ARTIFACT_URN =
>
Hi,
I have come across a bug with timer output timestamp - when using event
time and relative timers, setting the timer can arbitrarily throw
IllegalArgumentException if the firing timestamp (input watermark) is
ahead of the output timestamp (like .java.lang.IllegalArgumentException:
We have 10 open issues for Fix Version 2.30.0:
https://issues.apache.org/jira/browse/BEAM-12242?jql=project%20%3D%20BEAM%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20%22In%20Implementation%22%2C%20%22Triage%20Needed%22)%20AND%20fixVersion%20%3D%202.30.0
On Thu, Apr 29, 2021 at 12:30
FYI, I just cut the 2.30.0 release branch. From now on, late commits for
2.30.0 need to be cherry-picked. If you have any late commits, please make
sure that their Jira issues have the correct Fix Version, 2.30.0.
On Tue, Apr 27, 2021 at 7:52 AM Kenneth Knowles wrote:
> SGTM. Thanks!
>
> On
