[BEAM-14004] Support for CosmosDB/API MongoDB

Hi guys, I'm generate the pull request ( https://github.com/apache/beam/pull/16972) for fix the issue: [BEAM-14004] [CosmosDB] com.mongodb.MongoCommandException: Command failed with error 115: Command is not supported How can I make them take it

Re: Possible bug in ElasticsearchIO

Thanks Jan for confirming that the fix looks alright. I also found a PR[1] that appears to be a good case study of the Timer watermark hold technique that you previously mentioned so I'll study that a bit for my own understanding and future use. I was also previously missing the notion that a

Re: DRAFT - Beam board report March 2022

Typo: March 9, aka Wednesday, aka tomorrow On Mon, Mar 7, 2022 at 5:36 AM Kenneth Knowles wrote: > Hi all, > > The March board report is due March 7. Please help me to write it here: > https://s.apache.org/beam-draft-report-2022-03 > > Ideas: > > - highlights from CHANGES.md > - interesting

P1 issues report (70)

This is your daily summary of Beam's current P1 issues, not including flaky tests (https://issues.apache.org/jira/issues/?jql=project%20%3D%20BEAM%20AND%20statusCategory%20!%3D%20Done%20AND%20priority%20%3D%20P1%20AND%20(labels%20is%20EMPTY%20OR%20labels%20!%3D%20flake). See

Flaky test issue report (51)

This is your daily summary of Beam's current flaky tests (https://issues.apache.org/jira/issues/?jql=project%20%3D%20BEAM%20AND%20statusCategory%20!%3D%20Done%20AND%20labels%20%3D%20flake) These are P1 issues because they have a major negative impact on the community and make it hard to

Re: Possible bug in ElasticsearchIO

Hi Evan, the fix looks good to me, as long as the timestamp of the buffered data need to be preserved downstream. Generally I think it *should* be possible to output in-memory buffered data in @ProcessElement (and @FinishBundle), the case where you need timers is when your buffer needs to

Re: Possible bug in ElasticsearchIO

Thanks Jan, it's interesting to read about the handling of timestamp in cases employing a buffering pattern. In the case of the ES write transform, buffered data could be output from ProcessElement or FinishBundle. It's the case where data is output from ProcessElement that the error reported at

Re: Possible bug in ElasticsearchIO

Ah, sorry, the code flushes in @FinishBundle. Is it allowed to update output watermark while a bundle is being processed? That seems it could also cause the "watermark skip" problem, which is definitely an issue (and is probably the reason why the check fails?). On 3/8/22 09:35, Jan Lukavský

Re: Possible bug in ElasticsearchIO

The buffering seems incorrect to me. Whenever there is a buffer, we need to make sure we hold the output watermark, otherwise the watermark might "jump over" a buffered element transitioning it from "on-time" to "late", which would be a correctness bug (we can transition elements only from