[ https://issues.apache.org/jira/browse/BEAM-1330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16371080#comment-16371080 ]
Julien Sobczak commented on BEAM-1330: -------------------------------------- Hi, Have someone find a workaround for this problem? Removing duplicates inside the window does not work because it seems several windows are being sent in the same batch. Julien > DatastoreIO Writes should flush early when duplicate keys arrive. > ----------------------------------------------------------------- > > Key: BEAM-1330 > URL: https://issues.apache.org/jira/browse/BEAM-1330 > Project: Beam > Issue Type: Improvement > Components: io-java-gcp > Reporter: Vikas Kedigehalli > Assignee: Vikas Kedigehalli > Priority: Minor > > DatastoreIO writes batches upto 500 entities (rpc limit for Cloud Datastore), > before flushing them out. The writes are non-transactional and thus do not > support duplicate keys in the writes. This can be problem, especially when > using a non global windowing, where multiple windows for the same key end up > in the same batch, and prevents the writes from succeeding. -- This message was sent by Atlassian JIRA (v7.6.3#76005)