[
https://issues.apache.org/jira/browse/BEAM-5757?focusedWorklogId=478438&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-478438
]
ASF GitHub Bot logged work on BEAM-5757:
----------------------------------------
Author: ASF GitHub Bot
Created on: 03/Sep/20 10:00
Start Date: 03/Sep/20 10:00
Worklog Time Spent: 10m
Work Description: jithin97 commented on a change in pull request #12670:
URL: https://github.com/apache/beam/pull/12670#discussion_r482859763
##########
File path:
sdks/java/io/elasticsearch-tests/elasticsearch-tests-2/src/test/java/org/apache/beam/sdk/io/elasticsearch/ElasticsearchIOIT.java
##########
@@ -140,4 +140,38 @@ public void testWritePartialUpdate() throws Exception {
elasticsearchIOTestCommonUpdate.setPipeline(pipeline);
elasticsearchIOTestCommonUpdate.testWritePartialUpdate();
}
+
+ /**
+ * This test verifies volume deletes of Elasticsearch. The test dataset
index is cloned and then
+ * around half of the documents are deleted and the other half is partially
updated using bulk
+ * delete request. The test then asserts the documents were deleted
successfully.
+ */
+ @Test
+ public void testWriteWithIsDeletedFnWithPartialUpdates() throws Exception {
+ ElasticsearchIOTestUtils.copyIndex(
+ restClient,
+ readConnectionConfiguration.getIndex(),
+ updateConnectionConfiguration.getIndex());
+ ElasticsearchIOTestCommon elasticsearchIOTestCommonDeleteFn =
+ new ElasticsearchIOTestCommon(updateConnectionConfiguration,
restClient, true);
+ elasticsearchIOTestCommonDeleteFn.setPipeline(pipeline);
+
elasticsearchIOTestCommonDeleteFn.testWriteWithIsDeletedFnWithPartialUpdates();
+ }
+
+ /**
+ * This test verifies volume deletes of Elasticsearch. The test dataset
index is cloned and then
+ * around half of the documents are deleted using bulk delete request. The
test then asserts the
+ * documents were deleted successfully.
+ */
+ @Test
+ public void testWriteWithIsDeletedFnWithoutPartialUpdate() throws Exception {
+ ElasticsearchIOTestUtils.copyIndex(
+ restClient,
+ readConnectionConfiguration.getIndex(),
+ updateConnectionConfiguration.getIndex());
+ ElasticsearchIOTestCommon elasticsearchIOTestCommonDeleteFn =
+ new ElasticsearchIOTestCommon(updateConnectionConfiguration,
restClient, true);
+ elasticsearchIOTestCommonDeleteFn.setPipeline(pipeline);
+
elasticsearchIOTestCommonDeleteFn.testWriteWithIsDeletedFnWithoutPartialUpdate();
+ }
Review comment:
Hi @TheNeuralBit I am also new to the Apache Beam code base 😅. But I
also think there is a good opportunity to remove duplicate codes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 478438)
Time Spent: 1.5h (was: 1h 20m)
> Elasticsearch IO provide delete function
> ----------------------------------------
>
> Key: BEAM-5757
> URL: https://issues.apache.org/jira/browse/BEAM-5757
> Project: Beam
> Issue Type: New Feature
> Components: io-java-elasticsearch
> Affects Versions: 2.7.0
> Reporter: Prasad Marne
> Priority: P3
> Labels: stale-assigned
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Hello beam maintainers,
> I am trying to delete some index from Elasticsearch using beam pipeline. I
> couldn't find and delete function in ElasticsearchIO. It would be nice to
> have and would make sense for cleaning up indexes overtime.
> Â I also checked some other IO classes and they also don't support delete. Not
> sure if beam has some policy against supporting delete.Â
> Â Please guide me on this. I am willing to create pull request if this feature
> makes sense to project contributors.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
