jithin97 commented on a change in pull request #12670:
URL: https://github.com/apache/beam/pull/12670#discussion_r482860614
##########
File path:
sdks/java/io/elasticsearch/src/main/java/org/apache/beam/sdk/io/elasticsearch/ElasticsearchIO.java
##########
@@ -1346,17 +1359,36 @@ private static String lowerCaseOrNull(String input) {
@ProcessElement
public void processElement(ProcessContext context) throws Exception {
- String document = context.element();
- String documentMetadata = getDocumentMetadata(document);
-
- // index is an insert/upsert and update is a partial update (or insert
if not existing)
- if (spec.getUsePartialUpdate()) {
- batch.add(
- String.format(
- "{ \"update\" : %s }%n{ \"doc\" : %s, \"doc_as_upsert\" :
true }%n",
- documentMetadata, document));
+ String document = context.element(); // use configuration and
auto-generated document IDs
+ String documentMetadata = "{}";
+ boolean isDelete = false;
+ if (spec.getIndexFn() != null || spec.getTypeFn() != null ||
spec.getIdFn() != null) {
+ // parse once and reused for efficiency
+ JsonNode parsedDocument = OBJECT_MAPPER.readTree(document);
+ documentMetadata = getDocumentMetadata(parsedDocument);
+ if (spec.getIsDeleteFn() != null) {
+ isDelete = spec.getIsDeleteFn().apply(parsedDocument);
+ // if it is a delete opration, then it is mandatory to specify the
document id using
+ // getIdFn
+ checkArgument(
+ !(isDelete && spec.getIdFn() == null),
+ "Id needs to be specified by withIdFn for delete operation");
Review comment:
Yes, I think that is a better way to handle it. I fixed it.
Thanks.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
