Github user justinleet commented on a diff in the pull request:
https://github.com/apache/metron/pull/824#discussion_r150230733
--- Diff:
metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchDao.java
---
@@ -256,59 +256,91 @@ public Document getLatest(final String guid, final
String sensorType) throws IOE
return ret.orElse(null);
}
+ @Override
+ public Iterable<Document> getAllLatest(final Collection<String> guids,
final Collection<String> sensorTypes) throws IOException {
+ List<Document> documents = searchByGuids(
+ guids
+ , sensorTypes
+ , hit -> {
+ Long ts = 0L;
+ String doc = hit.getSourceAsString();
+ String sourceType =
Iterables.getFirst(Splitter.on("_doc").split(hit.getType()), null);
+ try {
+ return Optional.of(new Document(doc, hit.getId(), sourceType,
ts));
+ } catch (IOException e) {
+ throw new IllegalStateException("Unable to retrieve latest: "
+ e.getMessage(), e);
+ }
+ }
+
+ );
+ return documents;
+ }
+
+ <T> Optional<T> searchByGuid(String guid, String sensorType,
+ Function<SearchHit, Optional<T>> callback) {
+ Collection<String> sensorTypes = sensorType != null ?
Collections.singleton(sensorType) : null;
+ List<T> results = searchByGuids(Collections.singleton(guid),
sensorTypes, callback);
+ if (results.size() > 0) {
+ return Optional.of(results.get(0));
+ } else {
+ return Optional.empty();
+ }
+ }
+
/**
* Return the search hit based on the UUID and sensor type.
* A callback can be specified to transform the hit into a type T.
* If more than one hit happens, the first one will be returned.
*/
- <T> Optional<T> searchByGuid(String guid, String sensorType,
+ <T> List<T> searchByGuids(Collection<String> guids, Collection<String>
sensorTypes,
Function<SearchHit, Optional<T>> callback) {
QueryBuilder query;
- if (sensorType != null) {
- query = QueryBuilders.idsQuery(sensorType + "_doc").ids(guid);
+ if (sensorTypes != null) {
+ String[] types = sensorTypes.stream().map(sensorType -> sensorType +
"_doc").toArray(String[]::new);
+ query = QueryBuilders.idsQuery(types).ids(guids);
} else {
- query = QueryBuilders.idsQuery().ids(guid);
+ query = QueryBuilders.idsQuery().ids(guids);
}
SearchRequestBuilder request = client.prepareSearch()
--- End diff --
This is only going to return 10 max results, from the default size on the
query. This was fine before (since we only returned one), but it's going to be
incomplete with the refactor.
---
