Github user cestella commented on a diff in the pull request:
https://github.com/apache/metron/pull/824#discussion_r150887809
--- Diff:
metron-platform/metron-indexing/src/main/java/org/apache/metron/indexing/dao/HBaseDao.java
---
@@ -135,8 +138,9 @@ private Document getDocumentFromResult(Result result)
throws IOException {
Map.Entry<byte[], byte[]> entry= columns.lastEntry();
Long ts = Bytes.toLong(entry.getKey());
if(entry.getValue()!= null) {
- String json = new String(entry.getValue());
- return new Document(json, Bytes.toString(result.getRow()), null, ts);
+ Map<String, Object> json = JSONUtils.INSTANCE.load(new
String(entry.getValue()), new TypeReference<Map<String, Object>>() {
+ });
+ return new Document(json, Bytes.toString(result.getRow()), (String)
json.get(SOURCE_TYPE), ts);
--- End diff --
The problem with the "we're going to revert after ES 5" argument is that it
is likely not going to be immediate. Until then we have coupled HBaseDao and
this feels like a kludge.
If you don't like this solution, can I propose a compromise that will
sidestep this entirely? How about changing the hbase key in HBaseDao to pass
along the sensor (guid + sensor type). This has a couple of advantages:
* It requires no parsing of the document as it can retrieve the sensor type
from the key
* It does not couple our field name conversion from ES with HBase in any way
* It's a small change
I think this neatly sidesteps the issue. Thoughts?
---
