Github user cestella commented on a diff in the pull request:
https://github.com/apache/metron/pull/824#discussion_r150868283
--- Diff:
metron-platform/metron-indexing/src/main/java/org/apache/metron/indexing/dao/HBaseDao.java
---
@@ -135,8 +138,9 @@ private Document getDocumentFromResult(Result result)
throws IOException {
Map.Entry<byte[], byte[]> entry= columns.lastEntry();
Long ts = Bytes.toLong(entry.getKey());
if(entry.getValue()!= null) {
- String json = new String(entry.getValue());
- return new Document(json, Bytes.toString(result.getRow()), null, ts);
+ Map<String, Object> json = JSONUtils.INSTANCE.load(new
String(entry.getValue()), new TypeReference<Map<String, Object>>() {
+ });
+ return new Document(json, Bytes.toString(result.getRow()), (String)
json.get(SOURCE_TYPE), ts);
--- End diff --
I will amend my constant briefly in that I don't like the first option
there as the major flaw in all of these designs is coupling the ES logic within
HBase logic. I think given this is all going away after ES 5 (IMO), we should
probably just use the existing abstractions used in the writers to handle field
name transformations here where we need to transform field names.
---
