Github user cestella commented on a diff in the pull request:
https://github.com/apache/metron/pull/824#discussion_r150867558
--- Diff:
metron-platform/metron-indexing/src/main/java/org/apache/metron/indexing/dao/HBaseDao.java
---
@@ -135,8 +138,9 @@ private Document getDocumentFromResult(Result result)
throws IOException {
Map.Entry<byte[], byte[]> entry= columns.lastEntry();
Long ts = Bytes.toLong(entry.getKey());
if(entry.getValue()!= null) {
- String json = new String(entry.getValue());
- return new Document(json, Bytes.toString(result.getRow()), null, ts);
+ Map<String, Object> json = JSONUtils.INSTANCE.load(new
String(entry.getValue()), new TypeReference<Map<String, Object>>() {
+ });
+ return new Document(json, Bytes.toString(result.getRow()), (String)
json.get(SOURCE_TYPE), ts);
--- End diff --
I submitted a PR against this PR with my proposed solution of coupling the
HBaseDao with the Elasticsearch specific field name transformation logic. I
propose that we do this:
* Migrate the FieldNameConverter as a field in the REST config
* Make the IndexDao take the FieldNameConverter in the AccessConfig
* Have the HBaseDao use the parameterized field name converter to transform
`source.type` to the appropriate field.
---
