[jira] [Commented] (METRON-1005) Create Decodable Row Key for Profiler
[ https://issues.apache.org/jira/browse/METRON-1005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088597#comment-16088597 ] ASF GitHub Bot commented on METRON-1005: Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/622 > Create a Profile Audit Log table in HBase This is an interesting idea, Matt. I like it. Along these same lines I was thinking of a table of contents. I think your audit log idea is one way to implement a table of contents. I have looked at a TSDB implementation backed by HBase, OpenTSDB, and they use a table of contents approach. The ToC records metadata about the time series data that is stored. But I don't think these ideas are mutually exclusive with a decodable row key. The decodable row key would allow us to rebuild the ToC should it become corrupted or lost. Are you thinking that a decodable row key is not needed at all? > Create Decodable Row Key for Profiler > - > > Key: METRON-1005 > URL: https://issues.apache.org/jira/browse/METRON-1005 > Project: Metron > Issue Type: Improvement >Affects Versions: 0.3.0 >Reporter: Nick Allen >Assignee: Nick Allen > Fix For: Next + 1 > > > To be able to answer the types of questions that I outlined in METRON-450, we > need a row key that is decodable. Right now there is no logic to decode a > row key, nor is the existing row key easily decodable. > Once the row keys can be decoded, you could scan all of the row keys in the > Profiler's HBase table, decode each of them and extract things like, the > names of all your profiles, the names of entities within a profile, the > period duration of a given profile. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (METRON-1005) Create Decodable Row Key for Profiler
[ https://issues.apache.org/jira/browse/METRON-1005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088589#comment-16088589 ] ASF GitHub Bot commented on METRON-1005: Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/622 > Your proposal has the advantage of making data in HBase self-identifying (if one has the key), which I always like. However, it's a large change and induces yet more complexity What do you find unnecessarily complex here? The code base was already designed to accept different row key implementations. So this change involves the following. 1. The new decodable row key 2. Profiler client logic to instantiate row key builders 3. Profiler client logic to pass parameters to the instantiated row key builders I would agree that I think item 3 is unnecessarily complex. That's where I wanted feedback. I think just passing parameters through an interface method would simplify this a lot. > Create Decodable Row Key for Profiler > - > > Key: METRON-1005 > URL: https://issues.apache.org/jira/browse/METRON-1005 > Project: Metron > Issue Type: Improvement >Affects Versions: 0.3.0 >Reporter: Nick Allen >Assignee: Nick Allen > Fix For: Next + 1 > > > To be able to answer the types of questions that I outlined in METRON-450, we > need a row key that is decodable. Right now there is no logic to decode a > row key, nor is the existing row key easily decodable. > Once the row keys can be decoded, you could scan all of the row keys in the > Profiler's HBase table, decode each of them and extract things like, the > names of all your profiles, the names of entities within a profile, the > period duration of a given profile. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
