[ https://issues.apache.org/jira/browse/METRON-1005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16088727#comment-16088727 ]
ASF GitHub Bot commented on METRON-1005: ---------------------------------------- Github user mattf-horton commented on the issue: https://github.com/apache/metron/pull/622 Hi @nickwallen , - I didn't say "unnecessarily complex", I just said "adds complexity". If we are to have a decodable row key, then something like this would be necessary. The point of my comments on this is to shift to a top-down perspective: - The requirements you're trying to address are, I think, to be able to query metadata about existing Profiles, for multiple reasons including to not have old Profiles become inaccessible due to loss of metadata needed for Profile queries. - I think a ToC is a better way to do that than a decodable rowkey. It localizes the metadata instead of having to do a full scan and reason about all the rowkeys for every metadata query. I proposed the Profile audit log as a very simple way to implement a ToC, and thanks for saying that better than I did. - I'm not against doing a _more_ decodable row key, it just in my mind is not the simplest nor best solution to the evident requirement. But it does constitute an architectural improvement. - As I briefly mentioned, I don't think our current row key is totally opaque, it just needs a brute-force approach to figure out. Not suitable for interactive queries, but would be acceptable for a one-time pass to build (or re-build) the ToC. > Create Decodable Row Key for Profiler > ------------------------------------- > > Key: METRON-1005 > URL: https://issues.apache.org/jira/browse/METRON-1005 > Project: Metron > Issue Type: Improvement > Affects Versions: 0.3.0 > Reporter: Nick Allen > Assignee: Nick Allen > Fix For: Next + 1 > > > To be able to answer the types of questions that I outlined in METRON-450, we > need a row key that is decodable. Right now there is no logic to decode a > row key, nor is the existing row key easily decodable. > Once the row keys can be decoded, you could scan all of the row keys in the > Profiler's HBase table, decode each of them and extract things like, the > names of all your profiles, the names of entities within a profile, the > period duration of a given profile. -- This message was sent by Atlassian JIRA (v6.4.14#64029)