Github user mattf-horton commented on the issue:
https://github.com/apache/metron/pull/622
Hi @nickwallen ,
- I didn't say "unnecessarily complex", I just said "adds complexity". If
we are to have a decodable row key, then something like this would be
necessary. The point of my comments on this is to shift to a top-down
perspective:
- The requirements you're trying to address are, I think, to be able to
query metadata about existing Profiles, for multiple reasons including to not
have old Profiles become inaccessible due to loss of metadata needed for
Profile queries.
- I think a ToC is a better way to do that than a decodable rowkey. It
localizes the metadata instead of having to do a full scan and reason about all
the rowkeys for every metadata query. I proposed the Profile audit log as a
very simple way to implement a ToC, and thanks for saying that better than I
did.
- I'm not against doing a _more_ decodable row key, it just in my mind is
not the simplest nor best solution to the evident requirement. But it does
constitute an architectural improvement.
- As I briefly mentioned, I don't think our current row key is totally
opaque, it just needs a brute-force approach to figure out. Not suitable for
interactive queries, but would be acceptable for a one-time pass to build (or
re-build) the ToC.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
