I think we are on the same page
On May 23, 2018 at 14:00:25, Simon Elliston Ball (
si...@simonellistonball.com) wrote:
There is certainly a lot of value in the idea of tagging the data with a
config version of some sort for traceability. This is probably a per
topology it goes through thing
There is certainly a lot of value in the idea of tagging the data with a config
version of some sort for traceability. This is probably a per topology it goes
through thing that gives us detailed lineage. Maybe something like the NiFi
provenance approach and a link to a lineage store like atlas
I’ve also talked with J. Zeolla conceptually storing data in hdfs relative
to the version of the schema to produced it, but that may not matter….
So Simon, do you mean that as part of taking a configuration change (
either startup or live while running ) we ‘update’ the metadata/schema, or
