[ https://issues.apache.org/jira/browse/ATLAS-2915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Barbara Eckman updated ATLAS-2915: ---------------------------------- Description: Currently the base types in Atlas do not include AWS Kinesis Stream objects. It would be nice to add a typedef for a kinesis stream. Attributes would include: * streamType string, eg ""Single Region Stream". * awsRegion string: the AWS region in which the kinesis stream endpoint is deployed * shardCount int: number of shards (uniquely identified sequence of data records) in the stream * streamEnvironment enum. Valid values are "unknown", "production", "staging", "QA" and "development" * containsPII boolean: does this stream's data contain Personally Identifiable Information? * aggregationFormat enum. Indicates if/how the records are aggregated within a single kinesis record. Valid values are "none" or "kpl". * contentType enum: serialization format used by the producer of the stream. Valid values are "unknown", "avro", "bson", "csv", "json", "key-value", "kryo", "protobuf", "raw" [ie no consistent schema], "sdp" [confluent-style avro with envelope that specifies schema id surrounding the payload], "thrift", "tlv", "xml", "other". * schemaURL string: A URL to the data schema used by the producer, to facilitate consumption. * avroSchemas: array of avro schema objects (see ATLAS-XXXX) associated with the kinesis stream. was: Currently the base types in Atlas do not include AWS Kinesis Stream objects. It would be nice to add a typedef for a kinesis stream. Attributes would include: * streamType string, eg ""Single Region Stream". * awsRegion string: the AWS region in which the kinesis stream endpoint is deployed * shardCount int: number of shards (uniquely identified sequence of data records) in the stream * streamEnvironment enum. Valid values are "unknown", "production", "staging", "QA" and "development" * containsPII boolean: does this stream's data contain Personally Identifiable Information? * aggregationFormat enum. Indicates if/how the records are aggregated within a single kinesis record. Valid values are "none" or "kpl". * contentType enum: serialization format used by the producer of the stream. Valid values are "unknown", "avro", "bson", "csv", "json", "key-value", "kryo", "protobuf", "raw" [ie no consistent schema], "sdp" [confluent-style avro with envelope that specifies schema id surrounding the payload], "thrift", "tlv", "xml", "other". * schemaURL string: A URL to the data schema used by the producer, to facilitate consumption. * avroSchemas: array of avro schema objects (see ATLAS-associated with the kinesis stream. > AWS Kinesis Stream Typedef for Atlas > ------------------------------------ > > Key: ATLAS-2915 > URL: https://issues.apache.org/jira/browse/ATLAS-2915 > Project: Atlas > Issue Type: New Feature > Reporter: Barbara Eckman > Priority: Critical > > Currently the base types in Atlas do not include AWS Kinesis Stream objects. > It would be nice to add a typedef for a kinesis stream. Attributes would > include: > * streamType string, eg ""Single Region Stream". > * awsRegion string: the AWS region in which the kinesis stream endpoint is > deployed > * shardCount int: number of shards (uniquely identified sequence of data > records) in the stream > * streamEnvironment enum. Valid values are "unknown", "production", > "staging", "QA" and "development" > * containsPII boolean: does this stream's data contain Personally > Identifiable Information? > * aggregationFormat enum. Indicates if/how the records are aggregated within > a single kinesis record. Valid values are "none" or "kpl". > * contentType enum: serialization format used by the producer of the stream. > Valid values are "unknown", "avro", "bson", "csv", "json", "key-value", > "kryo", "protobuf", "raw" [ie no consistent schema], "sdp" [confluent-style > avro with envelope that specifies schema id surrounding the payload], > "thrift", "tlv", "xml", "other". > * schemaURL string: A URL to the data schema used by the producer, to > facilitate consumption. > * avroSchemas: array of avro schema objects (see ATLAS-XXXX) associated with > the kinesis stream. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)