Mike Percy created KUDU-1882:
--------------------------------
Summary: Configuration improvements for Flume Kudu Sink regexp
operations producer
Key: KUDU-1882
URL: https://issues.apache.org/jira/browse/KUDU-1882
Project: Kudu
Issue Type: Bug
Components: flume-sink, integration
Affects Versions: 1.2.0
Reporter: Mike Percy
The RegexpKuduOperationsProducer currently has the following configuration
options that could be improved:
||Property Name || Default || Required? || Description ||
| producer.skipMissingColumn | false | No | What to do if a column in the
Kudu table has no corresponding capture group. If set to true, a warning
message is logged and the operation is still attempted. If set to false, an
exception is thrown and the sink will not process the Event, causing a Flume
Channel rollback.|
| producer.skipBadColumnValue | false | No | What to do if a value in the
pattern match cannot be coerced to the required type. If set to true, a warning
message is logged and the operation is still attempted. If set to false, an
exception is thrown and the sink will not process the Event, causing a Flume
Channel rollback. |
| producer.warnUnmatchedRows | true | No | Whether to log a warning about
payloads that do not match the pattern. If set to false, event bodies with no
matches will be silently dropped. |
It would be an improvement if each of these concepts had the the following
options: {{warn}}, {{ignore}}, {{reject}}
Where {{warn}} would log a warning to the Flume log and continue processing,
{{ignore}} would attempt to continue processing without issuing a warning, and
{{reject}} would throw an exception.
It may be that some fields are nullable or have defaults, potentially due to an
ALTER TABLE, and we don't want to fill the Flume logs with useless warnings.
Users may also want to reject any Events that don't match their regex so they
can correct the configuration and restart Flume without losing those Events.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
