-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51182/
-----------------------------------------------------------
(Updated Aug. 24, 2016, 8:46 a.m.)
Review request for Flume.
Bugs: FLUME-2954
https://issues.apache.org/jira/browse/FLUME-2954
Repository: flume-git
Description
-------
--------------------------------------------------------------------------------
flume-ng-channel ---
flume-jdbc-channel ---
JdbcChannelProviderImpl#98 <- fail properties <REMOVED>
JdbcChannelProviderImpl#261 #431 <- fail properties: jdbc url
might include password <KEPT><FOLLOWUP IN JIRA>
flume-kafka-channel ---
KafkaChannel#230 #253 <- fail properties <REMOVED>
--------------------------------------------------------------------------------
flume-ng-configuration ---
FlumeConfiguration#315 #372 <- fail properties <DRIVE BY
PROPERTY>
--------------------------------------------------------------------------------
flume-ng-core ---
SyslogAvroEventSerializer#150 <- fail data: SyslogEvent.message
gets logged <DRIVE BY PROPERTY>
GangliaServer#224 #245 <- safe data: only flume
component metrics data <KEPT>
LoggerSink#95 <- fail data: on purpose <KEPT>
AvroSource#347 <- fail data: log whole message
<DRIVE BY PROPERTY>
MultiportSyslogTCPSource#360 <- fail data: log whole message
<DRIVE BY PROPERTY>
BLOBHandler#70 <- fail data: logs http request
headers <DRIVE BY PROPERTY>
-------------------------------------------------------------------q-------------
flume-ng-embedded-agent ---
EmbeddedAgent#155 <- fail properties: printing all
config <DRIVE BY PROPERTY>
--------------------------------------------------------------------------------
flume-ng-sinks ---
flume-hive-sink ---
HiveEndPoint has an URI field. <- fail properties
<KEPT><FOLLOWUP IN JIRA>
It may contain private data
(URI string may contain password) as it is
excessively logged within this module.
Appears in HiveSink#298 #342 #400 #403 #428,
HiveWriter#210 #319 #330 #337 #353 #365 #368 #407...)
HiveEndPoint is also attached to exception logs as well
flume-ng-hbase-sink ---
AsyncHBaseSink#641 <- safe data: error details gets
logged in case of failure <KEPT>
flume-ng-kafka-sink ---
KafkaSink#179 <- fail data: log whole message
<REMOVED>
KafkaSink#304 <- fail properties <REMOVED>
flume-ng-morphline-solr-sink ---
BlobHandler#98 #113 <- fail data: log http request
headers <DRIVE BY PROPERTY>
MorphlineSink#139 <- fail data: logs event <DRIVE
BY PROPERTY>
--------------------------------------------------------------------------------
flume-ng-sources ---
flume-kafka-source ---
KafkaSource#247 <- fail data: log whole <DRIVE BY
PROPERTY>
flume-twitter-source ---
TwitterSource#110-113 <- fail properties <REMOVED>
--------------------------------------------------------------------------------
Diffs (updated)
-----
conf/flume-env.ps1.template 8bf535a
conf/flume-env.sh.template c8b660f
flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/impl/JdbcChannelProviderImpl.java
845b794
flume-ng-channels/flume-kafka-channel/src/main/java/org/apache/flume/channel/kafka/KafkaChannel.java
90e3288
flume-ng-configuration/src/main/java/org/apache/flume/conf/FlumeConfiguration.java
9b3a434
flume-ng-core/src/main/java/org/apache/flume/source/AvroSource.java 8b9b956
flume-ng-core/src/main/java/org/apache/flume/source/MultiportSyslogTCPSource.java
b9f2438
flume-ng-core/src/main/java/org/apache/flume/source/http/BLOBHandler.java
e24d4c6
flume-ng-core/src/test/java/org/apache/flume/serialization/SyslogAvroEventSerializer.java
05af3b1
flume-ng-doc/sphinx/FlumeUserGuide.rst 5e677c6
flume-ng-embedded-agent/src/main/java/org/apache/flume/agent/embedded/EmbeddedAgent.java
ad3e138
flume-ng-sdk/src/main/java/org/apache/flume/util/LogRawDataUtil.java
PRE-CREATION
flume-ng-sinks/flume-ng-kafka-sink/src/main/java/org/apache/flume/sink/kafka/KafkaSink.java
9453546
flume-ng-sinks/flume-ng-morphline-solr-sink/src/main/java/org/apache/flume/sink/solr/morphline/BlobHandler.java
ca7614a
flume-ng-sinks/flume-ng-morphline-solr-sink/src/main/java/org/apache/flume/sink/solr/morphline/MorphlineSink.java
f7a73f3
flume-ng-sources/flume-kafka-source/src/main/java/org/apache/flume/source/kafka/KafkaSource.java
90e4715
flume-ng-sources/flume-twitter-source/src/main/java/org/apache/flume/source/twitter/TwitterSource.java
f5c8328
Diff: https://reviews.apache.org/r/51182/diff/
Testing
-------
compiles, site builds, all unit test passes, distribution target handles the
system properties as expected:
bin/flume-ng agent --conf conf --conf-file
../../../../../flume-conf/flume-log.conf --name a1
-Dflume.root.logger=DEBUG,console -Dorg.apache.flume.log.printconfig=true
-Dorg.apache.flume.log.rawdata=true (with and without the extra properties)
Thanks,
Attila Simon
