Mubarak Seyed created FLUME-1180:
------------------------------------
Summary: ChannelSelectorFactory creates incorrect selector for
multiplexing selector type
Key: FLUME-1180
URL: https://issues.apache.org/jira/browse/FLUME-1180
Project: Flume
Issue Type: Bug
Components: Sinks+Sources
Reporter: Mubarak Seyed
loadSources() of PropertiesFileConfigurationProvider creates a config with "."
for key values and ChannelSelectorFactory tries to get value for key "type",
which defaults to ReplicatingChannelSelector as "type" will not be found in
config.
{code}
agent.sources = avro-source1
agent.sinks = hdfs-sink1 hdfs-sink2
agent.channels = mem-channel-1 mem-channel-2
#source
agent.sources.avro-source1.channels = mem-channel-1 mem-channel-2
agent.sources.avro-source1.type = avro
agent.sources.avro-source1.bind = 0.0.0.0
agent.sources.avro-source1.port = 9090
agent.sources.avro-source1.selector.type = multiplexing
agent.sources.avro-source1.selector.header = tableName
agent.sources.avro-source1.selector.mapping.table1 = mem-channel-1
agent.sources.avro-source1.selector.mapping.table2 = mem-channel-2
agent.sources.avro-source1.selector.default = mem-channel-1
#sinks
agent.sinks.hdfs-sink1.channel = mem-channel-1
agent.sinks.hdfs-sink2.channel = mem-channel-2
agent.sinks.hdfs-sink1.type = hdfs
agent.sinks.hdfs-sink1.hdfs.batchSize = 1000
agent.sinks.hdfs-sink1.hdfs.txnEventMax = 1000
agent.sinks.hdfs-sink1.hdfs.fileType = DataStream
agent.sinks.hdfs-sink1.hdfs.rollSize = 1073741824
agent.sinks.hdfs-sink1.hdfs.rollCount = 0
agent.sinks.hdfs-sink1.hdfs.rollInterval = 0
agent.sinks.hdfs-sink1.serializer =
com.mycompany.Table1AvroEventSerializer$Builder
agent.sinks.hdfs-sink1.serializer.compressionCodec = snappy
agent.sinks.hdfs-sink1.serializer.syncIntervalBytes = 2048000
agent.sinks.hdfs-sink1.hdfs.path = hdfs://NN/logs/table1/%m%d%Y/%H
agent.sinks.hdfs-sink2.type = hdfs
agent.sinks.hdfs-sink2.hdfs.batchSize = 1000
agent.sinks.hdfs-sink2.hdfs.txnEventMax = 1000
agent.sinks.hdfs-sink2.hdfs.fileType = DataStream
agent.sinks.hdfs-sink2.hdfs.rollSize = 1073741824
agent.sinks.hdfs-sink2.hdfs.rollCount = 0
agent.sinks.hdfs-sink2.hdfs.rollInterval = 300
agent.sinks.hdfs-sink2.serializer =
com.mycompany.Table2AvroEventSerializer$Builder
agent.sinks.hdfs-sink2.serializer.compressionCodec = snappy
agent.sinks.hdfs-sink2.serializer.syncIntervalBytes = 2048000
agent.sinks.hdfs-sink2.hdfs.path = hdfs://NN/logs/table2/%m%d%Y/%H
agent.channels.mem-channel-1.type = memory
agent.channels.mem-channel-1.capacity = 100000
agent.channels.mem-channel-1.transactionCapacity = 1000
agent.channels.mem-channel-2.type = memory
agent.channels.mem-channel-2.capacity = 100000
agent.channels.mem-channel-2.transactionCapacity = 1000
{code}
but the config for ??selector?? gets parsed as
{code}
{.type=multiplexing, .default=mem-channel-1, .mapping.table2=mem-channel-2,
.mapping.table1=mem-channel-1, .header=tableName}
{code}
The fix is in ??loadSources()?? of ??PropertiesFileConfigurationProvider??
{code}
Map<String, String> selectorConfig = context.getSubProperties("selector" +
"."); //fix
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
