[
https://issues.apache.org/jira/browse/FLUME-1150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13262264#comment-13262264
]
Mubarak Seyed commented on FLUME-1150:
--------------------------------------
Had a discussion with Mike Percy in IRC
[16:47] <mpercy> maybe syntax like sinkName.hdfs.path =
/logs/%{tableName|UNKNOWN_TABLE}/%Y%m%d/%H:%M:%S
[16:48] <mpercy> where tableName is the key of a header
[16:48] <mpercy> and "UNKNOWN_TABLE" is the fallback value
[16:48] <mpercy> the rest is already implemented
[16:48] <mpercy> in BucketPath
[16:59] <mubarak_> ok, make sense
[17:00] <mpercy> there is some logic in BucketPath that might already almost do
this
[17:00] <mubarak_> ok
> Need a way to specify bucket in HDFSEventSink from client
> ---------------------------------------------------------
>
> Key: FLUME-1150
> URL: https://issues.apache.org/jira/browse/FLUME-1150
> Project: Flume
> Issue Type: Improvement
> Components: Sinks+Sources
> Reporter: Mubarak Seyed
>
> We need a way to specify the bucket when event is streamed from client.
> For instance, if i have 5 different log categories (like category [1-5]), if
> i use single source->channel->hdfsSink then all the logs goes to one
> specified directory in HDFS (/logs/<destination>) but we need something like
> /logs/category1
> /category2
> /category3
> /category4
> /category5
> I can use multiplexing (1 source, N channel, and N sinks) but i need to use a
> multiplexing channel processor to route the events(categories) to the proper
> sink.
> What i am thinking is if i specify the category name in header in Event,
> HDFSEventSink (and its helper classes) can create a bucket based on header
> value (and then escape-sequence needs to be honored).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
