[
https://issues.apache.org/jira/browse/FLUME-2960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16325926#comment-16325926
]
lixiaosong edited comment on FLUME-2960 at 1/15/18 6:45 AM:
------------------------------------------------------------
[~wenqiao]
你好
我按照
[https://github.com/tinawenqiao/flume]
把编译好的taildir.jar拷贝到flume的lib里
按照这个配置 flume会把符合条件的历史文件从头到尾全部读一遍 这不符合tail的方式啊 是我配置的有问题么
a1.sources.moments.type = taildir
a1.sources.moments.positionFile =
/data/inf_service/flume/offset_json/moments.json
a1.sources.moments.channels = c1
a1.sources.moments.filegroups = f1
a1.sources.moments.filegroups.f1 = /data/logs/uc_public/moments/**/.*
was (Author: handsome7038):
[~wenqiao]
你好
我按照
[https://github.com/tinawenqiao/flume]
把编译好的taildir.jar拷贝到flume的lib里
按照这个配置 flume会把符合条件的历史文件从头到尾全部读一遍 这不符合tail的方式啊 是我配置的有问题么
a1.sources.moments.type = taildir
a1.sources.moments.positionFile =
/data/inf_service/flume/offset_json/moments.json
a1.sources.moments.channels = c1
a1.sources.moments.filegroups = f1
a1.sources.moments.filegroups.f1 = /data/logs/uc_public/moments/*/.*
> Support Wildcards in directory name in TaildirSource
> ----------------------------------------------------
>
> Key: FLUME-2960
> URL: https://issues.apache.org/jira/browse/FLUME-2960
> Project: Flume
> Issue Type: Improvement
> Components: Sinks+Sources
> Affects Versions: 1.7.0
> Reporter: tinawenqiao
> Assignee: tinawenqiao
> Priority: Major
> Labels: wildcards
> Fix For: 1.9.0
>
> Attachments: FLUME-2960_1.patch, FLUME-2960_2.patch,
> FLUME-2960_3.patch, FLUME-2960_4.patch, FLUME-2960_5.patch,
> FLUME-2960_6.patch, FLUME-2960_7.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> In our log management project, we wan't to track many log files like this:
> /app/dir1/log.*
> /app/dir2/log.*
> ...
> /app/dirn/log.*
> But TaildirSource can't support wildcards in filegroup directory name. The
> following config is expected:
> a1.sources.r1.filegroups.fg = /app/\*/log.*
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
