[ https://issues.apache.org/jira/browse/FLUME-2960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048628#comment-16048628 ]
junjie SHI commented on FLUME-2960: ----------------------------------- Thanks for [~wenqiao], but I found that the new configuration can not be compatible with the old, I think it is best to be compatible, in my own code I modified the TaildirSource selectByKeys method. private Table<String, String, String> selectByKeys(Map<String, String> map, String[] keys) { Table<String, String, String> result = HashBasedTable.create(); for (String key : keys) { if (map.containsKey(key + FILE_GROUPS_SUFFIX_DIR) && map.containsKey(key + FILE_GROUPS_SUFFIX_FILE)) { result.put(key, FILE_GROUPS_SUFFIX_DIR.substring(1), map.get(key + FILE_GROUPS_SUFFIX_DIR)); result.put(key, FILE_GROUPS_SUFFIX_FILE.substring(1), map.get(key + FILE_GROUPS_SUFFIX_FILE)); } else if (map.containsKey(key)) { // translate the old configuration File fg = new File(map.get(key)); Log.warn("{} is deprecated. Please use the parameter {}", "filegroups." + key, new StringBuffer("filegroups.").append(key) .append(FILE_GROUPS_SUFFIX_DIR) .append(" and filegroups.") .append(key) .append(FILE_GROUPS_SUFFIX_FILE)); result.put(key, FILE_GROUPS_SUFFIX_DIR.substring(1), fg.getParent()); result.put(key, FILE_GROUPS_SUFFIX_FILE.substring(1), fg.getName()); } else { Preconditions.checkState(map.containsKey(key + FILE_GROUPS_SUFFIX_DIR), "Mapping for tailing files is empty or invalid: '" + FILE_GROUPS_PREFIX + (key + FILE_GROUPS_SUFFIX_DIR) + "'"); Preconditions.checkState(map.containsKey(key + FILE_GROUPS_SUFFIX_FILE), "Mapping for tailing files is empty or invalid: '" + FILE_GROUPS_PREFIX + (key + FILE_GROUPS_SUFFIX_FILE + "'")); } } return result; } > Support Wildcards in directory name in TaildirSource > ---------------------------------------------------- > > Key: FLUME-2960 > URL: https://issues.apache.org/jira/browse/FLUME-2960 > Project: Flume > Issue Type: Improvement > Components: Sinks+Sources > Affects Versions: 1.7.0 > Reporter: tinawenqiao > Assignee: tinawenqiao > Labels: wildcards > Fix For: 1.8.0 > > Attachments: FLUME-2960_1.patch, FLUME-2960_2.patch, > FLUME-2960_3.patch, FLUME-2960_4.patch, FLUME-2960_5.patch, FLUME-2960_6.patch > > Original Estimate: 168h > Remaining Estimate: 168h > > In our log management project, we wan't to track many log files like this: > /app/dir1/log.* > /app/dir2/log.* > ... > /app/dirn/log.* > But TaildirSource can't support wildcards in filegroup directory name. The > following config is expected: > a1.sources.r1.filegroups.fg = /app/\*/log.* -- This message was sent by Atlassian JIRA (v6.4.14#64029)