[jira] [Resolved] (FLUME-199) Unit tests should hunt for available ports if defaults are in use
[ https://issues.apache.org/jira/browse/FLUME-199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-199. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Unit tests should hunt for available ports if defaults are in use - Key: FLUME-199 URL: https://issues.apache.org/jira/browse/FLUME-199 Project: Flume Issue Type: Improvement Components: Test Affects Versions: v0.9.0 Reporter: E. Sammer Priority: Minor Fix For: v0.9.5 Currently, some of the unit tests fail if flume is running while tests are running. This leads to false positives in failure detection and slows down the dev cycle as one has to remember not to run flume while tests are in progress. It would be an improvement to hunt for an open port if the default port(s) are already in use during testing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-233) Provide configuration option to allow date rollover on different timezone
[ https://issues.apache.org/jira/browse/FLUME-233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-233. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Provide configuration option to allow date rollover on different timezone - Key: FLUME-233 URL: https://issues.apache.org/jira/browse/FLUME-233 Project: Flume Issue Type: New Feature Components: Sinks+Sources Affects Versions: v0.9.1 Reporter: Disabled imported user Priority: Minor Fix For: v0.9.5 It would be nice to have a configuration option to allow output bucketing to function on a timezone other than the local one. This would allow the following sink configuration to move to the next day based on GMT rather than the local time zone: collectorSink( hdfs://namenode/web/%Y/%m/%d/, web-) I think this can be accomplished currently by setting UOPTS=-Duser.timezone=UTC when starting flume but it would be nice to have a config option rather than having to edit the flume service init script to make the change persistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-225) Can we remove syslogTcp1?
[ https://issues.apache.org/jira/browse/FLUME-225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-225. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Can we remove syslogTcp1? - Key: FLUME-225 URL: https://issues.apache.org/jira/browse/FLUME-225 Project: Flume Issue Type: Task Components: Sinks+Sources Affects Versions: v0.9.4 Reporter: Bruce Mitchener Priority: Minor Fix For: v0.9.5 syslogTcp1 seems to something that could go away... is it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-219) Recipe for batching and wire compression.
[ https://issues.apache.org/jira/browse/FLUME-219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-219. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Recipe for batching and wire compression. - Key: FLUME-219 URL: https://issues.apache.org/jira/browse/FLUME-219 Project: Flume Issue Type: Documentation Components: Docs, Easy Affects Versions: v0.9.4 Reporter: Jonathan Hsieh Fix For: v0.9.5 Batching is a way to greatly improve the performance and efficiency of the rpc calls used to transfer data between nodes. There should be a quick write up about how to configure this in the cookbook. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-236) Builders shouldn't just rely on Boolean.parseBoolean.
[ https://issues.apache.org/jira/browse/FLUME-236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-236. -- Resolution: Won't Fix Won't fix. 0.X branch not maintained anymore Builders shouldn't just rely on Boolean.parseBoolean. -- Key: FLUME-236 URL: https://issues.apache.org/jira/browse/FLUME-236 Project: Flume Issue Type: Bug Affects Versions: v0.9.0, v0.9.1 Reporter: Jonathan Hsieh Fix For: v0.9.5 The semantics of parseBoolean (http://download-llnw.oracle.com/javase/6/docs/api/java/lang/Boolean.html#booleanValue()) says that and string is parsable as a boolean. So Boolean.parseBoolean(yes) = false; This may cause unexpected results for the user. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-237) documentation should describe flume installation issues - in particular specifying JVM heap sizes
[ https://issues.apache.org/jira/browse/FLUME-237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-237. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore documentation should describe flume installation issues - in particular specifying JVM heap sizes - Key: FLUME-237 URL: https://issues.apache.org/jira/browse/FLUME-237 Project: Flume Issue Type: Improvement Components: Docs Affects Versions: v0.9.0, v0.9.1 Reporter: Patrick Hunt Fix For: v0.9.5 I've updated the faq with some basic detail: Why is Flume running out of memory? http://github.com/cloudera/flume/wiki/FAQ however the flume documentation (man page?) should provide this detail. Probably it should include other aspects of setting up flume, gotchas etc... ---FAQ entry - Ensure that you have provided the jvm with sufficient heap space. By default Flume starts the jvm with it's default heap allocation, which differs depending on the jvm version, the host type (os, 32/64 bit, etc...), total host memory available, as well as other issues. The environment variable UOPTS can be used to pass additional jvm parameters when running Flume. e.g. $ UOPTS=-Xms1g -Xmx2g bin/flume node which starts a flume node with an initial heap of one gig and a max heap of two gig. See java -h or java -X for more details on available jvm options. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-238) Add more details in source/sink/decorator semantics
[ https://issues.apache.org/jira/browse/FLUME-238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-238. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Add more details in source/sink/decorator semantics --- Key: FLUME-238 URL: https://issues.apache.org/jira/browse/FLUME-238 Project: Flume Issue Type: Improvement Affects Versions: v0.9.2 Reporter: Jonathan Hsieh Fix For: v0.9.5 FLUME-155 is a good first cut but more improvements can be done. This jira is the place holder for some of these issues. Feedback from Vibhor on first cut. (FLUME-155) Most of it looks good and very interesting. I'll give the high level comments here, gave minor comments on paper to Jon. 1) Missing the main Threading detail, it should be explained that there is a main driver thread, and different decorators can create their own threads too. 2) In the section talking about the sinks with buffered memories it says close() from a closing state should blocks until resources are freed, it should attempt to flush its buffers before returning. For example, so if a network source has some buffered data, the network connection should be closed to prevent new data from entering, and then the buferred data should be flushed, but at the same time it is said that the next() should keep on pulling data from the buffer even in the closing state. As both these methods (close() and next()) can be from different threads, it should be clearly mentioned how are we handling the race conditions while operating on the same data. 3) In both Source/Sink descriptions, it is said that if a timeout happens in a closing state, then we move to a close state. But do we ensure that we have given up all the resources before it? Else there will be an issue when someone later will try to open this decorator. 4) It should be clearly explained why and how we handle the Thread-Interrupted-Exception when a append() on sink gets interrupted while it is blocked. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-249) Use Hadoop's org.apache.hadoop.hdfs.MiniDFSCluster to test namenode failures.
[ https://issues.apache.org/jira/browse/FLUME-249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-249. -- Resolution: Won't Fix Won't fix. 0.X branch not maintained anymore Use Hadoop's org.apache.hadoop.hdfs.MiniDFSCluster to test namenode failures. - Key: FLUME-249 URL: https://issues.apache.org/jira/browse/FLUME-249 Project: Flume Issue Type: Improvement Components: Sinks+Sources, Technical Debt, Test Affects Versions: v0.9.4 Reporter: Jonathan Hsieh Fix For: v0.9.5 FLUME-229 fixed a problem having to do with name node failures in a flume collector. Some tests were lacking because of dependencies on an external system (Hadoop Namenode). This issue suggests using a Hadoop internal testing DFS for the purpose of unit tests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-245) Default naming for sequence files breaks ability to use s3n as a sink
[ https://issues.apache.org/jira/browse/FLUME-245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-245. -- Resolution: Won't Fix Won't fix. 0.X branch not maintained anymore Default naming for sequence files breaks ability to use s3n as a sink - Key: FLUME-245 URL: https://issues.apache.org/jira/browse/FLUME-245 Project: Flume Issue Type: Bug Components: Sinks+Sources Affects Versions: v0.9.1 Reporter: Disabled imported user Priority: Minor Fix For: v0.9.5 The default naming scheme for file includes a + character before the timezone. This creates an invalid URL when trying to access the file on the S3 service. A log of the problem is here: http://gist.github.com/599979 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-251) time filter for tail-dir
[ https://issues.apache.org/jira/browse/FLUME-251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-251. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore time filter for tail-dir Key: FLUME-251 URL: https://issues.apache.org/jira/browse/FLUME-251 Project: Flume Issue Type: New Feature Affects Versions: v0.9.4 Reporter: Anurag Phadke Fix For: v0.9.5 tailDir currently supports tail'ing all file names that match a regex for a given directory. Adding time-stamp support to only tail files created after a given time-period would be nice. for eg: Archive folders get data that's added daily and also contain 'n' months old data. There are instances where we only need files that are 'x' days old (x n) sample tailDir call would look something like: tailDir(path[,fileregex=*.gz, createdAfter=2010-09-20]); -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (FLUME-199) Unit tests should hunt for available ports if defaults are in use
[ https://issues.apache.org/jira/browse/FLUME-199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Santiago M. Mola updated FLUME-199: --- Affects Version/s: (was: v0.9.0) v1.5.0.1 Fix Version/s: (was: v0.9.5) v1.6.0 Unit tests should hunt for available ports if defaults are in use - Key: FLUME-199 URL: https://issues.apache.org/jira/browse/FLUME-199 Project: Flume Issue Type: Improvement Components: Test Affects Versions: v1.5.0.1 Reporter: E. Sammer Assignee: Santiago M. Mola Priority: Minor Fix For: v1.6.0 Currently, some of the unit tests fail if flume is running while tests are running. This leads to false positives in failure detection and slows down the dev cycle as one has to remember not to run flume while tests are in progress. It would be an improvement to hunt for an open port if the default port(s) are already in use during testing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (FLUME-199) Unit tests should hunt for available ports if defaults are in use
[ https://issues.apache.org/jira/browse/FLUME-199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Santiago M. Mola reopened FLUME-199: Assignee: Santiago M. Mola Unit tests should hunt for available ports if defaults are in use - Key: FLUME-199 URL: https://issues.apache.org/jira/browse/FLUME-199 Project: Flume Issue Type: Improvement Components: Test Affects Versions: v1.5.0.1 Reporter: E. Sammer Assignee: Santiago M. Mola Priority: Minor Fix For: v1.6.0 Currently, some of the unit tests fail if flume is running while tests are running. This leads to false positives in failure detection and slows down the dev cycle as one has to remember not to run flume while tests are in progress. It would be an improvement to hunt for an open port if the default port(s) are already in use during testing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLUME-199) Unit tests should hunt for available ports if defaults are in use
[ https://issues.apache.org/jira/browse/FLUME-199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14197839#comment-14197839 ] Santiago M. Mola commented on FLUME-199: Reopening this issue. I started to work on this for 1.6.0 on a branch. Unit tests should hunt for available ports if defaults are in use - Key: FLUME-199 URL: https://issues.apache.org/jira/browse/FLUME-199 Project: Flume Issue Type: Improvement Components: Test Affects Versions: v1.5.0.1 Reporter: E. Sammer Assignee: Santiago M. Mola Priority: Minor Fix For: v1.6.0 Currently, some of the unit tests fail if flume is running while tests are running. This leads to false positives in failure detection and slows down the dev cycle as one has to remember not to run flume while tests are in progress. It would be an improvement to hunt for an open port if the default port(s) are already in use during testing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-269) Recipe for bucketing data using value decorators
[ https://issues.apache.org/jira/browse/FLUME-269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-269. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Recipe for bucketing data using value decorators Key: FLUME-269 URL: https://issues.apache.org/jira/browse/FLUME-269 Project: Flume Issue Type: Documentation Components: Docs, Easy Affects Versions: v0.9.4 Reporter: Jonathan Hsieh Fix For: v0.9.5 This general question has been asked several times now and should be written up and committed as a recipe. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-261) Watchdog should use raw bytes instead of encoding/decoding character sets.
[ https://issues.apache.org/jira/browse/FLUME-261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-261. -- Resolution: Won't Fix Won't fix. 0.X branch not maintained anymore Watchdog should use raw bytes instead of encoding/decoding character sets. -- Key: FLUME-261 URL: https://issues.apache.org/jira/browse/FLUME-261 Project: Flume Issue Type: Bug Components: Shell Affects Versions: v0.9.0, v0.9.1 Reporter: Jonathan Hsieh Fix For: v0.9.5 I sent some chinese characters (我是一個中國人) into the body of a flume event (via tail), and when running a node that dumps to console with the watchdog, the UTF-8 bytes get re-UTF-8'ed. When running without the watchdog, we get exepected chinese characters. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLUME-199) Unit tests should hunt for available ports if defaults are in use
[ https://issues.apache.org/jira/browse/FLUME-199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14197844#comment-14197844 ] Ashish Paliwal commented on FLUME-199: -- Lot of tests already use findAvailablePort() API to get the port to bind to. Can you elaborate what you intend to fix? Unit tests should hunt for available ports if defaults are in use - Key: FLUME-199 URL: https://issues.apache.org/jira/browse/FLUME-199 Project: Flume Issue Type: Improvement Components: Test Affects Versions: v1.5.0.1 Reporter: E. Sammer Assignee: Santiago M. Mola Priority: Minor Fix For: v1.6.0 Currently, some of the unit tests fail if flume is running while tests are running. This leads to false positives in failure detection and slows down the dev cycle as one has to remember not to run flume while tests are in progress. It would be an improvement to hunt for an open port if the default port(s) are already in use during testing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-279) Add options so that auto* will pick specific network interfaces
[ https://issues.apache.org/jira/browse/FLUME-279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-279. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Add options so that auto* will pick specific network interfaces --- Key: FLUME-279 URL: https://issues.apache.org/jira/browse/FLUME-279 Project: Flume Issue Type: New Feature Components: Master, Node Affects Versions: v0.9.4 Reporter: Jonathan Hsieh Fix For: v0.9.5 Tobias Schlottke I would find it useful if there was a startup-parameter to change 0.0.0.0 to eg. 192.168.1.1. Like there is -n for the nodes name. Maybe I'm confused but I see no real alternatives to that if I want Flume NOT to use a specific interface (eg: my public interface) in combination with auto sinks sources. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-282) Add Jabber/XMPP Sink
[ https://issues.apache.org/jira/browse/FLUME-282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-282. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore We can reopen the issue, if this Sink is needed. Haven't seen a related thread for a while Add Jabber/XMPP Sink Key: FLUME-282 URL: https://issues.apache.org/jira/browse/FLUME-282 Project: Flume Issue Type: Wish Components: Sinks+Sources Affects Versions: v0.9.4 Reporter: Disabled imported user Priority: Minor Fix For: v0.9.5 User specified Jabber/XMPP server can be added as a Flume sink. Sink will send the incoming event data as a instant message into a configured IM address(JID) using XMPP protocol. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-287) flume to support using template on syslog-ng messages
[ https://issues.apache.org/jira/browse/FLUME-287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-287. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore flume to support using template on syslog-ng messages - Key: FLUME-287 URL: https://issues.apache.org/jira/browse/FLUME-287 Project: Flume Issue Type: New Feature Affects Versions: v0.9.1 Reporter: Disabled imported user Fix For: v0.9.5 We would like to see flume collect the formatted messages from syslog-ng PS. Some anonymous user created a jira in HUE-273. I'm re-creating it in Flume. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-294) Flume shell should use logging instead of System.out and System.err
[ https://issues.apache.org/jira/browse/FLUME-294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-294. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Flume shell should use logging instead of System.out and System.err --- Key: FLUME-294 URL: https://issues.apache.org/jira/browse/FLUME-294 Project: Flume Issue Type: Improvement Components: Shell Affects Versions: v0.9.4 Reporter: Jonathan Hsieh Fix For: v0.9.5 Currently the shell uses System.out and System.err. Some folks say that this should use logging, even in the interactive shell environment. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-288) Support chown and chmod in post log roll hook
[ https://issues.apache.org/jira/browse/FLUME-288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-288. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Support chown and chmod in post log roll hook - Key: FLUME-288 URL: https://issues.apache.org/jira/browse/FLUME-288 Project: Flume Issue Type: New Feature Components: Sinks+Sources Affects Versions: v0.9.4 Reporter: Disabled imported user Priority: Minor Fix For: v0.9.5 We'd like to be able to change the owner and permissions of files created by flume on HDFS using a post roll hook of some kind. We're using CDH3b2 CFR https://groups.google.com/a/cloudera.org/group/flume-user/browse_thread/thread/3d1fd2534c3f3b99/a9b37fd75ee2e473#a9b37fd75ee2e473 Thanks! -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLUME-173) Create standard unit tests for Sinks
[ https://issues.apache.org/jira/browse/FLUME-173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14197858#comment-14197858 ] Santiago M. Mola commented on FLUME-173: Reopening. This is still applicable and would be a great improvement for testing. Create standard unit tests for Sinks Key: FLUME-173 URL: https://issues.apache.org/jira/browse/FLUME-173 Project: Flume Issue Type: New Feature Components: Sinks+Sources Affects Versions: v1.5.0.1 Reporter: Jonathan Hsieh Fix For: v1.6.0 Create standard tests for sinks -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (FLUME-173) Create standard unit tests for Sinks
[ https://issues.apache.org/jira/browse/FLUME-173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Santiago M. Mola updated FLUME-173: --- Affects Version/s: (was: v0.9.4) v1.5.0.1 Fix Version/s: (was: v0.9.5) v1.6.0 Create standard unit tests for Sinks Key: FLUME-173 URL: https://issues.apache.org/jira/browse/FLUME-173 Project: Flume Issue Type: New Feature Components: Sinks+Sources Affects Versions: v1.5.0.1 Reporter: Jonathan Hsieh Fix For: v1.6.0 Create standard tests for sinks -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-298) Try getting rid of things that use Apache Commons Logging
[
https://issues.apache.org/jira/browse/FLUME-298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashish Paliwal resolved FLUME-298.
--
Resolution: Won't Fix
Fix Version/s: v0.9.5
Won't fix. 0.X branch not maintained anymore
Try getting rid of things that use Apache Commons Logging
-
Key: FLUME-298
URL: https://issues.apache.org/jira/browse/FLUME-298
Project: Flume
Issue Type: Improvement
Affects Versions: v0.9.4
Reporter: Bruce Mitchener
Fix For: v0.9.5
With the change from log4j over to slf4j, it would be nice to also get rid of
all usages of Apache Commons Logging.
I folded a change for GangliaSink into my fixes for FLUME-283, but that
leaves HttpStatusServer. Unfortunately, this one isn't as easy to fix due to:
{code}
ReflectionUtils.logThreadInfo(LOG, jsp requested, 1);
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (FLUME-300) Master web page should report all partial configuration translation information
[ https://issues.apache.org/jira/browse/FLUME-300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-300. -- Resolution: Won't Fix Won't fix. 0.X branch not maintained anymore Master web page should report all partial configuration translation information --- Key: FLUME-300 URL: https://issues.apache.org/jira/browse/FLUME-300 Project: Flume Issue Type: New Feature Components: Master Affects Versions: v0.9.4 Reporter: Jonathan Hsieh Fix For: v0.9.5 There are occasionally some strange config translation failures and exposing intermediate translations would make diagnosing these problems much easier. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-312) agentDFOSink returns attempt failed.
[
https://issues.apache.org/jira/browse/FLUME-312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashish Paliwal resolved FLUME-312.
--
Resolution: Won't Fix
Fix Version/s: v0.9.5
Won't fix. 0.X branch not maintained anymore
agentDFOSink returns attempt failed.
Key: FLUME-312
URL: https://issues.apache.org/jira/browse/FLUME-312
Project: Flume
Issue Type: Bug
Affects Versions: v0.9.4
Reporter: Anurag Phadke
Fix For: v0.9.5
h1: exec(/usr/bin/perl /home/aphadke/buildbot/rpc.pl,false,true,3) |
agentDFOSink(h2,35853);
h2 : collectorSource(35853) |
collectorSink(hdfs://h3/flume/%{host}/%Y-%m-%d/%H,build-logs);
The above config returns following error on h1:
10/11/03 10:49:45 INFO thrift.ThriftEventSink: ThriftEventSink on port 35853
closed
10/11/03 10:49:45 INFO diskfailover.DiskFailoverSource: end of file
com.cloudera.flume.agent.diskfailover.NaiveFileFailoverManager$StateChangeDeco@6d581e80
10/11/03 10:49:45 INFO diskfailover.NaiveFileFailoverManager: opening new
file for log.0025.20101103-104921136-0700.4302031757228167.seq
10/11/03 10:49:45 INFO debug.StubbornAppendSink: Opening Stubborn Append Sink
com.cloudera.flume.handlers.debug.StubbornAppendSink@154e45b3
10/11/03 10:49:45 INFO debug.InsistentOpenDecorator: open attempt 0 failed,
backoff (1000ms): Failed to open thrift event sink at h2:35853 :
java.net.ConnectException: Connection refused
10/11/03 10:49:46 INFO debug.InsistentOpenDecorator: open attempt 1 failed,
backoff (2000ms): Failed to open thrift event sink at h2:35853 :
java.net.ConnectException: Connection refused
10/11/03 10:49:48 INFO debug.InsistentOpenDecorator: open attempt 2 failed,
backoff (4000ms): Failed to open thrift event sink at h2:35853 :
java.net.ConnectException: Connection refused
10/11/03 10:49:52 INFO debug.InsistentOpenDecorator: open attempt 3 failed,
backoff (8000ms): Failed to open thrift event sink at h2:35853 :
java.net.ConnectException: Connection refused
10/11/03 10:49:55 INFO hdfs.SeqfileEventSink: closed
/tmp/flume/agent/h1/dfo_writing/log.0019.20101103-104945436-0700.4302056056679167.seq
10/11/03 10:49:55 INFO diskfailover.NaiveFileFailoverManager: File lives in
/tmp/flume/agent/h1/dfo_writing/log.0019.20101103-104945436-0700.4302056056679167.seq
10/11/03 10:49:55 INFO hdfs.SeqfileEventSink: constructed new seqfile event
sink:
file=/tmp/flume/agent/h1/dfo_writing/log.0025.20101103-104955517-0700.4302066137680167.seq
Any idea why h1 is trying to open port 35853 and failing? Do I need to add
something to my config?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (FLUME-314) Flume webpages become unavailable when using Avro RPC and node gets physically disconnected
[ https://issues.apache.org/jira/browse/FLUME-314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-314. -- Resolution: Won't Fix Won't fix. 0.X branch not maintained anymore Flume webpages become unavailable when using Avro RPC and node gets physically disconnected --- Key: FLUME-314 URL: https://issues.apache.org/jira/browse/FLUME-314 Project: Flume Issue Type: Bug Components: Master, Node Affects Versions: v0.9.2 Reporter: Jonathan Hsieh Fix For: v0.9.5 Web page not available error page happens if a node using Avro rpc is physically disconnected from the network. Ex: Bring connected node or master and go to its web page. (it works!) Physically disconnect machine from network (this webpage is not available) Reconnect node (it works!) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-319) Logical nodes should act as roles and be spawned on multiple hosts
[ https://issues.apache.org/jira/browse/FLUME-319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-319. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Logical nodes should act as roles and be spawned on multiple hosts -- Key: FLUME-319 URL: https://issues.apache.org/jira/browse/FLUME-319 Project: Flume Issue Type: Improvement Affects Versions: v0.9.4 Reporter: Mathias Herberts Fix For: v0.9.5 Today logical nodes allow multiple configurations to be run in a single JVM. When running clusters of similar machines, it happens quite often that all machines behave the same as far as flume is concerned, they will for example all collect similar logs and metrics. Today a logical configuration has to be defined for each machine and then spawned on the host. This is tedious and error prone. It would be way better if we could define a logical node once (let's call it a role) and spawn it on each node of the cluster. exec config myrole myflow mysource mysink exec spawn node1 myrole exec spawn node2 myrole exec spawn node3 myrole ... This could probably be done by creating a 'logicalnode@physicalnode' virtual node upon spawn. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-321) Sources should document the metadata fields it provides on events.
[ https://issues.apache.org/jira/browse/FLUME-321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-321. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Sources should document the metadata fields it provides on events. --- Key: FLUME-321 URL: https://issues.apache.org/jira/browse/FLUME-321 Project: Flume Issue Type: Improvement Components: Docs, Easy, Sinks+Sources Affects Versions: v0.9.4 Reporter: Jonathan Hsieh Fix For: v0.9.5 For example, tail and tailDir add a attribute for the filename of the source file, exec includes the command line used to generate data, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-323) Exec should use Selector part of NIO API to be more responsive
[ https://issues.apache.org/jira/browse/FLUME-323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-323. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Exec should use Selector part of NIO API to be more responsive --- Key: FLUME-323 URL: https://issues.apache.org/jira/browse/FLUME-323 Project: Flume Issue Type: Improvement Components: Sinks+Sources Affects Versions: v0.9.2 Reporter: Jonathan Hsieh Fix For: v0.9.5 The reimplementation of 'exec' found in ExecNIOSource (FLUME-254) is works but uses spin loops instead of select based waits. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-324) Exec should be refactored to be more readable
[ https://issues.apache.org/jira/browse/FLUME-324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-324. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Exec should be refactored to be more readable - Key: FLUME-324 URL: https://issues.apache.org/jira/browse/FLUME-324 Project: Flume Issue Type: Improvement Components: Sinks+Sources, Technical Debt Affects Versions: v0.9.2 Reporter: Jonathan Hsieh Fix For: v0.9.5 Exec found in FLUME-254's ExecNIOSource refactored some its predecessor, ExecEventSource but is still cumbersome. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-325) Add something to test physical network partition failure recovery.
[ https://issues.apache.org/jira/browse/FLUME-325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-325. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Add something to test physical network partition failure recovery. -- Key: FLUME-325 URL: https://issues.apache.org/jira/browse/FLUME-325 Project: Flume Issue Type: New Feature Components: Technical Debt, Test Affects Versions: v0.9.4 Reporter: Jonathan Hsieh Fix For: v0.9.5 FLUME-286 highlights that some physical failures act differently than software failures. In that particular case, a machine being up provided icmp feedback which is different than if the machine was just powered down. We probably need to add some linux firewall tricks to simulate physical network partition failures (drop all packets, and fake out os level responses like icmp) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-326) Document properties for flume-site.xml file
[ https://issues.apache.org/jira/browse/FLUME-326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-326. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Document properties for flume-site.xml file --- Key: FLUME-326 URL: https://issues.apache.org/jira/browse/FLUME-326 Project: Flume Issue Type: Documentation Components: Docs, Easy Affects Versions: v0.9.4 Reporter: Jonathan Hsieh Fix For: v0.9.5 Flume should document the different configuration variables that can be set in flume-site.xml, and flume-conf.xml. It should also document which of these values are stable, unstable, or deprecated. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-334) syslogUdp to console configuration error
[ https://issues.apache.org/jira/browse/FLUME-334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-334. -- Resolution: Won't Fix Won't fix. 0.X branch not maintained anymore syslogUdp to console configuration error Key: FLUME-334 URL: https://issues.apache.org/jira/browse/FLUME-334 Project: Flume Issue Type: Bug Components: Sinks+Sources Affects Versions: v0.9.1 Environment: CentOS 5.4 Reporter: Disabled imported user Fix For: v0.9.5 Hello, We are evaluating flume to get data from syslog source: syslogUdp(5140) and output to console: flume dump 'syslogUdp(5140)'. If message is generated by the command 'echo 37hello via syslog | nc -u localhost 5140' it prints on the console but if generated by logger or by the system it does not prints on the console. We have tried to source to a file using 'text(/data/flume_output.txt)' but no luck. Is there any specific configuration needed to get syslogs to the sink? By the way we followed flume cookbook and configured syslog.conf file accordingly. Any help would be appreciated. Thanks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-344) Pluginify IRC source/sinks.
[ https://issues.apache.org/jira/browse/FLUME-344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-344. -- Resolution: Won't Fix Won't fix. 0.X branch not maintained anymore Pluginify IRC source/sinks. --- Key: FLUME-344 URL: https://issues.apache.org/jira/browse/FLUME-344 Project: Flume Issue Type: Task Components: Sinks+Sources Affects Versions: v0.9.4 Reporter: Jonathan Hsieh Fix For: v0.9.5 Because the irc sink/sources are registered by nodes and masters, the classes in the irc library are always class loaded on startup. This feature in likely not used often and results in extra memory footprint and startup delay. This should be pluginified to be an optional addition. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-346) Show logs being collected in web page (Websockets sink?)
[ https://issues.apache.org/jira/browse/FLUME-346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-346. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Show logs being collected in web page (Websockets sink?) Key: FLUME-346 URL: https://issues.apache.org/jira/browse/FLUME-346 Project: Flume Issue Type: New Feature Affects Versions: v0.9.4 Reporter: Jonathan Hsieh Fix For: v0.9.5 This could possibly be implemented as a websocket sink a la http://www.youtube.com/watch?v=Fi7OaiNqPCc#t=2m50s -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-351) Add Documentation about required escaping with windows filenames.
[ https://issues.apache.org/jira/browse/FLUME-351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-351. -- Resolution: Won't Fix Won't fix. 0.X branch not maintained anymore Add Documentation about required escaping with windows filenames. - Key: FLUME-351 URL: https://issues.apache.org/jira/browse/FLUME-351 Project: Flume Issue Type: Documentation Components: Docs, Windows Affects Versions: v0.9.4 Reporter: Jonathan Hsieh Fix For: v0.9.5 Flume specs with '\' in docs need to be esacaped to be '\\'. Would be good to document this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-355) Information regarding parameters for sources/sinks must be displayed
[ https://issues.apache.org/jira/browse/FLUME-355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-355. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Information regarding parameters for sources/sinks must be displayed - Key: FLUME-355 URL: https://issues.apache.org/jira/browse/FLUME-355 Project: Flume Issue Type: Improvement Affects Versions: v0.9.4 Reporter: Dani Abel Rayan Fix For: v0.9.5 http://localhost:35862/extension.jsp just lists the source and sink names. It will be wonderful if it also displays the parameter info. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-358) hubsubcriber: This consumes events from hubs of pubsubhubbub
[ https://issues.apache.org/jira/browse/FLUME-358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-358. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore hubsubcriber: This consumes events from hubs of pubsubhubbub - Key: FLUME-358 URL: https://issues.apache.org/jira/browse/FLUME-358 Project: Flume Issue Type: New Feature Components: Sinks+Sources Affects Versions: v0.9.4 Reporter: Dani Abel Rayan Assignee: Dani Abel Rayan Fix For: v0.9.5 Whats Pubsubhubbub ? A simple, open, server-to-server web-hook-based pubsub (publish/subscribe) protocol as an extension to Atom and RSS. http://code.google.com/p/pubsubhubbub/ Web Hooks: User-defined HTTP callbacks for push, pipes and plugins. Event callbacks over Web, the design pattern/architecture of letting users receive notifications over http. This creates Evented Web. Feedburner, blogger, google buzz, livejournal, Shopify, Paypal etc are some of the sites that serve their content with webhooks. I have developed a flume-source that subscribes to superfeedr. Why superfeedr? It is the most active hub. Their clients include Posterous, Tumblr and twitterfeed and friend-feed and they leverage low-latency push notifications. https://github.com/superfeedr is very active. I even see a couchdb client : https://github.com/superfeedr/couchpubtato: It reads Turn couchdb into a pubsubhubbub subscriber so it can eat your feeds like potato chips So, I have developed a hubsubcriber for flume. Will be submitting the patch soon. Superfeedr has hackr plan where they give unlimited credits. For more info: http://superfeedr.com/ Other hub implementations: http://code.google.com/p/pubsubhubbub/wiki/Hubs Also, we can have private hubs, need to hack on that later for making it as source for flume. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-359) Be able to specify the bind address for each listening socket
[ https://issues.apache.org/jira/browse/FLUME-359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-359. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Be able to specify the bind address for each listening socket - Key: FLUME-359 URL: https://issues.apache.org/jira/browse/FLUME-359 Project: Flume Issue Type: Improvement Components: Sinks+Sources Affects Versions: v0.9.4 Reporter: Bruce Mitchener Fix For: v0.9.5 In some production environments, it is required to be able to specify what address is being listened on so that you can be sure that you're using the correct network interfaces. This bug will end up being a tracking bug linked to a separate bug for each of the current problematic sources / sinks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-369) HBase sink must use the value of default attribute-prefix:2hb_ as rowkey for empty attribute-prefix as parameter
[ https://issues.apache.org/jira/browse/FLUME-369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-369. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore HBase sink must use the value of default attribute-prefix:2hb_ as rowkey for empty attribute-prefix as parameter -- Key: FLUME-369 URL: https://issues.apache.org/jira/browse/FLUME-369 Project: Flume Issue Type: Bug Affects Versions: v0.9.4 Reporter: Dani Abel Rayan Assignee: Dani Abel Rayan Fix For: v0.9.5 As per the design discussion on https://issues.cloudera.org/browse/FLUME-247 we must look for the value of 2hb_ attribute, if the parameter attribute-prefix of attr2hbase is (empty) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-368) HBase sink must throw proper errors based on preliminary check of parameters.
[ https://issues.apache.org/jira/browse/FLUME-368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-368. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore HBase sink must throw proper errors based on preliminary check of parameters. - Key: FLUME-368 URL: https://issues.apache.org/jira/browse/FLUME-368 Project: Flume Issue Type: Improvement Affects Versions: v0.9.4 Reporter: Dani Abel Rayan Assignee: Dani Abel Rayan Fix For: v0.9.5 The existence of table/colfams and other possible parameters must be validated before calling the Put method. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-371) Add flume icon to flume windows installer
[ https://issues.apache.org/jira/browse/FLUME-371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-371. -- Resolution: Won't Fix Won't fix. 0.X branch not maintained anymore Add flume icon to flume windows installer - Key: FLUME-371 URL: https://issues.apache.org/jira/browse/FLUME-371 Project: Flume Issue Type: Improvement Components: Build Affects Versions: v0.9.4 Reporter: Jonathan Hsieh Fix For: v0.9.5 We should use icons for the windows flume app's start menu items. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-372) Flume windows uninstaller should use install log to only delete necessary files.
[ https://issues.apache.org/jira/browse/FLUME-372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-372. -- Resolution: Won't Fix Won't fix. 0.X branch not maintained anymore Flume windows uninstaller should use install log to only delete necessary files. Key: FLUME-372 URL: https://issues.apache.org/jira/browse/FLUME-372 Project: Flume Issue Type: Improvement Components: Build, Technical Debt, Windows Affects Versions: v0.9.4 Reporter: Jonathan Hsieh Fix For: v0.9.5 Comments from Carl Steinbach. RMDir $INSTDIR will fail if the directory isn't empty, which is a possibility if the user placed any additional files in $INSTDIR after running the installer. Calling RMDIR /r $INSTDIR isn't safe since it's possible that the user selected a non-standard installation location like C:\WINDOWS It looks like the best practice for avoiding this problem is to use the the uninstall macro located here: http://nsis.sourceforge.net/Uninstall_only_installed_files -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-370) Update README, javadocs and create Event to HBase-Sink parameter mapping docs
[ https://issues.apache.org/jira/browse/FLUME-370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-370. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Update README, javadocs and create Event to HBase-Sink parameter mapping docs - Key: FLUME-370 URL: https://issues.apache.org/jira/browse/FLUME-370 Project: Flume Issue Type: Improvement Components: Docs Affects Versions: v0.9.4 Reporter: Dani Abel Rayan Assignee: Dani Abel Rayan Fix For: v0.9.5 The README needs to be updated with changes for the new attr2hbase sink. The javadocs need to be updated, since we now don't use nanos as default rowkey. We simply don't write events without the specified attribute-prefix or default-attribute-prefix. The Event-parameter to HBase-sink mapping needs the table in HTML for easier comprehension. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-374) Test running flume in 32- bit mode to minimize memory footprint.
[ https://issues.apache.org/jira/browse/FLUME-374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-374. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Test running flume in 32- bit mode to minimize memory footprint. Key: FLUME-374 URL: https://issues.apache.org/jira/browse/FLUME-374 Project: Flume Issue Type: Improvement Components: Node, Test Affects Versions: v0.9.4 Reporter: Jonathan Hsieh Fix For: v0.9.5 Running flume in 64 bit mode may consume more memory than running flume in 32 bit mode. It would be good to compare the memory footprint while running in the two modes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-381) Configs from peer masters are not translated
[ https://issues.apache.org/jira/browse/FLUME-381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-381. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Configs from peer masters are not translated Key: FLUME-381 URL: https://issues.apache.org/jira/browse/FLUME-381 Project: Flume Issue Type: Bug Components: Master Affects Versions: v0.9.2, v0.9.3, v0.9.4 Reporter: Jonathan Hsieh Fix For: v0.9.5 This also affects configurations that does not require translations. An 'exec refreshAll' must be executed in order to translate configs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-380) When using multimaster, an individual master's web page does not work until all zk nodes are up
[ https://issues.apache.org/jira/browse/FLUME-380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-380. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore When using multimaster, an individual master's web page does not work until all zk nodes are up --- Key: FLUME-380 URL: https://issues.apache.org/jira/browse/FLUME-380 Project: Flume Issue Type: Improvement Components: Master, Technical Debt Affects Versions: v0.9.4 Reporter: Jonathan Hsieh Fix For: v0.9.5 Ideally, the individual master web pages would come up even if the zk quorum hasn't been met yet. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-383) E2E agent keeps dying when using batch and gzip decorators.
[
https://issues.apache.org/jira/browse/FLUME-383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashish Paliwal resolved FLUME-383.
--
Resolution: Won't Fix
Won't fix. 0.X branch not maintained anymore
E2E agent keeps dying when using batch and gzip decorators.
---
Key: FLUME-383
URL: https://issues.apache.org/jira/browse/FLUME-383
Project: Flume
Issue Type: Bug
Components: Node
Affects Versions: v0.9.2
Environment: CentOS 5.4
Java build 1.6.0_13-b03
Reporter: Disabled imported user
Fix For: v0.9.5
When running in E2E using batch and gzip decorators, the agent dies.
Configs:
exec config 'xxx' 'syslogTcp(5140)' '{delay (500) = {batch(1,3) =
{gzip = autoE2EChain}}}'
exec config 'xxx' 'autoCollectorSource' '{gunzip = {unbatch =
collectorSink(hdfs://dc1-had01-m001.dc01.revsci.net:9000/data/pixlogs,,6)}
}'
Link to Flume user group thread
https://groups.google.com/a/cloudera.org/group/flume-user/browse_thread/thread/c3195f16324f43c8#
2010-12-06 15:29:57,669 ERROR
com.cloudera.flume.handlers.batch.BatchingDecorator: IOException when
ending batch!
java.io.IOException: java.lang.NullPointerException
at com.cloudera.flume.agent.durability.NaiveFileWALDeco
$1.fireError(NaiveFileWALDeco.java:250)
at
com.cloudera.flume.core.connector.DirectDriver.fireError(DirectDriver.java:
216)
at com.cloudera.flume.core.connector.DirectDriver
$PumperThread.run(DirectDriver.java:102)
Caused by: java.lang.NullPointerException
at
com.cloudera.flume.handlers.thrift.ThriftEventSink.append(ThriftEventSink.java:
76)
at
com.cloudera.flume.core.BackOffFailOverSink.append(BackOffFailOverSink.java:
144)
at
com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:
58)
at
com.cloudera.flume.handlers.debug.InsistentOpenDecorator.append(InsistentOpenDecorator.java:
163)
at
com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:
58)
at
com.cloudera.flume.handlers.debug.StubbornAppendSink.append(StubbornAppendSink.java:
84)
at
com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:
58)
at com.cloudera.flume.agent.durability.NaiveFileWALDeco
$AckChecksumRegisterer.append(NaiveFileWALDeco.java:99)
at
com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:
58)
at
com.cloudera.flume.handlers.debug.LazyOpenDecorator.append(LazyOpenDecorator.java:
70)
at
com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:
58)
at com.cloudera.flume.core.connector.DirectDriver
$PumperThread.run(DirectDriver.java:93)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (FLUME-396) A Master peer that goes down can not always rejoin the ensemble.
[ https://issues.apache.org/jira/browse/FLUME-396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-396. -- Resolution: Won't Fix Won't fix. 0.X branch not maintained anymore A Master peer that goes down can not always rejoin the ensemble. Key: FLUME-396 URL: https://issues.apache.org/jira/browse/FLUME-396 Project: Flume Issue Type: Bug Components: Master Affects Versions: v0.9.4 Reporter: Jonathan Hsieh Fix For: v0.9.5 * Scenario 1: Have a 3 master setup. Have some nodes heart beating across the nodes. Kill one of the masters. Nodes that once heartbeated with dead master move to other masters. Attempt to bring dead master back up. - Revived master eventually get into exception loop that never stops. * Scenario 2: Have a 3 master setup. Have some nodes heart beating across the nodes. Kill one of the masters. Nodes that once heartbeated with dead master move to other masters. Kill a second master. All nodes now heartbeat with single remaining master. Bring up a dead master. It rejoins. Bring up other dead master. It rejoins. (All heart beats remain with the master that initally survived) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-397) Nodes should rebalance themselves across masters when a master peer returns from dead.
[ https://issues.apache.org/jira/browse/FLUME-397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-397. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Nodes should rebalance themselves across masters when a master peer returns from dead. -- Key: FLUME-397 URL: https://issues.apache.org/jira/browse/FLUME-397 Project: Flume Issue Type: Improvement Components: Master, Node Affects Versions: v0.9.4 Reporter: Jonathan Hsieh Fix For: v0.9.5 Currently, nodes randomly select a master from the master list when it starts heartbeating. If that master goes down, it randomly fails over to another master. If the dead master comes back, no nodes shift back over to it unless another master dies, or if a new node randomly chooses it. Meanwhile, the remaining masters may bear an unfair share of heartbeating responsibilities. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-400) Parameterize rolltag in roll sink.
[ https://issues.apache.org/jira/browse/FLUME-400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-400. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Parameterize rolltag in roll sink. -- Key: FLUME-400 URL: https://issues.apache.org/jira/browse/FLUME-400 Project: Flume Issue Type: Improvement Affects Versions: v0.9.4 Reporter: Jonathan Hsieh Fix For: v0.9.5 Currently, the roller always tags each event with a rolltag attribute. Attributes names should be user configurable and visible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-401) Allow agent*Sink/Chain to have decorators expressed internally
[ https://issues.apache.org/jira/browse/FLUME-401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-401. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Allow agent*Sink/Chain to have decorators expressed internally Key: FLUME-401 URL: https://issues.apache.org/jira/browse/FLUME-401 Project: Flume Issue Type: New Feature Components: Node Affects Versions: v0.9.4 Reporter: Jonathan Hsieh Fix For: v0.9.5 Currently, batching and gzip wire compression requires some custom work if you want to use it between an e2e agent and collector. It should be made possible to express in the configuration language. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-404) Node status table should indicate if a node is in a state that is behind what it has been assigned.
[ https://issues.apache.org/jira/browse/FLUME-404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-404. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Node status table should indicate if a node is in a state that is behind what it has been assigned. - Key: FLUME-404 URL: https://issues.apache.org/jira/browse/FLUME-404 Project: Flume Issue Type: New Feature Affects Versions: v0.9.4 Reporter: Jonathan Hsieh Fix For: v0.9.5 In some situations, reconfigurations take a while to take effect. The status table should make it easy to see if a node is behind (and possibly stuck). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-409) Use avro 1.4.1 for source-sink SASL capable RPC
[ https://issues.apache.org/jira/browse/FLUME-409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-409. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore This is available in Flume 1.4 onwards Use avro 1.4.1 for source-sink SASL capable RPC --- Key: FLUME-409 URL: https://issues.apache.org/jira/browse/FLUME-409 Project: Flume Issue Type: Task Components: Sinks+Sources Affects Versions: v0.9.3 Reporter: Disabled imported user Fix For: v0.9.5 AVRO-641, support for SASL in avro, was released in avro-1.4.1. We should use the new SASL feature for source-sink RPC over WAN. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-421) Add filename convention info to docs/faq
[ https://issues.apache.org/jira/browse/FLUME-421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-421. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Add filename convention info to docs/faq Key: FLUME-421 URL: https://issues.apache.org/jira/browse/FLUME-421 Project: Flume Issue Type: Documentation Components: Docs Affects Versions: v0.9.4 Reporter: Jonathan Hsieh Fix For: v0.9.5 Extract info from post here: https://groups.google.com/a/cloudera.org/group/flume-user/browse_thread/thread/d0904b979dab82e4# -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-431) Add attribute with dir path to events produced by tailing sources
[
https://issues.apache.org/jira/browse/FLUME-431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashish Paliwal resolved FLUME-431.
--
Resolution: Won't Fix
Fix Version/s: v0.9.5
Won't fix. 0.X branch not maintained anymore
Add attribute with dir path to events produced by tailing sources
-
Key: FLUME-431
URL: https://issues.apache.org/jira/browse/FLUME-431
Project: Flume
Issue Type: Improvement
Components: Sinks+Sources
Affects Versions: v0.9.4
Reporter: Alex Baranau
Priority: Minor
Fix For: v0.9.5
Currently tailing sources (tail, taildir) add attribute with filename to
event. Looks like we should put dir names too, since this can be critical
info for event's processing logic.
E.g. tailing collectD output - it outputs stats data into dirs like
/ubuntu904desktop.localdomain/cpu-0, /ubuntu904desktop.localdomain/cpu-1,
etc. Here, path info is important for logs processing logic and should be
available too.
Jon's suggestion:
My suggestion is to use a kwarg to make recording the dir a open, and to give
the user option to pick attribute name.
So something like this:
tailDir(/dir,.*, 0, dirAttr=tailfiledir, fileAttr=tailfilename)
So a file /dir/file would produce events with metadata like:
event { taildirfile=/dir, tailfilename=file }
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (FLUME-432) Add ability to provide directory filter to tailDir sink in case of recursing into subdirs
[ https://issues.apache.org/jira/browse/FLUME-432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-432. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Add ability to provide directory filter to tailDir sink in case of recursing into subdirs - Key: FLUME-432 URL: https://issues.apache.org/jira/browse/FLUME-432 Project: Flume Issue Type: Improvement Components: Sinks+Sources Affects Versions: v0.9.4 Reporter: Alex Baranau Priority: Minor Fix For: v0.9.5 Currently tailDir accepts regex parameter that is used to filter files to be tailed. In case of recursing into subdirs (FLUME-410) filter is not applied to directories. Ability to filter directories to recurse into (i.e. the ones to tail) should be provided. Suggestions: * add extra param for filtering dirs on regex * filter files dirs based on full path. E.g. user could use subdir[12](/.*.log)? to tail subdir1/*.log and subdir2/*.log -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-429) Create GEN sink in configuration language.
[ https://issues.apache.org/jira/browse/FLUME-429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-429. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Create GEN sink in configuration language. -- Key: FLUME-429 URL: https://issues.apache.org/jira/browse/FLUME-429 Project: Flume Issue Type: Improvement Components: Sinks+Sources, Technical Debt Affects Versions: v0.9.4 Reporter: Jonathan Hsieh Fix For: v0.9.5 Roll and collectorDeco are currently special cases in the configuration language. This should be made into a new class of sink called a generatorSink -- a sink that can instantiate new sinks. (decorators can only continue to push data through, they cannot be changed or reinstantiated). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-447) Add category name attribute parameter to scribe source.
[ https://issues.apache.org/jira/browse/FLUME-447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-447. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Add category name attribute parameter to scribe source. --- Key: FLUME-447 URL: https://issues.apache.org/jira/browse/FLUME-447 Project: Flume Issue Type: Improvement Components: Easy, Sinks+Sources Affects Versions: v0.9.4 Reporter: Jonathan Hsieh Fix For: v0.9.5 In a response to Vlad's question here: https://groups.google.com/a/cloudera.org/group/flume-user/msg/4cad5b707a24a146 I think one thing we may do, is to start using keyword arguments to enable users to set attribute names on their own, and use the old names as defaults. This would help readability by making attributes explicitly defined at the same level they are used at in the configuration language. It would look something like: scribe(1464, categoryAttr=scribe.category) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-442) File rename technical debt.
[ https://issues.apache.org/jira/browse/FLUME-442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-442. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore File rename technical debt. --- Key: FLUME-442 URL: https://issues.apache.org/jira/browse/FLUME-442 Project: Flume Issue Type: Improvement Components: Build, Technical Debt Affects Versions: v0.9.4 Reporter: Jonathan Hsieh Fix For: v0.9.5 Over time, certain source files or package names have changed purposes and look misnamed, or inconsistent. This will be an umbrella jira to fix these. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-441) Update docs to take advantage of master id auto detect.
[ https://issues.apache.org/jira/browse/FLUME-441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-441. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Update docs to take advantage of master id auto detect. --- Key: FLUME-441 URL: https://issues.apache.org/jira/browse/FLUME-441 Project: Flume Issue Type: Documentation Components: Docs, Master Affects Versions: v0.9.3 Reporter: Jonathan Hsieh Fix For: v0.9.5 FLUME-424 implemented a feature that should make starting masters simpler, however it is not currently documentated. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-448) Create sharable MemoryQueueSink and MemoryQueueSource.
[ https://issues.apache.org/jira/browse/FLUME-448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-448. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Create sharable MemoryQueueSink and MemoryQueueSource. --- Key: FLUME-448 URL: https://issues.apache.org/jira/browse/FLUME-448 Project: Flume Issue Type: New Feature Components: Node Affects Versions: v0.9.4 Reporter: Jonathan Hsieh Fix For: v0.9.5 When two logical nodes are communicating to each other on the same physical node, we could just use a queue to manage communications without having to do extra encoding or going through some serialized transport mechanism. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-451) Exec Sink
[ https://issues.apache.org/jira/browse/FLUME-451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-451. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Exec Sink - Key: FLUME-451 URL: https://issues.apache.org/jira/browse/FLUME-451 Project: Flume Issue Type: New Feature Components: Sinks+Sources Affects Versions: v0.9.4 Reporter: Jonathan Hsieh Fix For: v0.9.5 This is the dual to a exec source -- this would write data out to a specified process -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-449) Log4net source connector for windows
[ https://issues.apache.org/jira/browse/FLUME-449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-449. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Log4net source connector for windows - Key: FLUME-449 URL: https://issues.apache.org/jira/browse/FLUME-449 Project: Flume Issue Type: New Feature Components: Sinks+Sources, Windows Affects Versions: v0.9.4 Reporter: Jonathan Hsieh Fix For: v0.9.5 Would like a story and support for window's log4net logging infrastructure. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-453) Non-threadsafe methods in ZooKepperConfigStore
[ https://issues.apache.org/jira/browse/FLUME-453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-453. -- Resolution: Won't Fix Won't fix. 0.X branch not maintained anymore Non-threadsafe methods in ZooKepperConfigStore -- Key: FLUME-453 URL: https://issues.apache.org/jira/browse/FLUME-453 Project: Flume Issue Type: Bug Components: Master Affects Versions: v0.9.4 Reporter: Jonathan Hsieh Fix For: v0.9.5 Henry: Both getConfigs and getLogicalNodeMap return unmodifiable* which read-through to the underlying map. They are therefore prone to concurrent modification exceptions. Both should be defensively copying the maps in question. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-452) Consolidate all calls in liveness manager into a fewer rpc calls.
[ https://issues.apache.org/jira/browse/FLUME-452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-452. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Consolidate all calls in liveness manager into a fewer rpc calls. - Key: FLUME-452 URL: https://issues.apache.org/jira/browse/FLUME-452 Project: Flume Issue Type: New Feature Components: Master, Node Affects Versions: v0.9.4 Reporter: Jonathan Hsieh Fix For: v0.9.5 This is an optimization. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-455) Add testcase for that shows availability of lgoical node when there is an illegal config
[ https://issues.apache.org/jira/browse/FLUME-455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-455. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Add testcase for that shows availability of lgoical node when there is an illegal config - Key: FLUME-455 URL: https://issues.apache.org/jira/browse/FLUME-455 Project: Flume Issue Type: Improvement Components: Technical Debt Affects Versions: v0.9.4 Reporter: Jonathan Hsieh Fix For: v0.9.5 This was a bug that was fixed without a unit test to verify. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-456) Support Windows Event Tracing for Windows logging
[ https://issues.apache.org/jira/browse/FLUME-456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-456. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Support Windows Event Tracing for Windows logging -- Key: FLUME-456 URL: https://issues.apache.org/jira/browse/FLUME-456 Project: Flume Issue Type: New Feature Components: Sinks+Sources, Windows Affects Versions: v0.9.4 Reporter: Jonathan Hsieh Priority: Minor Fix For: v0.9.5 ETW is another major windows event logging source. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-457) Persist some tails state to allow nodes to continue from previous positions.
[ https://issues.apache.org/jira/browse/FLUME-457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-457. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Persist some tails state to allow nodes to continue from previous positions. -- Key: FLUME-457 URL: https://issues.apache.org/jira/browse/FLUME-457 Project: Flume Issue Type: New Feature Components: Sinks+Sources Affects Versions: v0.9.4 Reporter: Jonathan Hsieh Fix For: v0.9.5 This is likely not something that will be completely robust it is an oft requested feature. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-472) Replace tagger with roller specific kw arg for adding a tag.
[ https://issues.apache.org/jira/browse/FLUME-472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-472. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Replace tagger with roller specific kw arg for adding a tag. Key: FLUME-472 URL: https://issues.apache.org/jira/browse/FLUME-472 Project: Flume Issue Type: Improvement Components: Sinks+Sources, Technical Debt Affects Versions: v0.9.4 Reporter: Jonathan Hsieh Fix For: v0.9.5 now with kw args, we can get rid of hard coded attribute names, and cleanup certain pieces of code used to generate unique tags. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-475) Remove flume.master.savefile.autoload option
[ https://issues.apache.org/jira/browse/FLUME-475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-475. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Remove flume.master.savefile.autoload option Key: FLUME-475 URL: https://issues.apache.org/jira/browse/FLUME-475 Project: Flume Issue Type: Improvement Affects Versions: v0.9.4 Reporter: Disabled imported user Priority: Minor Fix For: v0.9.5 I would like to bootstrap my Flume configuration from a file and came across the setting 'flume.master.savefile.autoload'. It seems this setting was only used for debugging with memory-backed masters. This setting should either be updated to allow bootstrapping using a file or removed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-480) Issuing an unconfig then reconfiguring a node leaves node in a decommissioned state
[ https://issues.apache.org/jira/browse/FLUME-480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-480. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Issuing an unconfig then reconfiguring a node leaves node in a decommissioned state - Key: FLUME-480 URL: https://issues.apache.org/jira/browse/FLUME-480 Project: Flume Issue Type: Bug Components: Master, Node Affects Versions: v0.9.4 Reporter: Nicholas Verbeck Priority: Minor Fix For: v0.9.5 If you issue an unconfig and then post new configurations for a node the node shows that its decommissioned on the master. However the node is well and still working just fine with the new configs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-478) unconfig command in shell does not have usage message with bad input.
[ https://issues.apache.org/jira/browse/FLUME-478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-478. -- Resolution: Won't Fix Won't fix. 0.X branch not maintained anymore unconfig command in shell does not have usage message with bad input. - Key: FLUME-478 URL: https://issues.apache.org/jira/browse/FLUME-478 Project: Flume Issue Type: Bug Components: Easy, Shell Affects Versions: v0.9.4 Reporter: Jonathan Hsieh Fix For: v0.9.5 When using shell, unconfig doesn't give usage information when no/bad arguments are specified. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-481) Unmapping a logic node leaves node in node status list
[ https://issues.apache.org/jira/browse/FLUME-481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-481. -- Resolution: Won't Fix Won't fix. 0.X branch not maintained anymore Unmapping a logic node leaves node in node status list -- Key: FLUME-481 URL: https://issues.apache.org/jira/browse/FLUME-481 Project: Flume Issue Type: Bug Components: Master Affects Versions: v0.9.4 Reporter: Nicholas Verbeck Priority: Minor Fix For: v0.9.5 If you unmap a logic node it remains in the master node status list until the master is restarted. Without looking at code I'd imagine this may be partially related to FLUME-108/FLUME-150. I think it should eather remove right away, or after a delayed time. However it should always remove from 1 master if it shows up on another. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-476) Ability to clear an agents config
[ https://issues.apache.org/jira/browse/FLUME-476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-476. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Ability to clear an agents config - Key: FLUME-476 URL: https://issues.apache.org/jira/browse/FLUME-476 Project: Flume Issue Type: New Feature Components: Master, Node Affects Versions: v0.9.4 Reporter: Nicholas Verbeck Priority: Minor Fix For: v0.9.5 Add the ability to both the web interface as well as the command interface to clear an agent/collectors configs. This would allow for easy switching from using named hosts to logic node without needing to go into zookeeper and reseting all the data. If this already exists lets expose it to the web interface and document it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-485) Time-based/Scheduled batch updates
[
https://issues.apache.org/jira/browse/FLUME-485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashish Paliwal resolved FLUME-485.
--
Resolution: Won't Fix
Fix Version/s: v0.9.5
Won't fix. 0.X branch not maintained anymore
Time-based/Scheduled batch updates
--
Key: FLUME-485
URL: https://issues.apache.org/jira/browse/FLUME-485
Project: Flume
Issue Type: New Feature
Affects Versions: v0.9.4
Reporter: Disabled imported user
Fix For: v0.9.5
Currently, batching waits for N records and sends them. Would be nice to
translate this into time-based batching like every hour or once a day at 1
am. The syntax may be:
pre class=code
source | { batch(0 30 1 ? * MON-FRI) = sink };
/pre
Where 0 30 1 ? * MON-FRI is cron/quartz standard for describing scheduling
intervals (at 1:30 am during weekdays)
a href=http://www.quartz-scheduler.org/;Quartz/a is an obvious
implementation choice.
Alex K
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (FLUME-484) Need a way to take node offline
[ https://issues.apache.org/jira/browse/FLUME-484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-484. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Need a way to take node offline --- Key: FLUME-484 URL: https://issues.apache.org/jira/browse/FLUME-484 Project: Flume Issue Type: Improvement Components: Node Affects Versions: v0.9.4 Reporter: Disabled imported user Labels: collector Fix For: v0.9.5 We need a way to take down a node without un-configuring for maintenance (mostly a collector in multi-collector configuration). The node should be brought back up on restart. Unconfiguring the node, while an option, requires entering the configuration information again. Alternatively a command like `exec shutdown physicalnode` will shutdown any logical node on the machine (flush the output, stop input) and optionally terminate the Flume process. The same may apply to master or an agent (an agent will have to stop accepting or reading the data). Currently we can just shut down the Java process on machine, which potentially leads to either data loss or data duplication in different modes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-486) When using taildir in a single node configuration records are being lost
[ https://issues.apache.org/jira/browse/FLUME-486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-486. -- Resolution: Won't Fix Won't fix. 0.X branch not maintained anymore When using taildir in a single node configuration records are being lost Key: FLUME-486 URL: https://issues.apache.org/jira/browse/FLUME-486 Project: Flume Issue Type: Bug Components: Node, Sinks+Sources Affects Versions: v0.9.1 Environment: CentOS 5.5 Sun JDK 1.6.0_b22 Reporter: Disabled imported user Priority: Critical Labels: events, flume, missing, records, sink, tailDir, text Fix For: v0.9.5 When using tailDir to ingest file using flumes we're seeing large numbers of records go missing from the beginning of the file. If we use the text source at the same file all records ingest successfully. The workflow we're using is very simple, one node is used per source and sink. The node reads the data using tailDir and writes out using escapedCustomDfs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-492) Windows reinstall should stop existing flume node
[ https://issues.apache.org/jira/browse/FLUME-492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-492. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Windows reinstall should stop existing flume node - Key: FLUME-492 URL: https://issues.apache.org/jira/browse/FLUME-492 Project: Flume Issue Type: New Feature Components: Windows Affects Versions: v0.9.4 Reporter: Jonathan Hsieh Fix For: v0.9.5 if a flumenode.exe is not stopped when reinstalling, files cannot be deleted, and updated. install should stop any running verisons. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-490) mismatch between hadoop versions causes flume to break silently
[ https://issues.apache.org/jira/browse/FLUME-490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-490. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore mismatch between hadoop versions causes flume to break silently --- Key: FLUME-490 URL: https://issues.apache.org/jira/browse/FLUME-490 Project: Flume Issue Type: Improvement Components: Build Affects Versions: v0.9.2 Reporter: Kim Vogt Fix For: v0.9.5 If there's a mismatch between hadoop versions on the agents and hdfs flume writes to, flume stops working. This is the log message that tipped us off: 2011-01-31 05:33:21,748 INFO com.cloudera.flume.handlers.debug.InsistentAppendDecorator: append attempt 122 failed, backoff (6ms): Protocol org.apache.hadoop.hdfs.protocol.ClientProtocol version mismatch. (client = 61, server = 62) Maybe this should be an ERROR instead of INFO statement. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-491) Windows install should allow user to specify FLUME_CONF_DIR environment variable
[ https://issues.apache.org/jira/browse/FLUME-491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-491. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Windows install should allow user to specify FLUME_CONF_DIR environment variable Key: FLUME-491 URL: https://issues.apache.org/jira/browse/FLUME-491 Project: Flume Issue Type: New Feature Components: Windows Affects Versions: v0.9.4 Reporter: Jonathan Hsieh Fix For: v0.9.5 When installing in windows, we don't always want to use a generic flume-site.xml -- we might already have a proper version of the file present and want to just point to a dir with the flume-site.xml file already in it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-493) Windows install has README file open option that does nothing.
[ https://issues.apache.org/jira/browse/FLUME-493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-493. -- Resolution: Won't Fix Won't fix. 0.X branch not maintained anymore Windows install has README file open option that does nothing. -- Key: FLUME-493 URL: https://issues.apache.org/jira/browse/FLUME-493 Project: Flume Issue Type: Bug Components: Windows Affects Versions: v0.9.4 Reporter: Jonathan Hsieh Fix For: v0.9.5 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-495) flumenodew.exe (tray program) should not be run by every user who logs into machine with flumenode installed.
[ https://issues.apache.org/jira/browse/FLUME-495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-495. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore flumenodew.exe (tray program) should not be run by every user who logs into machine with flumenode installed. --- Key: FLUME-495 URL: https://issues.apache.org/jira/browse/FLUME-495 Project: Flume Issue Type: Improvement Components: Windows Affects Versions: v0.9.4 Reporter: Jonathan Hsieh Fix For: v0.9.5 Ideally this should only be run by an administrator/installer or not be run unless manually started. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-497) Failure to heartbeat with master message should not log as often.
[ https://issues.apache.org/jira/browse/FLUME-497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-497. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Failure to heartbeat with master message should not log as often. - Key: FLUME-497 URL: https://issues.apache.org/jira/browse/FLUME-497 Project: Flume Issue Type: Improvement Affects Versions: v0.9.4 Reporter: Jonathan Hsieh Fix For: v0.9.5 Currently this logs every heartbeat, which is bulky. Ideally it would taper off the number of warning messages. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-494) Flume node should give better warning/error when flume's hadoop doesn't match hadoop cluster's hadoop.
[ https://issues.apache.org/jira/browse/FLUME-494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-494. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Flume node should give better warning/error when flume's hadoop doesn't match hadoop cluster's hadoop. -- Key: FLUME-494 URL: https://issues.apache.org/jira/browse/FLUME-494 Project: Flume Issue Type: Improvement Affects Versions: v0.9.0, v0.9.1, v0.9.1u1, v0.9.2 Reporter: Jonathan Hsieh Fix For: v0.9.5 This is a problem with windows and linux installations. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-498) wal/dfo logs should add space used to report.
[ https://issues.apache.org/jira/browse/FLUME-498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-498. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore wal/dfo logs should add space used to report. - Key: FLUME-498 URL: https://issues.apache.org/jira/browse/FLUME-498 Project: Flume Issue Type: New Feature Affects Versions: v0.9.4 Reporter: Jonathan Hsieh Fix For: v0.9.5 before we can limit disk space usage, we need to be able to find out how much space is used. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-501) document how to write to multiple places at different reliability levels.
[ https://issues.apache.org/jira/browse/FLUME-501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-501. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore document how to write to multiple places at different reliability levels. - Key: FLUME-501 URL: https://issues.apache.org/jira/browse/FLUME-501 Project: Flume Issue Type: Documentation Affects Versions: v0.9.4 Reporter: Jonathan Hsieh Fix For: v0.9.5 One request is to write to a local hdfs reliably while writing to a remote datacenter hdfs at best effort level. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-502) Corrupted gzip files are being created in HDFS (file attached)
[ https://issues.apache.org/jira/browse/FLUME-502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-502. -- Resolution: Won't Fix Won't fix. 0.X branch not maintained anymore Corrupted gzip files are being created in HDFS (file attached) -- Key: FLUME-502 URL: https://issues.apache.org/jira/browse/FLUME-502 Project: Flume Issue Type: Bug Components: Sinks+Sources Affects Versions: v0.9.2 Reporter: Kim Vogt Priority: Critical Fix For: v0.9.5 Attachments: ASF.LICENSE.NOT.GRANTED--Screen shot 2011-06-23 at 3.47.59 PM.png, ASF.LICENSE.NOT.GRANTED--bad_logs.gz There are corrupted gzip files being created in HDFS. These files appear to be truncated gzip files. We've had lots of datanodes go down, so not sure if this is a contributor. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-503) Use HDFS sync API instead of rolling for durability
[ https://issues.apache.org/jira/browse/FLUME-503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-503. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Use HDFS sync API instead of rolling for durability --- Key: FLUME-503 URL: https://issues.apache.org/jira/browse/FLUME-503 Project: Flume Issue Type: New Feature Components: Sinks+Sources Affects Versions: v0.9.4 Reporter: Disabled imported user Fix For: v0.9.5 Some versions of Hadoop (CDH3b2 or 0.20-append branch) support a sync() API that guarantees data has been flushed to all of the nodes in the write pipeline. This should be equally as durable as closing an HDFS file. Flume should allow the use of sync() to make data durable on a regular basis without having to create lots of tiny files on HDFS. Related is the ability to use the getNumCurrentReplicas() API to detect when the number of replicas falls below the desired replication factor, and roll at that point (to pick up a new DN) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-517) FlumeMaster deadlock during configuration of logical nodes
[
https://issues.apache.org/jira/browse/FLUME-517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashish Paliwal resolved FLUME-517.
--
Resolution: Won't Fix
Won't fix. 0.X branch not maintained anymore
FlumeMaster deadlock during configuration of logical nodes
--
Key: FLUME-517
URL: https://issues.apache.org/jira/browse/FLUME-517
Project: Flume
Issue Type: Bug
Affects Versions: v0.9.4
Reporter: Disabled imported user
Fix For: v0.9.5
Attachments: ASF.LICENSE.NOT.GRANTED--flume_dump.txt
Occasionally I notice the FlumeMaster deadlocking when I run commands against
the shell.
I have attached the Thread Dump during one of the deadlocks.
The following are the commands I run which 80% of the time execute without
any problem.
0 SUCCEEDED decommission [collector0_rpc]
1 SUCCEEDED decommission [collector0_log_dir]
2 SUCCEEDED noop [1]
3 SUCCEEDED config [collector0_rpc, rpcFlow, autoCollectorSource,
collectorSink(hdfs://hadoop-name-node1.dc1.blah.net:9000/rpc/%Y-%m-%d/%H00/%{host},
data-, 6)]
4 SUCCEEDED config [collector0_log_dir, logDirFlow,
autoCollectorSource,
collectorSink(hdfs://hadoop-name-node1.dc1.blah.net:9000/logs/%Y-%m-%d/%H00/%{host},
log-, 6)]
5 SUCCEEDED spawn [hadoop-data-node5.dc1.blah.net, collector0_rpc]
6 SUCCEEDED spawn [hadoop-data-node5.dc1.blah.net,
collector0_log_dir]
7 SUCCEEDED decommission [hadoop_agent0_log_dir_0]
8 SUCCEEDED decommission [hadoop_agent1_log_dir_0]
9 SUCCEEDED decommission [hadoop_agent2_rpc]
10SUCCEEDED decommission [hadoop_agent0_rpc]
11SUCCEEDED decommission [hadoop_agent1_rpc]
12SUCCEEDED decommission [hadoop_agent2_log_dir_0]
13SUCCEEDED noop [1]
14SUCCEEDED noop [1]
15SUCCEEDED noop [1]
16SUCCEEDED config [hadoop_agent0_log_dir_0, logDirFlow,
tailDir(/opt/flume/logs), autoE2EChain ]
17SUCCEEDED config [hadoop_agent0_rpc, rpcFlow, rpcSource(36893),
autoE2EChain ]
18SUCCEEDED config [hadoop_agent1_log_dir_0, logDirFlow,
tailDir(/opt/flume/logs), autoE2EChain ]
19SUCCEEDED config [hadoop_agent1_rpc, rpcFlow, rpcSource(36893),
autoE2EChain ]
20SUCCEEDED config [hadoop_agent2_rpc, rpcFlow, rpcSource(36893),
autoE2EChain ]
21SUCCEEDED config [hadoop_agent2_log_dir_0, logDirFlow,
tailDir(/opt/flume/logs), autoE2EChain ]
22SUCCEEDED spawn [hadoop-data-node1.dc1.blah.net,
hadoop_agent0_log_dir_0]
23SUCCEEDED spawn [hadoop-data-node2.dc1.blah.net,
hadoop_agent1_log_dir_0]
24SUCCEEDED spawn [hadoop-data-node3.dc1.blah.net,
hadoop_agent2_rpc]
25SUCCEEDED spawn [hadoop-data-node1.dc1.blah.net,
hadoop_agent0_rpc]
26SUCCEEDED spawn [hadoop-data-node2.dc1.blah.net,
hadoop_agent1_rpc]
27SUCCEEDED spawn [hadoop-data-node3.dc1.blah.net,
hadoop_agent2_log_dir_0]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (FLUME-507) RollSink should roll based on text in the event's body
[ https://issues.apache.org/jira/browse/FLUME-507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-507. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore RollSink should roll based on text in the event's body -- Key: FLUME-507 URL: https://issues.apache.org/jira/browse/FLUME-507 Project: Flume Issue Type: New Feature Components: Sinks+Sources Affects Versions: v0.9.4 Reporter: Disabled imported user Priority: Minor Labels: sink Fix For: v0.9.5 Currently a roll is based on either time or size( I guess this knob is not ON yet). It would be great if I had a custom RollSink based on the content of the event. This generally would be useful to flume XML files. For eg., I may need to roll a file if the body contains /root in it. I am expecting a RollSink that takes /root as an argument to do the roll. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-522) move flume-env.sh / flume-env.sh.template script to conf/ instead of bin/
[ https://issues.apache.org/jira/browse/FLUME-522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-522. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Flume 1.3+ have absorbed the change mentioned move flume-env.sh / flume-env.sh.template script to conf/ instead of bin/ - Key: FLUME-522 URL: https://issues.apache.org/jira/browse/FLUME-522 Project: Flume Issue Type: Improvement Affects Versions: v0.9.4 Reporter: Jonathan Hsieh Assignee: Alexander Alten-Lorenz Fix For: v0.9.5 This is for consistency with the hadoop project. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-526) Unit Test system needs ability to extend java.library.path without modifying build.xml
[ https://issues.apache.org/jira/browse/FLUME-526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-526. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Unit Test system needs ability to extend java.library.path without modifying build.xml -- Key: FLUME-526 URL: https://issues.apache.org/jira/browse/FLUME-526 Project: Flume Issue Type: Bug Components: Test Affects Versions: v0.9.4 Reporter: Nicholas Verbeck Priority: Minor Fix For: v0.9.5 Currently build.xml overrides the java.library.path without pulling in the env variables in the test section. This needs to be extend to load in JAVA_LIBRARY_PATH and/or LD_LIBRARY_PATH. This allows for Compression Codecs access to there native libs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-528) Be able to suppress the env tab of the flume-master and flume-node web pages.
[ https://issues.apache.org/jira/browse/FLUME-528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-528. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Be able to suppress the env tab of the flume-master and flume-node web pages. --- Key: FLUME-528 URL: https://issues.apache.org/jira/browse/FLUME-528 Project: Flume Issue Type: New Feature Affects Versions: v0.9.4 Reporter: Disabled imported user Priority: Minor Fix For: v0.9.5 I am using the java configuation variables fs.s3.awsAccessKeyId and fs.s3.awsSecretAccessKey to be able to communicate data to s3 buckets, and I would kind of like to keep this information private. Unfortunately, it shows up on the static configuration page on my flume-master's web server (and probably on the nodes...). It would be nice to have a way to suppress some information from the static config page. It would also be sufficient to suppress the entire static config page. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-534) flume.agent.logdir shouldn't default to /tmp
[ https://issues.apache.org/jira/browse/FLUME-534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-534. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore flume.agent.logdir shouldn't default to /tmp Key: FLUME-534 URL: https://issues.apache.org/jira/browse/FLUME-534 Project: Flume Issue Type: Bug Components: Build Affects Versions: v0.9.1, v0.9.3 Reporter: Disabled imported user Fix For: v0.9.5 The flume.agent.logdir, flume.collector.dfs.dir, and flume.master.zk.logdir properties default to logging in /tmp. They should default to /var/lib/ or /var/spool. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-535) Make agent not connecting to master due to host name error easier to detect.
[ https://issues.apache.org/jira/browse/FLUME-535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-535. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Make agent not connecting to master due to host name error easier to detect. Key: FLUME-535 URL: https://issues.apache.org/jira/browse/FLUME-535 Project: Flume Issue Type: Improvement Affects Versions: v0.9.4 Reporter: Jonathan Hsieh Fix For: v0.9.5 One suggestion : expose counters to show how often a config has been read. (in configuration table) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-529) HBase sink needs a mechanism to allow using event timestamp as the hbase entry timestamp.
[ https://issues.apache.org/jira/browse/FLUME-529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-529. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore HBase sink needs a mechanism to allow using event timestamp as the hbase entry timestamp. - Key: FLUME-529 URL: https://issues.apache.org/jira/browse/FLUME-529 Project: Flume Issue Type: New Feature Affects Versions: v0.9.4 Reporter: Disabled imported user Fix For: v0.9.5 HBase can store time sequences of values - every entry has a timestamp. By default, hbase will attach the insertion time to the values added to a table. In many cases, the timestamp attached to the event would be more appropriate - especially if you are trying to keep a history of when events occurred instead of when they appeared in the database. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-536) Turn Physical Node name in the Master's Status page into a link to Node status page.
[ https://issues.apache.org/jira/browse/FLUME-536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-536. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore Turn Physical Node name in the Master's Status page into a link to Node status page. - Key: FLUME-536 URL: https://issues.apache.org/jira/browse/FLUME-536 Project: Flume Issue Type: New Feature Components: Master, Web Affects Versions: v0.9.4 Reporter: Nicholas Verbeck Priority: Trivial Fix For: v0.9.5 Turn the Physical Node Name in the Physical/Logical Node mapping table on the Master Status page into links to that nodes status pages. Ideally should open in a new window. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-538) exec source should cause node to fail out if it cannot find the program it is attempting to exec.
[ https://issues.apache.org/jira/browse/FLUME-538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-538. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore exec source should cause node to fail out if it cannot find the program it is attempting to exec. - Key: FLUME-538 URL: https://issues.apache.org/jira/browse/FLUME-538 Project: Flume Issue Type: Improvement Affects Versions: v0.9.3 Reporter: Jonathan Hsieh Fix For: v0.9.5 when a typo is made in an exec command, the exec source just sits. It requires a user to look at logs to determine if there is a problem. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
