[jira] [Created] (FLUME-2088) Minor typo in Flume User Guide JSON Reporting section
Ashish Paliwal created FLUME-2088: - Summary: Minor typo in Flume User Guide JSON Reporting section Key: FLUME-2088 URL: https://issues.apache.org/jira/browse/FLUME-2088 Project: Flume Issue Type: Bug Components: Docs Affects Versions: v1.3.1 Reporter: Ashish Paliwal Priority: Minor In the JSOn reporting section http://flume.apache.org/FlumeUserGuide.html#json-reporting, before the command details, the doc says "We can start Flume with Ganglia support as follows:", which corresponds to Ganglia section above. Need to rephrase as "We can start Flume with HTTP JSON Metrics support as follows:" -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-2088) Minor typo in Flume User Guide JSON Reporting section
[ https://issues.apache.org/jira/browse/FLUME-2088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal updated FLUME-2088: -- Affects Version/s: (was: v1.3.1) v1.3.0 > Minor typo in Flume User Guide JSON Reporting section > - > > Key: FLUME-2088 > URL: https://issues.apache.org/jira/browse/FLUME-2088 > Project: Flume > Issue Type: Bug > Components: Docs >Affects Versions: v1.3.0 >Reporter: Ashish Paliwal >Priority: Minor > > In the JSOn reporting section > http://flume.apache.org/FlumeUserGuide.html#json-reporting, before the > command details, the doc says > "We can start Flume with Ganglia support as follows:", which corresponds to > Ganglia section above. > Need to rephrase as > "We can start Flume with HTTP JSON Metrics support as follows:" -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (FLUME-2112) Dead link on http://flume.apache.org/getinvolved.html
Ashish Paliwal created FLUME-2112: - Summary: Dead link on http://flume.apache.org/getinvolved.html Key: FLUME-2112 URL: https://issues.apache.org/jira/browse/FLUME-2112 Project: Flume Issue Type: Bug Components: Docs Affects Versions: v1.3.1 Reporter: Ashish Paliwal Priority: Minor On the page http://flume.apache.org/getinvolved.html, the link https://cwiki.apache.org/FLUME/flume-ng.html returns 404. Probably it should point to https://cwiki.apache.org/confluence/display/FLUME/Flume+NG -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-2088) Minor typo in Flume User Guide JSON Reporting section
[ https://issues.apache.org/jira/browse/FLUME-2088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal updated FLUME-2088: -- Attachment: FLUME-2088-0.patch Flume-2088 patch > Minor typo in Flume User Guide JSON Reporting section > - > > Key: FLUME-2088 > URL: https://issues.apache.org/jira/browse/FLUME-2088 > Project: Flume > Issue Type: Bug > Components: Docs >Affects Versions: v1.3.0 >Reporter: Ashish Paliwal >Priority: Minor > Attachments: FLUME-2088-0.patch > > > In the JSOn reporting section > http://flume.apache.org/FlumeUserGuide.html#json-reporting, before the > command details, the doc says > "We can start Flume with Ganglia support as follows:", which corresponds to > Ganglia section above. > Need to rephrase as > "We can start Flume with HTTP JSON Metrics support as follows:" -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-2112) Dead link on http://flume.apache.org/getinvolved.html
[ https://issues.apache.org/jira/browse/FLUME-2112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal updated FLUME-2112: -- Attachment: FLUME-2112-0.patch Fixed Dead Link > Dead link on http://flume.apache.org/getinvolved.html > - > > Key: FLUME-2112 > URL: https://issues.apache.org/jira/browse/FLUME-2112 > Project: Flume > Issue Type: Bug > Components: Docs >Affects Versions: v1.3.1 >Reporter: Ashish Paliwal >Priority: Minor > Attachments: FLUME-2112-0.patch > > > On the page http://flume.apache.org/getinvolved.html, the link > https://cwiki.apache.org/FLUME/flume-ng.html returns 404. Probably it should > point to https://cwiki.apache.org/confluence/display/FLUME/Flume+NG -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-2109) HTTPS support in HTTP Source
[ https://issues.apache.org/jira/browse/FLUME-2109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal updated FLUME-2109: -- Attachment: jettykeystore FLUME-2109-0.patch Patch for HTTPS source > HTTPS support in HTTP Source > > > Key: FLUME-2109 > URL: https://issues.apache.org/jira/browse/FLUME-2109 > Project: Flume > Issue Type: Bug >Reporter: Hari Shreedharan > Attachments: FLUME-2109-0.patch, jettykeystore > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-613) Add Cobertura support to the maven build
[ https://issues.apache.org/jira/browse/FLUME-613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal updated FLUME-613: - Attachment: FLUME-613-0.patch Patch to add Cobertura support with maven build > Add Cobertura support to the maven build > > > Key: FLUME-613 > URL: https://issues.apache.org/jira/browse/FLUME-613 > Project: Flume > Issue Type: New Feature > Components: Build, Technical Debt, Test >Affects Versions: v0.9.4 >Reporter: E. Sammer > Labels: maven, regression > Fix For: v0.9.5 > > Attachments: FLUME-613-0.patch > > > The maven version of the Flume build doesn't yet support Cobertura. It should. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-2109) HTTPS support in HTTP Source
[ https://issues.apache.org/jira/browse/FLUME-2109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal updated FLUME-2109: -- Attachment: FLUME-2109-1.patch Patch with Documentation fix. The Jetty keystore needs to be added to following location flume-ng-core/src/test/resources/ like flume-ng-core/src/test/resources/jettykeystore > HTTPS support in HTTP Source > > > Key: FLUME-2109 > URL: https://issues.apache.org/jira/browse/FLUME-2109 > Project: Flume > Issue Type: Bug >Affects Versions: v1.3.1 >Reporter: Hari Shreedharan > Attachments: FLUME-2109-0.patch, FLUME-2109-1.patch, jettykeystore > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-179) Recipe for hooking log4j to flume
[ https://issues.apache.org/jira/browse/FLUME-179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13699833#comment-13699833 ] Ashish Paliwal commented on FLUME-179: -- [~jmhsieh]This issue can be closed, as we have log4j support. Anything else needed to resolve this issue? > Recipe for hooking log4j to flume > - > > Key: FLUME-179 > URL: https://issues.apache.org/jira/browse/FLUME-179 > Project: Flume > Issue Type: Documentation > Components: Docs >Affects Versions: v0.9.4 >Reporter: Jonathan Hsieh > Fix For: v0.9.5 > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-2022) Executing flume agent on the remote machine
[ https://issues.apache.org/jira/browse/FLUME-2022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13700517#comment-13700517 ] Ashish Paliwal commented on FLUME-2022: --- This doesn't seem to be an issue. Let's close this. > Executing flume agent on the remote machine > --- > > Key: FLUME-2022 > URL: https://issues.apache.org/jira/browse/FLUME-2022 > Project: Flume > Issue Type: Question > Components: Node >Affects Versions: v1.3.1 > Environment: apache flume >Reporter: ChanduReddy >Priority: Critical > > Is it possible to run an agent on the remote machine through IP address of > the remote machine -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (FLUME-2022) Executing flume agent on the remote machine
[ https://issues.apache.org/jira/browse/FLUME-2022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-2022. --- Resolution: Invalid Fix Version/s: v1.3.1 This is more of a ML question posted here. Resolving it as invalid. Admin's please close this. > Executing flume agent on the remote machine > --- > > Key: FLUME-2022 > URL: https://issues.apache.org/jira/browse/FLUME-2022 > Project: Flume > Issue Type: Question > Components: Node >Affects Versions: v1.3.1 > Environment: apache flume >Reporter: ChanduReddy >Priority: Critical > Fix For: v1.3.1 > > > Is it possible to run an agent on the remote machine through IP address of > the remote machine -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-1246) FileChannel hangs silently when Hadoop libs not found
[ https://issues.apache.org/jira/browse/FLUME-1246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13700523#comment-13700523 ] Ashish Paliwal commented on FLUME-1246: --- [~hshreedharan] Shall I work on a patch for this or you already have a solved this. > FileChannel hangs silently when Hadoop libs not found > - > > Key: FLUME-1246 > URL: https://issues.apache.org/jira/browse/FLUME-1246 > Project: Flume > Issue Type: Bug > Components: Channel >Affects Versions: v1.2.0 > Environment: CentOS 5.4 >Reporter: Juhani Connolly >Priority: Blocker > Fix For: v1.2.0 > > Attachments: FLUME-1246.patch, flume-hari-2012May312217PST.log, > flume.log, flume.log, flume.log, flume.log.20120601, test_conf.conf > > > FileChannels Event implementation uses Hadoop IO's Writable for > serialization. Without it a class loader error is thrown but never caught and > logged killing the thread invisibly. This repeats and puts flume into a state > where interrupts are not responded to, making clean shutdown impossible > old desc for posterity: > Reduced to a minimal configuration for simplicity. I can recreate this on > some machines, and not others. I wouldn't be surprised if it is some machines > specific issue(test machines on CentOS5.4. On some it worked, others not), > however whatever exception was thrown when attempting to get created is > consumed and never passed onwards > Config: > test.channels.ch1.type = file > test.channels.ch1.checkpointDir = > /home/share/juhani_connolly/flume-1.2.0-incubating-SNAPSHOT/check > test.channels.ch1.dataDirs = > /home/share/juhani_connolly/flume-1.2.0-incubating-SNAPSHOT/filechdata > test.sources.top.type = exec > test.sources.top.command = /usr/bin top -b -d 1 > test.sources.top.restart = true > test.sources.top.restartThrottle = 1000 > test.sources.top.interceptors = ts > test.sources.top.interceptors.ts.type = > org.apache.flume.interceptor.TimestampInterceptor$Builder > test.sources.top.channels = ch1 > test.sinks.log.type = logger > test.sinks.log.channel = ch1 > test.channels = ch1 > test.sources = top > test.sinks = log > attaching logs with general/lifecycle loglevel down to debug. > A solution to this is probably going to be just improving error reporting. > Another possibly more important element is that flume enters a state from > which it cannot shutdown without kill -9. It looks like the Interrupts are > getting swallowed up silently somewhere -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Deleted] (FLUME-1246) FileChannel hangs silently when Hadoop libs not found
[ https://issues.apache.org/jira/browse/FLUME-1246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal updated FLUME-1246: -- Comment: was deleted (was: [~hshreedharan] Shall I work on a patch for this or you already have a solved this.) > FileChannel hangs silently when Hadoop libs not found > - > > Key: FLUME-1246 > URL: https://issues.apache.org/jira/browse/FLUME-1246 > Project: Flume > Issue Type: Bug > Components: Channel >Affects Versions: v1.2.0 > Environment: CentOS 5.4 >Reporter: Juhani Connolly >Priority: Blocker > Fix For: v1.2.0 > > Attachments: FLUME-1246.patch, flume-hari-2012May312217PST.log, > flume.log, flume.log, flume.log, flume.log.20120601, test_conf.conf > > > FileChannels Event implementation uses Hadoop IO's Writable for > serialization. Without it a class loader error is thrown but never caught and > logged killing the thread invisibly. This repeats and puts flume into a state > where interrupts are not responded to, making clean shutdown impossible > old desc for posterity: > Reduced to a minimal configuration for simplicity. I can recreate this on > some machines, and not others. I wouldn't be surprised if it is some machines > specific issue(test machines on CentOS5.4. On some it worked, others not), > however whatever exception was thrown when attempting to get created is > consumed and never passed onwards > Config: > test.channels.ch1.type = file > test.channels.ch1.checkpointDir = > /home/share/juhani_connolly/flume-1.2.0-incubating-SNAPSHOT/check > test.channels.ch1.dataDirs = > /home/share/juhani_connolly/flume-1.2.0-incubating-SNAPSHOT/filechdata > test.sources.top.type = exec > test.sources.top.command = /usr/bin top -b -d 1 > test.sources.top.restart = true > test.sources.top.restartThrottle = 1000 > test.sources.top.interceptors = ts > test.sources.top.interceptors.ts.type = > org.apache.flume.interceptor.TimestampInterceptor$Builder > test.sources.top.channels = ch1 > test.sinks.log.type = logger > test.sinks.log.channel = ch1 > test.channels = ch1 > test.sources = top > test.sinks = log > attaching logs with general/lifecycle loglevel down to debug. > A solution to this is probably going to be just improving error reporting. > Another possibly more important element is that flume enters a state from > which it cannot shutdown without kill -9. It looks like the Interrupts are > getting swallowed up silently somewhere -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-1086) duplicate lib in Flume's lib dir
[ https://issues.apache.org/jira/browse/FLUME-1086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13701866#comment-13701866 ] Ashish Paliwal commented on FLUME-1086: --- [~w...@cloudera.com] Flume 1.4.0 doesn't have duplicate lib. Can this issue can be marked as fixed. > duplicate lib in Flume's lib dir > > > Key: FLUME-1086 > URL: https://issues.apache.org/jira/browse/FLUME-1086 > Project: Flume > Issue Type: Bug > Components: Build >Affects Versions: v1.2.0 > Environment: [CentOS 6.2 64-bit] >Reporter: Will McQueen > > Flume NG has duplicate servlet libs in lib dir: > servlet-api-2.5-20081211.jar > servlet-api-2.5.jar > Is this is transitive dep issue? Which one should be used? Just > "servlet-api-2.5.jar"? > For reference, the following are all the libs in Flume's lib dir: > avro-1.6.1.jar > avro-ipc-1.6.1.jar > commons-cli-1.2.jar > commons-codec-1.3.jar > commons-collections-3.2.1.jar > commons-dbcp-1.4.jar > commons-io-2.1.jar > commons-lang-2.4.jar > commons-logging-1.1.1.jar > commons-pool-1.5.4.jar > derby-10.8.2.2.jar > flume-avro-source-1.2.0-incubating-SNAPSHOT.jar > flume-file-channel-1.2.0-incubating-SNAPSHOT.jar > flume-hdfs-sink-1.2.0-incubating-SNAPSHOT.jar > flume-irc-sink-1.2.0-incubating-SNAPSHOT.jar > flume-jdbc-channel-1.2.0-incubating-SNAPSHOT.jar > flume-ng-core-1.2.0-incubating-SNAPSHOT.jar > flume-ng-log4jappender-1.2.0-incubating-SNAPSHOT.jar > flume-ng-node-1.2.0-incubating-SNAPSHOT.jar > flume-ng-sdk-1.2.0-incubating-SNAPSHOT.jar > flume-thrift-source-1.2.0-incubating-SNAPSHOT.jar > guava-10.0.1.jar > httpclient-4.0.1.jar > httpcore-4.0.1.jar > irclib-1.10.jar > jackson-core-asl-1.9.3.jar > jackson-mapper-asl-1.9.3.jar > jetty-6.1.26.jar > jetty-util-6.1.26.jar > jsr305-1.3.9.jar > libthrift-0.6.1.jar > log4j-1.2.16.jar > netty-3.2.6.Final.jar > paranamer-2.3.jar > servlet-api-2.5-20081211.jar > servlet-api-2.5.jar > slf4j-api-1.6.1.jar > slf4j-log4j12-1.6.1.jar > snappy-java-1.0.4.1.jar > velocity-1.7.jar -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-1144) Update Flume User Guide with a section on Flume properties such as flume.called.from.service
[ https://issues.apache.org/jira/browse/FLUME-1144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13701987#comment-13701987 ] Ashish Paliwal commented on FLUME-1144: --- Flume 1.4.0 User Guide has details on flume.called.from.service. From JIRA it's not clear what other properties need to be documented. Is it Ok to mark it as resolved? > Update Flume User Guide with a section on Flume properties such as > flume.called.from.service > > > Key: FLUME-1144 > URL: https://issues.apache.org/jira/browse/FLUME-1144 > Project: Flume > Issue Type: Documentation > Components: Docs >Affects Versions: v1.2.0 >Reporter: Will McQueen > > Flume User Guide is at flume-ng-doc/xhtml/FlumeUserGuide.xhtml -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-1093) HDFSEventSink gives no indication of failure when Hadoop jars aren't in the classpath
[ https://issues.apache.org/jira/browse/FLUME-1093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13702028#comment-13702028 ] Ashish Paliwal commented on FLUME-1093: --- This issue has been fixed as part of FLUME-1630. Tried reproducing it and a proper error is logged, when dependent classes are not found. [~hshreedharan] We can mark this issue as resolved. > HDFSEventSink gives no indication of failure when Hadoop jars aren't in the > classpath > - > > Key: FLUME-1093 > URL: https://issues.apache.org/jira/browse/FLUME-1093 > Project: Flume > Issue Type: Bug > Components: Sinks+Sources >Affects Versions: v1.2.0 > Environment: CentOS 5.7 64-bit >Reporter: Will McQueen >Priority: Critical > Attachments: FLUME-1093.trunk.v1.patch > > > If HDFS sink starts up, but the hadoop jars aren't in the classpath, then the > HDFS sink doesn't fail-fast with an error. I think we should fail the > validation of the config file if it specifies an HDFS Sink when the hadoop > jars aren't in the classpath. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-2109) HTTPS support in HTTP Source
[ https://issues.apache.org/jira/browse/FLUME-2109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal updated FLUME-2109: -- Attachment: FLUME-2109-2.patch Patch with review comments incorporated > HTTPS support in HTTP Source > > > Key: FLUME-2109 > URL: https://issues.apache.org/jira/browse/FLUME-2109 > Project: Flume > Issue Type: Bug >Affects Versions: v1.3.1 >Reporter: Hari Shreedharan > Attachments: FLUME-2109-0.patch, FLUME-2109-1.patch, > FLUME-2109-2.patch, jettykeystore > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-2064) Typo/Grammar in flume main user doc under Scribe
[ https://issues.apache.org/jira/browse/FLUME-2064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal updated FLUME-2064: -- Attachment: FLUME-2064-0.patch > Typo/Grammar in flume main user doc under Scribe > > > Key: FLUME-2064 > URL: https://issues.apache.org/jira/browse/FLUME-2064 > Project: Flume > Issue Type: Documentation > Components: Docs >Affects Versions: v1.3.1 >Reporter: Hari Sekhon >Priority: Trivial > Attachments: FLUME-2064-0.patch > > > "The deployment of Scribe please following guide from Facebook" > Typo/grammar needs changing. > Thanks > Hari -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-1851) User Guide grammar mistake
[ https://issues.apache.org/jira/browse/FLUME-1851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal updated FLUME-1851: -- Attachment: FLUME-1851-0.patch Patch for 1851 > User Guide grammar mistake > -- > > Key: FLUME-1851 > URL: https://issues.apache.org/jira/browse/FLUME-1851 > Project: Flume > Issue Type: Documentation > Components: Docs >Affects Versions: v1.3.1 >Reporter: Steve Stogner >Priority: Trivial > Attachments: FLUME-1851-0.patch, FLUME-1851-2064-0.patch > > > Setup -> Setting up an agent > top of page 2 > actual: > This is a text file which has a format follows the Java properties file > format. > expected: > This is a text file which has a format that follows the Java properties file > format. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-1851) User Guide grammar mistake
[ https://issues.apache.org/jira/browse/FLUME-1851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal updated FLUME-1851: -- Attachment: FLUME-1851-2064-0.patch Patch for Flume-1851 & Flume-2064 in one file > User Guide grammar mistake > -- > > Key: FLUME-1851 > URL: https://issues.apache.org/jira/browse/FLUME-1851 > Project: Flume > Issue Type: Documentation > Components: Docs >Affects Versions: v1.3.1 >Reporter: Steve Stogner >Priority: Trivial > Attachments: FLUME-1851-0.patch, FLUME-1851-2064-0.patch > > > Setup -> Setting up an agent > top of page 2 > actual: > This is a text file which has a format follows the Java properties file > format. > expected: > This is a text file which has a format that follows the Java properties file > format. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (FLUME-1851) User Guide grammar mistake
[ https://issues.apache.org/jira/browse/FLUME-1851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal reassigned FLUME-1851: - Assignee: Ashish Paliwal > User Guide grammar mistake > -- > > Key: FLUME-1851 > URL: https://issues.apache.org/jira/browse/FLUME-1851 > Project: Flume > Issue Type: Documentation > Components: Docs >Affects Versions: v1.3.1 >Reporter: Steve Stogner >Assignee: Ashish Paliwal >Priority: Trivial > Attachments: FLUME-1851-0.patch, FLUME-1851-2064-0.patch > > > Setup -> Setting up an agent > top of page 2 > actual: > This is a text file which has a format follows the Java properties file > format. > expected: > This is a text file which has a format that follows the Java properties file > format. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-2006) in Avro the batch size is called batch-size, in all other sources batchSize
[ https://issues.apache.org/jira/browse/FLUME-2006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13710985#comment-13710985 ] Ashish Paliwal commented on FLUME-2006: --- Do we plan to update it to batchSize, making it uniform with other Sink? I can take this up. Seems a trivial change in constants file and docs update. > in Avro the batch size is called batch-size, in all other sources batchSize > --- > > Key: FLUME-2006 > URL: https://issues.apache.org/jira/browse/FLUME-2006 > Project: Flume > Issue Type: Bug > Components: Sinks+Sources >Affects Versions: v1.3.1 >Reporter: Alexander Alten-Lorenz >Assignee: Jeff Lord >Priority: Trivial > > http://mail-archives.apache.org/mod_mbox/flume-user/201304.mbox/%3c746ae600-7783-40f4-9817-25617370c...@gmail.com%3e -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-2006) in Avro the batch size is called batch-size, in all other sources batchSize
[ https://issues.apache.org/jira/browse/FLUME-2006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13712010#comment-13712010 ] Ashish Paliwal commented on FLUME-2006: --- For 1) Agree on this. We can mark it deprecated for a next few releases and then remove it 2.0 or whenever decided 2) Do you mean to fail the initialization of Agent? IMHO, this would be better, as this would fix the config file. Alternative is to use the supported property and log a warning message for the unsupported property. > in Avro the batch size is called batch-size, in all other sources batchSize > --- > > Key: FLUME-2006 > URL: https://issues.apache.org/jira/browse/FLUME-2006 > Project: Flume > Issue Type: Bug > Components: Sinks+Sources >Affects Versions: v1.3.1 >Reporter: Alexander Alten-Lorenz >Assignee: Jeff Lord >Priority: Trivial > > http://mail-archives.apache.org/mod_mbox/flume-user/201304.mbox/%3c746ae600-7783-40f4-9817-25617370c...@gmail.com%3e -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (FLUME-2109) HTTPS support in HTTP Source
[ https://issues.apache.org/jira/browse/FLUME-2109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal reassigned FLUME-2109: - Assignee: Ashish Paliwal > HTTPS support in HTTP Source > > > Key: FLUME-2109 > URL: https://issues.apache.org/jira/browse/FLUME-2109 > Project: Flume > Issue Type: Bug >Affects Versions: v1.3.1 >Reporter: Hari Shreedharan >Assignee: Ashish Paliwal > Attachments: FLUME-2109-0.patch, FLUME-2109-1.patch, > FLUME-2109-2.patch, jettykeystore > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (FLUME-2112) Dead link on http://flume.apache.org/getinvolved.html
[ https://issues.apache.org/jira/browse/FLUME-2112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal closed FLUME-2112. - > Dead link on http://flume.apache.org/getinvolved.html > - > > Key: FLUME-2112 > URL: https://issues.apache.org/jira/browse/FLUME-2112 > Project: Flume > Issue Type: Bug > Components: Docs >Reporter: Ashish Paliwal >Priority: Minor > Fix For: v1.4.0 > > Attachments: FLUME-2112-0.patch > > > On the page http://flume.apache.org/getinvolved.html, the link > https://cwiki.apache.org/FLUME/flume-ng.html returns 404. Probably it should > point to https://cwiki.apache.org/confluence/display/FLUME/Flume+NG -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (FLUME-2065) Regex Extractor Interceptor config agent name inconsistent with rest of docs
[ https://issues.apache.org/jira/browse/FLUME-2065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal reassigned FLUME-2065: - Assignee: Ashish Paliwal > Regex Extractor Interceptor config agent name inconsistent with rest of docs > > > Key: FLUME-2065 > URL: https://issues.apache.org/jira/browse/FLUME-2065 > Project: Flume > Issue Type: Documentation > Components: Docs >Affects Versions: v1.3.1 >Reporter: Hari Sekhon >Assignee: Ashish Paliwal >Priority: Trivial > Attachments: FLUME-2065-0.patch > > > Throughout the documentation the agent is a1.* in all examples, suddenly at > the end of the documentation, it is agent.* for Regex Filtering Interceptor. > Suggest this be changed from agent.* to a1.* in the config examples to be > consistent with rest of docs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-2065) Regex Extractor Interceptor config agent name inconsistent with rest of docs
[ https://issues.apache.org/jira/browse/FLUME-2065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal updated FLUME-2065: -- Attachment: FLUME-2065-0.patch Doc patch > Regex Extractor Interceptor config agent name inconsistent with rest of docs > > > Key: FLUME-2065 > URL: https://issues.apache.org/jira/browse/FLUME-2065 > Project: Flume > Issue Type: Documentation > Components: Docs >Affects Versions: v1.3.1 >Reporter: Hari Sekhon >Assignee: Ashish Paliwal >Priority: Trivial > Attachments: FLUME-2065-0.patch > > > Throughout the documentation the agent is a1.* in all examples, suddenly at > the end of the documentation, it is agent.* for Regex Filtering Interceptor. > Suggest this be changed from agent.* to a1.* in the config examples to be > consistent with rest of docs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (FLUME-2064) Typo/Grammar in flume main user doc under Scribe
[ https://issues.apache.org/jira/browse/FLUME-2064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal reassigned FLUME-2064: - Assignee: Ashish Paliwal > Typo/Grammar in flume main user doc under Scribe > > > Key: FLUME-2064 > URL: https://issues.apache.org/jira/browse/FLUME-2064 > Project: Flume > Issue Type: Documentation > Components: Docs >Affects Versions: v1.3.1 >Reporter: Hari Sekhon >Assignee: Ashish Paliwal >Priority: Trivial > Fix For: v1.4.0 > > Attachments: FLUME-2064-0.patch > > > "The deployment of Scribe please following guide from Facebook" > Typo/grammar needs changing. > Thanks > Hari -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-2006) in Avro the batch size is called batch-size, in all other sources batchSize
[ https://issues.apache.org/jira/browse/FLUME-2006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal updated FLUME-2006: -- Description: http://mail-archives.apache.org/mod_mbox/flume-user/201304.mbox/%3c746ae600-7783-40f4-9817-25617370c...@gmail.com%3e The mismatch is with Avro Sink as well as Thrift sink. Other sinks use batchSize as param name was:http://mail-archives.apache.org/mod_mbox/flume-user/201304.mbox/%3c746ae600-7783-40f4-9817-25617370c...@gmail.com%3e > in Avro the batch size is called batch-size, in all other sources batchSize > --- > > Key: FLUME-2006 > URL: https://issues.apache.org/jira/browse/FLUME-2006 > Project: Flume > Issue Type: Bug > Components: Sinks+Sources >Affects Versions: v1.3.1 >Reporter: Alexander Alten-Lorenz >Assignee: Jeff Lord >Priority: Trivial > > http://mail-archives.apache.org/mod_mbox/flume-user/201304.mbox/%3c746ae600-7783-40f4-9817-25617370c...@gmail.com%3e > The mismatch is with Avro Sink as well as Thrift sink. Other sinks use > batchSize as param name -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-2006) in Avro the batch size is called batch-size, in all other sources batchSize
[ https://issues.apache.org/jira/browse/FLUME-2006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13715084#comment-13715084 ] Ashish Paliwal commented on FLUME-2006: --- Looked at files for the fixes. The fixes are in sdk package, and doc changes in core. There is one thing before we fix this. Javadoc for NettyAvroRpcClient#configure() says "Optionally it can also have a batch-size = batchSize" This means there might be a specific reason for this. Can anyone provide insight into this? > in Avro the batch size is called batch-size, in all other sources batchSize > --- > > Key: FLUME-2006 > URL: https://issues.apache.org/jira/browse/FLUME-2006 > Project: Flume > Issue Type: Bug > Components: Sinks+Sources >Affects Versions: v1.3.1 >Reporter: Alexander Alten-Lorenz >Assignee: Jeff Lord >Priority: Trivial > > http://mail-archives.apache.org/mod_mbox/flume-user/201304.mbox/%3c746ae600-7783-40f4-9817-25617370c...@gmail.com%3e > The mismatch is with Avro Sink as well as Thrift sink. Other sinks use > batchSize as param name -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-2141) Documentation fix
[ https://issues.apache.org/jira/browse/FLUME-2141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727219#comment-13727219 ] Ashish Paliwal commented on FLUME-2141: --- There already a JIRA for this https://issues.apache.org/jira/browse/FLUME-2064 > Documentation fix > - > > Key: FLUME-2141 > URL: https://issues.apache.org/jira/browse/FLUME-2141 > Project: Flume > Issue Type: Bug > Components: Docs >Reporter: anon >Priority: Minor > > The 1.4.x documentation contains the phrase: > "The deployment of Scribe please following guide from Facebook. " > (I'm just looking at flume for the first time, great stuff > and the docs are very well done as well, thanks!) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (FLUME-2142) HTTPS tests for http source
[ https://issues.apache.org/jira/browse/FLUME-2142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal reassigned FLUME-2142: - Assignee: Ashish Paliwal > HTTPS tests for http source > --- > > Key: FLUME-2142 > URL: https://issues.apache.org/jira/browse/FLUME-2142 > Project: Flume > Issue Type: Bug >Affects Versions: v1.4.0 >Reporter: Hari Shreedharan >Assignee: Ashish Paliwal > > We should add at least 2 more tests for HTTPS: > 1. Negative test where non-https client fails to connect to https source > 2. Test that events are available on the channel after being written to https -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-2109) HTTPS support in HTTP Source
[ https://issues.apache.org/jira/browse/FLUME-2109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal updated FLUME-2109: -- Attachment: FLUME-2109-3.patch Patch #3 > HTTPS support in HTTP Source > > > Key: FLUME-2109 > URL: https://issues.apache.org/jira/browse/FLUME-2109 > Project: Flume > Issue Type: Bug >Affects Versions: v1.3.1 >Reporter: Hari Shreedharan >Assignee: Ashish Paliwal > Attachments: FLUME-2109-0.patch, FLUME-2109-1.patch, > FLUME-2109-2.patch, FLUME-2109-3.patch, jettykeystore > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-2142) HTTPS tests for http source
[ https://issues.apache.org/jira/browse/FLUME-2142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal updated FLUME-2142: -- Attachment: FLUME-2142-0.patch > HTTPS tests for http source > --- > > Key: FLUME-2142 > URL: https://issues.apache.org/jira/browse/FLUME-2142 > Project: Flume > Issue Type: Bug >Affects Versions: v1.4.0 >Reporter: Hari Shreedharan >Assignee: Ashish Paliwal > Attachments: FLUME-2142-0.patch > > > We should add at least 2 more tests for HTTPS: > 1. Negative test where non-https client fails to connect to https source > 2. Test that events are available on the channel after being written to https -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-2064) Typo/Grammar in flume main user doc under Scribe
[ https://issues.apache.org/jira/browse/FLUME-2064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal updated FLUME-2064: -- Attachment: FLUME-2064-1.patch Minor Fix > Typo/Grammar in flume main user doc under Scribe > > > Key: FLUME-2064 > URL: https://issues.apache.org/jira/browse/FLUME-2064 > Project: Flume > Issue Type: Documentation > Components: Docs >Affects Versions: v1.3.1 >Reporter: Hari Sekhon >Assignee: Ashish Paliwal >Priority: Trivial > Fix For: v1.5.0 > > Attachments: FLUME-2064-0.patch, FLUME-2064-1.patch > > > "The deployment of Scribe please following guide from Facebook" > Typo/grammar needs changing. > Thanks > Hari -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (FLUME-2197) Memory Channel has GC issues
[ https://issues.apache.org/jira/browse/FLUME-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13799230#comment-13799230 ] Ashish Paliwal commented on FLUME-2197: --- Did anyone worked on this? Curious on the benchmark with new implementation vs the LBQ one. IMHO, GC would be an issue if the Memory Channel is not cleaned up quickly (LinkedList nodes become eligible for GC), before Objects are promoted to Old Gen. For Young Gen GC time is proportional to Live objects. This could very well happen if Sink is not able to clean up Channel fast enough. This could very well be a real case, but missing the details here. > Memory Channel has GC issues > > > Key: FLUME-2197 > URL: https://issues.apache.org/jira/browse/FLUME-2197 > Project: Flume > Issue Type: Bug >Reporter: Hari Shreedharan >Assignee: Roshan Naik > > Due to the fact that we use a LinkedBlockingDeque as the backing queue for > the MemoryChannel, we end up hitting GC issues more often than we should. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (FLUME-2064) Typo/Grammar in flume main user doc under Scribe
[ https://issues.apache.org/jira/browse/FLUME-2064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal updated FLUME-2064: -- Attachment: FLUME-2064-2.patch > Typo/Grammar in flume main user doc under Scribe > > > Key: FLUME-2064 > URL: https://issues.apache.org/jira/browse/FLUME-2064 > Project: Flume > Issue Type: Documentation > Components: Docs >Affects Versions: v1.3.1 >Reporter: Hari Sekhon >Assignee: Ashish Paliwal >Priority: Trivial > Fix For: v1.5.0 > > Attachments: FLUME-2064-0.patch, FLUME-2064-1.patch, > FLUME-2064-2.patch > > > "The deployment of Scribe please following guide from Facebook" > Typo/grammar needs changing. > Thanks > Hari -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (FLUME-2064) Typo/Grammar in flume main user doc under Scribe
[ https://issues.apache.org/jira/browse/FLUME-2064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13801387#comment-13801387 ] Ashish Paliwal commented on FLUME-2064: --- Thanks Roshan! > Typo/Grammar in flume main user doc under Scribe > > > Key: FLUME-2064 > URL: https://issues.apache.org/jira/browse/FLUME-2064 > Project: Flume > Issue Type: Documentation > Components: Docs >Affects Versions: v1.3.1 >Reporter: Hari Sekhon >Assignee: Ashish Paliwal >Priority: Trivial > Fix For: v1.5.0 > > Attachments: FLUME-2064-0.patch, FLUME-2064-1.patch, > FLUME-2064-2.patch > > > "The deployment of Scribe please following guide from Facebook" > Typo/grammar needs changing. > Thanks > Hari -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (FLUME-1851) User Guide grammar mistake
[ https://issues.apache.org/jira/browse/FLUME-1851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal updated FLUME-1851: -- Attachment: FLUME-1851-1.patch > User Guide grammar mistake > -- > > Key: FLUME-1851 > URL: https://issues.apache.org/jira/browse/FLUME-1851 > Project: Flume > Issue Type: Documentation > Components: Docs >Affects Versions: v1.3.1 >Reporter: Steve Stogner >Assignee: Ashish Paliwal >Priority: Trivial > Attachments: FLUME-1851-0.patch, FLUME-1851-1.patch > > > Setup -> Setting up an agent > top of page 2 > actual: > This is a text file which has a format follows the Java properties file > format. > expected: > This is a text file which has a format that follows the Java properties file > format. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (FLUME-1851) User Guide grammar mistake
[ https://issues.apache.org/jira/browse/FLUME-1851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal updated FLUME-1851: -- Attachment: (was: FLUME-1851-2064-0.patch) > User Guide grammar mistake > -- > > Key: FLUME-1851 > URL: https://issues.apache.org/jira/browse/FLUME-1851 > Project: Flume > Issue Type: Documentation > Components: Docs >Affects Versions: v1.3.1 >Reporter: Steve Stogner >Assignee: Ashish Paliwal >Priority: Trivial > Attachments: FLUME-1851-0.patch, FLUME-1851-1.patch > > > Setup -> Setting up an agent > top of page 2 > actual: > This is a text file which has a format follows the Java properties file > format. > expected: > This is a text file which has a format that follows the Java properties file > format. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Assigned] (FLUME-1892) IRC Sink NPE
[ https://issues.apache.org/jira/browse/FLUME-1892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal reassigned FLUME-1892: - Assignee: Ashish Paliwal > IRC Sink NPE > > > Key: FLUME-1892 > URL: https://issues.apache.org/jira/browse/FLUME-1892 > Project: Flume > Issue Type: Bug > Components: Docs, Sinks+Sources > Environment: Ubuntu 12.10 >Reporter: Jeff Collell >Assignee: Ashish Paliwal >Priority: Minor > > IRCSink.java does not initialize splitlines to a default value if it is not > contained in the config. It does check if splitChars is NULL, and initialize > it to the default constant, but it does not do so for splitlines. > The user guide found > [here|http://flume.apache.org/FlumeUserGuide.html#irc-sink] does show that > splitlines has no default value, but it is not bold (indicating it must be > defined in the config). > Obviously if splitlines is not configured, you'll get a NPE when sendLine is > called, since it does not check for NULL values either. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Assigned] (FLUME-1281) No test case exists for flume-irc-sink
[ https://issues.apache.org/jira/browse/FLUME-1281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal reassigned FLUME-1281: - Assignee: Ashish Paliwal > No test case exists for flume-irc-sink > -- > > Key: FLUME-1281 > URL: https://issues.apache.org/jira/browse/FLUME-1281 > Project: Flume > Issue Type: Test > Components: Sinks+Sources >Reporter: Rajesh Koilpillai >Assignee: Ashish Paliwal >Priority: Trivial > > IRC sink doesn't seem to have any test cases > https://svn.apache.org/repos/asf/incubator/flume/trunk/flume-ng-sinks/flume-irc-sink/src/ -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (FLUME-1281) No test case exists for flume-irc-sink
[ https://issues.apache.org/jira/browse/FLUME-1281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal updated FLUME-1281: -- Attachment: FLUME-1281-1892-0.patch Patch > No test case exists for flume-irc-sink > -- > > Key: FLUME-1281 > URL: https://issues.apache.org/jira/browse/FLUME-1281 > Project: Flume > Issue Type: Test > Components: Sinks+Sources >Reporter: Rajesh Koilpillai >Assignee: Ashish Paliwal >Priority: Trivial > Attachments: FLUME-1281-1892-0.patch > > > IRC sink doesn't seem to have any test cases > https://svn.apache.org/repos/asf/incubator/flume/trunk/flume-ng-sinks/flume-irc-sink/src/ -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (FLUME-1892) IRC Sink NPE
[ https://issues.apache.org/jira/browse/FLUME-1892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal updated FLUME-1892: -- Attachment: FLUME-1281-1892-0.patch Patch > IRC Sink NPE > > > Key: FLUME-1892 > URL: https://issues.apache.org/jira/browse/FLUME-1892 > Project: Flume > Issue Type: Bug > Components: Docs, Sinks+Sources > Environment: Ubuntu 12.10 >Reporter: Jeff Collell >Assignee: Ashish Paliwal >Priority: Minor > Attachments: FLUME-1281-1892-0.patch > > > IRCSink.java does not initialize splitlines to a default value if it is not > contained in the config. It does check if splitChars is NULL, and initialize > it to the default constant, but it does not do so for splitlines. > The user guide found > [here|http://flume.apache.org/FlumeUserGuide.html#irc-sink] does show that > splitlines has no default value, but it is not bold (indicating it must be > defined in the config). > Obviously if splitlines is not configured, you'll get a NPE when sendLine is > called, since it does not check for NULL values either. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (FLUME-1501) Flume Scribe Source needs unit tests.
[ https://issues.apache.org/jira/browse/FLUME-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1380#comment-1380 ] Ashish Paliwal commented on FLUME-1501: --- Taking over this JIRA. Please reassign in case some work has already been done. > Flume Scribe Source needs unit tests. > - > > Key: FLUME-1501 > URL: https://issues.apache.org/jira/browse/FLUME-1501 > Project: Flume > Issue Type: Test >Reporter: Hari Shreedharan >Assignee: Denny Ye > -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Assigned] (FLUME-1501) Flume Scribe Source needs unit tests.
[ https://issues.apache.org/jira/browse/FLUME-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal reassigned FLUME-1501: - Assignee: Ashish Paliwal (was: Denny Ye) > Flume Scribe Source needs unit tests. > - > > Key: FLUME-1501 > URL: https://issues.apache.org/jira/browse/FLUME-1501 > Project: Flume > Issue Type: Test >Reporter: Hari Shreedharan >Assignee: Ashish Paliwal > -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (FLUME-1286) Storm Sink
[ https://issues.apache.org/jira/browse/FLUME-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13806667#comment-13806667 ] Ashish Paliwal commented on FLUME-1286: --- Anyone working on this? Would like to work on this. > Storm Sink > -- > > Key: FLUME-1286 > URL: https://issues.apache.org/jira/browse/FLUME-1286 > Project: Flume > Issue Type: New Feature > Components: Sinks+Sources >Reporter: Mubarak Seyed >Assignee: Mubarak Seyed > > Storm contrib project explains how to push/emit events to storm spout [1]. It > would be nice to create a flume-storm sink to emit events to storm spout > output collector [2] > [1] https://github.com/nathanmarz/storm-contrib/tree/master/storm-scribe > [2] > https://github.com/nathanmarz/storm-contrib/blob/master/storm-scribe/src/jvm/storm/scribe/ScribeSpout.java -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Resolved] (FLUME-2141) Documentation fix
[ https://issues.apache.org/jira/browse/FLUME-2141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-2141. --- Resolution: Duplicate Fix Version/s: v1.5.0 > Documentation fix > - > > Key: FLUME-2141 > URL: https://issues.apache.org/jira/browse/FLUME-2141 > Project: Flume > Issue Type: Bug > Components: Docs >Reporter: anon >Priority: Minor > Fix For: v1.5.0 > > > The 1.4.x documentation contains the phrase: > "The deployment of Scribe please following guide from Facebook. " > (I'm just looking at flume for the first time, great stuff > and the docs are very well done as well, thanks!) -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Assigned] (FLUME-1286) Storm Sink
[ https://issues.apache.org/jira/browse/FLUME-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal reassigned FLUME-1286: - Assignee: Ashish Paliwal (was: Mubarak Seyed) > Storm Sink > -- > > Key: FLUME-1286 > URL: https://issues.apache.org/jira/browse/FLUME-1286 > Project: Flume > Issue Type: New Feature > Components: Sinks+Sources >Reporter: Mubarak Seyed >Assignee: Ashish Paliwal > > Storm contrib project explains how to push/emit events to storm spout [1]. It > would be nice to create a flume-storm sink to emit events to storm spout > output collector [2] > [1] https://github.com/nathanmarz/storm-contrib/tree/master/storm-scribe > [2] > https://github.com/nathanmarz/storm-contrib/blob/master/storm-scribe/src/jvm/storm/scribe/ScribeSpout.java -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (FLUME-2223) is NettyAvroRpcClient methods threadsafe
[ https://issues.apache.org/jira/browse/FLUME-2223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13807605#comment-13807605 ] Ashish Paliwal commented on FLUME-2223: --- Please use User Mailing List to ask questions http://flume.apache.org/mailinglists.html > is NettyAvroRpcClient methods threadsafe > > > Key: FLUME-2223 > URL: https://issues.apache.org/jira/browse/FLUME-2223 > Project: Flume > Issue Type: Question > Components: Client SDK >Reporter: Praveen Nair > > Hi, > I would like to know if NettyAvroRpcClient methods are thread safe , so that > same client instance can be used by multiple threads to post log events from > a java application. > Could you provide an example code snippet that mentions the usage of > NettyAvroRpcClient methods in multithreaded situation? > Also the append method in org.apache.flume.clients.log4jappender.Log4jAppender > seems to be synchronised. Does that create a bottleneck if Log4jAppender > is used from application for appending log messages to Flume? > Thanks & Regards, > Praveen -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (FLUME-2223) is NettyAvroRpcClient methods threadsafe
[ https://issues.apache.org/jira/browse/FLUME-2223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13807606#comment-13807606 ] Ashish Paliwal commented on FLUME-2223: --- Please use User Mailing List to ask questions http://flume.apache.org/mailinglists.html > is NettyAvroRpcClient methods threadsafe > > > Key: FLUME-2223 > URL: https://issues.apache.org/jira/browse/FLUME-2223 > Project: Flume > Issue Type: Question > Components: Client SDK >Reporter: Praveen Nair > > Hi, > I would like to know if NettyAvroRpcClient methods are thread safe , so that > same client instance can be used by multiple threads to post log events from > a java application. > Could you provide an example code snippet that mentions the usage of > NettyAvroRpcClient methods in multithreaded situation? > Also the append method in org.apache.flume.clients.log4jappender.Log4jAppender > seems to be synchronised. Does that create a bottleneck if Log4jAppender > is used from application for appending log messages to Flume? > Thanks & Regards, > Praveen -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Created] (FLUME-2231) Add details in Flume Ganglia config in User Guide
Ashish Paliwal created FLUME-2231: - Summary: Add details in Flume Ganglia config in User Guide Key: FLUME-2231 URL: https://issues.apache.org/jira/browse/FLUME-2231 Project: Flume Issue Type: Improvement Components: Docs Affects Versions: v1.4.0 Reporter: Ashish Paliwal Assignee: Ashish Paliwal Priority: Minor In Flume User Guide,http://flume.apache.org/FlumeUserGuide.html#ganglia-reporting, add details to hosts parameter to make it explicit that it represents List of Ganglia Servers to report to -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (FLUME-2232) Add findbugs to Flume build
[ https://issues.apache.org/jira/browse/FLUME-2232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13811884#comment-13811884 ] Ashish Paliwal commented on FLUME-2232: --- Is this JIRA meant to integrate Findbugs with local maven build? You can get the Findbugs report for Flume at https://analysis.apache.org/dashboard/index/137513 The Analysis last ran on 24th April 2013, IMHO, we just need to get it fixed. > Add findbugs to Flume build > --- > > Key: FLUME-2232 > URL: https://issues.apache.org/jira/browse/FLUME-2232 > Project: Flume > Issue Type: Bug > Components: Build >Reporter: Hari Shreedharan >Assignee: Hari Shreedharan >Priority: Minor > Attachments: FLUME-2232.patch > > -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (FLUME-2065) Regex Extractor Interceptor config agent name inconsistent with rest of docs
[ https://issues.apache.org/jira/browse/FLUME-2065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal updated FLUME-2065: -- Attachment: FLUME-2065-2231-0.patch Cumulative Patch > Regex Extractor Interceptor config agent name inconsistent with rest of docs > > > Key: FLUME-2065 > URL: https://issues.apache.org/jira/browse/FLUME-2065 > Project: Flume > Issue Type: Documentation > Components: Docs >Affects Versions: v1.4.0 >Reporter: Hari Sekhon >Assignee: Ashish Paliwal >Priority: Trivial > Attachments: FLUME-2065-0.patch, FLUME-2065-2231-0.patch > > > Throughout the documentation the agent is a1.* in all examples, suddenly at > the end of the documentation, it is agent.* for Regex Filtering Interceptor. > Suggest this be changed from agent.* to a1.* in the config examples to be > consistent with rest of docs. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (FLUME-2231) Add details in Flume Ganglia config in User Guide
[ https://issues.apache.org/jira/browse/FLUME-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal updated FLUME-2231: -- Attachment: FLUME-2065-2231-0.patch Cumulative Patch > Add details in Flume Ganglia config in User Guide > - > > Key: FLUME-2231 > URL: https://issues.apache.org/jira/browse/FLUME-2231 > Project: Flume > Issue Type: Improvement > Components: Docs >Affects Versions: v1.4.0 >Reporter: Ashish Paliwal >Assignee: Ashish Paliwal >Priority: Minor > Labels: documentation > Attachments: FLUME-2065-2231-0.patch > > > In Flume User > Guide,http://flume.apache.org/FlumeUserGuide.html#ganglia-reporting, add > details to hosts parameter to make it explicit that it represents List of > Ganglia Servers to report to -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (FLUME-2065) Regex Extractor Interceptor config agent name inconsistent with rest of docs
[ https://issues.apache.org/jira/browse/FLUME-2065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13814588#comment-13814588 ] Ashish Paliwal commented on FLUME-2065: --- Individual patch is already there. The only reason I made it cumulative was to save time for committers, as both are trivial patches, with minor changes and can be tracked easily. I am ok with individual patches as well. > Regex Extractor Interceptor config agent name inconsistent with rest of docs > > > Key: FLUME-2065 > URL: https://issues.apache.org/jira/browse/FLUME-2065 > Project: Flume > Issue Type: Documentation > Components: Docs >Affects Versions: v1.4.0 >Reporter: Hari Sekhon >Assignee: Ashish Paliwal >Priority: Trivial > Attachments: FLUME-2065-0.patch, FLUME-2065-2231-0.patch > > > Throughout the documentation the agent is a1.* in all examples, suddenly at > the end of the documentation, it is agent.* for Regex Filtering Interceptor. > Suggest this be changed from agent.* to a1.* in the config examples to be > consistent with rest of docs. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (FLUME-2231) Add details in Flume Ganglia config in User Guide
[ https://issues.apache.org/jira/browse/FLUME-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal updated FLUME-2231: -- Attachment: FLUME-2231-0.patch Individual Patch for 2231 > Add details in Flume Ganglia config in User Guide > - > > Key: FLUME-2231 > URL: https://issues.apache.org/jira/browse/FLUME-2231 > Project: Flume > Issue Type: Improvement > Components: Docs >Affects Versions: v1.4.0 >Reporter: Ashish Paliwal >Assignee: Ashish Paliwal >Priority: Minor > Labels: documentation > Attachments: FLUME-2065-2231-0.patch, FLUME-2231-0.patch > > > In Flume User > Guide,http://flume.apache.org/FlumeUserGuide.html#ganglia-reporting, add > details to hosts parameter to make it explicit that it represents List of > Ganglia Servers to report to -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (FLUME-2231) Add details in Flume Ganglia config in User Guide
[ https://issues.apache.org/jira/browse/FLUME-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13815643#comment-13815643 ] Ashish Paliwal commented on FLUME-2231: --- [~roshan_naik] Individual patch uploaded. Please close the review request as well. Thanks for all the help! > Add details in Flume Ganglia config in User Guide > - > > Key: FLUME-2231 > URL: https://issues.apache.org/jira/browse/FLUME-2231 > Project: Flume > Issue Type: Improvement > Components: Docs >Affects Versions: v1.4.0 >Reporter: Ashish Paliwal >Assignee: Ashish Paliwal >Priority: Minor > Labels: documentation > Attachments: FLUME-2065-2231-0.patch, FLUME-2231-0.patch > > > In Flume User > Guide,http://flume.apache.org/FlumeUserGuide.html#ganglia-reporting, add > details to hosts parameter to make it explicit that it represents List of > Ganglia Servers to report to -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (FLUME-1501) Flume Scribe Source needs unit tests.
[ https://issues.apache.org/jira/browse/FLUME-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal updated FLUME-1501: -- Attachment: FLUME-1501-0.patch > Flume Scribe Source needs unit tests. > - > > Key: FLUME-1501 > URL: https://issues.apache.org/jira/browse/FLUME-1501 > Project: Flume > Issue Type: Test >Affects Versions: v1.4.0 >Reporter: Hari Shreedharan >Assignee: Ashish Paliwal > Attachments: FLUME-1501-0.patch > > -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Resolved] (FLUME-1833) Embedded Agent docs are missing from website
[ https://issues.apache.org/jira/browse/FLUME-1833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-1833. --- Resolution: Fixed Fix Version/s: v1.4.0 The issue is resolved with 1.4.0 release. The Developer Guide now has Embedded Agent section as discussed. > Embedded Agent docs are missing from website > > > Key: FLUME-1833 > URL: https://issues.apache.org/jira/browse/FLUME-1833 > Project: Flume > Issue Type: Documentation > Components: Web >Affects Versions: v1.4.0 >Reporter: Will McQueen > Fix For: v1.4.0 > > > http://flume.apache.org/FlumeDeveloperGuide.html > ...doesn't contain the embedded agent documentation, but that documentation > is already checked-in here: > > https://github.com/apache/flume/blob/trunk/flume-ng-doc/sphinx/FlumeDeveloperGuide.rst > ...so it looks like the website needs to be updated with the latest dev guide > (and maybe the latest user guide). -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Assigned] (FLUME-2053) TestExecSource Tests Fail
[ https://issues.apache.org/jira/browse/FLUME-2053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal reassigned FLUME-2053: - Assignee: Ashish Paliwal > TestExecSource Tests Fail > - > > Key: FLUME-2053 > URL: https://issues.apache.org/jira/browse/FLUME-2053 > Project: Flume > Issue Type: Bug > Components: Build >Affects Versions: v1.4.0 > Environment: Ubuntu 13.04 > Java(TM) SE Runtime Environment (build 1.7.0_13-b20) > Java HotSpot(TM) 64-Bit Server VM (build 23.7-b01, mixed mode) > Maven 3.0.5 >Reporter: Hans Uhlig >Assignee: Ashish Paliwal > > Failed tests: > testMonitoredCounterGroup(org.apache.flume.source.TestExecSource): Expected > Value: 5 expected:<[5]> but was:<[0]> -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Assigned] (FLUME-1521) Document the StressSource
[ https://issues.apache.org/jira/browse/FLUME-1521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal reassigned FLUME-1521: - Assignee: Ashish Paliwal (was: Denny Ye) > Document the StressSource > - > > Key: FLUME-1521 > URL: https://issues.apache.org/jira/browse/FLUME-1521 > Project: Flume > Issue Type: Improvement > Components: Docs >Affects Versions: v1.2.0 >Reporter: Denny Ye >Assignee: Ashish Paliwal >Priority: Minor > Attachments: FLUME-1521.patch > > > StressSource is absent at UserGuide, we should fix it -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (FLUME-1521) Document the StressSource
[ https://issues.apache.org/jira/browse/FLUME-1521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13823508#comment-13823508 ] Ashish Paliwal commented on FLUME-1521: --- Taking over the JIRA, seems like review comments need to be incorporated > Document the StressSource > - > > Key: FLUME-1521 > URL: https://issues.apache.org/jira/browse/FLUME-1521 > Project: Flume > Issue Type: Improvement > Components: Docs >Affects Versions: v1.2.0 >Reporter: Denny Ye >Assignee: Ashish Paliwal >Priority: Minor > Attachments: FLUME-1521.patch > > > StressSource is absent at UserGuide, we should fix it -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (FLUME-1542) Flume User Guide sample configurations need fixing
[ https://issues.apache.org/jira/browse/FLUME-1542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13823516#comment-13823516 ] Ashish Paliwal commented on FLUME-1542: --- Just checked for 1.4.0 docs, configuration is good for Avro sink. Can you please check and mark the JIRA as resolved if the change has been done. > Flume User Guide sample configurations need fixing > -- > > Key: FLUME-1542 > URL: https://issues.apache.org/jira/browse/FLUME-1542 > Project: Flume > Issue Type: Improvement > Components: Docs >Affects Versions: v1.2.0 >Reporter: Aaron Baff >Priority: Trivial > > The Flume Users Guide, specifically for Sinks (e.x. > http://flume.apache.org/FlumeUserGuide.html#avro-sink) have the sample > configurations using .channels = instead of just .channel as the correct > property name is. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Assigned] (FLUME-1542) Flume User Guide sample configurations need fixing
[ https://issues.apache.org/jira/browse/FLUME-1542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal reassigned FLUME-1542: - Assignee: Ashish Paliwal > Flume User Guide sample configurations need fixing > -- > > Key: FLUME-1542 > URL: https://issues.apache.org/jira/browse/FLUME-1542 > Project: Flume > Issue Type: Improvement > Components: Docs >Affects Versions: v1.2.0 >Reporter: Aaron Baff >Assignee: Ashish Paliwal >Priority: Trivial > > The Flume Users Guide, specifically for Sinks (e.x. > http://flume.apache.org/FlumeUserGuide.html#avro-sink) have the sample > configurations using .channels = instead of just .channel as the correct > property name is. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Assigned] (FLUME-1612) TestMultiportSyslogTCPSource is a little racy
[
https://issues.apache.org/jira/browse/FLUME-1612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashish Paliwal reassigned FLUME-1612:
-
Assignee: Ashish Paliwal
> TestMultiportSyslogTCPSource is a little racy
> -
>
> Key: FLUME-1612
> URL: https://issues.apache.org/jira/browse/FLUME-1612
> Project: Flume
> Issue Type: Improvement
> Components: Test
>Affects Versions: v1.3.0
>Reporter: Brock Noland
>Assignee: Ashish Paliwal
>Priority: Minor
>
> It failed by a ms. We should see if we can get it passing all the time.
> {noformat}
> Failed tests:
> testMultiplePorts(org.apache.flume.source.TestMultiportSyslogTCPSource):
> Timestamps must match expected:<134862061951[1]> but was:<134862061951[0]>
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Created] (FLUME-2237) Example config in User Guide to use same notation for Agent name in ExecSource
Ashish Paliwal created FLUME-2237: - Summary: Example config in User Guide to use same notation for Agent name in ExecSource Key: FLUME-2237 URL: https://issues.apache.org/jira/browse/FLUME-2237 Project: Flume Issue Type: Improvement Components: Docs Affects Versions: v1.4.0 Reporter: Ashish Paliwal Priority: Minor Agent name (using a1) used in User Guide to follow same convention for following 1. ExecSource http://flume.apache.org/FlumeUserGuide.html#exec-source - uses agent_foo 2. Spooling directory Source http://flume.apache.org/FlumeUserGuide.html#spooling-directory-source - uses agent-1 3. Regex Extractor Interceptor - http://flume.apache.org/FlumeUserGuide.html#regex-extractor-interceptor - Example 1 & 2 uses agent -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (FLUME-2237) Example config in User Guide to use same notation for Agent name
[ https://issues.apache.org/jira/browse/FLUME-2237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal updated FLUME-2237: -- Summary: Example config in User Guide to use same notation for Agent name (was: Example config in User Guide to use same notation for Agent name in ExecSource) > Example config in User Guide to use same notation for Agent name > > > Key: FLUME-2237 > URL: https://issues.apache.org/jira/browse/FLUME-2237 > Project: Flume > Issue Type: Improvement > Components: Docs >Affects Versions: v1.4.0 >Reporter: Ashish Paliwal >Priority: Minor > Labels: documentation > > Agent name (using a1) used in User Guide to follow same convention for > following > 1. ExecSource > http://flume.apache.org/FlumeUserGuide.html#exec-source - uses agent_foo > 2. Spooling directory Source > http://flume.apache.org/FlumeUserGuide.html#spooling-directory-source - uses > agent-1 > 3. Regex Extractor Interceptor - > http://flume.apache.org/FlumeUserGuide.html#regex-extractor-interceptor - > Example 1 & 2 uses agent -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Assigned] (FLUME-2237) Example config in User Guide to use same notation for Agent name
[ https://issues.apache.org/jira/browse/FLUME-2237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal reassigned FLUME-2237: - Assignee: Ashish Paliwal > Example config in User Guide to use same notation for Agent name > > > Key: FLUME-2237 > URL: https://issues.apache.org/jira/browse/FLUME-2237 > Project: Flume > Issue Type: Improvement > Components: Docs >Affects Versions: v1.4.0 >Reporter: Ashish Paliwal >Assignee: Ashish Paliwal >Priority: Minor > Labels: documentation > > Agent name (using a1) used in User Guide to follow same convention for > following > 1. ExecSource > http://flume.apache.org/FlumeUserGuide.html#exec-source - uses agent_foo > 2. Spooling directory Source > http://flume.apache.org/FlumeUserGuide.html#spooling-directory-source - uses > agent-1 > 3. Regex Extractor Interceptor - > http://flume.apache.org/FlumeUserGuide.html#regex-extractor-interceptor - > Example 1 & 2 uses agent -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (FLUME-2053) TestExecSource Tests Fail
[ https://issues.apache.org/jira/browse/FLUME-2053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13823625#comment-13823625 ] Ashish Paliwal commented on FLUME-2053: --- Not able to proceed on this. Hitting https://issues.apache.org/jira/browse/AVRO-1311 on Mac OSX with Oracle Java 7. Would need to bump up Avro version to fix this. > TestExecSource Tests Fail > - > > Key: FLUME-2053 > URL: https://issues.apache.org/jira/browse/FLUME-2053 > Project: Flume > Issue Type: Bug > Components: Build >Affects Versions: v1.4.0 > Environment: Ubuntu 13.04 > Java(TM) SE Runtime Environment (build 1.7.0_13-b20) > Java HotSpot(TM) 64-Bit Server VM (build 23.7-b01, mixed mode) > Maven 3.0.5 >Reporter: Hans Uhlig >Assignee: Ashish Paliwal > > Failed tests: > testMonitoredCounterGroup(org.apache.flume.source.TestExecSource): Expected > Value: 5 expected:<[5]> but was:<[0]> -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (FLUME-2237) Example config in User Guide to use same notation for Agent name
[ https://issues.apache.org/jira/browse/FLUME-2237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal updated FLUME-2237: -- Attachment: FLUME-2237-0.patch Patch > Example config in User Guide to use same notation for Agent name > > > Key: FLUME-2237 > URL: https://issues.apache.org/jira/browse/FLUME-2237 > Project: Flume > Issue Type: Improvement > Components: Docs >Affects Versions: v1.4.0 >Reporter: Ashish Paliwal >Assignee: Ashish Paliwal >Priority: Minor > Labels: documentation > Attachments: FLUME-2237-0.patch > > > Agent name (using a1) used in User Guide to follow same convention for > following > 1. ExecSource > http://flume.apache.org/FlumeUserGuide.html#exec-source - uses agent_foo > 2. Spooling directory Source > http://flume.apache.org/FlumeUserGuide.html#spooling-directory-source - uses > agent-1 > 3. Regex Extractor Interceptor - > http://flume.apache.org/FlumeUserGuide.html#regex-extractor-interceptor - > Example 1 & 2 uses agent -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (FLUME-1892) IRC Sink NPE
[ https://issues.apache.org/jira/browse/FLUME-1892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13824449#comment-13824449 ] Ashish Paliwal commented on FLUME-1892: --- Gentle reminder to help with Review > IRC Sink NPE > > > Key: FLUME-1892 > URL: https://issues.apache.org/jira/browse/FLUME-1892 > Project: Flume > Issue Type: Bug > Components: Docs, Sinks+Sources >Affects Versions: v1.4.0 > Environment: Ubuntu 12.10 >Reporter: Jeff Collell >Assignee: Ashish Paliwal >Priority: Minor > Labels: patch > Attachments: FLUME-1281-1892-0.patch > > > IRCSink.java does not initialize splitlines to a default value if it is not > contained in the config. It does check if splitChars is NULL, and initialize > it to the default constant, but it does not do so for splitlines. > The user guide found > [here|http://flume.apache.org/FlumeUserGuide.html#irc-sink] does show that > splitlines has no default value, but it is not bold (indicating it must be > defined in the config). > Obviously if splitlines is not configured, you'll get a NPE when sendLine is > called, since it does not check for NULL values either. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Created] (FLUME-2238) Provide option to configure worker threads in NettyAvroRpcClient
Ashish Paliwal created FLUME-2238: - Summary: Provide option to configure worker threads in NettyAvroRpcClient Key: FLUME-2238 URL: https://issues.apache.org/jira/browse/FLUME-2238 Project: Flume Issue Type: Bug Components: Sinks+Sources Affects Versions: v1.4.0 Reporter: Ashish Paliwal >From dev ML, Creadits to Cameron Gandevia Embedded flume agents were using around 50 netty IO worker threads. The issue was tracked down to the NettyAvroRpcClient which instantiates the NioClientSocketChannelFactory with the default workerCount (2 * the number of available processors). We would like finer control over the number of IO threads we dedicate to flume -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (FLUME-2238) Provide option to configure worker threads in NettyAvroRpcClient
[ https://issues.apache.org/jira/browse/FLUME-2238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal updated FLUME-2238: -- Description: >From dev ML, Credits to Cameron Gandevia Embedded flume agents were using around 50 netty IO worker threads. The issue was tracked down to the NettyAvroRpcClient which instantiates the NioClientSocketChannelFactory with the default workerCount (2 * the number of available processors). We would like finer control over the number of IO threads we dedicate to flume was: >From dev ML, Creadits to Cameron Gandevia Embedded flume agents were using around 50 netty IO worker threads. The issue was tracked down to the NettyAvroRpcClient which instantiates the NioClientSocketChannelFactory with the default workerCount (2 * the number of available processors). We would like finer control over the number of IO threads we dedicate to flume > Provide option to configure worker threads in NettyAvroRpcClient > > > Key: FLUME-2238 > URL: https://issues.apache.org/jira/browse/FLUME-2238 > Project: Flume > Issue Type: Bug > Components: Sinks+Sources >Affects Versions: v1.4.0 >Reporter: Ashish Paliwal >Assignee: Cameron Gandevia > > From dev ML, Credits to Cameron Gandevia > Embedded flume agents were using around 50 netty IO worker threads. > The issue was tracked down to the NettyAvroRpcClient which instantiates > the NioClientSocketChannelFactory with the default workerCount (2 * the > number of available processors). We would like finer control over the > number of IO threads we dedicate to flume -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (FLUME-2238) Provide option to configure worker threads in NettyAvroRpcClient
[ https://issues.apache.org/jira/browse/FLUME-2238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13826200#comment-13826200 ] Ashish Paliwal commented on FLUME-2238: --- Would be good to add a new variable like maxIoWorkers, which if specified initializes the appropriate NioClientSocketChannelFactory constructor. > Provide option to configure worker threads in NettyAvroRpcClient > > > Key: FLUME-2238 > URL: https://issues.apache.org/jira/browse/FLUME-2238 > Project: Flume > Issue Type: Bug > Components: Sinks+Sources >Affects Versions: v1.4.0 >Reporter: Ashish Paliwal >Assignee: Cameron Gandevia > > From dev ML, Credits to Cameron Gandevia > Embedded flume agents were using around 50 netty IO worker threads. > The issue was tracked down to the NettyAvroRpcClient which instantiates > the NioClientSocketChannelFactory with the default workerCount (2 * the > number of available processors). We would like finer control over the > number of IO threads we dedicate to flume -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Created] (FLUME-2244) AvroSource to use TransceiverThreadFactory for Thread naming while initializing NioServerSocketChannelFactory
Ashish Paliwal created FLUME-2244: - Summary: AvroSource to use TransceiverThreadFactory for Thread naming while initializing NioServerSocketChannelFactory Key: FLUME-2244 URL: https://issues.apache.org/jira/browse/FLUME-2244 Project: Flume Issue Type: Improvement Components: Sinks+Sources Affects Versions: v1.4.0 Reporter: Ashish Paliwal Priority: Minor Currently in Avro source, we use socketChannelFactory = new NioServerSocketChannelFactory (Executors .newCachedThreadPool(), Executors.newCachedThreadPool()); This would create generic Thread names like pool-1-thread-1. It would be good to use a ThreadFactory, like we use in NettyAvroRpcClient for better identification of Threads in log file ExecutorService bossExecutor = Executors.newCachedThreadPool(new TransceiverThreadFactory( "Avro " + NettyTransceiver.class.getSimpleName() + " Boss")); ExecutorService workerExecutor = Executors.newCachedThreadPool(new TransceiverThreadFactory( "Avro " + NettyTransceiver.class.getSimpleName() + " I/O Worker")); -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Created] (FLUME-2243) AvroSource to use TransceiverThreadFactory for Thread naming while initializing NioServerSocketChannelFactory
Ashish Paliwal created FLUME-2243: - Summary: AvroSource to use TransceiverThreadFactory for Thread naming while initializing NioServerSocketChannelFactory Key: FLUME-2243 URL: https://issues.apache.org/jira/browse/FLUME-2243 Project: Flume Issue Type: Improvement Components: Sinks+Sources Affects Versions: v1.4.0 Reporter: Ashish Paliwal Priority: Minor Currently in Avro source, we use socketChannelFactory = new NioServerSocketChannelFactory (Executors .newCachedThreadPool(), Executors.newCachedThreadPool()); This would create generic Thread names like pool-1-thread-1. It would be good to use a ThreadFactory, like we use in NettyAvroRpcClient for better identification of Threads in log file ExecutorService bossExecutor = Executors.newCachedThreadPool(new TransceiverThreadFactory( "Avro " + NettyTransceiver.class.getSimpleName() + " Boss")); ExecutorService workerExecutor = Executors.newCachedThreadPool(new TransceiverThreadFactory( "Avro " + NettyTransceiver.class.getSimpleName() + " I/O Worker")); -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (FLUME-2242) Flume Sink and Source for LinkedIn Kafka
[ https://issues.apache.org/jira/browse/FLUME-2242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal updated FLUME-2242: -- Affects Version/s: (was: v1.5.0) v1.4.0 Updated Affected Release version to 1.4.0 as 1.5.0 is not release yet > Flume Sink and Source for LinkedIn Kafka > > > Key: FLUME-2242 > URL: https://issues.apache.org/jira/browse/FLUME-2242 > Project: Flume > Issue Type: New Feature > Components: Sinks+Sources >Affects Versions: v1.4.0 > Environment: kafka, scala jar dependencies >Reporter: Frank Yao >Priority: Minor > Labels: kafka, sink, source > > Source and Sinks for LinkedIn Kafka 0.7.2. I use my code in our company and > runs in 100k msg/sec. > My Project:https://github.com/baniuyao/flume-kafka -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Assigned] (FLUME-2243) AvroSource to use TransceiverThreadFactory for Thread naming while initializing NioServerSocketChannelFactory
[ https://issues.apache.org/jira/browse/FLUME-2243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal reassigned FLUME-2243: - Assignee: Ashish Paliwal > AvroSource to use TransceiverThreadFactory for Thread naming while > initializing NioServerSocketChannelFactory > - > > Key: FLUME-2243 > URL: https://issues.apache.org/jira/browse/FLUME-2243 > Project: Flume > Issue Type: Improvement > Components: Sinks+Sources >Affects Versions: v1.4.0 >Reporter: Ashish Paliwal >Assignee: Ashish Paliwal >Priority: Minor > > Currently in Avro source, we use > socketChannelFactory = new NioServerSocketChannelFactory > (Executors .newCachedThreadPool(), Executors.newCachedThreadPool()); > This would create generic Thread names like pool-1-thread-1. It would be good > to use a ThreadFactory, like we use in NettyAvroRpcClient for better > identification of Threads in log file > ExecutorService bossExecutor = Executors.newCachedThreadPool(new > TransceiverThreadFactory( > "Avro " + NettyTransceiver.class.getSimpleName() + " Boss")); > ExecutorService workerExecutor = Executors.newCachedThreadPool(new > TransceiverThreadFactory( > "Avro " + NettyTransceiver.class.getSimpleName() + " I/O Worker")); -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Resolved] (FLUME-2244) AvroSource to use TransceiverThreadFactory for Thread naming while initializing NioServerSocketChannelFactory
[ https://issues.apache.org/jira/browse/FLUME-2244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-2244. --- Resolution: Duplicate 2 JIRA's got created with same content, some local issue > AvroSource to use TransceiverThreadFactory for Thread naming while > initializing NioServerSocketChannelFactory > - > > Key: FLUME-2244 > URL: https://issues.apache.org/jira/browse/FLUME-2244 > Project: Flume > Issue Type: Improvement > Components: Sinks+Sources >Affects Versions: v1.4.0 >Reporter: Ashish Paliwal >Priority: Minor > > Currently in Avro source, we use > socketChannelFactory = new NioServerSocketChannelFactory > (Executors .newCachedThreadPool(), Executors.newCachedThreadPool()); > This would create generic Thread names like pool-1-thread-1. It would be good > to use a ThreadFactory, like we use in NettyAvroRpcClient for better > identification of Threads in log file > ExecutorService bossExecutor = Executors.newCachedThreadPool(new > TransceiverThreadFactory( > "Avro " + NettyTransceiver.class.getSimpleName() + " Boss")); > ExecutorService workerExecutor = Executors.newCachedThreadPool(new > TransceiverThreadFactory( > "Avro " + NettyTransceiver.class.getSimpleName() + " I/O Worker")); -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (FLUME-2242) Flume Sink and Source for LinkedIn Kafka
[ https://issues.apache.org/jira/browse/FLUME-2242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13828853#comment-13828853 ] Ashish Paliwal commented on FLUME-2242: --- Is the project under ASL v2? > Flume Sink and Source for LinkedIn Kafka > > > Key: FLUME-2242 > URL: https://issues.apache.org/jira/browse/FLUME-2242 > Project: Flume > Issue Type: New Feature > Components: Sinks+Sources >Affects Versions: v1.4.0 > Environment: kafka, scala jar dependencies >Reporter: Frank Yao >Priority: Minor > Labels: kafka, sink, source > > Source and Sinks for LinkedIn Kafka 0.7.2. I use my code in our company and > runs in 100k msg/sec. > My Project:https://github.com/baniuyao/flume-kafka -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (FLUME-2242) Flume Sink and Source for LinkedIn Kafka
[ https://issues.apache.org/jira/browse/FLUME-2242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13828977#comment-13828977 ] Ashish Paliwal commented on FLUME-2242: --- No worries, we are closer in timezone, just an hour apart. Meanwhile, we will have more on the License stuff by then. > Flume Sink and Source for LinkedIn Kafka > > > Key: FLUME-2242 > URL: https://issues.apache.org/jira/browse/FLUME-2242 > Project: Flume > Issue Type: New Feature > Components: Sinks+Sources >Affects Versions: v1.4.0 > Environment: kafka, scala jar dependencies >Reporter: Frank Yao >Priority: Minor > Labels: kafka, sink, source > > Source and Sinks for LinkedIn Kafka 0.7.2. I use my code in our company and > runs in 100k msg/sec. > My Project:https://github.com/baniuyao/flume-kafka -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (FLUME-2242) Flume Sink and Source for LinkedIn Kafka
[ https://issues.apache.org/jira/browse/FLUME-2242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13828954#comment-13828954 ] Ashish Paliwal commented on FLUME-2242: --- AFAIK, you need to sign a ICLA. More here http://apache.org/licenses/ see section "Contributor License Agreements". I am not the expert on Licenses part and would let more experienced folks chime in for that. On other side, would be great if you can create a patch, attach to JIRA and open a review request. Once review is completed one of the committers can commit the code. More info on this https://cwiki.apache.org/confluence/display/FLUME/How+to+Contribute This would be a good addition to Flume. Few things that we would need to do are 1. Update the pkg structure and add ASL header 2. Add Unit Test cases > Flume Sink and Source for LinkedIn Kafka > > > Key: FLUME-2242 > URL: https://issues.apache.org/jira/browse/FLUME-2242 > Project: Flume > Issue Type: New Feature > Components: Sinks+Sources >Affects Versions: v1.4.0 > Environment: kafka, scala jar dependencies >Reporter: Frank Yao >Priority: Minor > Labels: kafka, sink, source > > Source and Sinks for LinkedIn Kafka 0.7.2. I use my code in our company and > runs in 100k msg/sec. > My Project:https://github.com/baniuyao/flume-kafka -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (FLUME-2238) Provide option to configure worker threads in NettyAvroRpcClient
[ https://issues.apache.org/jira/browse/FLUME-2238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13830958#comment-13830958 ] Ashish Paliwal commented on FLUME-2238: --- Can you please attach the updated patch to JIRA. We usually follow the following convention FLUME--.patch, where is num starts from 0 and is updated for each iteration if needed. For current case, post the review comments it shall be like FLUME-2238-1.patch > Provide option to configure worker threads in NettyAvroRpcClient > > > Key: FLUME-2238 > URL: https://issues.apache.org/jira/browse/FLUME-2238 > Project: Flume > Issue Type: Bug > Components: Sinks+Sources >Affects Versions: v1.4.0 >Reporter: Ashish Paliwal >Assignee: Cameron Gandevia > Fix For: v1.5.0 > > Attachments: FLUME-2238.patch > > > From dev ML, Credits to Cameron Gandevia > Embedded flume agents were using around 50 netty IO worker threads. > The issue was tracked down to the NettyAvroRpcClient which instantiates > the NioClientSocketChannelFactory with the default workerCount (2 * the > number of available processors). We would like finer control over the > number of IO threads we dedicate to flume -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (FLUME-2248) Redis sink
[ https://issues.apache.org/jira/browse/FLUME-2248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13830960#comment-13830960 ] Ashish Paliwal commented on FLUME-2248: --- Can you add doc changes to the patch? Meanwhile I shall try out the sink. > Redis sink > -- > > Key: FLUME-2248 > URL: https://issues.apache.org/jira/browse/FLUME-2248 > Project: Flume > Issue Type: New Feature > Components: Sinks+Sources >Reporter: Bruno Mahé >Assignee: Bruno Mahé > Attachments: FLUME-2248-1.patch > > > This ticket is about adding a new sink for Redis -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (FLUME-2248) Redis sink
[ https://issues.apache.org/jira/browse/FLUME-2248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13831170#comment-13831170 ] Ashish Paliwal commented on FLUME-2248: --- Got it. With doc changes I meant User Guide changes. > Redis sink > -- > > Key: FLUME-2248 > URL: https://issues.apache.org/jira/browse/FLUME-2248 > Project: Flume > Issue Type: New Feature > Components: Sinks+Sources >Reporter: Bruno Mahé >Assignee: Bruno Mahé > Attachments: FLUME-2248-1.patch > > > This ticket is about adding a new sink for Redis -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (FLUME-2242) Flume Sink and Source for LinkedIn Kafka
[ https://issues.apache.org/jira/browse/FLUME-2242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13831178#comment-13831178 ] Ashish Paliwal commented on FLUME-2242: --- Any progress with it? I am planning to work on it sometime today and add the work to this JIRA for further refinement. Will take a while to submit the patch for review. > Flume Sink and Source for LinkedIn Kafka > > > Key: FLUME-2242 > URL: https://issues.apache.org/jira/browse/FLUME-2242 > Project: Flume > Issue Type: New Feature > Components: Sinks+Sources >Affects Versions: v1.4.0 > Environment: kafka, scala jar dependencies >Reporter: Frank Yao >Priority: Minor > Labels: kafka, sink, source > > Source and Sinks for LinkedIn Kafka 0.7.2. I use my code in our company and > runs in 100k msg/sec. > My Project:https://github.com/baniuyao/flume-kafka -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Created] (FLUME-2250) Add support for Kafka Source
Ashish Paliwal created FLUME-2250: - Summary: Add support for Kafka Source Key: FLUME-2250 URL: https://issues.apache.org/jira/browse/FLUME-2250 Project: Flume Issue Type: Sub-task Components: Sinks+Sources Affects Versions: v1.4.0 Reporter: Ashish Paliwal Priority: Minor Add support for Kafka Source -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Created] (FLUME-2251) Add support for Kafka Sink
Ashish Paliwal created FLUME-2251: - Summary: Add support for Kafka Sink Key: FLUME-2251 URL: https://issues.apache.org/jira/browse/FLUME-2251 Project: Flume Issue Type: Sub-task Components: Sinks+Sources Affects Versions: v1.4.0 Reporter: Ashish Paliwal Priority: Minor Add support for Kafka Sink -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (FLUME-2242) Flume Sink and Source for LinkedIn Kafka
[ https://issues.apache.org/jira/browse/FLUME-2242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13831206#comment-13831206 ] Ashish Paliwal commented on FLUME-2242: --- I added sub-tasks for Source and Sink, as they shall go in separate packages. Kafka 0.7.2 is not in Maven, do you think we should upgrade to 0.8 beta version? > Flume Sink and Source for LinkedIn Kafka > > > Key: FLUME-2242 > URL: https://issues.apache.org/jira/browse/FLUME-2242 > Project: Flume > Issue Type: New Feature > Components: Sinks+Sources >Affects Versions: v1.4.0 > Environment: kafka, scala jar dependencies >Reporter: Frank Yao >Priority: Minor > Labels: kafka, sink, source > > Source and Sinks for LinkedIn Kafka 0.7.2. I use my code in our company and > runs in 100k msg/sec. > My Project:https://github.com/baniuyao/flume-kafka -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Comment Edited] (FLUME-2250) Add support for Kafka Source
[ https://issues.apache.org/jira/browse/FLUME-2250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13831274#comment-13831274 ] Ashish Paliwal edited comment on FLUME-2250 at 11/25/13 8:41 AM: - Not a final patch. Used Kafka 0.8 version, and it seems a lot more work would be needed to make it work as expected with Kafka 0.8. This is just to base further work on it. Test cases are still missing, need to add them. was (Author: paliwalashish): Not a final patch. Used Kafka 0.8 version, and it seems a lot more work would be needed to make it work as expected with Kafka 0.8. This is just to base further work on it. > Add support for Kafka Source > > > Key: FLUME-2250 > URL: https://issues.apache.org/jira/browse/FLUME-2250 > Project: Flume > Issue Type: Sub-task > Components: Sinks+Sources >Affects Versions: v1.4.0 >Reporter: Ashish Paliwal >Priority: Minor > Attachments: FLUME-2250-0.patch > > > Add support for Kafka Source -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (FLUME-2250) Add support for Kafka Source
[ https://issues.apache.org/jira/browse/FLUME-2250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal updated FLUME-2250: -- Attachment: FLUME-2250-0.patch Not a final patch. Used Kafka 0.8 version, and it seems a lot more work would be needed to make it work as expected with Kafka 0.8. This is just to base further work on it. > Add support for Kafka Source > > > Key: FLUME-2250 > URL: https://issues.apache.org/jira/browse/FLUME-2250 > Project: Flume > Issue Type: Sub-task > Components: Sinks+Sources >Affects Versions: v1.4.0 >Reporter: Ashish Paliwal >Priority: Minor > Attachments: FLUME-2250-0.patch > > > Add support for Kafka Source -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (FLUME-2242) Flume Sink and Source for LinkedIn Kafka
[ https://issues.apache.org/jira/browse/FLUME-2242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13831367#comment-13831367 ] Ashish Paliwal commented on FLUME-2242: --- Great ! The problem is Kafka 0.7.2 is not available in Maven repo, so am not sure how jenkins builds would work. We might have to find a way for this to work in automated environment. I am working on Sink part, shall attach the patch as soon as I am done. If you have any suggestions on 0.7.2 version's maven integration, let me know. I shall rebase the Source back to 0.7.2 > Flume Sink and Source for LinkedIn Kafka > > > Key: FLUME-2242 > URL: https://issues.apache.org/jira/browse/FLUME-2242 > Project: Flume > Issue Type: New Feature > Components: Sinks+Sources >Affects Versions: v1.4.0 > Environment: kafka, scala jar dependencies >Reporter: Frank Yao >Priority: Minor > Labels: kafka, sink, source > > Source and Sinks for LinkedIn Kafka 0.7.2. I use my code in our company and > runs in 100k msg/sec. > My Project:https://github.com/baniuyao/flume-kafka -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (FLUME-2251) Add support for Kafka Sink
[ https://issues.apache.org/jira/browse/FLUME-2251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal updated FLUME-2251: -- Attachment: FLUME-2251-0.patch Package changes, kept code with Kafka 0.7.2. Test cases are pending > Add support for Kafka Sink > -- > > Key: FLUME-2251 > URL: https://issues.apache.org/jira/browse/FLUME-2251 > Project: Flume > Issue Type: Sub-task > Components: Sinks+Sources >Affects Versions: v1.4.0 >Reporter: Ashish Paliwal >Priority: Minor > Attachments: FLUME-2251-0.patch > > > Add support for Kafka Sink -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (FLUME-2250) Add support for Kafka Source
[ https://issues.apache.org/jira/browse/FLUME-2250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal updated FLUME-2250: -- Attachment: FLUME-2250-1.patch Reverted to Kafka 0.7.2, minor fixes. Test cases still pending. > Add support for Kafka Source > > > Key: FLUME-2250 > URL: https://issues.apache.org/jira/browse/FLUME-2250 > Project: Flume > Issue Type: Sub-task > Components: Sinks+Sources >Affects Versions: v1.4.0 >Reporter: Ashish Paliwal >Priority: Minor > Attachments: FLUME-2250-0.patch, FLUME-2250-1.patch > > > Add support for Kafka Source -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (FLUME-2243) AvroSource to use TransceiverThreadFactory for Thread naming while initializing NioServerSocketChannelFactory
[ https://issues.apache.org/jira/browse/FLUME-2243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal updated FLUME-2243: -- Attachment: FLUME-2243-0.patch > AvroSource to use TransceiverThreadFactory for Thread naming while > initializing NioServerSocketChannelFactory > - > > Key: FLUME-2243 > URL: https://issues.apache.org/jira/browse/FLUME-2243 > Project: Flume > Issue Type: Improvement > Components: Sinks+Sources >Affects Versions: v1.4.0 >Reporter: Ashish Paliwal >Assignee: Ashish Paliwal >Priority: Minor > Attachments: FLUME-2243-0.patch > > > Currently in Avro source, we use > socketChannelFactory = new NioServerSocketChannelFactory > (Executors .newCachedThreadPool(), Executors.newCachedThreadPool()); > This would create generic Thread names like pool-1-thread-1. It would be good > to use a ThreadFactory, like we use in NettyAvroRpcClient for better > identification of Threads in log file > ExecutorService bossExecutor = Executors.newCachedThreadPool(new > TransceiverThreadFactory( > "Avro " + NettyTransceiver.class.getSimpleName() + " Boss")); > ExecutorService workerExecutor = Executors.newCachedThreadPool(new > TransceiverThreadFactory( > "Avro " + NettyTransceiver.class.getSimpleName() + " I/O Worker")); -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Assigned] (FLUME-2130) Handle larger payloads via SyslogUDPSource
[ https://issues.apache.org/jira/browse/FLUME-2130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal reassigned FLUME-2130: - Assignee: Ashish Paliwal > Handle larger payloads via SyslogUDPSource > -- > > Key: FLUME-2130 > URL: https://issues.apache.org/jira/browse/FLUME-2130 > Project: Flume > Issue Type: Improvement >Affects Versions: v1.2.0, v1.3.0, v1.4.0 >Reporter: Roman Scherer >Assignee: Ashish Paliwal > Attachments: 0001-Handle-larger-UDP-payloads-via-SyslogUDPSource.patch > > > Syslog messages sent via UDP can contain a payload up to 65507 bytes. > Payloads more than 768 bytes are currently truncated because the Netty's > buffer size set to this value. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (FLUME-2130) Handle larger payloads via SyslogUDPSource
[ https://issues.apache.org/jira/browse/FLUME-2130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13833501#comment-13833501 ] Ashish Paliwal commented on FLUME-2130: --- Working on rebasing the patch. > Handle larger payloads via SyslogUDPSource > -- > > Key: FLUME-2130 > URL: https://issues.apache.org/jira/browse/FLUME-2130 > Project: Flume > Issue Type: Improvement >Affects Versions: v1.2.0, v1.3.0, v1.4.0 >Reporter: Roman Scherer >Assignee: Ashish Paliwal > Attachments: 0001-Handle-larger-UDP-payloads-via-SyslogUDPSource.patch > > > Syslog messages sent via UDP can contain a payload up to 65507 bytes. > Payloads more than 768 bytes are currently truncated because the Netty's > buffer size set to this value. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (FLUME-2130) Handle larger payloads via SyslogUDPSource
[ https://issues.apache.org/jira/browse/FLUME-2130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal updated FLUME-2130: -- Attachment: FLUME-2130-0.patch > Handle larger payloads via SyslogUDPSource > -- > > Key: FLUME-2130 > URL: https://issues.apache.org/jira/browse/FLUME-2130 > Project: Flume > Issue Type: Improvement >Affects Versions: v1.2.0, v1.3.0, v1.4.0 >Reporter: Roman Scherer >Assignee: Ashish Paliwal > Attachments: > 0001-Handle-larger-UDP-payloads-via-SyslogUDPSource.patch, FLUME-2130-0.patch > > > Syslog messages sent via UDP can contain a payload up to 65507 bytes. > Payloads more than 768 bytes are currently truncated because the Netty's > buffer size set to this value. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (FLUME-2242) Flume Sink and Source for LinkedIn Kafka
[ https://issues.apache.org/jira/browse/FLUME-2242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13833844#comment-13833844 ] Ashish Paliwal commented on FLUME-2242: --- Great! Once done, please upload the patch for review (https://reviews.apache.org). More details here https://cwiki.apache.org/confluence/display/FLUME/How+to+Contribute > Flume Sink and Source for LinkedIn Kafka > > > Key: FLUME-2242 > URL: https://issues.apache.org/jira/browse/FLUME-2242 > Project: Flume > Issue Type: New Feature > Components: Sinks+Sources >Affects Versions: v1.4.0 > Environment: kafka, scala jar dependencies >Reporter: Frank Yao >Priority: Minor > Labels: kafka, sink, source > > Source and Sinks for LinkedIn Kafka 0.7.2. I use my code in our company and > runs in 100k msg/sec. > My Project:https://github.com/baniuyao/flume-kafka -- This message was sent by Atlassian JIRA (v6.1#6144)
