Re: New version of flume1.3
Hi, Yes it has been and I am just getting all the website items updated before announcing the release. The Staging website has the correct download links: http://flume.staging.apache.org/download.html Brock On Tue, Dec 4, 2012 at 5:17 AM, kashif khan drkashif8...@gmail.com wrote: HI Dear all, I just to want to know that the new version of Flume1.3 have been release or not ? As I am waiting for it. Many thanks Best regards -- Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
Re: New version of flume1.3
Dear Brock Noland. Many thanks for this good news. Now I have basic question that I have already setup the cluster and old version working properly. Can I install this new version parallel to old version or I should uninstall the old version and install this new version? The following documentation link does not work The change log and documentation are available on the 1.3.0 release page: http://flume.apache.org/releases/1.3.0.html Many thanks Best regards On Tue, Dec 4, 2012 at 12:19 PM, Brock Noland br...@cloudera.com wrote: Hi, Yes it has been and I am just getting all the website items updated before announcing the release. The Staging website has the correct download links: http://flume.staging.apache.org/download.html Brock On Tue, Dec 4, 2012 at 5:17 AM, kashif khan drkashif8...@gmail.com wrote: HI Dear all, I just to want to know that the new version of Flume1.3 have been release or not ? As I am waiting for it. Many thanks Best regards -- Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
Re: New version of flume1.3
Moving to u...@flume.apache.org You can install different versions of flume, for sure. Simply use different directories for the binaries and one directory for the configs. - Alex On Dec 4, 2012, at 1:54 PM, kashif khan drkashif8...@gmail.com wrote: Dear Brock Noland. Many thanks for this good news. Now I have basic question that I have already setup the cluster and old version working properly. Can I install this new version parallel to old version or I should uninstall the old version and install this new version? The following documentation link does not work The change log and documentation are available on the 1.3.0 release page: http://flume.apache.org/releases/1.3.0.html Many thanks Best regards On Tue, Dec 4, 2012 at 12:19 PM, Brock Noland br...@cloudera.com wrote: Hi, Yes it has been and I am just getting all the website items updated before announcing the release. The Staging website has the correct download links: http://flume.staging.apache.org/download.html Brock On Tue, Dec 4, 2012 at 5:17 AM, kashif khan drkashif8...@gmail.com wrote: HI Dear all, I just to want to know that the new version of Flume1.3 have been release or not ? As I am waiting for it. Many thanks Best regards -- Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/ -- Alexander Alten-Lorenz http://mapredit.blogspot.com German Hadoop LinkedIn Group: http://goo.gl/N8pCF
Re: New version of flume1.3
1.3 and 1.2 could be installed side by side but I would remove 1.2 when you install 1.3. Correct, that is one of pages I am working to update. -- Brock Noland Sent with Sparrow (http://www.sparrowmailapp.com/?sig) On Tuesday, December 4, 2012 at 6:54 AM, kashif khan wrote: Dear Brock Noland. Many thanks for this good news. Now I have basic question that I have already setup the cluster and old version working properly. Can I install this new version parallel to old version or I should uninstall the old version and install this new version? The following documentation link does not work The change log and documentation are available on the 1.3.0 release page: http://flume.apache.org/releases/1.3.0.html Many thanks Best regards On Tue, Dec 4, 2012 at 12:19 PM, Brock Noland br...@cloudera.com wrote: Hi, Yes it has been and I am just getting all the website items updated before announcing the release. The Staging website has the correct download links: http://flume.staging.apache.org/download.html Brock On Tue, Dec 4, 2012 at 5:17 AM, kashif khan drkashif8...@gmail.com wrote: HI Dear all, I just to want to know that the new version of Flume1.3 have been release or not ? As I am waiting for it. Many thanks Best regards -- Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
Re: New version of flume1.3
Many thanks Alex On Tue, Dec 4, 2012 at 1:18 PM, Alexander Alten-Lorenz wget.n...@gmail.comwrote: Moving to u...@flume.apache.org You can install different versions of flume, for sure. Simply use different directories for the binaries and one directory for the configs. - Alex On Dec 4, 2012, at 1:54 PM, kashif khan drkashif8...@gmail.com wrote: Dear Brock Noland. Many thanks for this good news. Now I have basic question that I have already setup the cluster and old version working properly. Can I install this new version parallel to old version or I should uninstall the old version and install this new version? The following documentation link does not work The change log and documentation are available on the 1.3.0 release page: http://flume.apache.org/releases/1.3.0.html Many thanks Best regards On Tue, Dec 4, 2012 at 12:19 PM, Brock Noland br...@cloudera.com wrote: Hi, Yes it has been and I am just getting all the website items updated before announcing the release. The Staging website has the correct download links: http://flume.staging.apache.org/download.html Brock On Tue, Dec 4, 2012 at 5:17 AM, kashif khan drkashif8...@gmail.com wrote: HI Dear all, I just to want to know that the new version of Flume1.3 have been release or not ? As I am waiting for it. Many thanks Best regards -- Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/ -- Alexander Alten-Lorenz http://mapredit.blogspot.com German Hadoop LinkedIn Group: http://goo.gl/N8pCF
[jira] [Commented] (FLUME-1764) File_roll type Sink how to achieve according to multiple files dynamically generated directory, and the file on the corresponding directory
[ https://issues.apache.org/jira/browse/FLUME-1764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13509820#comment-13509820 ] Brock Noland commented on FLUME-1764: - Echo'ing Roshan, could you explain again exactly what the problem is? File_roll type Sink how to achieve according to multiple files dynamically generated directory, and the file on the corresponding directory --- Key: FLUME-1764 URL: https://issues.apache.org/jira/browse/FLUME-1764 Project: Flume Issue Type: Improvement Components: Sinks+Sources Affects Versions: v1.4.0 Reporter: zhouqingfa Fix For: v1.4.0 I want to achieve such a function：Source terminal(using the 'tail -f' command of execSource)can dynamicly monitor a directory to get new files(such as access_1045.log),then transfer immediately the new file to the local directory（using FileRollSink type） and the file will reache such a directory（The directory will be based on the file name dynamically generated,for example,in the configuration file,sink.directory=/opt/apps_install,then the file will be put such a directory /opt/apps_install/1045）.Finally,Source terminal can not only minotor a directory and transter the new file to /opt/apps_install/..，but also transfer immediately the new content of the file to the Corresponding file in the directory.How to achieve such a function in the flume1.4.0. Thanks.. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-924) Implement a JMS source for Flume NG
[ https://issues.apache.org/jira/browse/FLUME-924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13509817#comment-13509817 ] Brock Noland commented on FLUME-924: Hi Bruno, Have you made any progress on this? If not I'd be interested in giving it a shot. Brock Implement a JMS source for Flume NG --- Key: FLUME-924 URL: https://issues.apache.org/jira/browse/FLUME-924 Project: Flume Issue Type: New Feature Affects Versions: v1.0.0 Reporter: Bruno Mahé Assignee: Bruno Mahé -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[ANNOUNCE] Apache Flume 1.3.0 released
The Apache Flume team is pleased to announce the release of Flume version 1.3.0. Flume is a distributed, reliable, and available service for efficiently collecting, aggregating, and moving large amounts of log data. This release can be downloaded from the Flume download page at: http://flume.apache.org/download.html The change log and documentation are available on the 1.3.0 release page: http://flume.apache.org/releases/1.3.0.html Your help and feedback is more than welcome. For more information on how to report problems and to get involved, visit the project website at http://flume.apache.org/ The Apache Flume Team
API Docs and PDFs on the website
Hi, I assume the PDFs and API Docs on the release page http://flume.apache.org/releases/1.3.0.html are just manually generated and then committed directly to the prod website? I just want to confirm as I am trying to put together a guide on how to edit the website and what items need to be updated as part of a release. Brock
[jira] [Assigned] (FLUME-1766) AvroSource throws confusing exception when configured without a port
[ https://issues.apache.org/jira/browse/FLUME-1766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Lord reassigned FLUME-1766: Assignee: Jeff Lord AvroSource throws confusing exception when configured without a port Key: FLUME-1766 URL: https://issues.apache.org/jira/browse/FLUME-1766 Project: Flume Issue Type: Bug Components: Sinks+Sources Affects Versions: v1.3.0 Reporter: Mike Percy Assignee: Jeff Lord Priority: Minor Labels: newbie Fix For: v1.4.0 2012-12-03 18:25:08,210 (conf-file-poller-0) [DEBUG - org.apache.flume.source.DefaultSourceFactory.create(DefaultSourceFactory.java:74)] Creating instance of source src-1, type AVRO 2012-12-03 18:25:08,235 (conf-file-poller-0) [ERROR - org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:204)] Failed to load configuration data. Exception follows. java.lang.NumberFormatException: null at java.lang.Integer.parseInt(Integer.java:417) at java.lang.Integer.parseInt(Integer.java:499) at org.apache.flume.source.AvroSource.configure(AvroSource.java:126) at org.apache.flume.conf.Configurables.configure(Configurables.java:41) at org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.loadSources(PropertiesFileConfigurationProvider.java:323) at org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.load(PropertiesFileConfigurationProvider.java:222) at org.apache.flume.conf.file.AbstractFileConfigurationProvider.doLoad(AbstractFileConfigurationProvider.java:123) at org.apache.flume.conf.file.AbstractFileConfigurationProvider.access$300(AbstractFileConfigurationProvider.java:38) at org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:202) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439) at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:680) ^C2012-12-03 18:25:17,989 (node-shutdownHook) [INFO - org.apache.flume.node.FlumeNode.stop(FlumeNode.java:67)] Flume node stopping - agent -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-951) Implement a direct S3 Sink for Flume
[ https://issues.apache.org/jira/browse/FLUME-951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland updated FLUME-951: --- Assignee: Brock Noland (was: Michael Katzenellenbogen) Implement a direct S3 Sink for Flume Key: FLUME-951 URL: https://issues.apache.org/jira/browse/FLUME-951 Project: Flume Issue Type: New Feature Affects Versions: v1.0.0 Reporter: Arvind Prabhakar Assignee: Brock Noland Attachments: flume-s3-sink.tar.gz -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [ANNOUNCE] Apache Flume 1.3.0 released
Thank you Brock for driving this release, you've done excellent job as a Release manager! Jarcec On Tue, Dec 04, 2012 at 10:13:58AM -0600, Brock Noland wrote: The Apache Flume team is pleased to announce the release of Flume version 1.3.0. Flume is a distributed, reliable, and available service for efficiently collecting, aggregating, and moving large amounts of log data. This release can be downloaded from the Flume download page at: http://flume.apache.org/download.html The change log and documentation are available on the 1.3.0 release page: http://flume.apache.org/releases/1.3.0.html Your help and feedback is more than welcome. For more information on how to report problems and to get involved, visit the project website at http://flume.apache.org/ The Apache Flume Team signature.asc Description: Digital signature
[jira] [Commented] (FLUME-1295) RollingFileSink need to be able to construct directory path based on escape sequence
[ https://issues.apache.org/jira/browse/FLUME-1295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13509844#comment-13509844 ] Brock Noland commented on FLUME-1295: - Looks like it has to be re-based Personally what I think we should do with this patch is not modify HDFSEventSink but get all the infrastructure ready so we could modify HDFSEventSink once we are sure the new code is working correctly. The reason is HDFSEventSink is used by a ton of people and it's been a source of bugs in the past. RollingFileSink need to be able to construct directory path based on escape sequence Key: FLUME-1295 URL: https://issues.apache.org/jira/browse/FLUME-1295 Project: Flume Issue Type: Bug Components: Sinks+Sources Affects Versions: v1.2.0 Reporter: Bhaskar Marthi Assignee: Ted Malaska Attachments: FLUME-1295-A.patch, FLUME-1295-B.patch Currently HDFSEventSink has an ability to construct the directory path based on escape sequence. It would be beneficial to have similar patch for RollingFileSink for people who just wants to use regular file system. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Review Request: RollingFileSink need to be able to construct directory path based on escape sequence
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/7131/#review14021 --- Echoing my comments from the JIRA. Personally what I think we should do with this patch is not modify HDFSEventSink but get all the infrastructure ready so we could modify HDFSEventSink once we are sure the new code is working correctly. The reason is HDFSEventSink is used by a ton of people and it's been a source of bugs in the past. - Brock Noland On Sept. 17, 2012, 10:34 p.m., Ted Malaska wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/7131/ --- (Updated Sept. 17, 2012, 10:34 p.m.) Review request for Flume. Description --- OK, first note this is still a work in progress, but I'm at the point where I need feedback. Initial goal was to allow RollingFileSink to do Escape Sequence the same way as HDFSEventSink. In order to reuse code, my change attempt to reuse bucketWriter and BucketWriterLinkedHashMap. BucketWriter has been broken up into three files. AbstractBucketWriter : Contains all the rollover and escape sequence logic. BucketWriter: Contains the IO logic to HDFS FileBucketWriter: contains the IO logic to normal files I also added a couple unit tests to RollingFileSink for new types of roll overs and escape sequences. This addresses bug FLUME-1295. https://issues.apache.org/jira/browse/FLUME-1295 Diffs - flume-ng-core/src/main/java/org/apache/flume/sink/AbstractBucketWriter.java PRE-CREATION flume-ng-core/src/main/java/org/apache/flume/sink/BucketWriterLinkedHashMap.java PRE-CREATION flume-ng-core/src/main/java/org/apache/flume/sink/FileBucketWriter.java PRE-CREATION flume-ng-core/src/main/java/org/apache/flume/sink/RollingFileSink.java be640bb flume-ng-core/src/test/java/org/apache/flume/sink/TestRollingFileSink.java 07fa644 flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/BucketWriter.java 6408eb9 flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSEventSink.java 9a76ecb Diff: https://reviews.apache.org/r/7131/diff/ Testing --- Thanks, Ted Malaska
Re: API Docs and PDFs on the website
On Dec 4, 2012, at 8:16 AM, Brock Noland wrote: Hi, I assume the PDFs and API Docs on the release page http://flume.apache.org/releases/1.3.0.html are just manually generated and then committed directly to the prod website? Yes - it looks like you did it correctly. Unfortunately, the maven-pdf-plugin doesn't support RST so you have to do it manually. I noticed the API Javadoc contains a reference to com.cloudera.flume.handlers.thrift. Why is that? Ralph I just want to confirm as I am trying to put together a guide on how to edit the website and what items need to be updated as part of a release. Brock
Re: API Docs and PDFs on the website
Hi, It looks like it's from the legacy thrift source: https://github.com/apache/flume/tree/trunk/flume-ng-legacy-sources/flume-thrift-source/src/main/java Does anything know if there is a valid reason it's not org.apache? I assume it was just missed. Brock On Tue, Dec 4, 2012 at 11:09 AM, Ralph Goers ralph.go...@dslextreme.com wrote: On Dec 4, 2012, at 8:16 AM, Brock Noland wrote: Hi, I assume the PDFs and API Docs on the release page http://flume.apache.org/releases/1.3.0.html are just manually generated and then committed directly to the prod website? Yes - it looks like you did it correctly. Unfortunately, the maven-pdf-plugin doesn't support RST so you have to do it manually. I noticed the API Javadoc contains a reference to com.cloudera.flume.handlers.thrift. Why is that? Ralph I just want to confirm as I am trying to put together a guide on how to edit the website and what items need to be updated as part of a release. Brock -- Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
Re: API Docs and PDFs on the website
That's there for Flume 0.9 backwards compatibility, thus being in legacy sources. It's already marked as @deprecated. Someone who is an expert with Thrift may have more insight into whether we can maintain backwards compat while changing the package/class names, but my understanding is that it was left that way for that reason. On Tue, Dec 4, 2012 at 9:12 AM, Brock Noland br...@cloudera.com wrote: Hi, It looks like it's from the legacy thrift source: https://github.com/apache/flume/tree/trunk/flume-ng-legacy-sources/flume-thrift-source/src/main/java Does anything know if there is a valid reason it's not org.apache? I assume it was just missed. Brock On Tue, Dec 4, 2012 at 11:09 AM, Ralph Goers ralph.go...@dslextreme.com wrote: On Dec 4, 2012, at 8:16 AM, Brock Noland wrote: Hi, I assume the PDFs and API Docs on the release page http://flume.apache.org/releases/1.3.0.html are just manually generated and then committed directly to the prod website? Yes - it looks like you did it correctly. Unfortunately, the maven-pdf-plugin doesn't support RST so you have to do it manually. I noticed the API Javadoc contains a reference to com.cloudera.flume.handlers.thrift. Why is that? Ralph I just want to confirm as I am trying to put together a guide on how to edit the website and what items need to be updated as part of a release. Brock -- Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
[jira] [Updated] (FLUME-1766) AvroSource throws confusing exception when configured without a port
[ https://issues.apache.org/jira/browse/FLUME-1766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Lord updated FLUME-1766: - Attachment: FLUME-1766-1.patch AvroSource throws confusing exception when configured without a port Key: FLUME-1766 URL: https://issues.apache.org/jira/browse/FLUME-1766 Project: Flume Issue Type: Bug Components: Sinks+Sources Affects Versions: v1.3.0 Reporter: Mike Percy Assignee: Jeff Lord Priority: Minor Labels: newbie Fix For: v1.4.0 Attachments: FLUME-1766-1.patch 2012-12-03 18:25:08,210 (conf-file-poller-0) [DEBUG - org.apache.flume.source.DefaultSourceFactory.create(DefaultSourceFactory.java:74)] Creating instance of source src-1, type AVRO 2012-12-03 18:25:08,235 (conf-file-poller-0) [ERROR - org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:204)] Failed to load configuration data. Exception follows. java.lang.NumberFormatException: null at java.lang.Integer.parseInt(Integer.java:417) at java.lang.Integer.parseInt(Integer.java:499) at org.apache.flume.source.AvroSource.configure(AvroSource.java:126) at org.apache.flume.conf.Configurables.configure(Configurables.java:41) at org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.loadSources(PropertiesFileConfigurationProvider.java:323) at org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.load(PropertiesFileConfigurationProvider.java:222) at org.apache.flume.conf.file.AbstractFileConfigurationProvider.doLoad(AbstractFileConfigurationProvider.java:123) at org.apache.flume.conf.file.AbstractFileConfigurationProvider.access$300(AbstractFileConfigurationProvider.java:38) at org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:202) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439) at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:680) ^C2012-12-03 18:25:17,989 (node-shutdownHook) [INFO - org.apache.flume.node.FlumeNode.stop(FlumeNode.java:67)] Flume node stopping - agent -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-1766) AvroSource throws confusing exception when configured without a port
[
https://issues.apache.org/jira/browse/FLUME-1766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13510267#comment-13510267
]
Mike Percy commented on FLUME-1766:
---
Hi Jeff, thanks for this patch!
Since you are essentially defining constants (portKey, bindKey), please declare
them in all-caps, like the following:
{noformat}
private final static String PORT_KEY = port;
private final static String BIND_KEY = bind;
{noformat}
and then use those constants in the rest of the configure() method. For
example, instead of
{noformat}
port = context.getInteger(port);
{noformat}
prefer:
{noformat}
port = context.getInteger(PORT_KEY);
{noformat}
Finally, please make sure your indentation is consistent with the rest of the
file. Flume uses 2 spaces everywhere.
Thanks!
AvroSource throws confusing exception when configured without a port
Key: FLUME-1766
URL: https://issues.apache.org/jira/browse/FLUME-1766
Project: Flume
Issue Type: Bug
Components: Sinks+Sources
Affects Versions: v1.3.0
Reporter: Mike Percy
Assignee: Jeff Lord
Priority: Minor
Labels: newbie
Fix For: v1.4.0
Attachments: FLUME-1766-1.patch
2012-12-03 18:25:08,210 (conf-file-poller-0) [DEBUG -
org.apache.flume.source.DefaultSourceFactory.create(DefaultSourceFactory.java:74)]
Creating instance of source src-1, type AVRO
2012-12-03 18:25:08,235 (conf-file-poller-0) [ERROR -
org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:204)]
Failed to load configuration data. Exception follows.
java.lang.NumberFormatException: null
at java.lang.Integer.parseInt(Integer.java:417)
at java.lang.Integer.parseInt(Integer.java:499)
at org.apache.flume.source.AvroSource.configure(AvroSource.java:126)
at org.apache.flume.conf.Configurables.configure(Configurables.java:41)
at
org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.loadSources(PropertiesFileConfigurationProvider.java:323)
at
org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.load(PropertiesFileConfigurationProvider.java:222)
at
org.apache.flume.conf.file.AbstractFileConfigurationProvider.doLoad(AbstractFileConfigurationProvider.java:123)
at
org.apache.flume.conf.file.AbstractFileConfigurationProvider.access$300(AbstractFileConfigurationProvider.java:38)
at
org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:202)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at
java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)
^C2012-12-03 18:25:17,989 (node-shutdownHook) [INFO -
org.apache.flume.node.FlumeNode.stop(FlumeNode.java:67)] Flume node stopping
- agent
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [ANNOUNCE] Apache Flume 1.3.0 released
Hear, hear! Brock, well done sir, thanks for all your excellent hard work on this release! Regards, Mike On Tue, Dec 4, 2012 at 8:30 AM, Jarek Jarcec Cecho jar...@apache.orgwrote: Thank you Brock for driving this release, you've done excellent job as a Release manager! Jarcec On Tue, Dec 04, 2012 at 10:13:58AM -0600, Brock Noland wrote: The Apache Flume team is pleased to announce the release of Flume version 1.3.0. Flume is a distributed, reliable, and available service for efficiently collecting, aggregating, and moving large amounts of log data. This release can be downloaded from the Flume download page at: http://flume.apache.org/download.html The change log and documentation are available on the 1.3.0 release page: http://flume.apache.org/releases/1.3.0.html Your help and feedback is more than welcome. For more information on how to report problems and to get involved, visit the project website at http://flume.apache.org/ The Apache Flume Team
[jira] [Commented] (FLUME-1766) AvroSource throws confusing exception when configured without a port
[ https://issues.apache.org/jira/browse/FLUME-1766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13510276#comment-13510276 ] Hari Shreedharan commented on FLUME-1766: - Also, currently bind is not a required value (as of v1.3.0). This patch makes it a required value - which we cannot do in a point release. And frankly, I don't think it should be required, it should be optional as it is now. AvroSource throws confusing exception when configured without a port Key: FLUME-1766 URL: https://issues.apache.org/jira/browse/FLUME-1766 Project: Flume Issue Type: Bug Components: Sinks+Sources Affects Versions: v1.3.0 Reporter: Mike Percy Assignee: Jeff Lord Priority: Minor Labels: newbie Fix For: v1.4.0 Attachments: FLUME-1766-1.patch 2012-12-03 18:25:08,210 (conf-file-poller-0) [DEBUG - org.apache.flume.source.DefaultSourceFactory.create(DefaultSourceFactory.java:74)] Creating instance of source src-1, type AVRO 2012-12-03 18:25:08,235 (conf-file-poller-0) [ERROR - org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:204)] Failed to load configuration data. Exception follows. java.lang.NumberFormatException: null at java.lang.Integer.parseInt(Integer.java:417) at java.lang.Integer.parseInt(Integer.java:499) at org.apache.flume.source.AvroSource.configure(AvroSource.java:126) at org.apache.flume.conf.Configurables.configure(Configurables.java:41) at org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.loadSources(PropertiesFileConfigurationProvider.java:323) at org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.load(PropertiesFileConfigurationProvider.java:222) at org.apache.flume.conf.file.AbstractFileConfigurationProvider.doLoad(AbstractFileConfigurationProvider.java:123) at org.apache.flume.conf.file.AbstractFileConfigurationProvider.access$300(AbstractFileConfigurationProvider.java:38) at org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:202) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439) at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:680) ^C2012-12-03 18:25:17,989 (node-shutdownHook) [INFO - org.apache.flume.node.FlumeNode.stop(FlumeNode.java:67)] Flume node stopping - agent -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [ANNOUNCE] Apache Flume 1.3.0 released
Great job Brock! And thank you to everyone who contributed! Cheers, Will On Tue, Dec 4, 2012 at 8:37 PM, Mike Percy mpe...@apache.org wrote: Hear, hear! Brock, well done sir, thanks for all your excellent hard work on this release! Regards, Mike On Tue, Dec 4, 2012 at 8:30 AM, Jarek Jarcec Cecho jar...@apache.org wrote: Thank you Brock for driving this release, you've done excellent job as a Release manager! Jarcec On Tue, Dec 04, 2012 at 10:13:58AM -0600, Brock Noland wrote: The Apache Flume team is pleased to announce the release of Flume version 1.3.0. Flume is a distributed, reliable, and available service for efficiently collecting, aggregating, and moving large amounts of log data. This release can be downloaded from the Flume download page at: http://flume.apache.org/download.html The change log and documentation are available on the 1.3.0 release page: http://flume.apache.org/releases/1.3.0.html Your help and feedback is more than welcome. For more information on how to report problems and to get involved, visit the project website at http://flume.apache.org/ The Apache Flume Team
Re: [ANNOUNCE] Apache Flume 1.3.0 released
Thanks for your hard work Brock! Appreciate your diligence and resolve in getting this through! Regards, Arvind Prabhakar On Tue, Dec 4, 2012 at 9:16 PM, Will McQueen w...@cloudera.com wrote: Great job Brock! And thank you to everyone who contributed! Cheers, Will On Tue, Dec 4, 2012 at 8:37 PM, Mike Percy mpe...@apache.org wrote: Hear, hear! Brock, well done sir, thanks for all your excellent hard work on this release! Regards, Mike On Tue, Dec 4, 2012 at 8:30 AM, Jarek Jarcec Cecho jar...@apache.org wrote: Thank you Brock for driving this release, you've done excellent job as a Release manager! Jarcec On Tue, Dec 04, 2012 at 10:13:58AM -0600, Brock Noland wrote: The Apache Flume team is pleased to announce the release of Flume version 1.3.0. Flume is a distributed, reliable, and available service for efficiently collecting, aggregating, and moving large amounts of log data. This release can be downloaded from the Flume download page at: http://flume.apache.org/download.html The change log and documentation are available on the 1.3.0 release page: http://flume.apache.org/releases/1.3.0.html Your help and feedback is more than welcome. For more information on how to report problems and to get involved, visit the project website at http://flume.apache.org/ The Apache Flume Team
[jira] [Updated] (FLUME-1766) AvroSource throws confusing exception when configured without a port
[
https://issues.apache.org/jira/browse/FLUME-1766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeff Lord updated FLUME-1766:
-
Attachment: FLUME-1766-2.patch
Ok. Feedback has been incorporated.
Regarding optional bind, it looks like this value is also assigned to hostname
and there is a check in LifecycleSupervisor which looks for that.
2012-12-04 22:33:04,356 (lifecycleSupervisor-1-3) [ERROR -
org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:238)]
Unable to start EventDrivenSourceRunner: { source:Avro source avrosource-1: {
bindAddress: null, port: 4141 } } - Exception follows.
java.lang.IllegalArgumentException: hostname can't be null
AvroSource throws confusing exception when configured without a port
Key: FLUME-1766
URL: https://issues.apache.org/jira/browse/FLUME-1766
Project: Flume
Issue Type: Bug
Components: Sinks+Sources
Affects Versions: v1.3.0
Reporter: Mike Percy
Assignee: Jeff Lord
Priority: Minor
Labels: newbie
Fix For: v1.4.0
Attachments: FLUME-1766-1.patch, FLUME-1766-2.patch
2012-12-03 18:25:08,210 (conf-file-poller-0) [DEBUG -
org.apache.flume.source.DefaultSourceFactory.create(DefaultSourceFactory.java:74)]
Creating instance of source src-1, type AVRO
2012-12-03 18:25:08,235 (conf-file-poller-0) [ERROR -
org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:204)]
Failed to load configuration data. Exception follows.
java.lang.NumberFormatException: null
at java.lang.Integer.parseInt(Integer.java:417)
at java.lang.Integer.parseInt(Integer.java:499)
at org.apache.flume.source.AvroSource.configure(AvroSource.java:126)
at org.apache.flume.conf.Configurables.configure(Configurables.java:41)
at
org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.loadSources(PropertiesFileConfigurationProvider.java:323)
at
org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.load(PropertiesFileConfigurationProvider.java:222)
at
org.apache.flume.conf.file.AbstractFileConfigurationProvider.doLoad(AbstractFileConfigurationProvider.java:123)
at
org.apache.flume.conf.file.AbstractFileConfigurationProvider.access$300(AbstractFileConfigurationProvider.java:38)
at
org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:202)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at
java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)
^C2012-12-03 18:25:17,989 (node-shutdownHook) [INFO -
org.apache.flume.node.FlumeNode.stop(FlumeNode.java:67)] Flume node stopping
- agent
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-1766) AvroSource throws confusing exception when configured without a port
[ https://issues.apache.org/jira/browse/FLUME-1766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13510317#comment-13510317 ] Mike Percy commented on FLUME-1766: --- Hari, are you sure? AvroSource throws confusing exception when configured without a port Key: FLUME-1766 URL: https://issues.apache.org/jira/browse/FLUME-1766 Project: Flume Issue Type: Bug Components: Sinks+Sources Affects Versions: v1.3.0 Reporter: Mike Percy Assignee: Jeff Lord Priority: Minor Labels: newbie Fix For: v1.4.0 Attachments: FLUME-1766-1.patch, FLUME-1766-2.patch 2012-12-03 18:25:08,210 (conf-file-poller-0) [DEBUG - org.apache.flume.source.DefaultSourceFactory.create(DefaultSourceFactory.java:74)] Creating instance of source src-1, type AVRO 2012-12-03 18:25:08,235 (conf-file-poller-0) [ERROR - org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:204)] Failed to load configuration data. Exception follows. java.lang.NumberFormatException: null at java.lang.Integer.parseInt(Integer.java:417) at java.lang.Integer.parseInt(Integer.java:499) at org.apache.flume.source.AvroSource.configure(AvroSource.java:126) at org.apache.flume.conf.Configurables.configure(Configurables.java:41) at org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.loadSources(PropertiesFileConfigurationProvider.java:323) at org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.load(PropertiesFileConfigurationProvider.java:222) at org.apache.flume.conf.file.AbstractFileConfigurationProvider.doLoad(AbstractFileConfigurationProvider.java:123) at org.apache.flume.conf.file.AbstractFileConfigurationProvider.access$300(AbstractFileConfigurationProvider.java:38) at org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:202) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439) at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:680) ^C2012-12-03 18:25:17,989 (node-shutdownHook) [INFO - org.apache.flume.node.FlumeNode.stop(FlumeNode.java:67)] Flume node stopping - agent -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-1766) AvroSource throws confusing exception when configured without a port
[ https://issues.apache.org/jira/browse/FLUME-1766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13510329#comment-13510329 ] Hari Shreedharan commented on FLUME-1766: - It seems like we check for hostname being null in the start method. I checked the configure method where the configuration is validated, so that the source can be removed if the hostname is null (the config system will remove the source if it throws an exception in the configure method). So we should move the check for hostname being non-null to the configure method, so the config system can remove the component when the configure method throws. Jeff - it would be nice to add that to this patch as well. Otherwise the lifecycle supervisor will try to start the component(there is some hack I put in for an error state, so it does not throw every 3 seconds - but I'd rather remove the component than use up resources). AvroSource throws confusing exception when configured without a port Key: FLUME-1766 URL: https://issues.apache.org/jira/browse/FLUME-1766 Project: Flume Issue Type: Bug Components: Sinks+Sources Affects Versions: v1.3.0 Reporter: Mike Percy Assignee: Jeff Lord Priority: Minor Labels: newbie Fix For: v1.4.0 Attachments: FLUME-1766-1.patch, FLUME-1766-2.patch 2012-12-03 18:25:08,210 (conf-file-poller-0) [DEBUG - org.apache.flume.source.DefaultSourceFactory.create(DefaultSourceFactory.java:74)] Creating instance of source src-1, type AVRO 2012-12-03 18:25:08,235 (conf-file-poller-0) [ERROR - org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:204)] Failed to load configuration data. Exception follows. java.lang.NumberFormatException: null at java.lang.Integer.parseInt(Integer.java:417) at java.lang.Integer.parseInt(Integer.java:499) at org.apache.flume.source.AvroSource.configure(AvroSource.java:126) at org.apache.flume.conf.Configurables.configure(Configurables.java:41) at org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.loadSources(PropertiesFileConfigurationProvider.java:323) at org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.load(PropertiesFileConfigurationProvider.java:222) at org.apache.flume.conf.file.AbstractFileConfigurationProvider.doLoad(AbstractFileConfigurationProvider.java:123) at org.apache.flume.conf.file.AbstractFileConfigurationProvider.access$300(AbstractFileConfigurationProvider.java:38) at org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:202) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439) at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:680) ^C2012-12-03 18:25:17,989 (node-shutdownHook) [INFO - org.apache.flume.node.FlumeNode.stop(FlumeNode.java:67)] Flume node stopping - agent -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
