[jira] [Updated] (FLUME-1734) Create a Hive Sink based on the new Hive Streaming support
[ https://issues.apache.org/jira/browse/FLUME-1734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roshan Naik updated FLUME-1734: --- Description: Hive 0.13 has introduced Streaming support which is itself transactional in nature and fits well with Flume's transaction model. Short overview of Hive's Streaming support on which this sink is based can be found here: https://issues.apache.org/jira/secure/attachment/12639278/package.html This jira is for creating a sink that would continuously stream data into Hive tables using the above APIs. The primary goal being that the data streamed by the sink should be instantly queryable (using say Hive or Pig) without requiring additional post-processing steps on behalf of the users. Sink should manage the creation of new partitions periodically if needed. was:Create a sink that would stream data into HCatalog partitions. The primary goal being that once the data is loaded into Hadoop, it should be automatically queryable (using say Hive or Pig) without requiring additional post processing steps on behalf of the users. Sink should manage the creation of new partitions and committing them periodically. > Create a Hive Sink based on the new Hive Streaming support > -- > > Key: FLUME-1734 > URL: https://issues.apache.org/jira/browse/FLUME-1734 > Project: Flume > Issue Type: New Feature > Components: Sinks+Sources >Affects Versions: v1.2.0 >Reporter: Roshan Naik >Assignee: Roshan Naik > Labels: features > Attachments: FLUME-1734.draft.1.patch, FLUME-1734.draft.2.patch, > FLUME-1734.v1.patch, FLUME-1734.v2.patch > > > Hive 0.13 has introduced Streaming support which is itself transactional in > nature and fits well with Flume's transaction model. > Short overview of Hive's Streaming support on which this sink is based can > be found here: > https://issues.apache.org/jira/secure/attachment/12639278/package.html > This jira is for creating a sink that would continuously stream data into > Hive tables using the above APIs. The primary goal being that the data > streamed by the sink should be instantly queryable (using say Hive or Pig) > without requiring additional post-processing steps on behalf of the users. > Sink should manage the creation of new partitions periodically if needed. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (FLUME-2393) [User Guide]Incorrect rendering of Twitter firehose Source header, displayed as Twitter 1% firehose Source
[ https://issues.apache.org/jira/browse/FLUME-2393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14031450#comment-14031450 ] Ashish Paliwal commented on FLUME-2393: --- Ahh got it, my mistake > [User Guide]Incorrect rendering of Twitter firehose Source header, displayed > as Twitter 1% firehose Source > -- > > Key: FLUME-2393 > URL: https://issues.apache.org/jira/browse/FLUME-2393 > Project: Flume > Issue Type: Documentation > Components: Docs >Affects Versions: v1.5.0 >Reporter: Ashish Paliwal >Assignee: Ashish Paliwal >Priority: Minor > Attachments: FLUME-2393-0.patch, > Screen-Shot-2014-06-14-at-8.47.34-am.jpg > > > Twitter firehose Source header is rendered incorrectly, has 1% in between and > is displayed as "Twitter 1% firehose Source" -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (FLUME-2393) [User Guide]Incorrect rendering of Twitter firehose Source header, displayed as Twitter 1% firehose Source
[ https://issues.apache.org/jira/browse/FLUME-2393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14031449#comment-14031449 ] Roshan Naik commented on FLUME-2393: Looks fine. It is supposed to be 'Twitter 1%' as AFAIKT from the description. > [User Guide]Incorrect rendering of Twitter firehose Source header, displayed > as Twitter 1% firehose Source > -- > > Key: FLUME-2393 > URL: https://issues.apache.org/jira/browse/FLUME-2393 > Project: Flume > Issue Type: Documentation > Components: Docs >Affects Versions: v1.5.0 >Reporter: Ashish Paliwal >Assignee: Ashish Paliwal >Priority: Minor > Attachments: FLUME-2393-0.patch, > Screen-Shot-2014-06-14-at-8.47.34-am.jpg > > > Twitter firehose Source header is rendered incorrectly, has 1% in between and > is displayed as "Twitter 1% firehose Source" -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Review Request 18544: Hive Streaming sink
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18544/ --- (Updated June 14, 2014, 4:19 a.m.) Review request for Flume. Changes --- Uploading new patch v2 : addressed loose ends in unit tests fixed issues related to recovering from intermittent HDFS and Hive failures. Bugs: FLUME-1734 https://issues.apache.org/jira/browse/FLUME-1734 Repository: flume-git Description --- Hive streaming sink. Diffs (updated) - bin/flume-ng e09e26b conf/log4j.properties 3918511 flume-ng-configuration/src/main/java/org/apache/flume/conf/sink/SinkConfiguration.java ac11558 flume-ng-configuration/src/main/java/org/apache/flume/conf/sink/SinkType.java 0a1cd7a flume-ng-dist/pom.xml 690fec5 flume-ng-doc/sphinx/FlumeUserGuide.rst b2058f5 flume-ng-legacy-sources/flume-thrift-source/src/main/java/com/cloudera/flume/handlers/thrift/ThriftFlumeEventServer.java ff32c45 flume-ng-legacy-sources/flume-thrift-source/src/test/java/org/apache/flume/source/thriftLegacy/TestThriftLegacySource.java 8e08f22 flume-ng-sdk/src/main/java/org/apache/flume/thrift/ThriftSourceProtocol.java 7f966b0 flume-ng-sinks/flume-hive-sink/pom.xml PRE-CREATION flume-ng-sinks/flume-hive-sink/src/main/java/org/apache/flume/sink/hive/HiveDelimitedTextSerializer.java PRE-CREATION flume-ng-sinks/flume-hive-sink/src/main/java/org/apache/flume/sink/hive/HiveEventSerializer.java PRE-CREATION flume-ng-sinks/flume-hive-sink/src/main/java/org/apache/flume/sink/hive/HiveJsonSerializer.java PRE-CREATION flume-ng-sinks/flume-hive-sink/src/main/java/org/apache/flume/sink/hive/HiveSink.java PRE-CREATION flume-ng-sinks/flume-hive-sink/src/main/java/org/apache/flume/sink/hive/HiveWriter.java PRE-CREATION flume-ng-sinks/flume-hive-sink/src/test/java/org/apache/flume/sink/hive/TestHiveSink.java PRE-CREATION flume-ng-sinks/flume-hive-sink/src/test/java/org/apache/flume/sink/hive/TestHiveWriter.java PRE-CREATION flume-ng-sinks/flume-hive-sink/src/test/java/org/apache/flume/sink/hive/TestUtil.java PRE-CREATION flume-ng-sinks/flume-hive-sink/src/test/resources/log4j.properties PRE-CREATION flume-ng-sinks/pom.xml 56c4dee flume-ng-sources/flume-scribe-source/src/main/java/org/apache/flume/source/scribe/Scribe.java eba8d2e pom.xml 4dae2c9 Diff: https://reviews.apache.org/r/18544/diff/ Testing --- This version lacks unit tests. Thanks, Roshan Naik
[jira] [Updated] (FLUME-1734) Create a Hive Sink based on the new Hive Streaming support
[ https://issues.apache.org/jira/browse/FLUME-1734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roshan Naik updated FLUME-1734: --- Attachment: FLUME-1734.v2.patch Uploading new patch v2 : - fleshed out the tests some more - fixed issues related to recovering from intermittent HDFS and Hive failures. > Create a Hive Sink based on the new Hive Streaming support > -- > > Key: FLUME-1734 > URL: https://issues.apache.org/jira/browse/FLUME-1734 > Project: Flume > Issue Type: New Feature > Components: Sinks+Sources >Affects Versions: v1.2.0 >Reporter: Roshan Naik >Assignee: Roshan Naik > Labels: features > Attachments: FLUME-1734.draft.1.patch, FLUME-1734.draft.2.patch, > FLUME-1734.v1.patch, FLUME-1734.v2.patch > > > Create a sink that would stream data into HCatalog partitions. The primary > goal being that once the data is loaded into Hadoop, it should be > automatically queryable (using say Hive or Pig) without requiring additional > post processing steps on behalf of the users. Sink should manage the creation > of new partitions and committing them periodically. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (FLUME-2402) Warning seen when overflow is disabled for Spillable Channel
[ https://issues.apache.org/jira/browse/FLUME-2402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roshan Naik updated FLUME-2402: --- Attachment: FLUME-2402.patch Uploading patch > Warning seen when overflow is disabled for Spillable Channel > > > Key: FLUME-2402 > URL: https://issues.apache.org/jira/browse/FLUME-2402 > Project: Flume > Issue Type: Bug > Components: Channel >Affects Versions: v1.5.0 >Reporter: Roshan Naik > Labels: SpillableChannel > Attachments: FLUME-2402.patch > > > When overflow is disabled by setting overflowCapacity=0 the below warning is > reported (by underlying File channel) > 2014-06-13 20:56:56,908i Invalid capacity specified, initializing channel to > default capacity of 100 > File channel's capacity should be overridden with some small number in such a > config. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (FLUME-2402) Warning seen when overflow is disabled for Spillable Channel
Roshan Naik created FLUME-2402: -- Summary: Warning seen when overflow is disabled for Spillable Channel Key: FLUME-2402 URL: https://issues.apache.org/jira/browse/FLUME-2402 Project: Flume Issue Type: Bug Components: Channel Affects Versions: v1.5.0 Reporter: Roshan Naik When overflow is disabled by setting overflowCapacity=0 the below warning is reported (by underlying File channel) 2014-06-13 20:56:56,908i Invalid capacity specified, initializing channel to default capacity of 100 File channel's capacity should be overridden with some small number in such a config. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (FLUME-2024) Add ExecSource flush timeout to Flume User Guide
[ https://issues.apache.org/jira/browse/FLUME-2024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14031439#comment-14031439 ] Hudson commented on FLUME-2024: --- SUCCESS: Integrated in flume-trunk #641 (See [https://builds.apache.org/job/flume-trunk/641/]) FLUME-2024. Add ExecSource flush timeout to Flume User Guide (roshan: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=b74e35aeb6aea87a29f062f55c79a499c6497abe) * flume-ng-core/src/main/java/org/apache/flume/source/ExecSource.java * flume-ng-doc/sphinx/FlumeUserGuide.rst > Add ExecSource flush timeout to Flume User Guide > > > Key: FLUME-2024 > URL: https://issues.apache.org/jira/browse/FLUME-2024 > Project: Flume > Issue Type: Bug >Affects Versions: v1.4.0 >Reporter: Hari Shreedharan >Assignee: Ashish Paliwal > Fix For: v1.6.0 > > Attachments: FLUME-2024-0.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (FLUME-2393) [User Guide]Incorrect rendering of Twitter firehose Source header, displayed as Twitter 1% firehose Source
[ https://issues.apache.org/jira/browse/FLUME-2393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal updated FLUME-2393: -- Attachment: Screen-Shot-2014-06-14-at-8.47.34-am.jpg Screenshot attached, see the title, made a circle to highlight 1% > [User Guide]Incorrect rendering of Twitter firehose Source header, displayed > as Twitter 1% firehose Source > -- > > Key: FLUME-2393 > URL: https://issues.apache.org/jira/browse/FLUME-2393 > Project: Flume > Issue Type: Documentation > Components: Docs >Affects Versions: v1.5.0 >Reporter: Ashish Paliwal >Assignee: Ashish Paliwal >Priority: Minor > Attachments: FLUME-2393-0.patch, > Screen-Shot-2014-06-14-at-8.47.34-am.jpg > > > Twitter firehose Source header is rendered incorrectly, has 1% in between and > is displayed as "Twitter 1% firehose Source" -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: [VOTE] Apache Flume 1.5.0.1 RC1
Ah yes.. now i recall. Thanks. +1 -roshan On Fri, Jun 13, 2014 at 6:30 PM, Hari Shreedharan wrote: > Did you compile hbase 94 against Hadoop 2 before running the build? Hbase > did not release Hadoop 2 compatible artifacts. > > On Friday, June 13, 2014, Roshan Naik wrote: > > > Do we plan to support -Dhadoop.profile=2 ? it appears to be failing on > > asynchbase tests: > > > > # mvn -Dhadoop.profile=2 > > > > -roshan > > > > > > > > *---* > > * T E S T S* > > *---* > > *Running org.apache.flume.sink.hbase.TestAsyncHBaseSink* > > *Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.549 > sec > > <<< FAILURE!* > > *org.apache.flume.sink.hbase.TestAsyncHBaseSink Time elapsed: 548 sec > <<< > > ERROR!* > > *java.lang.NoSuchMethodError: > > > > > org.apache.hadoop.hdfs.MiniDFSCluster.(ILorg/apache/hadoop/conf/Configuration;IZZZLorg/apache/hadoop/hdfs/server/common/HdfsConstants$StartupOption;[Ljava/lang/String;[Ljava/lang/String;[J)V* > > * at > > > > > org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSCluster(HBaseTestingUtility.java:433)* > > * at > > > > > org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:601)* > > * at > > > > > org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:557)* > > * at > > > > > org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:526)* > > * at > > > > > org.apache.flume.sink.hbase.TestAsyncHBaseSink.setUp(TestAsyncHBaseSink.java:69)* > > * at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)* > > * at > > > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)* > > * at > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)* > > * at java.lang.reflect.Method.invoke(Method.java:597)* > > * at > > > > > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)* > > * at > > > > > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)* > > * at > > > > > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)* > > * at > > > > > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27)* > > * at > > > > > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)* > > * at org.junit.runners.ParentRunner.run(ParentRunner.java:300)* > > * at > > > > > org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)* > > * at > > > > > org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)* > > * at > > > > > org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)* > > * at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)* > > * at > > > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)* > > * at > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)* > > * at java.lang.reflect.Method.invoke(Method.java:597)* > > * at > > > > > org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)* > > * at > > > > > org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)* > > * at > > > > > org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)* > > * at > > > > > org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)* > > * at > > org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)* > > > > *Running org.apache.flume.sink.hbase.TestHBaseSink* > > *Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.557 > sec > > <<< FAILURE!* > > *org.apache.flume.sink.hbase.TestHBaseSink Time elapsed: 557 sec <<< > > ERROR!* > > *java.lang.NoSuchMethodError: > > > > > org.apache.hadoop.hdfs.MiniDFSCluster.(ILorg/apache/hadoop/conf/Configuration;IZZZLorg/apache/hadoop/hdfs/server/common/HdfsConstants$StartupOption;[Ljava/lang/String;[Ljava/lang/String;[J)V* > > * at > > > > > org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSCluster(HBaseTestingUtility.java:433)* > > * at > > > > > org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:601)* > > * at > > > > > org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:557)* > > * at > > > > > org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:526)* > > * at > > > org.apache.flume.sink.hbase.TestHBaseSink.setUpOnce(TestHBaseSink.java:81)* > > * at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)* > > * at > > > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)* > > * at > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)* >
[jira] [Updated] (FLUME-2024) Add ExecSource flush timeout to Flume User Guide
[ https://issues.apache.org/jira/browse/FLUME-2024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roshan Naik updated FLUME-2024: --- Fix Version/s: v1.6.0 > Add ExecSource flush timeout to Flume User Guide > > > Key: FLUME-2024 > URL: https://issues.apache.org/jira/browse/FLUME-2024 > Project: Flume > Issue Type: Bug >Affects Versions: v1.4.0 >Reporter: Hari Shreedharan >Assignee: Ashish Paliwal > Fix For: v1.6.0 > > Attachments: FLUME-2024-0.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (FLUME-2024) Add ExecSource flush timeout to Flume User Guide
[ https://issues.apache.org/jira/browse/FLUME-2024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14031422#comment-14031422 ] ASF subversion and git services commented on FLUME-2024: Commit b74e35aeb6aea87a29f062f55c79a499c6497abe in flume's branch refs/heads/trunk from [~roshan_naik] [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=b74e35a ] FLUME-2024. Add ExecSource flush timeout to Flume User Guide (Ashish Paliwal via Roshan Naik) > Add ExecSource flush timeout to Flume User Guide > > > Key: FLUME-2024 > URL: https://issues.apache.org/jira/browse/FLUME-2024 > Project: Flume > Issue Type: Bug >Affects Versions: v1.4.0 >Reporter: Hari Shreedharan >Assignee: Ashish Paliwal > Attachments: FLUME-2024-0.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (FLUME-2024) Add ExecSource flush timeout to Flume User Guide
[ https://issues.apache.org/jira/browse/FLUME-2024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14031421#comment-14031421 ] ASF subversion and git services commented on FLUME-2024: Commit a7f0306919c1c7a80c7e0b586a93840cf9c0ec8e in flume's branch refs/heads/flume-1.6 from [~roshan_naik] [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=a7f0306 ] FLUME-2024. Add ExecSource flush timeout to Flume User Guide (Ashish Paliwal via Roshan Naik) > Add ExecSource flush timeout to Flume User Guide > > > Key: FLUME-2024 > URL: https://issues.apache.org/jira/browse/FLUME-2024 > Project: Flume > Issue Type: Bug >Affects Versions: v1.4.0 >Reporter: Hari Shreedharan >Assignee: Ashish Paliwal > Attachments: FLUME-2024-0.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (FLUME-2024) Add ExecSource flush timeout to Flume User Guide
[ https://issues.apache.org/jira/browse/FLUME-2024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14031410#comment-14031410 ] Roshan Naik commented on FLUME-2024: +1 > Add ExecSource flush timeout to Flume User Guide > > > Key: FLUME-2024 > URL: https://issues.apache.org/jira/browse/FLUME-2024 > Project: Flume > Issue Type: Bug >Affects Versions: v1.4.0 >Reporter: Hari Shreedharan >Assignee: Ashish Paliwal > Attachments: FLUME-2024-0.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (FLUME-2393) [User Guide]Incorrect rendering of Twitter firehose Source header, displayed as Twitter 1% firehose Source
[ https://issues.apache.org/jira/browse/FLUME-2393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14031397#comment-14031397 ] Roshan Naik commented on FLUME-2393: Does not look like there is a rendering issue AFAIKT from http://flume.apache.org/FlumeUserGuide.html > [User Guide]Incorrect rendering of Twitter firehose Source header, displayed > as Twitter 1% firehose Source > -- > > Key: FLUME-2393 > URL: https://issues.apache.org/jira/browse/FLUME-2393 > Project: Flume > Issue Type: Documentation > Components: Docs >Affects Versions: v1.5.0 >Reporter: Ashish Paliwal >Assignee: Ashish Paliwal >Priority: Minor > Attachments: FLUME-2393-0.patch > > > Twitter firehose Source header is rendered incorrectly, has 1% in between and > is displayed as "Twitter 1% firehose Source" -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Comment Edited] (FLUME-2197) Memory Channel has GC issues
[ https://issues.apache.org/jira/browse/FLUME-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14031389#comment-14031389 ] Roshan Naik edited comment on FLUME-2197 at 6/14/14 1:41 AM: - During the first 6 mins or so... the total size of allocated objects is still below the max heap size. Once it reaches the max heap size, the allocation graph flattens. You can see the Green portion does not bump up any more. In this state Java is basically doing a minimal GC (tiny peaks) and not a deep gc. So things stay kind of flat. In the flat section after 6mins, you can see more of the tiny peaks (small gc cycles) per unit time in the mem ch graph. was (Author: roshan_naik): During the first 6 mins or so... the total size of allocated objects is still below the max heap size. Once it reaches the max heap size, the allocation graph flattens. You can see the Green portion does bump up any more. In this state Java is basically doing a minimal GC (tiny peaks) and not a deep gc. So things stay kind of flat. In the flat section after 6mins, you can see more of the tiny peaks (small gc cycles) per unit time in the mem ch graph. > Memory Channel has GC issues > > > Key: FLUME-2197 > URL: https://issues.apache.org/jira/browse/FLUME-2197 > Project: Flume > Issue Type: Bug >Reporter: Hari Shreedharan >Assignee: Roshan Naik > Attachments: mem ch - mem alloc.png, spill ch - mem alloc.png > > > Due to the fact that we use a LinkedBlockingDeque as the backing queue for > the MemoryChannel, we end up hitting GC issues more often than we should. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Comment Edited] (FLUME-2197) Memory Channel has GC issues
[ https://issues.apache.org/jira/browse/FLUME-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14031389#comment-14031389 ] Roshan Naik edited comment on FLUME-2197 at 6/14/14 1:38 AM: - During the first 6 mins or so... the total size of allocated objects is still below the max heap size. Once it reaches the max heap size, the allocation graph flattens. You can see the Green portion does bump up any more. In this state Java is basically doing a minimal GC (tiny peaks) and not a deep gc. So things stay kind of flat. In the flat section after 6mins, you can see more of the tiny peaks (small gc cycles) per unit time in the mem ch graph. was (Author: roshan_naik): During the first 6 mins or so... the total size of allocated objects is still well below the max heap size. Once it reaches the max heap size, the allocation graph flattens. You can see the Green portion does bump up any more. In this state Java is basically doing a minimal GC (tiny peaks) and not a deep gc. So things stay kind of flat. In the flat section after 6mins, you can see more of the tiny peaks (small gc cycles) per unit time in the mem ch graph. > Memory Channel has GC issues > > > Key: FLUME-2197 > URL: https://issues.apache.org/jira/browse/FLUME-2197 > Project: Flume > Issue Type: Bug >Reporter: Hari Shreedharan >Assignee: Roshan Naik > Attachments: mem ch - mem alloc.png, spill ch - mem alloc.png > > > Due to the fact that we use a LinkedBlockingDeque as the backing queue for > the MemoryChannel, we end up hitting GC issues more often than we should. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (FLUME-2197) Memory Channel has GC issues
[ https://issues.apache.org/jira/browse/FLUME-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14031389#comment-14031389 ] Roshan Naik commented on FLUME-2197: During the first 6 mins or so... the total size of allocated objects is still well below the max heap size. Once it reaches the max heap size, the allocation graph flattens. You can see the Green portion does bump up any more. In this state Java is basically doing a minimal GC (tiny peaks) and not a deep gc. So things stay kind of flat. In the flat section after 6mins, you can see more of the tiny peaks (small gc cycles) per unit time in the mem ch graph. > Memory Channel has GC issues > > > Key: FLUME-2197 > URL: https://issues.apache.org/jira/browse/FLUME-2197 > Project: Flume > Issue Type: Bug >Reporter: Hari Shreedharan >Assignee: Roshan Naik > Attachments: mem ch - mem alloc.png, spill ch - mem alloc.png > > > Due to the fact that we use a LinkedBlockingDeque as the backing queue for > the MemoryChannel, we end up hitting GC issues more often than we should. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: [VOTE] Apache Flume 1.5.0.1 RC1
Did you compile hbase 94 against Hadoop 2 before running the build? Hbase did not release Hadoop 2 compatible artifacts. On Friday, June 13, 2014, Roshan Naik wrote: > Do we plan to support -Dhadoop.profile=2 ? it appears to be failing on > asynchbase tests: > > # mvn -Dhadoop.profile=2 > > -roshan > > > > *---* > * T E S T S* > *---* > *Running org.apache.flume.sink.hbase.TestAsyncHBaseSink* > *Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.549 sec > <<< FAILURE!* > *org.apache.flume.sink.hbase.TestAsyncHBaseSink Time elapsed: 548 sec <<< > ERROR!* > *java.lang.NoSuchMethodError: > > org.apache.hadoop.hdfs.MiniDFSCluster.(ILorg/apache/hadoop/conf/Configuration;IZZZLorg/apache/hadoop/hdfs/server/common/HdfsConstants$StartupOption;[Ljava/lang/String;[Ljava/lang/String;[J)V* > * at > > org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSCluster(HBaseTestingUtility.java:433)* > * at > > org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:601)* > * at > > org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:557)* > * at > > org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:526)* > * at > > org.apache.flume.sink.hbase.TestAsyncHBaseSink.setUp(TestAsyncHBaseSink.java:69)* > * at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)* > * at > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)* > * at > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)* > * at java.lang.reflect.Method.invoke(Method.java:597)* > * at > > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)* > * at > > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)* > * at > > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)* > * at > > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27)* > * at > > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)* > * at org.junit.runners.ParentRunner.run(ParentRunner.java:300)* > * at > > org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)* > * at > > org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)* > * at > > org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)* > * at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)* > * at > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)* > * at > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)* > * at java.lang.reflect.Method.invoke(Method.java:597)* > * at > > org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)* > * at > > org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)* > * at > > org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)* > * at > > org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)* > * at > org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)* > > *Running org.apache.flume.sink.hbase.TestHBaseSink* > *Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.557 sec > <<< FAILURE!* > *org.apache.flume.sink.hbase.TestHBaseSink Time elapsed: 557 sec <<< > ERROR!* > *java.lang.NoSuchMethodError: > > org.apache.hadoop.hdfs.MiniDFSCluster.(ILorg/apache/hadoop/conf/Configuration;IZZZLorg/apache/hadoop/hdfs/server/common/HdfsConstants$StartupOption;[Ljava/lang/String;[Ljava/lang/String;[J)V* > * at > > org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSCluster(HBaseTestingUtility.java:433)* > * at > > org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:601)* > * at > > org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:557)* > * at > > org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:526)* > * at > org.apache.flume.sink.hbase.TestHBaseSink.setUpOnce(TestHBaseSink.java:81)* > * at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)* > * at > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)* > * at > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)* > * at java.lang.reflect.Method.invoke(Method.java:597)* > * at > > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)* > * at > > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)* > * at > > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)* > * at > > org.junit.internal.runners.statements.RunBefores.evaluate(RunBe
[jira] [Commented] (FLUME-2197) Memory Channel has GC issues
[ https://issues.apache.org/jira/browse/FLUME-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14031381#comment-14031381 ] Otis Gospodnetic commented on FLUME-2197: - Thanks for the graphs, Roshan. bq. Spill Channel contains a modified implementation of mem channel and addresses this GC issue and some other locking issues. I don't get why you said this when the graphs for both mem and spillable channel look very similar and show the same "flat after ~6 minutrs" pattern? > Memory Channel has GC issues > > > Key: FLUME-2197 > URL: https://issues.apache.org/jira/browse/FLUME-2197 > Project: Flume > Issue Type: Bug >Reporter: Hari Shreedharan >Assignee: Roshan Naik > Attachments: mem ch - mem alloc.png, spill ch - mem alloc.png > > > Due to the fact that we use a LinkedBlockingDeque as the backing queue for > the MemoryChannel, we end up hitting GC issues more often than we should. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (FLUME-2197) Memory Channel has GC issues
[ https://issues.apache.org/jira/browse/FLUME-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roshan Naik updated FLUME-2197: --- Attachment: spill ch - mem alloc.png mem ch - mem alloc.png Hard to quantify the impact verbally. Attached images show GC activity of mem channel and Spillable Channel (with overflow disabled). Spill Channel contains a modified implementation of mem channel and addresses this GC issue and some other locking issues. In boths runs, java was setup to have a minHeap & maxHeap at 3GB and 6GB respectively. You can see the allocation graphs inching higher till max heap size is reached (after 6 mins) and then staying flat. Answer to your question lies in the number of peaks in each graph. Each peak indicates the start of a GC cycle. > Memory Channel has GC issues > > > Key: FLUME-2197 > URL: https://issues.apache.org/jira/browse/FLUME-2197 > Project: Flume > Issue Type: Bug >Reporter: Hari Shreedharan >Assignee: Roshan Naik > Attachments: mem ch - mem alloc.png, spill ch - mem alloc.png > > > Due to the fact that we use a LinkedBlockingDeque as the backing queue for > the MemoryChannel, we end up hitting GC issues more often than we should. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (FLUME-2365) Please create a DOAP file for your TLP
[ https://issues.apache.org/jira/browse/FLUME-2365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14031372#comment-14031372 ] Hudson commented on FLUME-2365: --- SUCCESS: Integrated in flume-trunk #640 (See [https://builds.apache.org/job/flume-trunk/640/]) FLUME-2365. Add DOAP file for Flume. (harishreedharan: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=f7e240ff3e376bac2dc0fb91e7c7ae32a933f899) * doap_Flume.rdf > Please create a DOAP file for your TLP > -- > > Key: FLUME-2365 > URL: https://issues.apache.org/jira/browse/FLUME-2365 > Project: Flume > Issue Type: Task >Reporter: Sebb >Assignee: Ashish Paliwal > Attachments: flume.rdf > > > As per my recent e-mail to your dev list, please can you set up a DOAP for > your project and get it added to files.xml? > Please see http://projects.apache.org/create.html > Once you have created the DOAP and committed it to your source code > repository, please submit it for inclusion in the Apache projects listing as > per: > http://projects.apache.org/create.html#submit > Remember, if you ever move or rename the doap file in future, please > ensure that files.xml is updated to point to the new location. > Thanks! -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (FLUME-2365) Please create a DOAP file for your TLP
[ https://issues.apache.org/jira/browse/FLUME-2365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Shreedharan updated FLUME-2365: Assignee: Ashish Paliwal > Please create a DOAP file for your TLP > -- > > Key: FLUME-2365 > URL: https://issues.apache.org/jira/browse/FLUME-2365 > Project: Flume > Issue Type: Task >Reporter: Sebb >Assignee: Ashish Paliwal > Attachments: flume.rdf > > > As per my recent e-mail to your dev list, please can you set up a DOAP for > your project and get it added to files.xml? > Please see http://projects.apache.org/create.html > Once you have created the DOAP and committed it to your source code > repository, please submit it for inclusion in the Apache projects listing as > per: > http://projects.apache.org/create.html#submit > Remember, if you ever move or rename the doap file in future, please > ensure that files.xml is updated to point to the new location. > Thanks! -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (FLUME-2365) Please create a DOAP file for your TLP
[ https://issues.apache.org/jira/browse/FLUME-2365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14031329#comment-14031329 ] Hari Shreedharan commented on FLUME-2365: - I committed the DOAP file after making some minor changes. [~aprabhakar] - Could you please make the change to the sites.xml page as described here:http://projects.apache.org/create.html (Submitting your file section) > Please create a DOAP file for your TLP > -- > > Key: FLUME-2365 > URL: https://issues.apache.org/jira/browse/FLUME-2365 > Project: Flume > Issue Type: Task >Reporter: Sebb > Attachments: flume.rdf > > > As per my recent e-mail to your dev list, please can you set up a DOAP for > your project and get it added to files.xml? > Please see http://projects.apache.org/create.html > Once you have created the DOAP and committed it to your source code > repository, please submit it for inclusion in the Apache projects listing as > per: > http://projects.apache.org/create.html#submit > Remember, if you ever move or rename the doap file in future, please > ensure that files.xml is updated to point to the new location. > Thanks! -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (FLUME-2365) Please create a DOAP file for your TLP
[ https://issues.apache.org/jira/browse/FLUME-2365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14031326#comment-14031326 ] ASF subversion and git services commented on FLUME-2365: Commit b0824298560faba2e15199c20d39bf4f9d64c2b4 in flume's branch refs/heads/flume-1.6 from [~hshreedharan] [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=b082429 ] FLUME-2365. Add DOAP file for Flume. (Ashish Paliwal via Hari Shreedharan) > Please create a DOAP file for your TLP > -- > > Key: FLUME-2365 > URL: https://issues.apache.org/jira/browse/FLUME-2365 > Project: Flume > Issue Type: Task >Reporter: Sebb > Attachments: flume.rdf > > > As per my recent e-mail to your dev list, please can you set up a DOAP for > your project and get it added to files.xml? > Please see http://projects.apache.org/create.html > Once you have created the DOAP and committed it to your source code > repository, please submit it for inclusion in the Apache projects listing as > per: > http://projects.apache.org/create.html#submit > Remember, if you ever move or rename the doap file in future, please > ensure that files.xml is updated to point to the new location. > Thanks! -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (FLUME-2365) Please create a DOAP file for your TLP
[ https://issues.apache.org/jira/browse/FLUME-2365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14031325#comment-14031325 ] ASF subversion and git services commented on FLUME-2365: Commit f7e240ff3e376bac2dc0fb91e7c7ae32a933f899 in flume's branch refs/heads/trunk from [~hshreedharan] [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=f7e240f ] FLUME-2365. Add DOAP file for Flume. (Ashish Paliwal via Hari Shreedharan) > Please create a DOAP file for your TLP > -- > > Key: FLUME-2365 > URL: https://issues.apache.org/jira/browse/FLUME-2365 > Project: Flume > Issue Type: Task >Reporter: Sebb > Attachments: flume.rdf > > > As per my recent e-mail to your dev list, please can you set up a DOAP for > your project and get it added to files.xml? > Please see http://projects.apache.org/create.html > Once you have created the DOAP and committed it to your source code > repository, please submit it for inclusion in the Apache projects listing as > per: > http://projects.apache.org/create.html#submit > Remember, if you ever move or rename the doap file in future, please > ensure that files.xml is updated to point to the new location. > Thanks! -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: [VOTE] Apache Flume 1.5.0.1 RC1
Do we plan to support -Dhadoop.profile=2 ? it appears to be failing on asynchbase tests: # mvn -Dhadoop.profile=2 -roshan *---* * T E S T S* *---* *Running org.apache.flume.sink.hbase.TestAsyncHBaseSink* *Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.549 sec <<< FAILURE!* *org.apache.flume.sink.hbase.TestAsyncHBaseSink Time elapsed: 548 sec <<< ERROR!* *java.lang.NoSuchMethodError: org.apache.hadoop.hdfs.MiniDFSCluster.(ILorg/apache/hadoop/conf/Configuration;IZZZLorg/apache/hadoop/hdfs/server/common/HdfsConstants$StartupOption;[Ljava/lang/String;[Ljava/lang/String;[J)V* * at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSCluster(HBaseTestingUtility.java:433)* * at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:601)* * at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:557)* * at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:526)* * at org.apache.flume.sink.hbase.TestAsyncHBaseSink.setUp(TestAsyncHBaseSink.java:69)* * at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)* * at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)* * at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)* * at java.lang.reflect.Method.invoke(Method.java:597)* * at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)* * at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)* * at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)* * at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27)* * at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)* * at org.junit.runners.ParentRunner.run(ParentRunner.java:300)* * at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)* * at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)* * at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)* * at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)* * at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)* * at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)* * at java.lang.reflect.Method.invoke(Method.java:597)* * at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)* * at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)* * at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)* * at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)* * at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)* *Running org.apache.flume.sink.hbase.TestHBaseSink* *Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.557 sec <<< FAILURE!* *org.apache.flume.sink.hbase.TestHBaseSink Time elapsed: 557 sec <<< ERROR!* *java.lang.NoSuchMethodError: org.apache.hadoop.hdfs.MiniDFSCluster.(ILorg/apache/hadoop/conf/Configuration;IZZZLorg/apache/hadoop/hdfs/server/common/HdfsConstants$StartupOption;[Ljava/lang/String;[Ljava/lang/String;[J)V* * at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSCluster(HBaseTestingUtility.java:433)* * at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:601)* * at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:557)* * at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:526)* * at org.apache.flume.sink.hbase.TestHBaseSink.setUpOnce(TestHBaseSink.java:81)* * at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)* * at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)* * at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)* * at java.lang.reflect.Method.invoke(Method.java:597)* * at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)* * at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)* * at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)* * at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27)* * at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)* * at org.junit.runners.ParentRunner.run(ParentRunner.java:300)* * at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)* * at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)* * at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)*
Re: [VOTE] Apache Flume 1.5.0.1 RC1
Thanks Cos! Thanks, Hari On Thursday, June 12, 2014 at 11:45 PM, Konstantin Boudnik wrote: > I will give it a sping in Bigtop tomorrow - sorry for the delay: been a crazy > week ;( > > Thanks guys! > Cos > > On Thu, Jun 12, 2014 at 10:36PM, Arvind Prabhakar wrote: > > +1 > > > > Thanks for shepherding this Hari. > > > > Regards, > > Arvind Prabhakar > > > > > > On Tue, Jun 10, 2014 at 3:40 PM, Hari Shreedharan > > mailto:hshreedha...@cloudera.com) > > > wrote: > > > > > > > This is a vote for the next release of Apache Flume, version 1.5.0.1. We > > > are voting on release candidate RC1. > > > > > > It fixes the following issues: > > > http://s.apache.org/v7X > > > > > > *** Please cast your vote within the next 72 hours *** > > > > > > The tarball (*.tar.gz), signature (*.asc), and checksums (*.md5, *.sha1) > > > for the source and binary artifacts can be found here: > > > https://people.apache.org/~hshreedharan/apache-flume-1.5.0.1-rc1/ > > > > > > Maven staging repo: > > > https://repository.apache.org/content/repositories/orgapacheflume-1004/ > > > > > > The tag to be voted on: > > > > > > > > > https://git-wip-us.apache.org/repos/asf?p=flume.git;a=commit;h=ceda6aa1126a01370641caf729d8b1dd6d80aa61 > > > > > > Flume's KEYS file containing PGP keys we use to sign the release: > > > http://www.apache.org/dist/flume/KEYS > > > > > > > > > Thanks, > > > Hari > > > > > > > > > >
Re: [VOTE] Apache Flume 1.5.0.1 RC1
Roshan, No. The released artifacts are going to be on hadoop-1 itself. Until we can come up with a naming scheme and the related build changes so that the artifacts for hadoop-1 and 2 can exist simultaneously - we probably should stick to just releasing the 1.0 compatible artifacts. Thanks, Hari On Thursday, June 12, 2014 at 8:04 PM, Roshan Naik wrote: > Will the released binaries be built with the new -Dhadoop.profile=hbase-98 > option ? > > > On Wed, Jun 11, 2014 at 8:54 AM, Ashish (mailto:paliwalash...@gmail.com)> wrote: > > > +1 > > > > Build works good and all tests pass > > > > > > On Wed, Jun 11, 2014 at 4:10 AM, Hari Shreedharan < > > hshreedha...@cloudera.com (mailto:hshreedha...@cloudera.com) > > > wrote: > > > > > > > This is a vote for the next release of Apache Flume, version 1.5.0.1. We > > > are voting on release candidate RC1. > > > > > > It fixes the following issues: > > > http://s.apache.org/v7X > > > > > > *** Please cast your vote within the next 72 hours *** > > > > > > The tarball (*.tar.gz), signature (*.asc), and checksums (*.md5, *.sha1) > > > for the source and binary artifacts can be found here: > > > https://people.apache.org/~hshreedharan/apache-flume-1.5.0.1-rc1/ > > > > > > Maven staging repo: > > https://repository.apache.org/content/repositories/orgapacheflume-1004/ > > > > > > The tag to be voted on: > > https://git-wip-us.apache.org/repos/asf?p=flume.git;a=commit;h=ceda6aa1126a01370641caf729d8b1dd6d80aa61 > > > > > > Flume's KEYS file containing PGP keys we use to sign the release: > > > http://www.apache.org/dist/flume/KEYS > > > > > > > > > Thanks, > > > Hari > > > > > > > > > > > > > -- > > thanks > > ashish > > > > Blog: http://www.ashishpaliwal.com/blog > > My Photo Galleries: http://www.pbase.com/ashishpaliwal > > > > > -- > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity to > which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You. > >
ApacheCon CFP closes June 25
Dear Flume enthusiast, As you may be aware, ApacheCon will be held this year in Budapest, on November 17-23. (See http://apachecon.eu for more info.) The Call For Papers for that conference is still open, but will be closing soon. We need you talk proposals, to represent Flume at ApacheCon. We need all kinds of talks - deep technical talks, hands-on tutorials, introductions for beginners, or case studies about the awesome stuff you're doing with Flume. Please consider submitting a proposal, at http://events.linuxfoundation.org//events/apachecon-europe/program/cfp Thanks, Arvind Prabhakar
