[ANNOUNCE] Change of Apache Flume PMC Chair
Dear Flume Users and Developers, I have had the pleasure of serving as the PMC Chair of Apache Flume since its graduation three years ago. I sincerely thank you and the Flume PMC for this opportunity. However, I have decided to step down from this responsibility due to personal reasons. I am very happy to announce that on the request of Flume PMC and with the approval from the board of directors at The Apache Software Foundation, Hari Shreedharan is hereby appointed as the new PMC Chair. I am confident that Hari will do everything possible to help further grow the community and adoption of Apache Flume. Please join me in congratulating Hari on his appointment and welcoming him to this role. Regards, Arvind Prabhakar
Re: contributing to flume
Hi Eran, I added you to the contributors list and assigned the Jira to you. You should be able to make the status changes yourself. Regards, Arvind Prabhakar On Sat, Sep 26, 2015 at 4:20 AM, IT CTO wrote: > Hi, > I am new to flume and want to be able to contribute to flume. > I opened a jira issue (flume-2802) but I can't assign it to me so I can't > change the status to get the patch reviewed. > Can someone help here? > > Eran > -- > Eran | "You don't need eyes to see, you need vision" (Faithless) >
[jira] [Assigned] (FLUME-2802) Folder name interceptor
[ https://issues.apache.org/jira/browse/FLUME-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arvind Prabhakar reassigned FLUME-2802: --- Assignee: Eran W > Folder name interceptor > --- > > Key: FLUME-2802 > URL: https://issues.apache.org/jira/browse/FLUME-2802 > Project: Flume > Issue Type: New Feature >Reporter: Eran W >Assignee: Eran W > Attachments: FLUME-2802.patch > > > This interceptor retrieve the last folder name from the > SpoolDir.fileHeaderKey and set it to the given folderKey. > This is allow users to set the target hdfs directory based on the source > directory and not the whole path or file name. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: [VOTE] Release Apache Flume version 1.6.0 RC1
Thanks Rufus for helping with this release. My vote: +1 * Verified signatures and checksums * Top level files look good (Nit: The NOTICE file copyright statement says 2012 instead of 2015) * The tag looks good (Nit: the DOAP file is not included in the source tar ball) * RAT test passes with configured exclusions (checked via mvn verify) Regards, Arvind Prabhakar On Sun, May 17, 2015 at 5:50 AM, 李响 wrote: > +1 > All test cases pass using the latest OpenJDK 1.7.0_79. Thanks Johny!! > > On Thu, May 14, 2015 at 9:44 PM, Ashish wrote: > > > +1 > > > > Build works good, all test cases pass > > Randomly picked few JIRA's and validate the commits looks good > > > > Thank You Johny for all the hard work. > > > > On Tue, May 12, 2015 at 11:13 PM, Johny Rufus > wrote: > > > Hi All, > > > > > > This is the ninth release for Apache Flume as a top-level project, > > > version 1.6.0. We are voting on release candidate RC1. > > > > > > It fixes the following issues: > > > > > > > > > https://git-wip-us.apache.org/repos/asf?p=flume.git;a=blob;f=CHANGELOG;h=53ea45cbd496b89fcd84c89f2ebd8d51e5bb8016;hb=f7560038a25430378f09ea631b6e472979d7988c > > > > > > *** Please cast your vote within the next 72 hours *** > > > > > > The tarball (*.tar.gz), signature (*.asc), and checksums (*.md5, > *.sha1) > > > for the source and binary artifacts can be found here: > > > > > > http://people.apache.org/~hshreedharan/apache-flume-1.6.0-rc1/ > > > > > > Maven staging repo: > > > > > > > https://repository.apache.org/content/repositories/orgapacheflume-1016/ > > > > > > The tag to be voted on: > > > > > > > > > https://git-wip-us.apache.org/repos/asf?p=flume.git;a=commit;h=f7560038a25430378f09ea631b6e472979d7988c > > > > > > > > > Flume's KEYS file containing PGP keys we use to sign the release: > > > http://www.apache.org/dist/flume/KEYS > > > > > > > > > Thanks, > > > Rufus > > > > > > > > -- > > thanks > > ashish > > > > Blog: http://www.ashishpaliwal.com/blog > > My Photo Galleries: http://www.pbase.com/ashishpaliwal > > > > > > -- > >李响 > > 手机 cellphone ：+86-1368-113-8972 > E-mail ：wate...@gmail.com > MSN ：wate...@hotmail.com >
Re: Flume performance measurements
Done. Please let me know if you run into any issues. Regards, Arvind On Wed, Apr 8, 2015 at 3:58 PM, Roshan Naik wrote: > roshan_naik is my login to cwiki.apache.org > > > > > On 4/8/15 3:55 PM, "Arvind Prabhakar" wrote: > > >Added Hari to the wiki. > > > >Roshan, I could not look you up on the wiki users, can you please tell me > >your username? If you don't have one yet, please register and let me know. > > > >Regards, > >Arvind Prabhakar > > > >On Wed, Apr 8, 2015 at 3:26 PM, Roshan Naik > >wrote: > > > >> Arvind, > >> Please do let me know once you have granted me permission to the > >>wiki. > >> -roshan > >> > >> From: Hari Shreedharan >> hshreedha...@cloudera.com>> > >> Date: Thursday, April 2, 2015 3:06 PM > >> To: Roshan Naik mailto:ros...@hortonworks.com>> > >> Cc: "dev@flume.apache.org<mailto:dev@flume.apache.org>" < > >> dev@flume.apache.org<mailto:dev@flume.apache.org>> > >> Subject: Re: Flume performance measurements > >> > >> Arvind - please could you grant Roshan access to the wiki. > >> > >> Thanks, > >> Hari > >> > >> > >> > >> On Thu, Apr 2, 2015 at 3:04 PM, Roshan Naik >> <mailto:ros...@hortonworks.com>> wrote: > >> > >> Could u grant me write access to wiki ? > >> username: roshannaik > >> > >> > >> > >> On 4/2/15 2:53 PM, "Hari Shreedharan" >> hshreedha...@cloudera.com>> wrote: > >> > >> >Roshan, > >> > > >> > > >> > > >> > > >> >Could you update the performance measurements page on our wiki with > >>this > >> >info? That would be more useful to reference. > >> > > >> > > >> > > >> > > >> >Thanks, Hari > >> > > >> >On Thu, Apr 2, 2015 at 2:34 PM, Roshan Naik >> <mailto:ros...@hortonworks.com>> > >> >wrote: > >> > > >> >> Sample Flume v1.4 Measurements for reference: > >> >> Here are some sample measurements taken with a single agent and 500 > >> >>byte events. > >> >> Cluster Config: 20-node Hadoop cluster (1 name node and 19 data > >>nodes). > >> >> Machine Config: 24 cores - Xeon E5-2640 v2 @ 2.00GHz, 164 GB RAM. > >> >> 1. File channel with HDFS Sink (Sequence File): > >> >> Source: 4 x Exec Source, 100k batchSize > >> >> HDFS Sink Batch size: 500,000 > >> >> Channel: File > >> >> Number of data dirs: 8 > >> >> Events/Sec > >> >> Sink Count > >> >> 1 data dirs > >> >> 2 data dirs > >> >> 4 data dirs > >> >> 6 data dirs > >> >> 8 data dirs > >> >> 10 data dirs > >> >> 1 > >> >> 14.3 k > >> >> 2 > >> >> 21.9 k > >> >> 4 > >> >> 35.8 k > >> >> 8 > >> >> 24.8 k > >> >> 43.8 k > >> >> 72.5 k > >> >> 77 k > >> >> 78.6 k > >> >> 76.6 k > >> >> 10 > >> >> 58 k > >> >> 12 > >> >> 49.3 k > >> >> 49 k > >> >> Was looking for sweet spot in perf. So did not take measurements for > >> >>all data points on grid. Only too for the ones that made sense. For > >> >>example: when perf dropped by adding more sinks, did not take more > >> >>measurements for those rows. > >> >> 2. HDFS Sink: > >> >> Channel: Memory > >> >> # of HDFS > >> >> Sinks > >> >> Snappy > >> >> BatchSz:1.2mill > >> >> Snappy > >> >> BatchSz:1.4mill > >> >> Sequence File > >> >> BatchSz:1.2mill > >> >> 1 > >> >> 34.3 k > >> >> 33 k > >> >> 33 k > >> >> 2 > >> >> 71 k > >> >> 75 k > >> >> 69 k > >> >> 4 > >> >> 141 k > >> >> 145 k > >> >> 141 k > >> >> 8 > >> >> 271 k > >> >> 273 k > >> >> 251 k > >> >> 12 > >> >> 382 k > >> >> 380 k > >> >> 370 k > >> >> 16 > >> >> 478 k > >> >> 538 k > >> >> 486 k > >> >> Some simple observations : > >> >> * increasing number of dataDirs helps FC perf even on single disk > >> >>systems > >> >> * Increasing number of sinks helps > >> >> * Max throughput observed was about 538k events/sec for HDFS sink > >> >>which is approx 240MB/s > >> > >> > >> > >
Re: Flume performance measurements
Added Hari to the wiki. Roshan, I could not look you up on the wiki users, can you please tell me your username? If you don't have one yet, please register and let me know. Regards, Arvind Prabhakar On Wed, Apr 8, 2015 at 3:26 PM, Roshan Naik wrote: > Arvind, > Please do let me know once you have granted me permission to the wiki. > -roshan > > From: Hari Shreedharan hshreedha...@cloudera.com>> > Date: Thursday, April 2, 2015 3:06 PM > To: Roshan Naik mailto:ros...@hortonworks.com>> > Cc: "dev@flume.apache.org<mailto:dev@flume.apache.org>" < > dev@flume.apache.org<mailto:dev@flume.apache.org>> > Subject: Re: Flume performance measurements > > Arvind - please could you grant Roshan access to the wiki. > > Thanks, > Hari > > > > On Thu, Apr 2, 2015 at 3:04 PM, Roshan Naik <mailto:ros...@hortonworks.com>> wrote: > > Could u grant me write access to wiki ? > username: roshannaik > > > > On 4/2/15 2:53 PM, "Hari Shreedharan" hshreedha...@cloudera.com>> wrote: > > >Roshan, > > > > > > > > > >Could you update the performance measurements page on our wiki with this > >info? That would be more useful to reference. > > > > > > > > > >Thanks, Hari > > > >On Thu, Apr 2, 2015 at 2:34 PM, Roshan Naik <mailto:ros...@hortonworks.com>> > >wrote: > > > >> Sample Flume v1.4 Measurements for reference: > >> Here are some sample measurements taken with a single agent and 500 > >>byte events. > >> Cluster Config: 20-node Hadoop cluster (1 name node and 19 data nodes). > >> Machine Config: 24 cores - Xeon E5-2640 v2 @ 2.00GHz, 164 GB RAM. > >> 1. File channel with HDFS Sink (Sequence File): > >> Source: 4 x Exec Source, 100k batchSize > >> HDFS Sink Batch size: 500,000 > >> Channel: File > >> Number of data dirs: 8 > >> Events/Sec > >> Sink Count > >> 1 data dirs > >> 2 data dirs > >> 4 data dirs > >> 6 data dirs > >> 8 data dirs > >> 10 data dirs > >> 1 > >> 14.3 k > >> 2 > >> 21.9 k > >> 4 > >> 35.8 k > >> 8 > >> 24.8 k > >> 43.8 k > >> 72.5 k > >> 77 k > >> 78.6 k > >> 76.6 k > >> 10 > >> 58 k > >> 12 > >> 49.3 k > >> 49 k > >> Was looking for sweet spot in perf. So did not take measurements for > >>all data points on grid. Only too for the ones that made sense. For > >>example: when perf dropped by adding more sinks, did not take more > >>measurements for those rows. > >> 2. HDFS Sink: > >> Channel: Memory > >> # of HDFS > >> Sinks > >> Snappy > >> BatchSz:1.2mill > >> Snappy > >> BatchSz:1.4mill > >> Sequence File > >> BatchSz:1.2mill > >> 1 > >> 34.3 k > >> 33 k > >> 33 k > >> 2 > >> 71 k > >> 75 k > >> 69 k > >> 4 > >> 141 k > >> 145 k > >> 141 k > >> 8 > >> 271 k > >> 273 k > >> 251 k > >> 12 > >> 382 k > >> 380 k > >> 370 k > >> 16 > >> 478 k > >> 538 k > >> 486 k > >> Some simple observations : > >> * increasing number of dataDirs helps FC perf even on single disk > >>systems > >> * Increasing number of sinks helps > >> * Max throughput observed was about 538k events/sec for HDFS sink > >>which is approx 240MB/s > > >
[jira] [Created] (FLUME-2564) Failover processor does not kick-in for HDFS sink on IOException
Arvind Prabhakar created FLUME-2564:
---
Summary: Failover processor does not kick-in for HDFS sink on
IOException
Key: FLUME-2564
URL: https://issues.apache.org/jira/browse/FLUME-2564
Project: Flume
Issue Type: Bug
Reporter: Arvind Prabhakar
Assignee: Arvind Prabhakar
>From a recent thread on the user mailing list:
{quote}
I have investigated the HDFSEventSink source code, found if the exception was
IOException , the exception would not throw to the upper layer,
So FailOverSinkProcessor would not mark this sink as dead.
{quote}
{code}
} catch (IOException eIO) {
transaction.rollback();
LOG.warn("HDFS IO error", eIO);
return Status.BACKOFF;
} catch (Throwable th) {
transaction.rollback();
LOG.error("process failed", th);
if (th instanceof Error) {
throw (Error) th;
} else {
throw new EventDeliveryException(th);
}
}
{code}
The failover processor should be able to use the backoff signal as indication
of failure and switch over to the next sink.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: [VOTE] Apache Flume 1.5.2 RC1
+1 * Verified checksums and signatures * Verified build Regards, Arvind Prabhakar On Fri, Nov 14, 2014 at 2:58 PM, Hari Shreedharan wrote: > +1. > > > > > - Verified signatures and checksums > > - Built and ran tests > > - Verified top-level files. > > > Thanks, > Hari > > On Thu, Nov 13, 2014 at 1:17 PM, Roshan Naik > wrote: > > > +1 > > verified the code change > > -roshan > > On Wed, Nov 12, 2014 at 8:03 PM, Jarek Jarcec Cecho > > wrote: > >> +1 > >> > >> * Verified checksums and signature files > >> * Verified that each jar in binary tarball is in the license > >> * Checked top level files (NOTICE, ...) > >> * Run tests > >> > >> (pretty much the same email I’ve sent for 1.5.1 :)) > >> > >> Jarcec > >> > On Nov 12, 2014, at 1:15 PM, Hari Shreedharan < > hshreedha...@cloudera.com> > >> wrote: > >> > > >> > This is the eighth release for Apache Flume as a top-level project, > >> > version 1.5.2. We are voting on release candidate RC1. > >> > > >> > This release fixes an incompatibility with Java 6 based clients found > >> > in Apache Flume 1.5.1 Release. > >> > > >> > It fixes the following > >> > issues: > >> > https://git-wip-us.apache.org/repos/asf?p=flume.git;a=blob;f=CHANGELOG;h=cc7321361d0b702ba870de20d6a3d2106987186a;hb=229442aa6835ee0faa17e3034bcab42754c460f5 > >> > > >> > *** Please cast your vote within the next 72 hours *** > >> > > >> > The tarball (*.tar.gz), signature (*.asc), and checksums (*.md5, > *.sha1) > >> > for the source and binary artifacts can be found here: > >> > *https://people.apache.org/~hshreedharan/apache-flume-1.5.2-rc1/ > >> > <https://people.apache.org/~hshreedharan/apache-flume-1.5.2-rc1/>* > >> > > >> > Maven staging repo: > >> > * > >> https://repository.apache.org/content/repositories/orgapacheflume-1008/ > >> > < > https://repository.apache.org/content/repositories/orgapacheflume-1008/ > >> >* > >> > > >> > The tag to be voted on: > >> > * > >> > https://git-wip-us.apache.org/repos/asf?p=flume.git;a=commit;h=229442aa6835ee0faa17e3034bcab42754c460f5 > >> > < > >> > https://git-wip-us.apache.org/repos/asf?p=flume.git;a=commit;h=229442aa6835ee0faa17e3034bcab42754c460f5 > >> >* > >> > > >> > Flume's KEYS file containing PGP keys we use to sign the release: > >> > http://www.apache.org/dist/flume/KEYS > >> > > >> > > >> > Thanks, > >> > Hari > >> > >> > > -- > > CONFIDENTIALITY NOTICE > > NOTICE: This message is intended for the use of the individual or entity > to > > which it is addressed and may contain information that is confidential, > > privileged and exempt from disclosure under applicable law. If the reader > > of this message is not the intended recipient, you are hereby notified > that > > any printing, copying, dissemination, distribution, disclosure or > > forwarding of this communication is strictly prohibited. If you have > > received this communication in error, please contact the sender > immediately > > and delete it from your system. Thank You. >
Re: [VOTE] Release Apache Flume 1.5.1 RC1
That sounds good to me. Thanks for working on this release Hari. Regards, Arvind Prabhakar On Mon, Nov 10, 2014 at 11:10 AM, Hari Shreedharan < hshreedha...@cloudera.com> wrote: > It does not look like we ever actually included the dev-support directory > in the source tarball (I checked 1.3.1,1.4.0 and 1.5.0.1). If we need a > re-spin for another reason, I will try to fix the release process to pull > this in and remove the iml files. > > > > > Arvind - does that sound good to you? Otherwise I will spin another RC. > > > Thanks, > Hari > > On Sun, Nov 9, 2014 at 8:47 PM, Arvind Prabhakar > wrote: > > > +1 > > * Verified signatures > > * Verified checksums > > * Verified the tag (minor issues noted below - would be good to address > if > > there is RC2) > > * Builds correctly > > * All tests run with default profile and avro version set to 1.7.5 (to > > avoid an issue with snappy on Mac OS) > > Nits: > > * The tag and sources match except that the src tarball contains the iml > > files and does not contain the dev-support directory. Since both the iml > > files and dev-support files are not related to product functionality, it > is > > OK for the tarball to not include them. However, if there is a respin it > > would be good to address that. > > * It is time we updated the avro version in the system to a newer > release, > > which among other things will allow people to build on Mac OS without > > running into the JDK7+Snappy 1.0.4 problem where tests because native > > library does not load. > > Regards, > > Arvind > > On Thu, Nov 6, 2014 at 3:17 PM, Hari Shreedharan < > hshreedha...@cloudera.com> > > wrote: > >> This is the seventh release for Apache Flume as a top-level project, > >> version 1.5.1. We are voting on release candidate RC1. > >> > >> It fixes the following issues: > >> > >> > https://git-wip-us.apache.org/repos/asf?p=flume.git;a=blob_plain;f=CHANGELOG;hb=c74804226bcee59823c0cbc09cdf803a3d9e6920 > >> > >> *** Please cast your vote within the next 72 hours *** > >> > >> The tarball (*.tar.gz), signature (*.asc), and checksums (*.md5, *.sha1) > >> for the source and binary artifacts can be found here: > >> https://people.apache.org/~hshreedharan/apache-flume-1.5.1-rc1/ > >> > >> Maven staging repo: > >> > https://repository.apache.org/content/repositories/orgapacheflume-1006/ > >> > >> The tag to be voted on: > >> > >> > https://git-wip-us.apache.org/repos/asf?p=flume.git;a=commit;h=c74804226bcee59823c0cbc09cdf803a3d9e6920 > >> > >> Flume's KEYS file containing PGP keys we use to sign the release: > >> http://www.apache.org/dist/flume/KEYS > >> > >> Thanks, > >> Hari >
Re: [VOTE] Release Apache Flume 1.5.1 RC1
+1 * Verified signatures * Verified checksums * Verified the tag (minor issues noted below - would be good to address if there is RC2) * Builds correctly * All tests run with default profile and avro version set to 1.7.5 (to avoid an issue with snappy on Mac OS) Nits: * The tag and sources match except that the src tarball contains the iml files and does not contain the dev-support directory. Since both the iml files and dev-support files are not related to product functionality, it is OK for the tarball to not include them. However, if there is a respin it would be good to address that. * It is time we updated the avro version in the system to a newer release, which among other things will allow people to build on Mac OS without running into the JDK7+Snappy 1.0.4 problem where tests because native library does not load. Regards, Arvind On Thu, Nov 6, 2014 at 3:17 PM, Hari Shreedharan wrote: > This is the seventh release for Apache Flume as a top-level project, > version 1.5.1. We are voting on release candidate RC1. > > It fixes the following issues: > > https://git-wip-us.apache.org/repos/asf?p=flume.git;a=blob_plain;f=CHANGELOG;hb=c74804226bcee59823c0cbc09cdf803a3d9e6920 > > *** Please cast your vote within the next 72 hours *** > > The tarball (*.tar.gz), signature (*.asc), and checksums (*.md5, *.sha1) > for the source and binary artifacts can be found here: > https://people.apache.org/~hshreedharan/apache-flume-1.5.1-rc1/ > > Maven staging repo: > https://repository.apache.org/content/repositories/orgapacheflume-1006/ > > The tag to be voted on: > > https://git-wip-us.apache.org/repos/asf?p=flume.git;a=commit;h=c74804226bcee59823c0cbc09cdf803a3d9e6920 > > Flume's KEYS file containing PGP keys we use to sign the release: > http://www.apache.org/dist/flume/KEYS > > Thanks, > Hari
[ANNOUNCE] New Flume PMC Member - Roshan Naik
On behalf of Apache Flume PMC, it is my pleasure to announce that Roshan Naik has been elected to the Flume Project Management Committee. Roshan has been active with the project for many years and has been a committer on the project since September of 2013. Please join me in congratulating Roshan and welcoming him to the Flume PMC. Regards, Arvind Prabhakar
Re: What Source/Sink would you want next?
(cross-posting this to dev@) While I do not speak for the availability of other committers of the project, I would like to spend sometime with the contributors to help identify what are the most important needs of the project, and see how best we can get those committed into the codebase. Santiago (and others who would like to contribute) - please go ahead and create the necessary Jiras if they do not exist already, and invite the community to vote on those. That way we can prioritize the review and commit for functionality that is aligned with community requirements. Regards, Arvind Prabhakar On Fri, Sep 26, 2014 at 5:13 AM, jean garutti wrote: > hi > This seems to be great. > I'll wait to have the 'production ready' flag for ELS mapping patch. > I think more effort should be done to have this sink more configurable > like what we can do with logstash. > > anyway it's nice to share your development to the community > i'd love to have the mongodb sink packaged in the official flume release. > > jean > > > Le Jeudi 25 septembre 2014 9h48, Santiago Mola a > écrit : > > > Hi Jean, > > 2014-09-24 22:44 GMT+02:00 Jean : > > A solid mongodb source would be Nice. > > > Definitely! > > > I wish the same for elasticsearch sink where we could specify the mapping > for the headers instead of sending everything as a string > > > We have a serializer that creates mappings for ElasticSearch [1]. It is > not ready for production [2] but it is one of our priorities. > > [1] > https://github.com/Stratio/stratio-ingestion/tree/develop/stratio-serializers/stratio-elasticsearch-serializer > [2] https://github.com/Stratio/stratio-ingestion/issues/21 > > Thanks for your feedback, > > -- > > Santiago M. Mola > > > <http://www.stratio.com/> > Avenida de Europa, 26. Ática 5. 3ª Planta > 28224 Pozuelo de Alarcón, Madrid > Tel: +34 91 352 59 42 // *@stratiobd <https://twitter.com/StratioBD>* > > >
[jira] [Commented] (FLUME-2365) Please create a DOAP file for your TLP
[ https://issues.apache.org/jira/browse/FLUME-2365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14032013#comment-14032013 ] Arvind Prabhakar commented on FLUME-2365: - [~hshreedharan] - I updated the files.xlm in the site repository. Once the project shows up correctly on http://projects.apache.org/indexes/alpha.html#F we can go ahead and close this Jira out. > Please create a DOAP file for your TLP > -- > > Key: FLUME-2365 > URL: https://issues.apache.org/jira/browse/FLUME-2365 > Project: Flume > Issue Type: Task >Reporter: Sebb >Assignee: Ashish Paliwal > Attachments: flume.rdf > > > As per my recent e-mail to your dev list, please can you set up a DOAP for > your project and get it added to files.xml? > Please see http://projects.apache.org/create.html > Once you have created the DOAP and committed it to your source code > repository, please submit it for inclusion in the Apache projects listing as > per: > http://projects.apache.org/create.html#submit > Remember, if you ever move or rename the doap file in future, please > ensure that files.xml is updated to point to the new location. > Thanks! -- This message was sent by Atlassian JIRA (v6.2#6252)
ApacheCon CFP closes June 25
Dear Flume enthusiast, As you may be aware, ApacheCon will be held this year in Budapest, on November 17-23. (See http://apachecon.eu for more info.) The Call For Papers for that conference is still open, but will be closing soon. We need you talk proposals, to represent Flume at ApacheCon. We need all kinds of talks - deep technical talks, hands-on tutorials, introductions for beginners, or case studies about the awesome stuff you're doing with Flume. Please consider submitting a proposal, at http://events.linuxfoundation.org//events/apachecon-europe/program/cfp Thanks, Arvind Prabhakar
Re: [VOTE] Apache Flume 1.5.0.1 RC1
+1 Thanks for shepherding this Hari. Regards, Arvind Prabhakar On Tue, Jun 10, 2014 at 3:40 PM, Hari Shreedharan wrote: > This is a vote for the next release of Apache Flume, version 1.5.0.1. We > are voting on release candidate RC1. > > It fixes the following issues: > http://s.apache.org/v7X > > *** Please cast your vote within the next 72 hours *** > > The tarball (*.tar.gz), signature (*.asc), and checksums (*.md5, *.sha1) > for the source and binary artifacts can be found here: >https://people.apache.org/~hshreedharan/apache-flume-1.5.0.1-rc1/ > > Maven staging repo: > https://repository.apache.org/content/repositories/orgapacheflume-1004/ > > The tag to be voted on: > > > https://git-wip-us.apache.org/repos/asf?p=flume.git;a=commit;h=ceda6aa1126a01370641caf729d8b1dd6d80aa61 > > Flume's KEYS file containing PGP keys we use to sign the release: > http://www.apache.org/dist/flume/KEYS > > > Thanks, > Hari >
Re: [VOTE] Apache Flume 1.5.0 RC1
+1 * Verified signatures and checksums for both binary and source tarballs * Rat check looks good on source tarball * Nit: Notice file has dated header, needs to be updated but not a blocker Regards, Arvind Prabhakar On Wed, May 7, 2014 at 3:28 PM, Hari Shreedharan wrote: > This is a vote for the next release of Apache Flume, version 1.5.0. We are > voting on release candidate RC1. > > It fixes the following issues: > http://s.apache.org/4eQ > > *** Please cast your vote within the next 72 hours *** > > The tarball (*.tar.gz), signature (*.asc), and checksums (*.md5, > *.sha1) for the source and binary artifacts can be found here: >https://people.apache.org/~hshreedharan/apache-flume-1.5.0-rc1/ > > Maven staging repo: > https://repository.apache.org/content/repositories/orgapacheflume-1001/ > > > The tag to be voted on: > > > https://git-wip-us.apache.org/repos/asf?p=flume.git;a=commit;h=8633220df808c4cd0c13d1cf0320454a94f1ea97 > > Flume's KEYS file containing PGP keys we use to sign the release: > http://www.apache.org/dist/flume/KEYS > > > Thanks, > Hari >
Re: [DISCUSS] Release Flume 1.5.0
Thanks for bringing this up Hari. A new release for Flume is overdue in my opinion. Regards, Arvind Prabhakar On Thu, Jan 30, 2014 at 9:43 AM, Chiwan Park wrote: > +1 on new release! > > -- > Regards, > Chiwan Park > > On Jan 31, 2014, at 2:17 AM, Hari Shreedharan > wrote: > > > Hi folks, > > > > It has been about 6 months since we did a release. We have added several > > new features and fixed a lot of bugs. What do you guys think about > > releasing Flume 1.5.0? > > > > > > Thanks > > Hari > >
Re: Phoenix- Hbase Sink
Apologies for the delay Ravi and Hari - last time I tried to add you the Wiki was being upgraded and was not ready. I have now added you in and you should be able to see the edit button. Regards, Arvind Prabhakar On Thu, Dec 19, 2013 at 5:18 PM, Hari Shreedharan wrote: > Arvind - Could you please give Ravi edit privileges. I don’t seem to have > access. > > > Thanks, > Hari > > > On Thursday, December 19, 2013 at 5:13 PM, Ravi Kiran wrote: > > > Hi Hari, > > > > Can you please grant me permissions to update the WIKI to have > pointers to Phoenix . > > > > Regards > > Ravi > > > > > > On Sun, Dec 15, 2013 at 7:25 AM, Ravi Kiran > > maghamraviki...@gmail.com)> wrote: > > > Hi Hari, > > > > > > Its maghamravikiran > > > > > > Thanks > > > Ravi. > > > > > > > > > > > > On Sat, Dec 14, 2013 at 7:03 AM, Hari Shreedharan < > hshreedha...@cloudera.com (mailto:hshreedha...@cloudera.com)> wrote: > > > > +dev@ > > > > > > > > Hi Ravi, > > > > > > > > Can you please send your confluence (wiki) login id? > > > > Thanks, > > > > Hari > > > > > > > > > > > > On Thursday, December 12, 2013 at 4:24 AM, Ravi Kiran wrote: > > > > > > > > > Hi Hari, > > > > > > > > > >I don't seem to have permissions to edit the page. Can you > please grant me permissions. > > > > > > > > > > Regards > > > > > Ravi > > > > > > > > > > > > > > > On Thu, Dec 12, 2013 at 10:40 AM, Hari Shreedharan < > hshreedha...@cloudera.com (mailto:hshreedha...@cloudera.com)> wrote: > > > > > > Hi Ravi, > > > > > > > > > > > > Thanks for the information. You could post a link to this on the > wiki here: > https://cwiki.apache.org/confluence/display/FLUME/Flume+NG+Plugins for > users to be able to find it. > > > > > > > > > > > > > > > > > > Thanks, > > > > > > Hari > > > > > > > > > > > > > > > > > > On Wednesday, December 11, 2013 at 8:26 PM, Ravi Kiran wrote: > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > >The Apache Phoenix project now provides a custom sink for > streaming Flume events into HBase. These events may be queried through SQL > using the Phoenix JDBC driver. > > > > > > > The detailed instructions can be found here (still on > github until we move to Apache): > https://github.com/forcedotcom/phoenix/wiki/Apache-Flume-Plugin. > > > > > > > > > > > > > > > > > > > > > Regards > > > > > > > Ravi > > > > > > > > > > > > > > > > > > > > > >
[jira] [Commented] (FLUME-2191) HDFS Minicluster tests failing after protobuf upgrade.
[
https://issues.apache.org/jira/browse/FLUME-2191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13785791#comment-13785791
]
Arvind Prabhakar commented on FLUME-2191:
-
+1 changes look good to me. Will commit after a sanity run.
> HDFS Minicluster tests failing after protobuf upgrade.
> --
>
> Key: FLUME-2191
> URL: https://issues.apache.org/jira/browse/FLUME-2191
> Project: Flume
> Issue Type: Bug
>Reporter: Hari Shreedharan
>Assignee: Hari Shreedharan
>Priority: Blocker
> Attachments: FLUME-2191.patch
>
>
> I ran the full build in hadoop-1 profile, but it looks like the protobuf
> upgrade broke the hadoop-2 profile. The HDFS Sink test on Minicluster fails
> with this:
> {code}
> Running org.apache.flume.sink.hdfs.TestHDFSEventSinkOnMiniCluster
> 2013-09-13 12:11:31.159 java[58566:1203] Unable to load realm info from
> SCDynamicStore
> 2013-09-13 12:11:31.208 java[58566:1203] Unable to load realm info from
> SCDynamicStore
> Tests run: 4, Failures: 0, Errors: 4, Skipped: 0, Time elapsed: 4.238 sec <<<
> FAILURE!
> simpleHDFSTest(org.apache.flume.sink.hdfs.TestHDFSEventSinkOnMiniCluster)
> Time elapsed: 1979 sec <<< ERROR!
> java.lang.UnsupportedOperationException: This is supposed to be overridden by
> subclasses.
> at
> com.google.protobuf.GeneratedMessage.getUnknownFields(GeneratedMessage.java:180)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$GetDatanodeReportRequestProto.getSerializedSize(ClientNamenodeProtocolProtos.java:21638)
> at
> com.google.protobuf.AbstractMessageLite.toByteString(AbstractMessageLite.java:49)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.constructRpcRequest(ProtobufRpcEngine.java:137)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:181)
> at com.sun.proxy.$Proxy15.getDatanodeReport(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:165)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:84)
> at com.sun.proxy.$Proxy15.getDatanodeReport(Unknown Source)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getDatanodeReport(ClientNamenodeProtocolTranslatorPB.java:488)
> at org.apache.hadoop.hdfs.DFSClient.datanodeReport(DFSClient.java:1642)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster.waitActive(MiniDFSCluster.java:1703)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster.waitActive(MiniDFSCluster.java:1722)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:1066)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:929)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:588)
> at org.apache.hadoop.hdfs.MiniDFSCluster.(MiniDFSCluster.java:527)
> at org.apache.hadoop.hdfs.MiniDFSCluster.(MiniDFSCluster.java:398)
> at
> org.apache.flume.sink.hdfs.TestHDFSEventSinkOnMiniCluster.simpleHDFSTest(TestHDFSEventSinkOnMiniCluster.java:85)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
> at org.junit.runn
[jira] [Assigned] (FLUME-2192) AbstractSinkProcessor stop incorrectly calls start
[ https://issues.apache.org/jira/browse/FLUME-2192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arvind Prabhakar reassigned FLUME-2192: --- Assignee: Jeremy Karlson > AbstractSinkProcessor stop incorrectly calls start > -- > > Key: FLUME-2192 > URL: https://issues.apache.org/jira/browse/FLUME-2192 > Project: Flume > Issue Type: Bug > Components: Sinks+Sources >Affects Versions: v1.4.0, v1.3.1 >Reporter: Jeremy Karlson >Assignee: Jeremy Karlson > Fix For: v1.4.1, v1.5.0 > > Attachments: FLUME-2192.patch > > > AbstractSinkProcessor incorrectly calls start when trying to stop. Patch is > attached. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (FLUME-2199) Flume builds with new version require mvn install before site can be generated
[ https://issues.apache.org/jira/browse/FLUME-2199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13783567#comment-13783567 ] Arvind Prabhakar commented on FLUME-2199: - Thanks for the patch Andrew. Do you mind publishing a review request? > Flume builds with new version require mvn install before site can be generated > -- > > Key: FLUME-2199 > URL: https://issues.apache.org/jira/browse/FLUME-2199 > Project: Flume > Issue Type: Bug > Components: Build >Affects Versions: v1.4.0 >Reporter: Andrew Bayer >Assignee: Andrew Bayer > Fix For: v1.5.0 > > Attachments: FLUME-2199.patch > > > At this point, if you change the version for Flume, you need to run a mvn > install before you can run with -Psite (or, for that matter, javadoc:javadoc) > enabled. This is because the top-level POM in flume.git/pom.xml is both the > parent POM and the root of the reactor - since it's the parent, it's got to > run before any of the children that inherit from it, but site generation > should be running *after* all the children, so that it probably pulls in the > reactor's build of each child module, rather than having to pull in one > already installed/deployed before the build starts. > There are a bunch of other reasons to split parent POM and top-level POM, but > that's the biggest one right there. > Also, the javadoc jar generation is a bit messed up - every module's javadoc > jar contains not only its own javadocs but the javadocs for every Flume > module it depends on. That, again, may make sense in a site context for the > top-level, but not for the individual modules. This results in unnecessary > bloat in the javadoc jars, and unnecessary time spent downloading the > "*-javadoc-resources.jar" for every dependency each module has, due to how > the javadoc plugin works. Also the whole site generation per-module thing, > which I am not a fan of in most cases. I don't think it's needed here. > Tweaking the site plugin not to run anywhere but the top-level and the > javadoc plugin to not do the dependency aggregation anywhere but the > top-level should make a big difference on build speed. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Assigned] (FLUME-2199) Flume builds with new version require mvn install before site can be generated
[ https://issues.apache.org/jira/browse/FLUME-2199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arvind Prabhakar reassigned FLUME-2199: --- Assignee: Andrew Bayer > Flume builds with new version require mvn install before site can be generated > -- > > Key: FLUME-2199 > URL: https://issues.apache.org/jira/browse/FLUME-2199 > Project: Flume > Issue Type: Bug > Components: Build >Affects Versions: v1.4.0 >Reporter: Andrew Bayer >Assignee: Andrew Bayer > Fix For: v1.5.0 > > Attachments: FLUME-2199.patch > > > At this point, if you change the version for Flume, you need to run a mvn > install before you can run with -Psite (or, for that matter, javadoc:javadoc) > enabled. This is because the top-level POM in flume.git/pom.xml is both the > parent POM and the root of the reactor - since it's the parent, it's got to > run before any of the children that inherit from it, but site generation > should be running *after* all the children, so that it probably pulls in the > reactor's build of each child module, rather than having to pull in one > already installed/deployed before the build starts. > There are a bunch of other reasons to split parent POM and top-level POM, but > that's the biggest one right there. > Also, the javadoc jar generation is a bit messed up - every module's javadoc > jar contains not only its own javadocs but the javadocs for every Flume > module it depends on. That, again, may make sense in a site context for the > top-level, but not for the individual modules. This results in unnecessary > bloat in the javadoc jars, and unnecessary time spent downloading the > "*-javadoc-resources.jar" for every dependency each module has, due to how > the javadoc plugin works. Also the whole site generation per-module thing, > which I am not a fan of in most cases. I don't think it's needed here. > Tweaking the site plugin not to run anywhere but the top-level and the > javadoc plugin to not do the dependency aggregation anywhere but the > top-level should make a big difference on build speed. -- This message was sent by Atlassian JIRA (v6.1#6144)
Re: [ANNOUNCE] New Flume Committer - Wolfgang Hoschek
Congratulations Wolfgang! Regards, Arvind Prabhakar On Tue, Sep 24, 2013 at 4:05 PM, Mike Percy wrote: > Congrats Wolfgang, and welcome! > > Mike > > > On Tue, Sep 24, 2013 at 3:46 PM, Jarek Jarcec Cecho wrote: > >> Congratulations Wolfgang, well done! >> >> Jarcec >> >> On Tue, Sep 24, 2013 at 03:39:12PM -0700, Hari Shreedharan wrote: >> > On behalf of the Apache Flume PMC, I am excited to welcome Wolfgang >> Hoschek >> > as a committer on the Apache Flume project. Wolfgang contributed a new >> sink >> > with the ability to do heavyweight ETL-style processing and writing to >> > Apache Solr indices. >> > >> > Congratulations and Welcome, Wolfgang! >> > >> > >> > Cheers, >> > Hari Shreedharan >> > >
Re: [ANNOUNCE] New Flume Committer - Roshan Naik
Congratulations Roshan! Regards, Arvind Prabhakar On Tue, Sep 24, 2013 at 4:05 PM, Mike Percy wrote: > Congrats Roshan, welcome! > > Mike > > > On Tue, Sep 24, 2013 at 3:47 PM, Jarek Jarcec Cecho wrote: > >> Congratulations Roshan, well done! >> >> Jarcec >> >> On Tue, Sep 24, 2013 at 03:39:13PM -0700, Hari Shreedharan wrote: >> > On behalf of the Apache Flume PMC, I am excited to welcome Roshan Naik >> as a >> > committer on the Apache Flume project. Roshan has actively contributed >> > several patches to the Flume project, including bug fixes, Windows >> support >> > and new features. >> > >> > Congratulations and Welcome, Roshan! >> > >> > >> > Cheers, >> > Hari Shreedharan >> > >
[jira] [Commented] (FLUME-2173) Exactly once semantics for Flume
[ https://issues.apache.org/jira/browse/FLUME-2173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13753284#comment-13753284 ] Arvind Prabhakar commented on FLUME-2173: - (continuing the discussion here instead of email) Thanks Hari. In the spirit of keeping processing components pluggable, it would make sense to have this de-dupe logic pluggable itself. One benefit of doing so would be the choice of different implementations that could provide broader degree of guarantees. For example, the ZK based approach over the enter pipeline could provide complete once-only delivery guarantee but as you pointed out could add latency to delivery. Alternatively there could be locally optimized implementation of this approach that act on subsets of the event stream and thus benefit partitioned deployments where events cannot cross wires. Another use-case to consider would be to locally optimize for multiple channels within the same Agent. That way an Agent that has a File Channel setup as the primary channel and a Memory Channel setup as a fall-back channel in case the primary is full - would need local deduping without having to store state in ZK. > Exactly once semantics for Flume > > > Key: FLUME-2173 > URL: https://issues.apache.org/jira/browse/FLUME-2173 > Project: Flume > Issue Type: Bug >Reporter: Hari Shreedharan >Assignee: Hari Shreedharan > > Currently Flume guarantees only at least once semantics. This jira is meant > to track exactly once semantics for Flume. My initial idea is to include uuid > event ids on events at the original source (use a config to mark a source an > original source) and identify destination sinks. At the destination sinks, > use a unique ZK Znode to track the events. If once seen (and configured), > pull the duplicate out. > This might need some refactoring, but my belief is we can do this in a > backward compatible way. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [jira] [Issue Comment Deleted] (FLUME-2173) Exactly once semantics for Flume
Hi Hari, Thanks for bringing this up for discussion. I think it will be tremendously beneficial to Flume users if we can extend once-only guarantee. Your initial suggestion seems reasonable of having a Sink trap the events and reference a global state to drop duplicates. Rather than pushing this functionality to Sinks is there any other way by which we can make it more generally available? The reason I raise this concern is because otherwise this becomes a feature of a particular sink and not every sink will have the necessary implementation opportunity to get this. Alternatively what do you think about this being done at the channel level? Since we normally do not see custom implementations of channels, an implementation that works with the channel will likely be more useful for the broader community of Flume users. Regards, Arvidn On Sun, Aug 25, 2013 at 9:07 AM, Hari Shreedharan wrote: > Hi Gabriel, > > Thanks for your input. The part where we use replicating channel selector > to purposefully replicate - we can easily make it configurable whether to > delete deplicate events or not. That should not be difficult to do. > > The 2nd point where multiple agents/sinks could write the same event can > be solved by namespacing the events into different namespaces. So each sink > checks one namespace for the event, and multiple sinks can belong to the > same namespace - this way, if multiple events are going to write to the > same HDFS cluster, then if a duplicate occurs we can easily drop it. > Unfortunately, this also does not work around the who > HDFS-writing-but-throwing issue. > > I agree updating ZK will hit latency, but that is the cost to build once > only semantics on a highly flexible system. If you look at the algorithm, > we actually go to ZK only once per event (to create, there are no updates) > - this can even happen per batch if needed to reduce ZK round trips (though > I am not sure if ZK provides a batch API). > > The two phase commit approach sounds good, but it might require interface > changes which can now only be made in Flume 2.x. Alse, if we use a single > UUID combined with several flags we might be able to work duplicates caused > by this replication. > > > Thanks, > Hari > > > On Sunday, August 25, 2013 at 7:24 AM, Gabriel Commeau wrote: > > > Hi Hari, > > > > > > I deleted my comment (again). The mailing list is probably a better > avenue > > to discuss this sorry about that! :) > > > > I can find at least one other way duplicate events can occur, and so what > > I provided helps to reduce duplicate events but is not sufficient to > > guaranty exactly once semantics. However, I still think that using a > > 2-phase commit when writing to multiple channels would benefit Flume. > This > > should probably be a different ticket though. > > > > Concerning the algorithm you offered, the case of replicating channel > > selector should probably be handled, by creating a new UUID for each > > duplicate message. > > I hope this helps. > > > > > > Regards, > > > > Gabriel > > > > > > On 8/25/13 7:27 AM, "Gabriel Commeau (JIRA)" j...@apache.org)> wrote: > > > > > > > > [ > > > > https://issues.apache.org/jira/browse/FLUME-2173?page=com.atlassian.jira.p > > > lugin.system.issuetabpanels:all-tabpanel ] > > > > > > Gabriel Commeau updated FLUME-2173: > > > --- > > > > > > Comment: was deleted > > > > > > (was: I would approach the problem from a different angle. The way I > see > > > it, there are two main places where duplicates can occur: when using > > > multiple channels for one source (using a replication channel > selector), > > > and when the "output" of a sink cannot guaranty whether the event has > > > truly been committed or not (as you pointed out for example, HDFS > writing > > > the event but throwing an exception). > > > Actually, I don¹t think there is a general solution to the problem of > > > output systems (e.g. HDFS) that do not guaranty whether the event is > > > truly committed or not, because we¹d need to enforce this requirement > on > > > 3rd party systems (relative to Flume). I see it as a problem to be > solved > > > on a case-by-case basis for each sink. > > > > > > However, I would like to suggest a solution to the first problem. Here > is > > > an example to illustrate it: Pretend an agent has a source that writes > to > > > two (required) channels. As part of a transaction, the channel > processor > > > will commit to the first channel, which succeeds, and then to the > second > > > channel, which fails. The whole transaction will fail, but the event > has > > > already been committed once to the first channel. When the transaction > is > > > retried, the event will be duplicated. > > > The solution I discussed a few months back with Mike P. was to use a > > > two-phase commit when writing to channels. This insures that the events > > > are not actually committed to a channel if the following ones fail. > This > > > however w
[jira] [Commented] (FLUME-2140) Support diverting bad events from pipeline
[ https://issues.apache.org/jira/browse/FLUME-2140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13732732#comment-13732732 ] Arvind Prabhakar commented on FLUME-2140: - [Discussion thread|http://flume.markmail.org/thread/y3cks6hdgof3kxu6#query:+page:1+mid:rx3zm53t4dhmqskk+state:results] on this subject in the user-list for reference. > Support diverting bad events from pipeline > -- > > Key: FLUME-2140 > URL: https://issues.apache.org/jira/browse/FLUME-2140 > Project: Flume > Issue Type: New Feature > Components: Node > Reporter: Arvind Prabhakar > > A *bad event* can be any event that causes persistent sink side processing > failure due to the inherent nature of the event itself. Note that failures > that are not related to the inherent nature of the event such as network > communication failure, downstream capacity failure etc., do not make the > event a bad-event. > The presence of a bad event in a channel can cause the entire pipleline to > choke and become unusable. Flume should therefore be able to identify bad > events and provide a facility to route them out of the pipleline in order to > ensure the transport of other events continues uninterrupted. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-2140) Support diverting bad events from pipeline
[ https://issues.apache.org/jira/browse/FLUME-2140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727912#comment-13727912 ] Arvind Prabhakar commented on FLUME-2140: - Another case - a downstream filter is buggy and causes a batch to fail repeatedly due to a malformed header or some other details. > Support diverting bad events from pipeline > -- > > Key: FLUME-2140 > URL: https://issues.apache.org/jira/browse/FLUME-2140 > Project: Flume > Issue Type: New Feature > Components: Node > Reporter: Arvind Prabhakar > > A *bad event* can be any event that causes persistent sink side processing > failure due to the inherent nature of the event itself. Note that failures > that are not related to the inherent nature of the event such as network > communication failure, downstream capacity failure etc., do not make the > event a bad-event. > The presence of a bad event in a channel can cause the entire pipleline to > choke and become unusable. Flume should therefore be able to identify bad > events and provide a facility to route them out of the pipleline in order to > ensure the transport of other events continues uninterrupted. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (FLUME-2140) Support diverting bad events from pipeline
Arvind Prabhakar created FLUME-2140: --- Summary: Support diverting bad events from pipeline Key: FLUME-2140 URL: https://issues.apache.org/jira/browse/FLUME-2140 Project: Flume Issue Type: New Feature Components: Node Reporter: Arvind Prabhakar A *bad event* can be any event that causes persistent sink side processing failure due to the inherent nature of the event itself. Note that failures that are not related to the inherent nature of the event such as network communication failure, downstream capacity failure etc., do not make the event a bad-event. The presence of a bad event in a channel can cause the entire pipleline to choke and become unusable. Flume should therefore be able to identify bad events and provide a facility to route them out of the pipleline in order to ensure the transport of other events continues uninterrupted. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [VOTE] Release Apache Flume version 1.4.0 RC1
+1 * Built the sources * Verified checksums and signatures Thanks for the hard work Mike! Regards, Arvind Prabhakar On Mon, Jun 24, 2013 at 7:30 PM, Mike Percy wrote: > This is the fourth release for Apache Flume as a top-level project, > version 1.4.0. We are voting on release candidate RC1. > > It fixes the following issues: > > > https://git-wip-us.apache.org/repos/asf?p=flume.git;a=blob_plain;f=CHANGELOG;hb=756924e96ace470289472a3bdb4d87e273ca74ef > > *** Please cast your vote within the next 72 hours *** > > The tarball (*.tar.gz), signature (*.asc), and checksums (*.md5, *.sha1) > for the source and binary artifacts can be found here: > http://people.apache.org/~mpercy/flume/apache-flume-1.4.0-RC1/ > > Maven staging repo: > https://repository.apache.org/content/repositories/orgapacheflume-067/ > > The tag to be voted on: > > > https://git-wip-us.apache.org/repos/asf?p=flume.git;a=commit;h=756924e96ace470289472a3bdb4d87e273ca74ef > > Flume's KEYS file containing PGP keys we use to sign the release is here: > https://svn.apache.org/repos/asf/flume/dist/KEYS > > Thanks, > Mike >
Re: [VOTE] Release Apache Flume version 1.4.0 RC1
Given that there is ambiguity in terms of which license applies, and given that one of these licenses is Apache Software License 2.0, my suggestion is to keep BSD on record for our release. That way, we cover the more restrictive case and ideally should not pose any problems. Regards, Arvind Prabhakar On Fri, Jun 28, 2013 at 4:43 PM, Mike Percy wrote: > Interesting find, Hari. These guys are really a licensing disaster. However > I believe Maven is wrong since the LICENSE file in their repository > contains this: > > > https://code.google.com/p/findbugs/source/browse/branches/1.3.9/findbugs/LICENSE-jsr305.txt > > -- > The JSR-305 reference implementation (lib/jsr305.jar) is > distributed under the terms of the New BSD license: > > http://www.opensource.org/licenses/bsd-license.php > > See the JSR-305 home page for more information: > > http://code.google.com/p/jsr-305/ -- So I think it really is BSD. > Thoughts? Thanks, Mike > > > On Fri, Jun 28, 2013 at 4:24 PM, Hari Shreedharan < > hshreedha...@cloudera.com > > wrote: > > > Hi, > > > > Looks like jsr305 is actually ASL2.0 (according to the mvn central pom > for > > the specific version: > > > http://search.maven.org/#artifactdetails%7Ccom.google.code.findbugs%7Cjsr305%7C1.3.9%7Cjar > ). > > The pom installed locally also has this: > > > > > > The Apache Software License, Version > > 2.0 > > > > http://www.apache.org/licenses/LICENSE-2.0.txt > > repo > > > > > > > > > > The webpage on the other hand says it is BSD licensed. Maybe we should > > verify this? I know the last few of our releases went out with BSD in the > > Licenses file. > > > > > > Thanks, > > Hari > > > > > > On Friday, June 28, 2013 at 1:37 PM, Jarek Jarcec Cecho wrote: > > > > > +1 > > > > > > * Checked license file > > > * Run tests > > > * Checked other top level files > > > * Checked checksums and signature > > > > > > Jarcec > > > > > > On Mon, Jun 24, 2013 at 07:30:18PM -0700, Mike Percy wrote: > > > > This is the fourth release for Apache Flume as a top-level project, > > > > version 1.4.0. We are voting on release candidate RC1. > > > > > > > > It fixes the following issues: > > > > > > > > > > > https://git-wip-us.apache.org/repos/asf?p=flume.git;a=blob_plain;f=CHANGELOG;hb=756924e96ace470289472a3bdb4d87e273ca74ef > > > > > > > > *** Please cast your vote within the next 72 hours *** > > > > > > > > The tarball (*.tar.gz), signature (*.asc), and checksums (*.md5, > > *.sha1) > > > > for the source and binary artifacts can be found here: > > > > http://people.apache.org/~mpercy/flume/apache-flume-1.4.0-RC1/ > > > > > > > > Maven staging repo: > > > > > https://repository.apache.org/content/repositories/orgapacheflume-067/ > > > > > > > > The tag to be voted on: > > > > > > > > > > > https://git-wip-us.apache.org/repos/asf?p=flume.git;a=commit;h=756924e96ace470289472a3bdb4d87e273ca74ef > > > > > > > > Flume's KEYS file containing PGP keys we use to sign the release is > > here: > > > > https://svn.apache.org/repos/asf/flume/dist/KEYS > > > > > > > > Thanks, > > > > Mike > > > > > > > > > > > > > > > > > > > >
Re: [DISCUSS] Flume 1.4 release plan
Thanks for taking this initiative Mike! +1 for 1.4 and Mike as RM. Regards, Arvind Prabhakar On Wed, May 22, 2013 at 12:45 AM, Hari Shreedharan < hshreedha...@cloudera.com> wrote: > +1 for Flume 1.4 > +1 for Mike being RM. > > > Cheers, > Hari > > > On Wednesday, May 22, 2013 at 12:33 AM, Mike Percy wrote: > > > Hi folks, > > We have had over 100 commits since 1.3.1, and a bunch of new features and > > improvements including a Thrift source, much improved ElasticSearch sink, > > support for a new plugins directory and layout, compression support in > the > > avro sink/source, improved checkpointing in the file channel and more, > plus > > a lot of bug fixes. > > > > It seems to me that it's time to start thinking about cutting a 1.4 > > release. I would be happy to volunteer to RM the release. Worth noting > that > > I will be unavailable for the next two weeks... but after that I'd be > happy > > to pick this up and run with it. That's also a decent amount of time for > > people to get moving on patches and reviews for their favorite features, > > bug fixes, etc. > > > > If this all sounds OK, I'd like to suggest targeting the last week of > June > > as a release date. If we can release in time for Hadoop Summit then that > > would be pretty nice. Otherwise, if something comes up and we can't get > the > > release out that week, let's shoot for the first week of July at the > latest. > > > > Please let me know your thoughts. > > > > Regards, > > Mike > > > > > > >
Re: Flume schedule
Hi Aline, Currently there is no discussion around the timing for Flume 1.4.0. Could you share your motivation behind asking for the release schedule? Regards, Arvind Prabhakar On Mon, Feb 4, 2013 at 9:10 AM, Aline Guedes wrote: > Hello, > > Is there a schedule for Flume available somewhere? I am interested in the > planned release date for Flume 1.4.0 (in case there is a planned date), but > I can't find it anywhere. > > Thanks! > Aline > >
Re: [VOTE] Release Apache Flume 1.3.1
+1 * Verified signatures and hash sums * Build and tests work fine * Top level files look good. Thanks for driving this Hari. Regards, Arvind Prabhakar On Fri, Dec 21, 2012 at 11:44 PM, Hari Shreedharan < hshreedha...@cloudera.com> wrote: > Hi all, > > This is the third release for Apache Flume as a top-level project, > version 1.3.1. We are voting on release candidate rc0. > > *** This vote will remain open for at least 72 hours *** > > The list of fixed issues: > http://s.apache.org/01x > > The tarball (*.tar.gz), signature (*.asc), and checksums (*.md5, *.sha1) > for the source and binary artifacts can be found at: > http://people.apache.org/~hshreedharan/apache-flume-1.3.1-rc0/ > > Nexus Staging Repository: > https://repository.apache.org/content/repositories/orgapacheflume-074/ > > The tag we are voting on: > http://s.apache.org/L8q > > The KEYS file can be found here: > https://dist.apache.org/repos/dist/release/flume/KEYS > > > Thanks, > Hari Shreedharan > > > > -- > Hari Shreedharan > >
Re: [ANNOUNCE] Apache Flume 1.3.0 released
Thanks for your hard work Brock! Appreciate your diligence and resolve in getting this through! Regards, Arvind Prabhakar On Tue, Dec 4, 2012 at 9:16 PM, Will McQueen wrote: > Great job Brock! And thank you to everyone who contributed! > > Cheers, > Will > > > On Tue, Dec 4, 2012 at 8:37 PM, Mike Percy wrote: > > > Hear, hear! Brock, well done sir, thanks for all your excellent hard work > > on this release! > > > > Regards, > > Mike > > > > > > On Tue, Dec 4, 2012 at 8:30 AM, Jarek Jarcec Cecho > >wrote: > > > > > Thank you Brock for driving this release, you've done excellent job as > a > > > Release manager! > > > > > > Jarcec > > > > > > On Tue, Dec 04, 2012 at 10:13:58AM -0600, Brock Noland wrote: > > > > The Apache Flume team is pleased to announce the release of Flume > > > version 1.3.0. > > > > > > > > Flume is a distributed, reliable, and available service for > efficiently > > > > collecting, aggregating, and moving large amounts of log data. > > > > > > > > This release can be downloaded from the Flume download page at: > > > > http://flume.apache.org/download.html > > > > > > > > The change log and documentation are available on the 1.3.0 release > > page: > > > > http://flume.apache.org/releases/1.3.0.html > > > > > > > > Your help and feedback is more than welcome. For more information on > > how > > > > to report problems and to get involved, visit the project website at > > > > http://flume.apache.org/ > > > > > > > > The Apache Flume Team > > > > > >
Re: [VOTE] Release Apache Flume version 1.3.0 rc5
Thanks for the effort in getting this through Brock. Here is my vote: +1 * Verified all signatures and checksums * Verified the build and tests * Did simple configuration testing with staged binaries Regards, Arvind Prabhakar On Wed, Nov 21, 2012 at 11:07 AM, Brock Noland wrote: > This is the second release for Apache Flume as a top-level project, > version 1.3.0. We are voting on release candidate rc5. > > *** Please cast your vote within the next 72 hours *** > > The list of fixed issues: > http://s.apache.org/RoN > > The tarball (*.tar.gz), signature (*.asc), and checksums (*.md5, *.sha1) > for the source and binary artifacts can be found at: > http://people.apache.org/~brock/flume-1.3.0-rc5/ > > Nexus Staging Repository: > https://repository.apache.org/content/repositories/orgapacheflume-059/ > > The tag to be voted on: > http://s.apache.org/gC > > The KEYS file can be found here: > https://svn.apache.org/repos/asf/flume/dist/KEYS >
Re: https://cwiki.apache.org/confluence/display/FLUME/Index
Hi Alex, I have granted you temporary administrator privileges for both the spaces. Please let me know as soon as you are done with the changes as I will have to revert the privileges back to normal. Regards, Arvind Prabhakar On Thu, Nov 15, 2012 at 10:40 PM, Alexander Alten-Lorenz < wget.n...@gmail.com> wrote: > Hi Arvid, > > Can I please have Space Admin rights for Sqoop and Flume Confluence spaces > to configure the index as well the favicon? > > Thanks, > Alex > > Begin forwarded message: > > > From: Alexander Alten-Lorenz > > Subject: Re: https://cwiki.apache.org/confluence/display/FLUME/Index > > Date: November 15, 2012 10:11:53 AM GMT+01:00 > > To: dev@flume.apache.org > > > > I did some changes today, but I guess confluence has a bug: > > > > https://cwiki.apache.org/FLUME/index.html - the h1. line is missing, > and the layout was bad. I did a ugly hack (adding 5 spaces as a own column). > > https://cwiki.apache.org/confluence/display/FLUME/Index - looks much > better, but will not shown as the index.html > > > > I didn't figured out what the heck is going on there, I guess a bug in > the html exporter (parser)? > > > > Anyway, looks now cleaner and I moved Mike's both article into the blog > section and linked that together to get better search engines results. > > > > Thanks, > > Alex > > > > > > On Nov 15, 2012, at 3:21 AM, Brock Noland wrote: > > > >> Big +1 > >> > >> Thank you very much! > >> > >> On Wed, Nov 14, 2012 at 6:05 PM, Hari Shreedharan > >> wrote: > >>> Excellent work! This was something I heard from many people - the wiki > is the top result if you search for Flume docs, and it pointed to OG stuff. > Thanks to you, now that is taken care of. > >>> > >>> Thanks a lot for this effort! > >>> > >>> > >>> Hari > >>> > >>> -- > >>> Hari Shreedharan > >>> > >>> > >>> On Wednesday, November 14, 2012 at 3:52 PM, Mike Percy wrote: > >>> > >>>> Alex, this looks great! Thanks so much for spending the time to > reorganize > >>>> the Wiki. It is way more useful. > >>>> > >>>> Regards, > >>>> Mike > >>>> > >>>> On Wed, Nov 14, 2012 at 6:27 AM, Alexander Alten-Lorenz < > wget.n...@gmail.com (mailto:wget.n...@gmail.com) > >>>>> wrote: > >>>> > >>>> > >>>>> Guys, > >>>>> > >>>>> I've spent most of my day today to reorganize the wiki, please have > a look > >>>>> and ping me with all stuff you miss or what we should organize > better. > >>>>> > >>>>> I was moving all the OG stuff into a new section, called Flume OG > (pre > >>>>> 1.0), the same I did with all Flume NG stuff. Also I added a > blogpost about > >>>>> flume's memory consumption and will add some from time time with > topics we > >>>>> figured out in our mailing list. Of course, add own stuff too! > >>>>> > >>>>> I did some cosmetic changes too (include our logo as example). > >>>>> > >>>>> best, > >>>>> Alex > >>>>> > >>>>> > >>>>> > >>>>> -- > >>>>> Alexander Alten-Lorenz > >>>>> http://mapredit.blogspot.com > >>>>> German Hadoop LinkedIn Group: http://goo.gl/N8pCF > >>>>> > >>>> > >>>> > >>>> > >>> > >>> > >> > >> > >> > >> -- > >> Apache MRUnit - Unit testing MapReduce - > http://incubator.apache.org/mrunit/ > > > > -- > > Alexander Alten-Lorenz > > http://mapredit.blogspot.com > > German Hadoop LinkedIn Group: http://goo.gl/N8pCF > > > > -- > Alexander Alten-Lorenz > http://mapredit.blogspot.com > German Hadoop LinkedIn Group: http://goo.gl/N8pCF > >
Re: [ANNOUNCE] New Apache Flume committer - Patrick Wendell
Congratulations Patrick! Well deserved! Regards, Arvind Prabhakar On Mon, Nov 12, 2012 at 1:04 PM, Hari Shreedharan wrote: > On behalf of the Apache Flume PMC, I am excited to welcome Patrick > Wendell as a committer on Flume! Patrick has contributed significantly to > the project, by adding new features, fixing bugs and helping users on the > Flume users list. Here is a list of jiras Patrick has worked on: > > http://s.apache.org/6EG > > Please join me in congratulating Patrick on his new role! > > > Thanks, > Hari > > > > >
[jira] [Commented] (FLUME-1502) Support for running simple configurations embedded in host process
[ https://issues.apache.org/jira/browse/FLUME-1502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13491105#comment-13491105 ] Arvind Prabhakar commented on FLUME-1502: - @Brock, thanks for the design document. On the point of File Channel, I do feel that it is important to have that support to ensure that we do not put excessive strain on memory for the host process, and that we do not lose events in the case of host process failure. Another point to consider is whether the source would be any different from a regular source when running in embedded mode. For example, does it make sense to have embedded agent with a network source like Avro working on it? For instance, it may make sense to have no source support, but a direct pass-through for the client API that directly talks with the channel in question. > Support for running simple configurations embedded in host process > -- > > Key: FLUME-1502 > URL: https://issues.apache.org/jira/browse/FLUME-1502 > Project: Flume > Issue Type: Improvement >Affects Versions: v1.2.0 > Reporter: Arvind Prabhakar >Assignee: Brock Noland > Attachments: embeeded-agent-1.pdf > > > Flume should provide a light-weight embeddable node manager that can be > started in process where necessary. This will allow the users to embed > light-weight agents within the host process where necessary. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-1573) Duplicated HDFS file name when multiple SinkRunner was existing
[
https://issues.apache.org/jira/browse/FLUME-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13465681#comment-13465681
]
Arvind Prabhakar commented on FLUME-1573:
-
@Denny - a sink is an independent, isolated component of Flume. It cannot
assume any knowledge of other sink(s) operating within the same agent. Having a
synchronization requirement across multiple sinks breaks this invariant.
However, if within the same sink there are problems due to collisions between
different bucket writers, that would be a bug and merits fixing. From the
explanation above that does not seem to be the case to me.
> Duplicated HDFS file name when multiple SinkRunner was existing
> ---
>
> Key: FLUME-1573
> URL: https://issues.apache.org/jira/browse/FLUME-1573
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
>Affects Versions: v1.2.0
>Reporter: Denny Ye
>Assignee: Denny Ye
> Fix For: v1.3.0
>
> Attachments: FLUME-1573.patch
>
>
> Multiple HDFS Sinks to write events into storage. Timeout exception is always
> happening:
> {code:xml}
> 11 Sep 2012 07:04:53,478 WARN
> [SinkRunner-PollingRunner-DefaultSinkProcessor]
> (org.apache.flume.sink.hdfs.HDFSEventSink.process:442) - HDFS IO error
> java.io.IOException: Callable timed out after 1 ms
> at
> org.apache.flume.sink.hdfs.HDFSEventSink.callWithTimeout(HDFSEventSink.java:342)
> at
> org.apache.flume.sink.hdfs.HDFSEventSink.append(HDFSEventSink.java:713)
> at
> org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:412)
> at
> org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
> at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
> at java.lang.Thread.run(Thread.java:619)
> Caused by: java.util.concurrent.TimeoutException
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:228)
> at java.util.concurrent.FutureTask.get(FutureTask.java:91)
> at
> org.apache.flume.sink.hdfs.HDFSEventSink.callWithTimeout(HDFSEventSink.java:335)
> ... 5 more
> {code}
> I doubted that there might be happened HDFS timeout or slowly response. As
> expected, I found the duplicated creation exception with same with at HDFS.
> Also, Flume recorded same case for duplicated file name.
> {code:xml}
> 13 Sep 2012 02:09:35,432 INFO [hdfs-hdfsSink-3-call-runner-7]
> (org.apache.flume.sink.hdfs.BucketWriter.doOpen:189) - Creating
> /FLUME/dt=2012-09-13/02-host.1347501924111.tmp
> 13 Sep 2012 02:09:36,425 INFO [hdfs-hdfsSink-4-call-runner-8]
> (org.apache.flume.sink.hdfs.BucketWriter.doOpen:189) - Creating
> /FLUME/dt=2012-09-13/02-host.1347501924111.tmp
> {code}
> Different threads were going to create same file without time conflict.
> I found the root cause might be wrong usage the AtomicLong property named
> 'fileExtensionCounter' at BucketWriter. Different threads should own same
> counter by protected with CAS, not multiple private property in each thread.
> It's useless to avoid conflict of HDFS path
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (FLUME-1502) Support for running simple configurations embedded in host process
Arvind Prabhakar created FLUME-1502: --- Summary: Support for running simple configurations embedded in host process Key: FLUME-1502 URL: https://issues.apache.org/jira/browse/FLUME-1502 Project: Flume Issue Type: Improvement Affects Versions: v1.2.0 Reporter: Arvind Prabhakar Flume should provide a light-weight embeddable node manager that can be started in process where necessary. This will allow the users to embed light-weight agents within the host process where necessary. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: FlumeNG & Twitter Storm
Hi, Perhaps an Interceptor [1] may be more appropriate to do this integration. [1] http://flume.apache.org/FlumeUserGuide.html#flume-interceptors Regards, Arvind Prabhakar On Fri, Aug 17, 2012 at 12:53 AM, Mac Fang wrote: > Hi, Team, > > I am trying to combine the Twitter Storm with FlumeNG to achieve the > realtime analytics ability. However, it seems Twitter Storm is appropriate > for 'Queue' based design, since it utilize the 'pull' design to fetch data > and to keep the message reliability. Whereas Sink of FlumeNG is 'pull' data > from 'Channel' and 'push' to others. It seems to be conflicted. Are there > any best practices or any plan for FlumeNG to support Continuous Analytics > or Realtime Analytics. Or I am trying to imlement a configured Spout in > Storm to act as a Sink in Flume, but that looks weird. > > Many thanks. > > > regards > macf >
Flume builds back online
The flume builds were previously disabled due to repository change. Updating the configuration and restricting it to the nodes that have Git support seems to have worked: https://builds.apache.org/job/flume-trunk/281/ I also took the liberty to enabling email notifications but in order to minimize the overall mails generated reduced the frequency to daily instead of the previous hourly frequency. Regards, Arvind Prabhakar
Re: [jira] [Commented] (FLUME-1424) File Channel should support encryption
@Mike - please go ahead. I have done some preliminary analysis but no patch yet. It is similar to what Ralph has suggested below but not based on input/output streams. My findings are that it is possible to use built in JCE supported algorithms such as PBEWithMD5AndDES but have not yet done a performance impact analysis. Regards, Arvind Prabhakar On Fri, Aug 10, 2012 at 3:31 PM, Ralph Goers wrote: > It would be great for you to pick this up. We need this at Intuit asap > and we are planning on having one of our engineers work on it next week if > no one here can do it first. What I had suggested was to have add a > FlumeEventFactory that could be configurable. We could then use an > EncryptedFlumeEventFactory that wraps the OutputStream with a > CipherOutputStream and likewise when reading. Ideally, the key should be > able to be stored either in the default keystore something pluggable. > > Ralph > > On Aug 10, 2012, at 3:22 PM, Mike Percy wrote: > > > @Arvind: JIRA is down but I would like to look @ this issue. Are you > > working on a patch or can I pick this up? > > > > Regards, > > Mike > > > > On Tue, Aug 7, 2012 at 12:37 AM, Arvind Prabhakar (JIRA) < > j...@apache.org>wrote: > > > >> > >>[ > >> > https://issues.apache.org/jira/browse/FLUME-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13429992#comment-13429992 > ] > >> > >> Arvind Prabhakar commented on FLUME-1424: > >> - > >> > >> Yes, the put records do store the data in them. We can perhaps start > with > >> that as a first step and if more requirements pop-up, we can address > them > >> in follow-up Jiras as necessary. > >> > >>> File Channel should support encryption > >>> ------ > >>> > >>>Key: FLUME-1424 > >>>URL: https://issues.apache.org/jira/browse/FLUME-1424 > >>>Project: Flume > >>> Issue Type: Bug > >>> Reporter: Arvind Prabhakar > >>> Assignee: Arvind Prabhakar > >>> > >>> When persisting the data to disk, the File Channel should allow some > >> form of encryption to ensure safety of data. > >> > >> -- > >> This message is automatically generated by JIRA. > >> If you think it was sent incorrectly, please contact your JIRA > >> administrators: > >> > https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa > >> For more information on JIRA, see: > http://www.atlassian.com/software/jira > >> > >> > >> > >
[jira] [Commented] (FLUME-1424) File Channel should support encryption
[ https://issues.apache.org/jira/browse/FLUME-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13429992#comment-13429992 ] Arvind Prabhakar commented on FLUME-1424: - Yes, the put records do store the data in them. We can perhaps start with that as a first step and if more requirements pop-up, we can address them in follow-up Jiras as necessary. > File Channel should support encryption > -- > > Key: FLUME-1424 > URL: https://issues.apache.org/jira/browse/FLUME-1424 > Project: Flume > Issue Type: Bug > Reporter: Arvind Prabhakar > Assignee: Arvind Prabhakar > > When persisting the data to disk, the File Channel should allow some form of > encryption to ensure safety of data. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-1424) File Channel should support encryption
[ https://issues.apache.org/jira/browse/FLUME-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13428457#comment-13428457 ] Arvind Prabhakar commented on FLUME-1424: - @Ralph - this is definitely one way to address this requirement. The advantage (and perhaps a disadvantage at the same time) of this approach is that it will only incorporate encryption for the put records. Another way to do this is to implement encryption at the LogFile.Writer/Reader level where the byte buffers are serialized between transaction boundaries. This approach will have a higher performance penalty but would encrypt every file channel record regardless of type. > File Channel should support encryption > -- > > Key: FLUME-1424 > URL: https://issues.apache.org/jira/browse/FLUME-1424 > Project: Flume > Issue Type: Bug > Reporter: Arvind Prabhakar > Assignee: Arvind Prabhakar > > When persisting the data to disk, the File Channel should allow some form of > encryption to ensure safety of data. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Sending avro data from other languages
Another alternative to consider for cross-platform/language support would be protocol buffers. That has relatively better tooling and integration than other similar systems and is used by other projects as well. Regards, Arvind Prabhakar On Thu, Aug 2, 2012 at 7:01 AM, Brock Noland wrote: > I cannot answer what made us move to Avro. However, I prefer Avro because > you don't have to build the thrift compiler and you aren't required to do > code generation. > > On Wed, Aug 1, 2012 at 11:06 PM, Juhani Connolly < > juhani_conno...@cyberagent.co.jp> wrote: > > > It looks to me like this was because of the transceiver I was using. > > > > Unfortunately it seems like avro doesn't have a python implementation of > a > > transceiver that fits the format expected by netty/avro(in fact it only > has > > one transceiver... HTTPTransceiver). > > > > To address this, I'm thinking of putting together a thrift source(the > > legacy source doesn't seem to be usable as it returns nothing, and lacks > > batching). Does this seem like a reasonable solution to making it > possible > > to send data to flume from other languages(and allowing backoff on > > failure?). Historically, what made us move away from thrift to avro? > > > > > > On 07/30/2012 05:34 PM, Juhani Connolly wrote: > > > >> I'm playing around with making a standalone tail client in python(so > that > >> I can access inode data) that tracks position in a file and then sends > it > >> across avro to an avro sink. > >> > >> However I'm having issues with the avro part of this and wondering if > >> anyone more familiar with it could help. > >> > >> I took the flume.avdl file and converted it using "java -jar > >> ~/Downloads/avro-tools-1.6.3.**jar idl flume.avdl flume.avpr" > >> > >> I then run it through a simple test program to see if its sending the > >> data correctly and it sends from the python client fine, but the sink > end > >> OOM's because presumably the wire format is wrong: > >> > >> 2012-07-30 17:22:57,565 INFO ipc.NettyServer: [id: 0x5fc6e818, / > >> 172.22.114.32:55671 => /172.28.19.112:41414] OPEN > >> 2012-07-30 17:22:57,565 INFO ipc.NettyServer: [id: 0x5fc6e818, / > >> 172.22.114.32:55671 => /172.28.19.112:41414] BOUND: / > 172.28.19.112:41414 > >> 2012-07-30 17:22:57,565 INFO ipc.NettyServer: [id: 0x5fc6e818, / > >> 172.22.114.32:55671 => /172.28.19.112:41414] CONNECTED: / > >> 172.22.114.32:55671 > >> 2012-07-30 17:22:57,646 WARN ipc.NettyServer: Unexpected exception from > >> downstream. > >> java.lang.OutOfMemoryError: Java heap space > >> at java.util.ArrayList.(**ArrayList.java:112) > >> at > org.apache.avro.ipc.**NettyTransportCodec$**NettyFrameDecoder. > >> **decodePackHeader(**NettyTransportCodec.java:154) > >> at org.apache.avro.ipc.**NettyTransportCodec$** > >> NettyFrameDecoder.decode(**NettyTransportCodec.java:131) > >> at > org.jboss.netty.handler.codec.**frame.FrameDecoder.callDecode( > >> **FrameDecoder.java:282) > >> at org.jboss.netty.handler.codec.**frame.FrameDecoder.** > >> messageReceived(FrameDecoder.**java:216) > >> at org.jboss.netty.channel.**Channels.fireMessageReceived(** > >> Channels.java:274) > >> at org.jboss.netty.channel.**Channels.fireMessageReceived(** > >> Channels.java:261) > >> at org.jboss.netty.channel.**socket.nio.NioWorker.read(** > >> NioWorker.java:351) > >> at org.jboss.netty.channel.**socket.nio.NioWorker.** > >> processSelectedKeys(NioWorker.**java:282) > >> at org.jboss.netty.channel.**socket.nio.NioWorker.run(** > >> NioWorker.java:202) > >> at java.util.concurrent.**ThreadPoolExecutor$Worker.** > >> runTask(ThreadPoolExecutor.**java:886) > >> at java.util.concurrent.**ThreadPoolExecutor$Worker.run(** > >> ThreadPoolExecutor.java:908) > >> at java.lang.Thread.run(Thread.**java:619) > >> 2012-07-30 17:22:57,647 INFO ipc.NettyServer: [id: 0x5fc6e818, / > >> 172.22.114.32:55671 :> /172.28.19.112:41414] DISCONNECTED > >> 2012-07-30 17:22:57,647 INFO ipc.NettyServer: [id: 0x5fc6e818, / > >> 172.22.114.32:55671 :> /172.28.19.112:41414] UNBOUND > >> 2012-07-30 17:22:57,647 INFO ipc.NettyServer: [id: 0x5fc6e818, / > >> 172.22.114.32:55671 :> /172.28.19.112:41414] CLOSED > >> > >> I've dumped the test program and its output > >> > >> http://pastebin.com/1DtXZyTu > >> http://pastebin.com/T9kaqKHY > >> > >> > > > > > -- > Apache MRUnit - Unit testing MapReduce - > http://incubator.apache.org/mrunit/ >
[jira] [Assigned] (FLUME-1424) File Channel should support encryption
[ https://issues.apache.org/jira/browse/FLUME-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arvind Prabhakar reassigned FLUME-1424: --- Assignee: Arvind Prabhakar > File Channel should support encryption > -- > > Key: FLUME-1424 > URL: https://issues.apache.org/jira/browse/FLUME-1424 > Project: Flume > Issue Type: Bug > Reporter: Arvind Prabhakar > Assignee: Arvind Prabhakar > > When persisting the data to disk, the File Channel should allow some form of > encryption to ensure safety of data. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (FLUME-1424) File Channel should support encryption
Arvind Prabhakar created FLUME-1424: --- Summary: File Channel should support encryption Key: FLUME-1424 URL: https://issues.apache.org/jira/browse/FLUME-1424 Project: Flume Issue Type: Bug Reporter: Arvind Prabhakar When persisting the data to disk, the File Channel should allow some form of encryption to ensure safety of data. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Please welcome Mubarak Seyed, our newest Flume committer
Congratulations Mubarak! Regards, Arvind Prabhakar On Tue, Jul 31, 2012 at 10:42 AM, Hari Shreedharan < hshreedha...@cloudera.com> wrote: > Welcome aboard, Mubarak! > > > Hari > > -- > Hari Shreedharan > > > On Tuesday, July 31, 2012 at 10:29 AM, Harish Mandala wrote: > > > Welcome, Mubarak! > > > > On Tue, Jul 31, 2012 at 1:28 PM, Ralph Goers > > ralph.go...@dslextreme.com)>wrote: > > > > > Welcome! > > > > > > Ralph > > > > > > On Jul 30, 2012, at 11:06 PM, Mingjie Lai wrote: > > > > > > > Hi Flume devs. > > > > > > > > Please welcome Mubarak Seyed, our newest Flume committer. He's been > > > doing great contributions to Flume project during the past several > months. > > > > > > > > Welcome aboard. > > > > > > > > Thanks, > > > > Mingjie > > > > > > > > > > > > > > > > > > >
[jira] [Updated] (FLUME-1380) File channel log can record the op code and not the operation in some cases
[
https://issues.apache.org/jira/browse/FLUME-1380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arvind Prabhakar updated FLUME-1380:
Attachment: FLUME-1380-1.patch
> File channel log can record the op code and not the operation in some cases
> ---
>
> Key: FLUME-1380
> URL: https://issues.apache.org/jira/browse/FLUME-1380
> Project: Flume
> Issue Type: Bug
> Reporter: Arvind Prabhakar
> Assignee: Arvind Prabhakar
> Attachments: FLUME-1380-1.patch
>
>
> There is a race condition in the system where the log file can record the
> beginning of a record and be shutdown before the remaining record is written
> out. This will lead to the system not starting up correctly again with
> exceptions like:
> {noformat}
> ERROR file.Log: Failed to initialize Log
> java.io.IOException: Header 80808080 not expected value: deadbeef
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
Review Request: FLUME-1380. File channel log write consistency
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/6038/ --- Review request for Flume. Description --- Fixed the log file writer to make a single call to the underlying channel in order to serialize the event. This addresses bug FLUME-1380. https://issues.apache.org/jira/browse/FLUME-1380 Diffs - /trunk/flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/LogFile.java 1363086 Diff: https://reviews.apache.org/r/6038/diff/ Testing --- Ran all tests. Manually populated and flushed a file channel. Thanks, Arvind Prabhakar
[jira] [Created] (FLUME-1380) File channel log can record the op code and not the operation in some cases
Arvind Prabhakar created FLUME-1380:
---
Summary: File channel log can record the op code and not the
operation in some cases
Key: FLUME-1380
URL: https://issues.apache.org/jira/browse/FLUME-1380
Project: Flume
Issue Type: Bug
Reporter: Arvind Prabhakar
Assignee: Arvind Prabhakar
There is a race condition in the system where the log file can record the
beginning of a record and be shutdown before the remaining record is written
out. This will lead to the system not starting up correctly again with
exceptions like:
{noformat}
ERROR file.Log: Failed to initialize Log
java.io.IOException: Header 80808080 not expected value: deadbeef
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [DISCUSS] Git as primary source control for Flume
+1 for using Git as primary source control system. Thanks Hari for following up on this. Regards, Arvind Prabhakar On Wed, Jul 11, 2012 at 7:16 PM, Leslin wrote: > +1 for this proposal. Git is fine for me. I never back to SVN after I > touched git. > > 2012/7/12 Mike Percy > > > On Wed, Jul 11, 2012 at 5:45 PM, Ralph Goers > >wrote: > > > > > IMO the person who wrote the code is the one who should get credit. > > > > > > > Of course they should get the credit for the work. > > > > Anyone who has ever performed a careful code review knows that it can be > > time-consuming work. I assume that's one reason why we currently list > both > > the author and the committer in the commit message. > > > > Regards, > > Mike > > > > > > -- > > > > Best Regards > > Leslin >
Re: [VOTE] Release Apache Flume version 1.2.0 (rc1)
+1 * Binary and Source distributions checksums and signatures match * LICENSE file accounts for all included Jars in the binary distribution * Sources build and test fine. * Top level files all look good * Jira is clean One slight concern (not a blocker): the tag contains sources in contrib that are not included in the source tar-ball. Since these sources are not used for build, we can do without those for now. Thanks for your hard work Mike! Regards, Arvind Prabhakar On Wed, Jul 11, 2012 at 4:57 AM, Mike Percy wrote: > This is the first release for Apache Flume as a top-level project, > version 1.2.0. We are voting on release candidate rc1. > > *** Please cast your vote within the next 72 hours *** > > The list of fixed issues: > https://svn.apache.org/repos/asf/flume/tags/flume-1.2.0-rc1/CHANGELOG > > The tarball (*.tar.gz), signature (*.asc), and checksums (*.md5, *.sha1) > for the source and binary artifacts can be found at: > https://people.apache.org/~mpercy/flume/apache-flume-1.2.0-rc1/ > > The tag to be voted on: > https://svn.apache.org/repos/asf/flume/tags/flume-1.2.0-rc1 > > The KEYS file can be found here: > https://svn.apache.org/repos/asf/flume/dist/KEYS > > Changes since rc0: > - Updated LICENSE file > - Updated DEVNOTES file > - Removed DISCLAIMER file from dist.xml and src.xml manifests > - pom.xml file updated with TLP info (FLUME-1359) > - A build fix to prevent multiple servlet-api jars in lib dir >
Re: Review Request: Intital version of the Flume web site.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5765/#review9130 --- Ship it! Ship It! - Arvind Prabhakar On July 8, 2012, 7:36 p.m., Ralph Goers wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/5765/ > --- > > (Updated July 8, 2012, 7:36 p.m.) > > > Review request for Flume. > > > Description > --- > > This contains the source to build the initial version of the web site using > the CMS, Maven and Sphinx. To test the build just run mvn site and the > output of the site will be in target/site. "mvn -P pdf package" is supposed > to package the users and developers guides as pdf's to be deployed to the > site as part of a release but that isn't quite working yet. > > A few notes: > 1. The site will be committed to > https://svn.apache.org/repos/asf/flume/site/trunk. > 2. The site is incomplete in that it is missing release information. This > will be directly added once the site is published to the production svn > location. > > > This addresses bug FLUME-813. > https://issues.apache.org/jira/browse/FLUME-813 > > > Diffs > - > > > Diff: https://reviews.apache.org/r/5765/diff/ > > > Testing > --- > > > Thanks, > > Ralph Goers > >
Re: Review Request: Intital version of the Flume web site.
> On July 13, 2012, 4:12 a.m., Mike Percy wrote: > > +1 > > > > I would like to see us release 1.2.0 from a TLP subdomain. Ralph since you > > have revamped the site, I am also preemptively +1 on any fixes to the CMS > > push procedure since that is hard to test. Let's get this checked in. > > Ralph Goers wrote: > Thanks Mike. However, I've been incorporating some of Arvind's > suggestions. I have one more change to make and then I'll repost it here > again. Unfortunately, that will probably be on Saturday as I have to fly to > Mountain View tomorrow for the day. > > Mike Percy wrote: > Let's post the initial version now and follow up with enhancements. We > need to unblock Joe S. on the INFRA ticket. Thanks for following up Ralph! I am perfectly happy with making the changes post-checkin. Will +1 to unblock. Regards, Arvind Prabhakar - Arvind --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5765/#review9126 --- On July 8, 2012, 7:36 p.m., Ralph Goers wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/5765/ > --- > > (Updated July 8, 2012, 7:36 p.m.) > > > Review request for Flume. > > > Description > --- > > This contains the source to build the initial version of the web site using > the CMS, Maven and Sphinx. To test the build just run mvn site and the > output of the site will be in target/site. "mvn -P pdf package" is supposed > to package the users and developers guides as pdf's to be deployed to the > site as part of a release but that isn't quite working yet. > > A few notes: > 1. The site will be committed to > https://svn.apache.org/repos/asf/flume/site/trunk. > 2. The site is incomplete in that it is missing release information. This > will be directly added once the site is published to the production svn > location. > > > This addresses bug FLUME-813. > https://issues.apache.org/jira/browse/FLUME-813 > > > Diffs > - > > > Diff: https://reviews.apache.org/r/5765/diff/ > > > Testing > --- > > > Thanks, > > Ralph Goers > >
Re: Review Request: Intital version of the Flume web site.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5765/#review9012 --- Thanks for the site Ralph. Overall the site looks great for a first cut. Some feedback for your consideration: * The license link on the side displays the license header. It should instead display or link to the normative AL v2.0 * Just like the Releases on the left side, the Documentation should also expand to itemize the releases and provide links to the static release documents. * The "How to get involved" details currently show up underneath the release 1.2.0 menu item on the left. Considering this generally applies to the entire project, it merits its own menu heading with necessary details inside. * The developer access under source repository mentions log4j instead of flume ($ svn checkout https://svn.apache.org/repos/asf/flume/trunk log4j) * Although the colors are bright and of high contrast, I feel it may be better to tone them down and use more of white/gray or other milder shades if possible. Regards, Arvind Prabhakar - Arvind Prabhakar On July 8, 2012, 7:36 p.m., Ralph Goers wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/5765/ > --- > > (Updated July 8, 2012, 7:36 p.m.) > > > Review request for Flume. > > > Description > --- > > This contains the source to build the initial version of the web site using > the CMS, Maven and Sphinx. To test the build just run mvn site and the > output of the site will be in target/site. "mvn -P pdf package" is supposed > to package the users and developers guides as pdf's to be deployed to the > site as part of a release but that isn't quite working yet. > > A few notes: > 1. The site will be committed to > https://svn.apache.org/repos/asf/flume/site/trunk. > 2. The site is incomplete in that it is missing release information. This > will be directly added once the site is published to the production svn > location. > > > This addresses bug FLUME-813. > https://issues.apache.org/jira/browse/FLUME-813 > > > Diffs > - > > > Diff: https://reviews.apache.org/r/5765/diff/ > > > Testing > --- > > > Thanks, > > Ralph Goers > >
IMPORTANT: Project Infrastructure Changes
As some of you already know, the project SVN repository and mailing lists have been moved to the top-level project. Some details on these changes and what you need to do to keep up are outlined below: SUBVERSION The subversion repository for Flume has moved from within incubator to its own dedicated root. This requires that you either update your existing repository to match the new subversion repository path, or checkout a fresh image. New Subversion Repo: https://svn.apache.org/repos/asf/flume/trunk To update your existing workspace to point to this repository, backup any changes that you have, and then execute the switch command as follows from within the root of your workspace: $ svn switch https://svn.apache.org/repos/asf/flume/trunk This command will update your repository to point to the new location. REVIEW BOARD The review board has been updated to use the new subversion configuration. As a result reviews that were in progress will have to be discarded and submitted again. If you have any pending reviews, please resubmit the patch generated from the updated repository root. Similarly, the previously submitted reviews may not be accessible anymore. MAILING LISTS The mailing lists have moved from their previous location to their new location. While the old mailing lists will continue to redirect to the new lists, it is better if you update your mail clients to use the new lists instead. This includes updating any filters etc that you may be using to sort through the list traffic. The specifics of mailing lists migration are as follows: flume-...@incubator.apache.org --> dev@flume.apache.org flume-u...@incubator.apache.org --> u...@flume.apache.org flume-comm...@incubator.apache.org --> comm...@flume.apache.org WEBSITE During incubation, the project website was hosted at http://incubator.apache.org/flume. The new website address will be http://flume.apache.org/. It will take a little while before the website is migrated completely and the work is currently underway. Once the migration is complete, the old website location will automatically redirect to the new location. If you have any issues due to this migration, please let me know. Regards, Arvind Prabhakar
Re: test 2
Jira notifications have been updated. https://issues.apache.org/jira/browse/INFRA-4985 Regards, Arvind Prabhakar On Sat, Jul 7, 2012 at 11:19 PM, Ralph Goers wrote: > I've already modified mine. Jira is still sending to incubator. > > Ralph > > On Jul 7, 2012, at 11:14 PM, Mike Percy wrote: > > > Yeah, time to change the old email filters :) > > > > On Sat, Jul 7, 2012 at 11:12 PM, Jarek Jarcec Cecho >wrote: > > > >> Seems as a dev@ > >> > >> Jarcec > >> > >> On Jul 8, 2012, at 8:10 AM, Mike Percy wrote: > >> > >>> Wondering which alias this appears from when I send to > >> flume-dev@incubator > >> > >> > >
Re: New SVN location for flume
I just updated the authorization file for the new subversion repo. Can you please try and check if this resolves the issue? Regards, Arvind Prabhakar On Sun, Jul 8, 2012 at 11:14 AM, Jarek Jarcec Cecho wrote: > Thank you Hari for your feedback, I've filled INFRA-5022 to get it > resolved. > > Jarcec > > https://issues.apache.org/jira/browse/INFRA-5022 > > On Sun, Jul 08, 2012 at 10:38:54AM -0700, Hari Shreedharan wrote: > > Same here. Permissions issue. > > > > -- > > Hari Shreedharan > > > > > > On Sunday, July 8, 2012 at 10:17 AM, Jarek Jarcec Cecho wrote: > > > > > Hi guys, > > > I've tried to commit FLUME-1348 today to our new SVN location on > https://svn.apache.org/repos/asf/flume/. Unfortunately, I have failed on > permission issue. Can anyone else try test commit just to see whether it's > my local issue or it's also affecting anyone else? > > > > > > Jarcec > > >
