[jira] [Commented] (FLUME-2994) flume-taildir-source: support for windows

2016-10-24 Thread Kamlesh Pandey (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLUME-2994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15603197#comment-15603197
 ] 

Kamlesh Pandey commented on FLUME-2994:
---

[~jkushmaul] This patch works fine for tailing a file on windows. But  I 
observed that when I set an application log file for the taildir source it does 
not perform the rotation as per the log4j configuration. In log4j config I gave 
the max file size as 100KB but the log file kept growing beyond 100KB.

> flume-taildir-source: support for windows
> -
>
> Key: FLUME-2994
> URL: https://issues.apache.org/jira/browse/FLUME-2994
> Project: Flume
>  Issue Type: Improvement
>  Components: Sinks+Sources, Windows
>Affects Versions: v1.7.0
>Reporter: Jason Kushmaul
>Assignee: Jason Kushmaul
>Priority: Trivial
> Fix For: v1.8.0
>
> Attachments: FLUME-2994-2.patch, taildir-mac.conf, taildir-win8.1.conf
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> The current implementation of flume-taildir-source does not support windows.
> The only reason for this from what I can see is a simple call to 
> Files.getAttribute(file.toPath(), "unix:ino");
> I've tested an equivalent for windows (which of course does not work on 
> non-windows).  With an OS switch we should be able to identify a file 
> independent of file name on either system.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Moving Developer Section of cwiki to the git repository

2016-10-24 Thread Mike Percy 
I don't think it would pollute the git log. I also don't think spamming the
dev list with PRs is a big deal... I use GMail filters so I can see the
human traffic... but maybe not everyone does that.

Anyway it's up to you how you want to organize it. :)

Mike

On Mon, Oct 24, 2016 at 4:39 PM, Balazs Donat Bessenyei  wrote:

> Thank you.
>
> Wouldn't committing (and PRing) them one-by-one pollute the git log and
> mailing lists too much?
>
> On Oct 24, 2016 5:26 PM, "Mike Percy"  wrote:
>
> > Feel free to do it per-file if you want. Looks good to me
> >
> > On Mon, Oct 24, 2016 at 4:14 PM, Balazs Donat Bessenyei <
> > bes...@cloudera.com
> > > wrote:
> >
> > > I have commenced moving the Developer Section to the git repo.
> > >
> > > There is a WIP pull request available at https://github.com/apache/
> > > flume/pull/77
> > >
> > > I'm planning to add more e-mail templates to the HowToRelease.md and
> > > commit all docs in a single commit, but reviews and comments are
> > > welcome in the meanwhile.
> > >
> > >
> > > Thank you,
> > >
> > > Donat
> > >
> > > On Fri, Oct 21, 2016 at 6:07 PM, Mike Percy  wrote:
> > > > Thanks, Donat. +1 on moving the docs.
> > > >
> > > > Regarding the web site source code, it can reside in svn or git
> however
> > > it
> > > > should continue using the standard ASF site infrastructure [1] for
> > > hosting
> > > > content which consists only of static rendered HTML content. Our
> > options
> > > > for pushing that rendered HTML are either svnpubsub[2] or
> gitpubsub[3]
> > > > which are basically the same thing - commit HTML and it gets pushed
> > live
> > > to
> > > > the web site.
> > > >
> > > > If someone wants to redesign the site or improve the site template, I
> > > would
> > > > certainly welcome that.
> > > >
> > > > As a comparison point with another ASF project I am involved in, on
> > > Apache
> > > > Kudu we use the software used for GitHub pages (Jekyll) to render the
> > > site.
> > > > The web site source code is in the gh-pages branch in Git [4]. The
> site
> > > is
> > > > mostly written in Markdown and we wrote a script [5] that locally
> > renders
> > > > the site to static HTML content and checks in the rendered content to
> > the
> > > > Kudu gitpubsub repository [6].
> > > >
> > > > In Flume, we basically do the same thing but but instead of Jekyll we
> > use
> > > > Sphinx and the site source code is primarily ReStructuredText. The
> site
> > > > lives in its own svn repo [7]. We push rendered HTML tent to an
> > svnpubsub
> > > > repository for Flume [8] which is controlled by the ASF CMS [9]...
> > IIRC,
> > > > the CMS takes care of invoking the rendering part instead of having a
> > > local
> > > > script to do it. This is kind of documented in the How to Release
> guide
> > > for
> > > > Flume [10].
> > > >
> > > > If someone just wanted to change the look & feel of the web site they
> > > could
> > > > easily do it with a little HTML/CSS knowledge and some reading up on
> > > > Sphinx. The web site template could be changed to something nicer
> (it's
> > > > using some kind of stock template right now). The existing .rst
> content
> > > > would not need to be modified at all, probably.
> > > >
> > > > If someone wanted to switch the whole web site to some other system,
> > like
> > > > Jekyll, obviously it's a bigger change but if they were determined
> > then I
> > > > don't think anyone would try to stop them, assuming it's an
> > improvement!
> > > >
> > > > Hope this helps,
> > > > Mike
> > > >
> > > > [1] https://www.apache.org/dev/project-site.html
> > > > [2] http://svn.apache.org/viewvc/subversion/trunk/tools/
> > > > server-side/svnpubsub/
> > > > [3] https://www.apache.org/dev/gitpubsub.html
> > > > [4] https://github.com/apache/kudu/tree/gh-pages
> > > > [5] https://github.com/apache/kudu/blob/gh-pages/site_tool
> > > > [6] https://git-wip-us.apache.org/repos/asf?p=kudu-site.git;a=
> > > > shortlog;h=refs/heads/asf-site
> > > > [7] https://svn.apache.org/repos/asf/flume/site/trunk/
> > > > [8] https://svn.apache.org/repos/infra/websites/production/flume/
> > > > [9] https://www.apache.org/dev/cms.html
> > > > [10]
> > > > https://cwiki.apache.org/confluence/display/FLUME/How+
> > > to+Release#HowtoRelease-Updatethewebsite
> > > >
> > > > On Thu, Oct 20, 2016 at 2:59 PM, Balazs Donat Bessenyei <
> > > bes...@cloudera.com
> > > >> wrote:
> > > >
> > > >> As nobody has objected in ~48 hours, I'll open a pull request about
> > this
> > > >> soon.
> > > >>
> > > >> Regarding the whole site thing: AFAIK, the site contents have to
> > > >> reside in SVN, because that's how ASF infrastructure works now.
> > > >>
> > > >> First, I'll work on moving the Developer Section to
> > > >> https://github.com/apache/flume , then we'll see what else we can
> do.
> > > >>
> > > >>
> > > >> Thank you,
> > > >>
> > > >> Donat
> > > >>
> > > >> On Thu, Oct 20, 2016 at 12:41 PM, UMESH CHAUDHARY <
>

[GitHub] flume pull request #77: [WIP] Add Developer Section from cwiki.apache.org

2016-10-24 Thread bessbd 
GitHub user bessbd opened a pull request:

https://github.com/apache/flume/pull/77

[WIP] Add Developer Section from cwiki.apache.org



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/bessbd/flume devdocs

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flume/pull/77.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #77


commit 8b894f39b0b86570b6cf24272e06311a539f9b95
Author: Bessenyei Balázs Donát 
Date:   2016-10-24T14:53:29Z

Add Developer Section / How to Release from cwiki.apache.org




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Moving Developer Section of cwiki to the git repository

2016-10-24 Thread Balazs Donat Bessenyei 
Thank you.

Wouldn't committing (and PRing) them one-by-one pollute the git log and
mailing lists too much?

On Oct 24, 2016 5:26 PM, "Mike Percy"  wrote:

> Feel free to do it per-file if you want. Looks good to me
>
> On Mon, Oct 24, 2016 at 4:14 PM, Balazs Donat Bessenyei <
> bes...@cloudera.com
> > wrote:
>
> > I have commenced moving the Developer Section to the git repo.
> >
> > There is a WIP pull request available at https://github.com/apache/
> > flume/pull/77
> >
> > I'm planning to add more e-mail templates to the HowToRelease.md and
> > commit all docs in a single commit, but reviews and comments are
> > welcome in the meanwhile.
> >
> >
> > Thank you,
> >
> > Donat
> >
> > On Fri, Oct 21, 2016 at 6:07 PM, Mike Percy  wrote:
> > > Thanks, Donat. +1 on moving the docs.
> > >
> > > Regarding the web site source code, it can reside in svn or git however
> > it
> > > should continue using the standard ASF site infrastructure [1] for
> > hosting
> > > content which consists only of static rendered HTML content. Our
> options
> > > for pushing that rendered HTML are either svnpubsub[2] or gitpubsub[3]
> > > which are basically the same thing - commit HTML and it gets pushed
> live
> > to
> > > the web site.
> > >
> > > If someone wants to redesign the site or improve the site template, I
> > would
> > > certainly welcome that.
> > >
> > > As a comparison point with another ASF project I am involved in, on
> > Apache
> > > Kudu we use the software used for GitHub pages (Jekyll) to render the
> > site.
> > > The web site source code is in the gh-pages branch in Git [4]. The site
> > is
> > > mostly written in Markdown and we wrote a script [5] that locally
> renders
> > > the site to static HTML content and checks in the rendered content to
> the
> > > Kudu gitpubsub repository [6].
> > >
> > > In Flume, we basically do the same thing but but instead of Jekyll we
> use
> > > Sphinx and the site source code is primarily ReStructuredText. The site
> > > lives in its own svn repo [7]. We push rendered HTML tent to an
> svnpubsub
> > > repository for Flume [8] which is controlled by the ASF CMS [9]...
> IIRC,
> > > the CMS takes care of invoking the rendering part instead of having a
> > local
> > > script to do it. This is kind of documented in the How to Release guide
> > for
> > > Flume [10].
> > >
> > > If someone just wanted to change the look & feel of the web site they
> > could
> > > easily do it with a little HTML/CSS knowledge and some reading up on
> > > Sphinx. The web site template could be changed to something nicer (it's
> > > using some kind of stock template right now). The existing .rst content
> > > would not need to be modified at all, probably.
> > >
> > > If someone wanted to switch the whole web site to some other system,
> like
> > > Jekyll, obviously it's a bigger change but if they were determined
> then I
> > > don't think anyone would try to stop them, assuming it's an
> improvement!
> > >
> > > Hope this helps,
> > > Mike
> > >
> > > [1] https://www.apache.org/dev/project-site.html
> > > [2] http://svn.apache.org/viewvc/subversion/trunk/tools/
> > > server-side/svnpubsub/
> > > [3] https://www.apache.org/dev/gitpubsub.html
> > > [4] https://github.com/apache/kudu/tree/gh-pages
> > > [5] https://github.com/apache/kudu/blob/gh-pages/site_tool
> > > [6] https://git-wip-us.apache.org/repos/asf?p=kudu-site.git;a=
> > > shortlog;h=refs/heads/asf-site
> > > [7] https://svn.apache.org/repos/asf/flume/site/trunk/
> > > [8] https://svn.apache.org/repos/infra/websites/production/flume/
> > > [9] https://www.apache.org/dev/cms.html
> > > [10]
> > > https://cwiki.apache.org/confluence/display/FLUME/How+
> > to+Release#HowtoRelease-Updatethewebsite
> > >
> > > On Thu, Oct 20, 2016 at 2:59 PM, Balazs Donat Bessenyei <
> > bes...@cloudera.com
> > >> wrote:
> > >
> > >> As nobody has objected in ~48 hours, I'll open a pull request about
> this
> > >> soon.
> > >>
> > >> Regarding the whole site thing: AFAIK, the site contents have to
> > >> reside in SVN, because that's how ASF infrastructure works now.
> > >>
> > >> First, I'll work on moving the Developer Section to
> > >> https://github.com/apache/flume , then we'll see what else we can do.
> > >>
> > >>
> > >> Thank you,
> > >>
> > >> Donat
> > >>
> > >> On Thu, Oct 20, 2016 at 12:41 PM, UMESH CHAUDHARY <
> umesh9...@gmail.com>
> > >> wrote:
> > >> > +1 for moving wiki and web contents into GitHub. It would be easier
> to
> > >> > manage. Also, +1 for improving our website.
> > >> >
> > >> > On Thu, 20 Oct 2016 at 15:42 Lior Zeno  wrote:
> > >> >
> > >> > +1 for moving the whole site as well. I wish we could improve our
> > >> frontend
> > >> > to become a bit more appealing.
> > >> >
> > >> > On Thu, Oct 20, 2016 at 11:53 AM, Attila Simon 
> > >> wrote:
> > >> >
> > >> >> +1
> > >> >> Essentially moving whole site (all web content) to scm would help
> > >> >>

Re: Moving Developer Section of cwiki to the git repository

2016-10-24 Thread Mike Percy 
Feel free to do it per-file if you want. Looks good to me

On Mon, Oct 24, 2016 at 4:14 PM, Balazs Donat Bessenyei  wrote:

> I have commenced moving the Developer Section to the git repo.
>
> There is a WIP pull request available at https://github.com/apache/
> flume/pull/77
>
> I'm planning to add more e-mail templates to the HowToRelease.md and
> commit all docs in a single commit, but reviews and comments are
> welcome in the meanwhile.
>
>
> Thank you,
>
> Donat
>
> On Fri, Oct 21, 2016 at 6:07 PM, Mike Percy  wrote:
> > Thanks, Donat. +1 on moving the docs.
> >
> > Regarding the web site source code, it can reside in svn or git however
> it
> > should continue using the standard ASF site infrastructure [1] for
> hosting
> > content which consists only of static rendered HTML content. Our options
> > for pushing that rendered HTML are either svnpubsub[2] or gitpubsub[3]
> > which are basically the same thing - commit HTML and it gets pushed live
> to
> > the web site.
> >
> > If someone wants to redesign the site or improve the site template, I
> would
> > certainly welcome that.
> >
> > As a comparison point with another ASF project I am involved in, on
> Apache
> > Kudu we use the software used for GitHub pages (Jekyll) to render the
> site.
> > The web site source code is in the gh-pages branch in Git [4]. The site
> is
> > mostly written in Markdown and we wrote a script [5] that locally renders
> > the site to static HTML content and checks in the rendered content to the
> > Kudu gitpubsub repository [6].
> >
> > In Flume, we basically do the same thing but but instead of Jekyll we use
> > Sphinx and the site source code is primarily ReStructuredText. The site
> > lives in its own svn repo [7]. We push rendered HTML tent to an svnpubsub
> > repository for Flume [8] which is controlled by the ASF CMS [9]... IIRC,
> > the CMS takes care of invoking the rendering part instead of having a
> local
> > script to do it. This is kind of documented in the How to Release guide
> for
> > Flume [10].
> >
> > If someone just wanted to change the look & feel of the web site they
> could
> > easily do it with a little HTML/CSS knowledge and some reading up on
> > Sphinx. The web site template could be changed to something nicer (it's
> > using some kind of stock template right now). The existing .rst content
> > would not need to be modified at all, probably.
> >
> > If someone wanted to switch the whole web site to some other system, like
> > Jekyll, obviously it's a bigger change but if they were determined then I
> > don't think anyone would try to stop them, assuming it's an improvement!
> >
> > Hope this helps,
> > Mike
> >
> > [1] https://www.apache.org/dev/project-site.html
> > [2] http://svn.apache.org/viewvc/subversion/trunk/tools/
> > server-side/svnpubsub/
> > [3] https://www.apache.org/dev/gitpubsub.html
> > [4] https://github.com/apache/kudu/tree/gh-pages
> > [5] https://github.com/apache/kudu/blob/gh-pages/site_tool
> > [6] https://git-wip-us.apache.org/repos/asf?p=kudu-site.git;a=
> > shortlog;h=refs/heads/asf-site
> > [7] https://svn.apache.org/repos/asf/flume/site/trunk/
> > [8] https://svn.apache.org/repos/infra/websites/production/flume/
> > [9] https://www.apache.org/dev/cms.html
> > [10]
> > https://cwiki.apache.org/confluence/display/FLUME/How+
> to+Release#HowtoRelease-Updatethewebsite
> >
> > On Thu, Oct 20, 2016 at 2:59 PM, Balazs Donat Bessenyei <
> bes...@cloudera.com
> >> wrote:
> >
> >> As nobody has objected in ~48 hours, I'll open a pull request about this
> >> soon.
> >>
> >> Regarding the whole site thing: AFAIK, the site contents have to
> >> reside in SVN, because that's how ASF infrastructure works now.
> >>
> >> First, I'll work on moving the Developer Section to
> >> https://github.com/apache/flume , then we'll see what else we can do.
> >>
> >>
> >> Thank you,
> >>
> >> Donat
> >>
> >> On Thu, Oct 20, 2016 at 12:41 PM, UMESH CHAUDHARY 
> >> wrote:
> >> > +1 for moving wiki and web contents into GitHub. It would be easier to
> >> > manage. Also, +1 for improving our website.
> >> >
> >> > On Thu, 20 Oct 2016 at 15:42 Lior Zeno  wrote:
> >> >
> >> > +1 for moving the whole site as well. I wish we could improve our
> >> frontend
> >> > to become a bit more appealing.
> >> >
> >> > On Thu, Oct 20, 2016 at 11:53 AM, Attila Simon 
> >> wrote:
> >> >
> >> >> +1
> >> >> Essentially moving whole site (all web content) to scm would help
> >> >> contribution.
> >> >>
> >> >> Cheers,
> >> >> Attila
> >> >>
> >> >> On Tuesday, 18 October 2016, Balazs Donat Bessenyei <
> >> bes...@cloudera.com>
> >> >> wrote:
> >> >>
> >> >> > Hi All,
> >> >> >
> >> >> > As it's kind of difficult to get permissions in the wiki to edit
> pages
> >> >> > like https://cwiki.apache.org/confluence/display/FLUME/How+to+
> >> Contribute
> >> >> > , I suggest moving contents to the git repository into files

Re: Moving Developer Section of cwiki to the git repository

2016-10-24 Thread Balazs Donat Bessenyei 
I have commenced moving the Developer Section to the git repo.

There is a WIP pull request available at https://github.com/apache/flume/pull/77

I'm planning to add more e-mail templates to the HowToRelease.md and
commit all docs in a single commit, but reviews and comments are
welcome in the meanwhile.


Thank you,

Donat

On Fri, Oct 21, 2016 at 6:07 PM, Mike Percy  wrote:
> Thanks, Donat. +1 on moving the docs.
>
> Regarding the web site source code, it can reside in svn or git however it
> should continue using the standard ASF site infrastructure [1] for hosting
> content which consists only of static rendered HTML content. Our options
> for pushing that rendered HTML are either svnpubsub[2] or gitpubsub[3]
> which are basically the same thing - commit HTML and it gets pushed live to
> the web site.
>
> If someone wants to redesign the site or improve the site template, I would
> certainly welcome that.
>
> As a comparison point with another ASF project I am involved in, on Apache
> Kudu we use the software used for GitHub pages (Jekyll) to render the site.
> The web site source code is in the gh-pages branch in Git [4]. The site is
> mostly written in Markdown and we wrote a script [5] that locally renders
> the site to static HTML content and checks in the rendered content to the
> Kudu gitpubsub repository [6].
>
> In Flume, we basically do the same thing but but instead of Jekyll we use
> Sphinx and the site source code is primarily ReStructuredText. The site
> lives in its own svn repo [7]. We push rendered HTML tent to an svnpubsub
> repository for Flume [8] which is controlled by the ASF CMS [9]... IIRC,
> the CMS takes care of invoking the rendering part instead of having a local
> script to do it. This is kind of documented in the How to Release guide for
> Flume [10].
>
> If someone just wanted to change the look & feel of the web site they could
> easily do it with a little HTML/CSS knowledge and some reading up on
> Sphinx. The web site template could be changed to something nicer (it's
> using some kind of stock template right now). The existing .rst content
> would not need to be modified at all, probably.
>
> If someone wanted to switch the whole web site to some other system, like
> Jekyll, obviously it's a bigger change but if they were determined then I
> don't think anyone would try to stop them, assuming it's an improvement!
>
> Hope this helps,
> Mike
>
> [1] https://www.apache.org/dev/project-site.html
> [2] http://svn.apache.org/viewvc/subversion/trunk/tools/
> server-side/svnpubsub/
> [3] https://www.apache.org/dev/gitpubsub.html
> [4] https://github.com/apache/kudu/tree/gh-pages
> [5] https://github.com/apache/kudu/blob/gh-pages/site_tool
> [6] https://git-wip-us.apache.org/repos/asf?p=kudu-site.git;a=
> shortlog;h=refs/heads/asf-site
> [7] https://svn.apache.org/repos/asf/flume/site/trunk/
> [8] https://svn.apache.org/repos/infra/websites/production/flume/
> [9] https://www.apache.org/dev/cms.html
> [10]
> https://cwiki.apache.org/confluence/display/FLUME/How+to+Release#HowtoRelease-Updatethewebsite
>
> On Thu, Oct 20, 2016 at 2:59 PM, Balazs Donat Bessenyei > wrote:
>
>> As nobody has objected in ~48 hours, I'll open a pull request about this
>> soon.
>>
>> Regarding the whole site thing: AFAIK, the site contents have to
>> reside in SVN, because that's how ASF infrastructure works now.
>>
>> First, I'll work on moving the Developer Section to
>> https://github.com/apache/flume , then we'll see what else we can do.
>>
>>
>> Thank you,
>>
>> Donat
>>
>> On Thu, Oct 20, 2016 at 12:41 PM, UMESH CHAUDHARY 
>> wrote:
>> > +1 for moving wiki and web contents into GitHub. It would be easier to
>> > manage. Also, +1 for improving our website.
>> >
>> > On Thu, 20 Oct 2016 at 15:42 Lior Zeno  wrote:
>> >
>> > +1 for moving the whole site as well. I wish we could improve our
>> frontend
>> > to become a bit more appealing.
>> >
>> > On Thu, Oct 20, 2016 at 11:53 AM, Attila Simon 
>> wrote:
>> >
>> >> +1
>> >> Essentially moving whole site (all web content) to scm would help
>> >> contribution.
>> >>
>> >> Cheers,
>> >> Attila
>> >>
>> >> On Tuesday, 18 October 2016, Balazs Donat Bessenyei <
>> bes...@cloudera.com>
>> >> wrote:
>> >>
>> >> > Hi All,
>> >> >
>> >> > As it's kind of difficult to get permissions in the wiki to edit pages
>> >> > like https://cwiki.apache.org/confluence/display/FLUME/How+to+
>> Contribute
>> >> > , I suggest moving contents to the git repository into files like
>> >> > CONTRIBUTING.md, etc.
>> >> >
>> >> > I'd be happy to create the files and move the current texts.
>> >> >
>> >> > Please, let me know your thoughts.
>> >> >
>> >> >
>> >> > Thank you,
>> >> >
>> >> > Donat
>> >> >
>> >>
>> >>
>> >> --
>> >>
>> >> *Attila Simon*
>> >> Software Engineer
>> >> Email:   s...@cloudera.com
>> >>
>> >> [image: Cloudera Inc.]
>> >>
>>

Re: make junit tests passing again

2016-10-24 Thread Lior Zeno 
We have another one:
testMinimumRequiredSpaceTooSmallForPut(org.apache.flume.channel.file.TestLog).
See
https://builds.apache.org/job/Flume-trunk-hbase-1/220/org.apache.flume.flume-ng-channels$flume-file-channel/console

On Fri, Oct 21, 2016 at 7:56 PM, Attila Simon  wrote:

> Hi,
>
> Thanks Lior! I just created jiras for the ones I have the stack trace.
>
> testShutdown(org.apache.flume.source.TestExecSource)  FLUME-3013
> 
> org.apache.flume.source.TestSyslogUdpSource.testSourceCounter  FLUME-3014
> 
> org.apache.flume.channel.jdbc.TestJdbcChannelProvider.
> testEventWithSimulatedSourceAndSinks
> (only reference in FLUME-3015
>  as it has very similar
> stack trace)
> org.apache.flume.channel.jdbc.TestJdbcChannelProviderNoFK.te
> stEventWithSimulatedSourceAndSinks  FLUME-3015
> 
> org.apache.flume.sink.hbase.TestAsyncHBaseSink FLUME-3016
> org.apache.flume.sink.hbase.TestHBaseSink  FLUME-3017
> 
> org.apache.flume.source.thriftLegacy.TestThriftLegacySource (I couldn't
> get
> the stack trace now)
> org.apache.flume.channel.TestSpillableMemoryChannel.
> testParallelMultipleSourcesAndSinks
> (sati is already addressing this with https://issues.apache.
> org/jira/browse/FLUME-2997)
> org.apache.flume.sink.hdfs.TestBucketWriter.testFileSuffixGiven (bessbd is
> already addressing this with https://issues.apache.
> org/jira/browse/FLUME-3002)
> org.apache.flume.sink.hdfs.TestBucketWriter.testFileSuffixNotGiven (bessbd
> is already addressing this with https://issues.apache.
> org/jira/browse/FLUME-3002)
> org.apache.flume.source.TestExecSource.testMonitoredCounterGroup (mpercy
> is
> already addressing this with https://github.com/apache/flume/pull/72)
>
> Cheers,
> Attila
>
> On Fri, Oct 14, 2016 at 8:07 PM, Lior Zeno  wrote:
>
> > You can add testShutdown(org.apache.flume.source.TestExecSource) to the
> > list.
> >
> > java.lang.IllegalStateException: Command [ps -ef] exited with 143
> > at org.apache.flume.source.TestExecSource.exec(TestExecSource.java:460)
> > at
> > org.apache.flume.source.TestExecSource.testShutdown(
> > TestExecSource.java:406)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> > sun.reflect.NativeMethodAccessorImpl.invoke(
> NativeMethodAccessorImpl.java:
> > 62)
> > at
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(
> > DelegatingMethodAccessorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:498)
> > at
> > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(
> > FrameworkMethod.java:45)
> > at
> > org.junit.internal.runners.model.ReflectiveCallable.run(
> > ReflectiveCallable.java:15)
> > at
> > org.junit.runners.model.FrameworkMethod.invokeExplosively(
> > FrameworkMethod.java:42)
> > at
> > org.junit.internal.runners.statements.InvokeMethod.
> > evaluate(InvokeMethod.java:20)
> > at
> > org.junit.internal.runners.statements.RunBefores.
> > evaluate(RunBefores.java:28)
> > at
> > org.junit.internal.runners.statements.RunAfters.evaluate(
> > RunAfters.java:30)
> > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
> > at
> > org.junit.runners.BlockJUnit4ClassRunner.runChild(
> > BlockJUnit4ClassRunner.java:68)
> > at
> > org.junit.runners.BlockJUnit4ClassRunner.runChild(
> > BlockJUnit4ClassRunner.java:47)
> > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
> > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
> > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
> > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
> > at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> > at
> > org.apache.maven.surefire.junit4.JUnit4Provider.execute(
> > JUnit4Provider.java:252)
> > at
> > org.apache.maven.surefire.junit4.JUnit4Provider.
> > executeTestSet(JUnit4Provider.java:141)
> > at
> > org.apache.maven.surefire.junit4.JUnit4Provider.invoke(
> > JUnit4Provider.java:112)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> > sun.reflect.NativeMethodAccessorImpl.invoke(
> NativeMethodAccessorImpl.java:
> > 62)
> > at
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(
> > DelegatingMethodAccessorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:498)
> > at
> > org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(
> > ReflectionUtils.java:189)
> > at
> > org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(
> > ProviderFactory.java:165)
> > at
> > org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(
> > ProviderFactory.java:85)
> > at
> > org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(
> >

Jenkins build is unstable: Flume-trunk-hbase-1 #220

2016-10-24 Thread Apache Jenkins Server 
See

[jira] [Commented] (FLUME-3002) Some tests in TestBucketWriter are flaky

2016-10-24 Thread Hudson (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLUME-3002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15602231#comment-15602231
 ] 

Hudson commented on FLUME-3002:
---

UNSTABLE: Integrated in Jenkins build Flume-trunk-hbase-1 #220 (See 
[https://builds.apache.org/job/Flume-trunk-hbase-1/220/])
FLUME-3002. Fix tests in TestBucketWriter (bessbd: 
[http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git=commit=2399329ee2ca2d9fc4ec0ec8fc5d16fb213795b2])
* (edit) 
flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestBucketWriter.java
* (edit) 
flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/BucketWriter.java


> Some tests in TestBucketWriter are flaky
> 
>
> Key: FLUME-3002
> URL: https://issues.apache.org/jira/browse/FLUME-3002
> Project: Flume
>  Issue Type: Test
>  Components: Sinks+Sources
>Affects Versions: v1.6.0
>Reporter: Bessenyei Balázs Donát
>Assignee: Bessenyei Balázs Donát
> Fix For: v1.8.0
>
>
> testFileSuffixNotGiven (and probably a few other tests) are flaky



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLUME-3002) Some tests in TestBucketWriter are flaky

2016-10-24 Thread ASF subversion and git services (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLUME-3002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15601946#comment-15601946
 ] 

ASF subversion and git services commented on FLUME-3002:


Commit 2399329ee2ca2d9fc4ec0ec8fc5d16fb213795b2 in flume's branch 
refs/heads/trunk from [~bessbd]
[ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=2399329 ]

FLUME-3002. Fix tests in TestBucketWriter

Some tests are flaky in TestBucketWriter.

This commit fixes the flakiness mentioned before by
adding a new constructor with an extra Clock parameter.

Reviewers: Attila Simon, Denes Arvay


> Some tests in TestBucketWriter are flaky
> 
>
> Key: FLUME-3002
> URL: https://issues.apache.org/jira/browse/FLUME-3002
> Project: Flume
>  Issue Type: Test
>  Components: Sinks+Sources
>Affects Versions: v1.6.0
>Reporter: Bessenyei Balázs Donát
>Assignee: Bessenyei Balázs Donát
> Fix For: v1.8.0
>
>
> testFileSuffixNotGiven (and probably a few other tests) are flaky



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] flume pull request #75: Set MAVEN_OPTS for Travis-CI

2016-10-24 Thread asfgit 
Github user asfgit closed the pull request at:

https://github.com/apache/flume/pull/75


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (FLUME-3002) Some tests in TestBucketWriter are flaky

2016-10-24 Thread JIRA 

[ 
https://issues.apache.org/jira/browse/FLUME-3002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15601815#comment-15601815
 ] 

Bessenyei Balázs Donát commented on FLUME-3002:
---

I'm about to commit this.

> Some tests in TestBucketWriter are flaky
> 
>
> Key: FLUME-3002
> URL: https://issues.apache.org/jira/browse/FLUME-3002
> Project: Flume
>  Issue Type: Test
>  Components: Sinks+Sources
>Affects Versions: v1.6.0
>Reporter: Bessenyei Balázs Donát
>Assignee: Bessenyei Balázs Donát
> Fix For: v1.8.0
>
>
> testFileSuffixNotGiven (and probably a few other tests) are flaky



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 53056: FLUME-2945: Bump java target version to 1.8

2016-10-24 Thread Lior Zeno 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53056/
---

(Updated Oct. 24, 2016, 11:27 a.m.)


Review request for Flume.


Changes
---

Udpate MAVEN_OPTS in travis.yml


Repository: flume-git


Description
---

We should move to Java 8 as a minimum requirement.

1. Java 7 is EOL'ed http://www.oracle.com/technetwork/java/eol-135779.html.
2. Many projects are adopting java 8 as a minimum requirement, for instance:

* Solr 6: https://issues.apache.org/jira/browse/LUCENE-6722
* Hbase 2: https://issues.apache.org/jira/browse/HBASE-15624
* elasticsearch 5: 
https://www.elastic.co/guide/en/elasticsearch/reference/master/setup.html


Diffs (updated)
-

  .travis.yml befc2c9 
  DEVNOTES 83cdf89 
  README.md 9ebb2a3 
  conf/flume-env.sh.template 07182ca 
  flume-ng-doc/sphinx/FlumeUserGuide.rst cd76bb3 
  flume-ng-sources/flume-taildir-source/pom.xml 96c2468 
  pom.xml f62c99a 

Diff: https://reviews.apache.org/r/53056/diff/


Testing
---

The attached patch was tested on Ubuntu 16.04 with openjdk version "1.8.0_91".
All unit tests, except for the known flaky ones, successfully passed.


Thanks,

Lior Zeno

Build failed in Jenkins: Flume-trunk-hbase-1 #219

2016-10-24 Thread Apache Jenkins Server 
See 

Changes:

[bessbd] Set MAVEN_OPTS for Travis-CI

--
[...truncated 3150 lines...]
[INFO] Exclude: **/test/resources/**
[INFO] Exclude: **/.settings/*
[INFO] Exclude: **/.classpath
[INFO] Exclude: **/.project
[INFO] Exclude: **/target/**
[INFO] Exclude: **/derby.log
[INFO] Exclude: **/metastore_db/
[INFO] 1 resources included (use -debug for more details)
[INFO] Rat check: Summary of files. Unapproved: 0 unknown: 0 generated: 0 
approved: 1 licence.
[INFO] 
[INFO] --- maven-checkstyle-plugin:2.17:check (verify) @ 
flume-ng-legacy-sources ---
[INFO] 
[INFO] --- maven-install-plugin:2.5.2:install (default-install) @ 
flume-ng-legacy-sources ---
[INFO] Installing 

 to 

[INFO] 
[INFO] 
[INFO] Skipping Flume legacy Avro source
[INFO] This project has been banned from the build due to previous failures.
[INFO] 
[INFO] 
[INFO] 
[INFO] Skipping Flume legacy Thrift Source
[INFO] This project has been banned from the build due to previous failures.
[INFO] 
[INFO] 
[INFO] 
[INFO] Building Flume NG Clients 1.8.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-clean-plugin:2.4.1:clean (default-clean) @ flume-ng-clients ---
[INFO] Deleting 

[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (process-resource-bundles) 
@ flume-ng-clients ---
[INFO] 
[INFO] --- maven-site-plugin:3.3:attach-descriptor (attach-descriptor) @ 
flume-ng-clients ---
[INFO] 
[INFO] --- apache-rat-plugin:0.11:check (verify.rat) @ flume-ng-clients ---
[INFO] 52 implicit excludes (use -debug for more details).
[INFO] Exclude: **/.idea/
[INFO] Exclude: **/*.iml
[INFO] Exclude: **/nb-configuration.xml
[INFO] Exclude: .git/
[INFO] Exclude: patchprocess/
[INFO] Exclude: .gitignore
[INFO] Exclude: .repository/
[INFO] Exclude: **/*.diff
[INFO] Exclude: **/*.patch
[INFO] Exclude: **/*.avsc
[INFO] Exclude: **/*.avro
[INFO] Exclude: **/docs/**
[INFO] Exclude: **/test/resources/**
[INFO] Exclude: **/.settings/*
[INFO] Exclude: **/.classpath
[INFO] Exclude: **/.project
[INFO] Exclude: **/target/**
[INFO] Exclude: **/derby.log
[INFO] Exclude: **/metastore_db/
[INFO] 1 resources included (use -debug for more details)
[INFO] Rat check: Summary of files. Unapproved: 0 unknown: 0 generated: 0 
approved: 1 licence.
[INFO] 
[INFO] --- maven-checkstyle-plugin:2.17:check (verify) @ flume-ng-clients ---
[INFO] 
[INFO] --- maven-install-plugin:2.5.2:install (default-install) @ 
flume-ng-clients ---
[INFO] Installing 
 
to 

[INFO] 
[INFO] 
[INFO] Skipping Flume NG Log4j Appender
[INFO] This project has been banned from the build due to previous failures.
[INFO] 
[INFO] 
[INFO] 
[INFO] Skipping Flume NG Tools
[INFO] This project has been banned from the build due to previous failures.
[INFO] 
[INFO] 
[INFO] 
[INFO] Skipping Flume NG distribution
[INFO] This project has been banned from the build due to previous failures.
[INFO] 
[INFO] 
[INFO] 
[INFO] Skipping Flume NG Integration Tests
[INFO] This

Re: Review Request 52510: Some tests in TestBucketWriter are flaky

2016-10-24 Thread Denes Arvay 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52510/#review153678
---


Ship it!




Ship It!

- Denes Arvay


On Oct. 4, 2016, 1:11 p.m., Balázs Donát Bessenyei wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52510/
> ---
> 
> (Updated Oct. 4, 2016, 1:11 p.m.)
> 
> 
> Review request for Flume.
> 
> 
> Bugs: FLUME-3002
> https://issues.apache.org/jira/browse/FLUME-3002
> 
> 
> Repository: flume-git
> 
> 
> Description
> ---
> 
> testFileSuffixNotGiven (and probably a few other tests) are flaky
> 
> 
> Diffs
> -
> 
>   
> c/flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/BucketWriter.java
>  b096410 
>   
> c/flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestBucketWriter.java
>  742deb0 
> 
> Diff: https://reviews.apache.org/r/52510/diff/
> 
> 
> Testing
> ---
> 
> mvn clean install runs successfully in flume-ng-sinks
> 
> 
> Thanks,
> 
> Balázs Donát Bessenyei
> 
>