[jira] [Assigned] (FLUME-1705) SpoolDirectory short name points at the wrong class

[Alexander Alten-Lorenz (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-1705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Alten-Lorenz reassigned FLUME-1705:
-

Assignee: Alexander Alten-Lorenz

> SpoolDirectory short name points at the wrong class
> ---
>
> Key: FLUME-1705
> URL: https://issues.apache.org/jira/browse/FLUME-1705
> Project: Flume
>  Issue Type: Bug
>Affects Versions: v1.3.0
>Reporter: Brock Noland
>Assignee: Alexander Alten-Lorenz
>Priority: Blocker
> Fix For: v1.3.0
>
> Attachments: FLUME-1705.patch
>
>
> is `org.apache.flume.conf.source.SpoolDirectorySource' should be 
> `org.apache.flume.source.SpoolDirectorySource'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (FLUME-1705) SpoolDirectory short name points at the wrong class

[Alexander Alten-Lorenz (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLUME-1705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13496924#comment-13496924
 ] 

Alexander Alten-Lorenz commented on FLUME-1705:
---

https://reviews.apache.org/r/8050/

> SpoolDirectory short name points at the wrong class
> ---
>
> Key: FLUME-1705
> URL: https://issues.apache.org/jira/browse/FLUME-1705
> Project: Flume
>  Issue Type: Bug
>Affects Versions: v1.3.0
>Reporter: Brock Noland
>Assignee: Alexander Alten-Lorenz
>Priority: Blocker
> Fix For: v1.3.0
>
> Attachments: FLUME-1705.patch
>
>
> is `org.apache.flume.conf.source.SpoolDirectorySource' should be 
> `org.apache.flume.source.SpoolDirectorySource'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: [VOTE] Release Apache Flume version 1.3.0-rc2

[Alexander Alten-Lorenz]

Patch attached. For this one -1 from me.

Thanks,
 Alex

On Nov 13, 2012, at 11:13 PM, Brock Noland  wrote:

> Blocker came up on the user list
> 
> https://issues.apache.org/jira/browse/FLUME-1705
> 
> Brock
> 
> On Tue, Nov 13, 2012 at 2:59 PM, Roshan Naik  wrote:
>> Looks good. Verified build and tests.
>> +1  (non binding)
>> -roshan
>> 
>> 
>> On Tue, Nov 13, 2012 at 8:41 AM, Brock Noland  wrote:
>> 
>>> +1 verified hashes, sigs, and performed single node sanity test
>>> 
>>> On Tue, Nov 13, 2012 at 10:39 AM, Brock Noland  wrote:
 This is the second release for Apache Flume as a top-level project,
 version 1.3.0. We are voting on release candidate rc2.
 
 *** Please cast your vote within the next 72 hours ***
 
 The list of fixed issues:
 http://s.apache.org/A9x
 
 The tarball (*.tar.gz), signature (*.asc), and checksums (*.md5, *.sha1)
 for the source and binary artifacts can be found at:
 http://people.apache.org/~brock/flume-1.3.0-rc2/
 
 The tag to be voted on:
 http://s.apache.org/aVa
 
 The KEYS file can be found here:
 https://dist.apache.org/repos/dist/release/flume/KEYS
>>> 
>>> 
>>> 
>>> --
>>> Apache MRUnit - Unit testing MapReduce -
>>> http://incubator.apache.org/mrunit/
>>> 
> 
> 
> 
> -- 
> Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/

--
Alexander Alten-Lorenz
http://mapredit.blogspot.com
German Hadoop LinkedIn Group: http://goo.gl/N8pCF

Review Request: SpoolDirectory short name points at the wrong class

[Alexander Alten-Lorenz]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8050/
---

Review request for Flume.


Description
---

SourceType.java 

`org.apache.flume.conf.source.SpoolDirectorySource' should be 
`org.apache.flume.source.SpoolDirectorySource'


This addresses bug FLUME-1705.
https://issues.apache.org/jira/browse/FLUME-1705


Diffs
-

  
flume-ng-configuration/src/main/java/org/apache/flume/conf/source/SourceType.java
 c7c511e 

Diff: https://reviews.apache.org/r/8050/diff/


Testing
---

SUCCESS


Thanks,

Alexander Alten-Lorenz

[jira] [Updated] (FLUME-1705) SpoolDirectory short name points at the wrong class

[Alexander Alten-Lorenz (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-1705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Alten-Lorenz updated FLUME-1705:
--

Attachment: FLUME-1705.patch

> SpoolDirectory short name points at the wrong class
> ---
>
> Key: FLUME-1705
> URL: https://issues.apache.org/jira/browse/FLUME-1705
> Project: Flume
>  Issue Type: Bug
>Affects Versions: v1.3.0
>Reporter: Brock Noland
>Priority: Blocker
> Fix For: v1.3.0
>
> Attachments: FLUME-1705.patch
>
>
> is `org.apache.flume.conf.source.SpoolDirectorySource' should be 
> `org.apache.flume.source.SpoolDirectorySource'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: [ANNOUNCE] New Apache Flume committer - Patrick Wendell

[Mubarak Seyed]

Congrats Patrick!

-Mubarak
On Nov 12, 2012, at 1:04 PM, Hari Shreedharan  wrote:

> On behalf of the Apache Flume PMC,  I am excited to welcome Patrick Wendell 
> as a committer on Flume! Patrick has contributed significantly to the 
> project, by adding new features, fixing bugs and helping users on the Flume 
> users list. Here is a list of jiras Patrick has worked on:
> 
> http://s.apache.org/6EG
> 
> Please join me in congratulating Patrick on his new role!
> 
> 
> Thanks,
> Hari
> 
> 
> 
> 

[jira] [Commented] (FLUME-1707) Update FlumeDevGuide and FlumeUserGuide

[Will McQueen (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLUME-1707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13496784#comment-13496784
 ] 

Will McQueen commented on FLUME-1707:
-

Ok




> Update FlumeDevGuide and FlumeUserGuide
> ---
>
> Key: FLUME-1707
> URL: https://issues.apache.org/jira/browse/FLUME-1707
> Project: Flume
>  Issue Type: Bug
>  Components: Docs
>Affects Versions: v1.3.0
>Reporter: Will McQueen
>Assignee: Will McQueen
>Priority: Blocker
> Fix For: v1.3.0
>
>
> There are several places in docs that have missing or incorrect info. These 
> need to be updated.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (FLUME-1707) Update FlumeDevGuide and FlumeUserGuide

[Will McQueen (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLUME-1707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13496783#comment-13496783
 ] 

Will McQueen commented on FLUME-1707:
-

Yup, understood;-)




> Update FlumeDevGuide and FlumeUserGuide
> ---
>
> Key: FLUME-1707
> URL: https://issues.apache.org/jira/browse/FLUME-1707
> Project: Flume
>  Issue Type: Bug
>  Components: Docs
>Affects Versions: v1.3.0
>Reporter: Will McQueen
>Assignee: Will McQueen
>Priority: Blocker
> Fix For: v1.3.0
>
>
> There are several places in docs that have missing or incorrect info. These 
> need to be updated.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: Review Request: FLUME-1657 - Regex Extractor Interceptor

[Hari Shreedharan]

> On Nov. 6, 2012, 5:05 p.m., Brock Noland wrote:
> > flume-ng-doc/sphinx/FlumeUserGuide.rst, line 1824
> > 
> >
> > Generally in flume when we are going to have multiple sub property 
> > items we give them a name and then have sub-properties under that name. 
> > Thoughts on doing that hear as opposed to using the class name?
> 
> Cameron Gandevia wrote:
> The idea I saw was allowing anyone to provide their own serializer 
> implementation outside of the ones provided by the flume project, maybe I am 
> not following how this would be done using sub property names.
> 
> Brock Noland wrote:
> Yes I think that is great a great idea!
> 
> So what I was thinking is that this:
> 
> agent.sources.r1.channels = c1
> agent.sources.r1.type = SEQ
> agent.sources.r1.interceptors = i1
> agent.sources.r1.interceptors.i1.type = REGEX_EXTRACTOR
> agent.sources.r1.interceptors.i1.regex = (WARNING)|(ERROR)|(FATAL)
> agent.sources.r1.interceptors.i1.serializer = 
> warning:com.blah.SomeSerializer,error,fatal:org.apache.flume.interceptor.RegexExtractorInterceptorTimestampSerializer
> 
> agent.sources.r1.interceptors.i1.org.apache.flume.interceptor.RegexExtractorInterceptorTimestampSerializer.dateFormat
>  = -MM-dd
> 
> becomes something  approximately like this:
> 
> agent.sources.r1.channels = c1
> agent.sources.r1.type = SEQ
> agent.sources.r1.interceptors = i1
> agent.sources.r1.interceptors.i1.type = REGEX_EXTRACTOR
> agent.sources.r1.interceptors.i1.regex = (WARNING)|(ERROR)|(FATAL)
> agent.sources.r1.interceptors.i1.serializers = 
> warning:s1,error,fatal:s1
> agent.sources.r1.interceptors.i1.serializers.s1.type = 
> org.apache.flume.interceptor.RegexExtractorInterceptorTimestampSerializer
> agent.sources.r1.interceptors.i1.serializers.s1.dateFormat = -MM-dd
> agent.sources.r1.interceptors.i1.serializers.s2.type = 
> com.blah.SomeSerializer
> 
> That is how other plugabble components are configured. Then s1.* can be 
> passed to the a configure method in the serializer and the plugin can do it's 
> own configuration.

I like Brock's idea. I'd also change this to be more uniform with other Flume 
components to something like this:
agent.sources.r1.interceptors.i1.serializers = s1 s2
agent.sources.r1.interceptors.i1.serializers.s1.type = 
org.apache.flume.interceptor.RegexExtractorInterceptorTimestampSerializer
agent.sources.r1.interceptors.i1.serializers.s1.dateFormat = -MM-dd
agent.sources.r1.interceptors.i1.serializers.s1.severity = warning
agent.sources.r1.interceptors.i1.serializers.s2.type = com.blah.SomeSerializer
agent.sources.r1.interceptors.i1.serializers.s2.severity = error fatal


- Hari


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7700/#review13156
---


On Nov. 14, 2012, 1:24 a.m., Cameron Gandevia wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/7700/
> ---
> 
> (Updated Nov. 14, 2012, 1:24 a.m.)
> 
> 
> Review request for Flume.
> 
> 
> Description
> ---
> 
> A RegexExtractor interceptor that will allow users to extract regex matches 
> and append them as header fields of the event.
> 
> 
> Diffs
> -
> 
>   
> flume-ng-core/src/main/java/org/apache/flume/interceptor/InterceptorType.java 
> c478337 
>   
> flume-ng-core/src/main/java/org/apache/flume/interceptor/RegexExtractorInterceptor.java
>  PRE-CREATION 
>   
> flume-ng-core/src/main/java/org/apache/flume/interceptor/RegexExtractorInterceptorMillisSerializer.java
>  PRE-CREATION 
>   
> flume-ng-core/src/main/java/org/apache/flume/interceptor/RegexExtractorInterceptorPassThroughSerializer.java
>  PRE-CREATION 
>   
> flume-ng-core/src/main/java/org/apache/flume/interceptor/RegexExtractorInterceptorSerializer.java
>  PRE-CREATION 
>   
> flume-ng-core/src/test/java/org/apache/flume/interceptor/TestRegexExtractorInterceptor.java
>  PRE-CREATION 
>   
> flume-ng-core/src/test/java/org/apache/flume/interceptor/TestRegexExtractorInterceptorMillisSerializer.java
>  PRE-CREATION 
>   
> flume-ng-core/src/test/java/org/apache/flume/interceptor/TestRegexExtractorInterceptorPassThroughSerializer.java
>  PRE-CREATION 
>   flume-ng-doc/sphinx/FlumeUserGuide.rst 0596473 
>   flume-ng-doc/sphinx/FlumeUserGuide.rst.orig PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/7700/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Cameron Gandevia
> 
>

Re: Review Request: FLUME-1657 - Regex Extractor Interceptor

[Brock Noland]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7700/#review13410
---



flume-ng-doc/sphinx/FlumeUserGuide.rst.orig


Looks like this .orig file can be removed.


- Brock Noland


On Nov. 14, 2012, 1:24 a.m., Cameron Gandevia wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/7700/
> ---
> 
> (Updated Nov. 14, 2012, 1:24 a.m.)
> 
> 
> Review request for Flume.
> 
> 
> Description
> ---
> 
> A RegexExtractor interceptor that will allow users to extract regex matches 
> and append them as header fields of the event.
> 
> 
> Diffs
> -
> 
>   
> flume-ng-core/src/main/java/org/apache/flume/interceptor/InterceptorType.java 
> c478337 
>   
> flume-ng-core/src/main/java/org/apache/flume/interceptor/RegexExtractorInterceptor.java
>  PRE-CREATION 
>   
> flume-ng-core/src/main/java/org/apache/flume/interceptor/RegexExtractorInterceptorMillisSerializer.java
>  PRE-CREATION 
>   
> flume-ng-core/src/main/java/org/apache/flume/interceptor/RegexExtractorInterceptorPassThroughSerializer.java
>  PRE-CREATION 
>   
> flume-ng-core/src/main/java/org/apache/flume/interceptor/RegexExtractorInterceptorSerializer.java
>  PRE-CREATION 
>   
> flume-ng-core/src/test/java/org/apache/flume/interceptor/TestRegexExtractorInterceptor.java
>  PRE-CREATION 
>   
> flume-ng-core/src/test/java/org/apache/flume/interceptor/TestRegexExtractorInterceptorMillisSerializer.java
>  PRE-CREATION 
>   
> flume-ng-core/src/test/java/org/apache/flume/interceptor/TestRegexExtractorInterceptorPassThroughSerializer.java
>  PRE-CREATION 
>   flume-ng-doc/sphinx/FlumeUserGuide.rst 0596473 
>   flume-ng-doc/sphinx/FlumeUserGuide.rst.orig PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/7700/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Cameron Gandevia
> 
>

Re: Review Request: FLUME-1657 - Regex Extractor Interceptor

[Brock Noland]

> On Nov. 6, 2012, 5:05 p.m., Brock Noland wrote:
> > flume-ng-doc/sphinx/FlumeUserGuide.rst, line 1824
> > 
> >
> > Generally in flume when we are going to have multiple sub property 
> > items we give them a name and then have sub-properties under that name. 
> > Thoughts on doing that hear as opposed to using the class name?
> 
> Cameron Gandevia wrote:
> The idea I saw was allowing anyone to provide their own serializer 
> implementation outside of the ones provided by the flume project, maybe I am 
> not following how this would be done using sub property names.

Yes I think that is great a great idea!

So what I was thinking is that this:

agent.sources.r1.channels = c1
agent.sources.r1.type = SEQ
agent.sources.r1.interceptors = i1
agent.sources.r1.interceptors.i1.type = REGEX_EXTRACTOR
agent.sources.r1.interceptors.i1.regex = (WARNING)|(ERROR)|(FATAL)
agent.sources.r1.interceptors.i1.serializer = 
warning:com.blah.SomeSerializer,error,fatal:org.apache.flume.interceptor.RegexExtractorInterceptorTimestampSerializer
agent.sources.r1.interceptors.i1.org.apache.flume.interceptor.RegexExtractorInterceptorTimestampSerializer.dateFormat
 = -MM-dd

becomes something  approximately like this:

agent.sources.r1.channels = c1
agent.sources.r1.type = SEQ
agent.sources.r1.interceptors = i1
agent.sources.r1.interceptors.i1.type = REGEX_EXTRACTOR
agent.sources.r1.interceptors.i1.regex = (WARNING)|(ERROR)|(FATAL)
agent.sources.r1.interceptors.i1.serializers = warning:s1,error,fatal:s1
agent.sources.r1.interceptors.i1.serializers.s1.type = 
org.apache.flume.interceptor.RegexExtractorInterceptorTimestampSerializer
agent.sources.r1.interceptors.i1.serializers.s1.dateFormat = -MM-dd
agent.sources.r1.interceptors.i1.serializers.s2.type = com.blah.SomeSerializer

That is how other plugabble components are configured. Then s1.* can be passed 
to the a configure method in the serializer and the plugin can do it's own 
configuration.


- Brock


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7700/#review13156
---


On Nov. 14, 2012, 1:24 a.m., Cameron Gandevia wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/7700/
> ---
> 
> (Updated Nov. 14, 2012, 1:24 a.m.)
> 
> 
> Review request for Flume.
> 
> 
> Description
> ---
> 
> A RegexExtractor interceptor that will allow users to extract regex matches 
> and append them as header fields of the event.
> 
> 
> Diffs
> -
> 
>   
> flume-ng-core/src/main/java/org/apache/flume/interceptor/InterceptorType.java 
> c478337 
>   
> flume-ng-core/src/main/java/org/apache/flume/interceptor/RegexExtractorInterceptor.java
>  PRE-CREATION 
>   
> flume-ng-core/src/main/java/org/apache/flume/interceptor/RegexExtractorInterceptorMillisSerializer.java
>  PRE-CREATION 
>   
> flume-ng-core/src/main/java/org/apache/flume/interceptor/RegexExtractorInterceptorPassThroughSerializer.java
>  PRE-CREATION 
>   
> flume-ng-core/src/main/java/org/apache/flume/interceptor/RegexExtractorInterceptorSerializer.java
>  PRE-CREATION 
>   
> flume-ng-core/src/test/java/org/apache/flume/interceptor/TestRegexExtractorInterceptor.java
>  PRE-CREATION 
>   
> flume-ng-core/src/test/java/org/apache/flume/interceptor/TestRegexExtractorInterceptorMillisSerializer.java
>  PRE-CREATION 
>   
> flume-ng-core/src/test/java/org/apache/flume/interceptor/TestRegexExtractorInterceptorPassThroughSerializer.java
>  PRE-CREATION 
>   flume-ng-doc/sphinx/FlumeUserGuide.rst 0596473 
>   flume-ng-doc/sphinx/FlumeUserGuide.rst.orig PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/7700/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Cameron Gandevia
> 
>

[jira] [Commented] (FLUME-1707) Update FlumeDevGuide and FlumeUserGuide

[Mike Percy (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLUME-1707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13496753#comment-13496753
 ] 

Mike Percy commented on FLUME-1707:
---

Will, if you are working on a docs patch, please consider building on top of 
FLUME-1706.

> Update FlumeDevGuide and FlumeUserGuide
> ---
>
> Key: FLUME-1707
> URL: https://issues.apache.org/jira/browse/FLUME-1707
> Project: Flume
>  Issue Type: Bug
>  Components: Docs
>Affects Versions: v1.3.0
>Reporter: Will McQueen
>Assignee: Will McQueen
>Priority: Blocker
> Fix For: v1.3.0
>
>
> There are several places in docs that have missing or incorrect info. These 
> need to be updated.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (FLUME-1707) Update FlumeDevGuide and FlumeUserGuide

[Mike Percy (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLUME-1707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13496754#comment-13496754
 ] 

Mike Percy commented on FLUME-1707:
---

(I mean the patch I posted there)

> Update FlumeDevGuide and FlumeUserGuide
> ---
>
> Key: FLUME-1707
> URL: https://issues.apache.org/jira/browse/FLUME-1707
> Project: Flume
>  Issue Type: Bug
>  Components: Docs
>Affects Versions: v1.3.0
>Reporter: Will McQueen
>Assignee: Will McQueen
>Priority: Blocker
> Fix For: v1.3.0
>
>
> There are several places in docs that have missing or incorrect info. These 
> need to be updated.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (FLUME-1706) Website for 1.3 fails to build

[Mike Percy (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-1706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mike Percy updated FLUME-1706:
--

Attachment: FLUME-1706-1.patch

Patch to fix the layout errors in the docs.

Also removed all tabs and replaced them with correctly indented spaces.

> Website for 1.3 fails to build
> --
>
> Key: FLUME-1706
> URL: https://issues.apache.org/jira/browse/FLUME-1706
> Project: Flume
>  Issue Type: Sub-task
>Reporter: Brock Noland
>Assignee: Mike Percy
>Priority: Blocker
> Fix For: v1.3.0
>
> Attachments: Flume 1.3 Developer Guide.pdf, Flume 1.3 User Guide.pdf, 
> FLUME-1706-1.patch, Screen Shot 2012-11-13 at 4.16.54 PM.png
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (FLUME-1706) Website for 1.3 fails to build

[Mike Percy (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-1706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mike Percy reassigned FLUME-1706:
-

Assignee: Mike Percy

> Website for 1.3 fails to build
> --
>
> Key: FLUME-1706
> URL: https://issues.apache.org/jira/browse/FLUME-1706
> Project: Flume
>  Issue Type: Sub-task
>Reporter: Brock Noland
>Assignee: Mike Percy
>Priority: Blocker
> Fix For: v1.3.0
>
> Attachments: Flume 1.3 Developer Guide.pdf, Flume 1.3 User Guide.pdf, 
> FLUME-1706-1.patch, Screen Shot 2012-11-13 at 4.16.54 PM.png
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: [VOTE] Release Apache Flume version 1.3.0-rc2

[Mike Percy]

Yeah, in
https://cwiki.apache.org/confluence/display/FLUME/How+to+Release#HowtoRelease-DeployMavenartifactsthere
is a section saying to do this at release time but you should
probably do it at RC vote time. Just push it to the staging repo, not the
release repo before the vote. It's a binary artifact, so technically we
aren't voting on it, but I agree with you guys that it's good practice to
do it early. Probably the wiki should be updated to make this process
explicit.

Regards,
Mike

On Tue, Nov 13, 2012 at 2:16 PM, Brock Noland  wrote:

> Hi,
>
> I actually thought the maven item as odd as well. MRUnit deploys to
> the staging repository (See "Deploy to Nexus Staging Repository"
> http://mrunit.apache.org/pmc/how_to_release.html).
>
> The documentation will be updated, I believe these can verified
> locally by using mvn site. Actually I have just looked at version I
> generated earlier today and there are some updates needed.
>
> Brock
>
> On Tue, Nov 13, 2012 at 3:29 PM, Ralph Goers 
> wrote:
> > How can I verify the Maven artifacts?  It really is a problem to deploy
> them after the vote since the can't be verified.
> >
> > What web site updates will be made?  How can we verify those?
> >
> > Ralph
> >
> > On Nov 13, 2012, at 8:39 AM, Brock Noland wrote:
> >
> >> This is the second release for Apache Flume as a top-level project,
> >> version 1.3.0. We are voting on release candidate rc2.
> >>
> >> *** Please cast your vote within the next 72 hours ***
> >>
> >> The list of fixed issues:
> >> http://s.apache.org/A9x
> >>
> >> The tarball (*.tar.gz), signature (*.asc), and checksums (*.md5, *.sha1)
> >> for the source and binary artifacts can be found at:
> >> http://people.apache.org/~brock/flume-1.3.0-rc2/
> >>
> >> The tag to be voted on:
> >> http://s.apache.org/aVa
> >>
> >> The KEYS file can be found here:
> >> https://dist.apache.org/repos/dist/release/flume/KEYS
> >
>
>
>
> --
> Apache MRUnit - Unit testing MapReduce -
> http://incubator.apache.org/mrunit/
>

Re: [ANNOUNCE] New Apache Flume committer - Patrick Wendell

[Juhani Connolly]

Congrats Patrick, keep up the good work!

On 11/13/2012 06:04 AM, Hari Shreedharan wrote:

On behalf of the Apache Flume PMC,  I am excited to welcome Patrick Wendell as 
a committer on Flume! Patrick has contributed significantly to the project, by 
adding new features, fixing bugs and helping users on the Flume users list. 
Here is a list of jiras Patrick has worked on:

http://s.apache.org/6EG

Please join me in congratulating Patrick on his new role!


Thanks,
Hari

Re: Review Request: FLUME-1657 - Regex Extractor Interceptor

[Cameron Gandevia]

> On Nov. 6, 2012, 5:05 p.m., Brock Noland wrote:
> > flume-ng-doc/sphinx/FlumeUserGuide.rst, line 1824
> > 
> >
> > Generally in flume when we are going to have multiple sub property 
> > items we give them a name and then have sub-properties under that name. 
> > Thoughts on doing that hear as opposed to using the class name?

The idea I saw was allowing anyone to provide their own serializer 
implementation outside of the ones provided by the flume project, maybe I am 
not following how this would be done using sub property names.


- Cameron


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7700/#review13156
---


On Nov. 14, 2012, 1:24 a.m., Cameron Gandevia wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/7700/
> ---
> 
> (Updated Nov. 14, 2012, 1:24 a.m.)
> 
> 
> Review request for Flume.
> 
> 
> Description
> ---
> 
> A RegexExtractor interceptor that will allow users to extract regex matches 
> and append them as header fields of the event.
> 
> 
> Diffs
> -
> 
>   
> flume-ng-core/src/main/java/org/apache/flume/interceptor/InterceptorType.java 
> c478337 
>   
> flume-ng-core/src/main/java/org/apache/flume/interceptor/RegexExtractorInterceptor.java
>  PRE-CREATION 
>   
> flume-ng-core/src/main/java/org/apache/flume/interceptor/RegexExtractorInterceptorMillisSerializer.java
>  PRE-CREATION 
>   
> flume-ng-core/src/main/java/org/apache/flume/interceptor/RegexExtractorInterceptorPassThroughSerializer.java
>  PRE-CREATION 
>   
> flume-ng-core/src/main/java/org/apache/flume/interceptor/RegexExtractorInterceptorSerializer.java
>  PRE-CREATION 
>   
> flume-ng-core/src/test/java/org/apache/flume/interceptor/TestRegexExtractorInterceptor.java
>  PRE-CREATION 
>   
> flume-ng-core/src/test/java/org/apache/flume/interceptor/TestRegexExtractorInterceptorMillisSerializer.java
>  PRE-CREATION 
>   
> flume-ng-core/src/test/java/org/apache/flume/interceptor/TestRegexExtractorInterceptorPassThroughSerializer.java
>  PRE-CREATION 
>   flume-ng-doc/sphinx/FlumeUserGuide.rst 0596473 
>   flume-ng-doc/sphinx/FlumeUserGuide.rst.orig PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/7700/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Cameron Gandevia
> 
>

Re: Review Request: FLUME-1657 - Regex Extractor Interceptor

[Cameron Gandevia]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7700/
---

(Updated Nov. 14, 2012, 1:24 a.m.)


Review request for Flume.


Changes
---

Code review comment fixes


Description
---

A RegexExtractor interceptor that will allow users to extract regex matches and 
append them as header fields of the event.


Diffs (updated)
-

  flume-ng-core/src/main/java/org/apache/flume/interceptor/InterceptorType.java 
c478337 
  
flume-ng-core/src/main/java/org/apache/flume/interceptor/RegexExtractorInterceptor.java
 PRE-CREATION 
  
flume-ng-core/src/main/java/org/apache/flume/interceptor/RegexExtractorInterceptorMillisSerializer.java
 PRE-CREATION 
  
flume-ng-core/src/main/java/org/apache/flume/interceptor/RegexExtractorInterceptorPassThroughSerializer.java
 PRE-CREATION 
  
flume-ng-core/src/main/java/org/apache/flume/interceptor/RegexExtractorInterceptorSerializer.java
 PRE-CREATION 
  
flume-ng-core/src/test/java/org/apache/flume/interceptor/TestRegexExtractorInterceptor.java
 PRE-CREATION 
  
flume-ng-core/src/test/java/org/apache/flume/interceptor/TestRegexExtractorInterceptorMillisSerializer.java
 PRE-CREATION 
  
flume-ng-core/src/test/java/org/apache/flume/interceptor/TestRegexExtractorInterceptorPassThroughSerializer.java
 PRE-CREATION 
  flume-ng-doc/sphinx/FlumeUserGuide.rst 0596473 
  flume-ng-doc/sphinx/FlumeUserGuide.rst.orig PRE-CREATION 

Diff: https://reviews.apache.org/r/7700/diff/


Testing
---


Thanks,

Cameron Gandevia

[jira] [Created] (FLUME-1707) Update FlumeDevGuide and FlumeUserGuide

[Will McQueen (JIRA)]

Will McQueen created FLUME-1707:
---

 Summary: Update FlumeDevGuide and FlumeUserGuide
 Key: FLUME-1707
 URL: https://issues.apache.org/jira/browse/FLUME-1707
 Project: Flume
  Issue Type: Bug
  Components: Docs
Affects Versions: v1.3.0
Reporter: Will McQueen
Assignee: Will McQueen
Priority: Blocker
 Fix For: v1.3.0


There are several places in docs that have missing or incorrect info. These 
need to be updated.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (FLUME-1706) Website for 1.3 fails to build

[Brock Noland (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-1706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brock Noland updated FLUME-1706:


 Priority: Blocker  (was: Major)
Fix Version/s: v1.3.0

> Website for 1.3 fails to build
> --
>
> Key: FLUME-1706
> URL: https://issues.apache.org/jira/browse/FLUME-1706
> Project: Flume
>  Issue Type: Sub-task
>Reporter: Brock Noland
>Priority: Blocker
> Fix For: v1.3.0
>
> Attachments: Flume 1.3 Developer Guide.pdf, Flume 1.3 User Guide.pdf, 
> Screen Shot 2012-11-13 at 4.16.54 PM.png
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (FLUME-1706) Website for 1.3 fails to build

[Brock Noland (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-1706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brock Noland updated FLUME-1706:


Attachment: Screen Shot 2012-11-13 at 4.16.54 PM.png
Flume 1.3 User Guide.pdf
Flume 1.3 Developer Guide.pdf

user/dev guides have some invalid entries.

> Website for 1.3 fails to build
> --
>
> Key: FLUME-1706
> URL: https://issues.apache.org/jira/browse/FLUME-1706
> Project: Flume
>  Issue Type: Sub-task
>Reporter: Brock Noland
> Fix For: v1.3.0
>
> Attachments: Flume 1.3 Developer Guide.pdf, Flume 1.3 User Guide.pdf, 
> Screen Shot 2012-11-13 at 4.16.54 PM.png
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (FLUME-1706) Website for 1.3 fails to build

[Brock Noland (JIRA)]

Brock Noland created FLUME-1706:
---

 Summary: Website for 1.3 fails to build
 Key: FLUME-1706
 URL: https://issues.apache.org/jira/browse/FLUME-1706
 Project: Flume
  Issue Type: Sub-task
Reporter: Brock Noland
 Attachments: Flume 1.3 Developer Guide.pdf, Flume 1.3 User Guide.pdf, 
Screen Shot 2012-11-13 at 4.16.54 PM.png



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: [VOTE] Release Apache Flume version 1.3.0-rc2

[Brock Noland]

Hi,

I actually thought the maven item as odd as well. MRUnit deploys to
the staging repository (See "Deploy to Nexus Staging Repository"
http://mrunit.apache.org/pmc/how_to_release.html).

The documentation will be updated, I believe these can verified
locally by using mvn site. Actually I have just looked at version I
generated earlier today and there are some updates needed.

Brock

On Tue, Nov 13, 2012 at 3:29 PM, Ralph Goers  wrote:
> How can I verify the Maven artifacts?  It really is a problem to deploy them 
> after the vote since the can't be verified.
>
> What web site updates will be made?  How can we verify those?
>
> Ralph
>
> On Nov 13, 2012, at 8:39 AM, Brock Noland wrote:
>
>> This is the second release for Apache Flume as a top-level project,
>> version 1.3.0. We are voting on release candidate rc2.
>>
>> *** Please cast your vote within the next 72 hours ***
>>
>> The list of fixed issues:
>> http://s.apache.org/A9x
>>
>> The tarball (*.tar.gz), signature (*.asc), and checksums (*.md5, *.sha1)
>> for the source and binary artifacts can be found at:
>> http://people.apache.org/~brock/flume-1.3.0-rc2/
>>
>> The tag to be voted on:
>> http://s.apache.org/aVa
>>
>> The KEYS file can be found here:
>> https://dist.apache.org/repos/dist/release/flume/KEYS
>



-- 
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/

Re: [VOTE] Release Apache Flume version 1.3.0-rc2

[Brock Noland]

Blocker came up on the user list

https://issues.apache.org/jira/browse/FLUME-1705

Brock

On Tue, Nov 13, 2012 at 2:59 PM, Roshan Naik  wrote:
> Looks good. Verified build and tests.
> +1  (non binding)
> -roshan
>
>
> On Tue, Nov 13, 2012 at 8:41 AM, Brock Noland  wrote:
>
>> +1 verified hashes, sigs, and performed single node sanity test
>>
>> On Tue, Nov 13, 2012 at 10:39 AM, Brock Noland  wrote:
>> > This is the second release for Apache Flume as a top-level project,
>> > version 1.3.0. We are voting on release candidate rc2.
>> >
>> > *** Please cast your vote within the next 72 hours ***
>> >
>> > The list of fixed issues:
>> > http://s.apache.org/A9x
>> >
>> > The tarball (*.tar.gz), signature (*.asc), and checksums (*.md5, *.sha1)
>> > for the source and binary artifacts can be found at:
>> > http://people.apache.org/~brock/flume-1.3.0-rc2/
>> >
>> > The tag to be voted on:
>> > http://s.apache.org/aVa
>> >
>> > The KEYS file can be found here:
>> > https://dist.apache.org/repos/dist/release/flume/KEYS
>>
>>
>>
>> --
>> Apache MRUnit - Unit testing MapReduce -
>> http://incubator.apache.org/mrunit/
>>



-- 
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/

[jira] [Created] (FLUME-1705) SpoolDirectory short name points at the wrong class

[Brock Noland (JIRA)]

Brock Noland created FLUME-1705:
---

 Summary: SpoolDirectory short name points at the wrong class
 Key: FLUME-1705
 URL: https://issues.apache.org/jira/browse/FLUME-1705
 Project: Flume
  Issue Type: Bug
Affects Versions: v1.3.0
Reporter: Brock Noland
 Fix For: v1.3.0


is `org.apache.flume.conf.source.SpoolDirectorySource' should be 
`org.apache.flume.source.SpoolDirectorySource'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (FLUME-1705) SpoolDirectory short name points at the wrong class

[Brock Noland (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-1705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brock Noland updated FLUME-1705:


Priority: Blocker  (was: Major)

> SpoolDirectory short name points at the wrong class
> ---
>
> Key: FLUME-1705
> URL: https://issues.apache.org/jira/browse/FLUME-1705
> Project: Flume
>  Issue Type: Bug
>Affects Versions: v1.3.0
>Reporter: Brock Noland
>Priority: Blocker
> Fix For: v1.3.0
>
>
> is `org.apache.flume.conf.source.SpoolDirectorySource' should be 
> `org.apache.flume.source.SpoolDirectorySource'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: [VOTE] Release Apache Flume version 1.3.0-rc2

[Ralph Goers]

How can I verify the Maven artifacts?  It really is a problem to deploy them 
after the vote since the can't be verified.

What web site updates will be made?  How can we verify those?

Ralph

On Nov 13, 2012, at 8:39 AM, Brock Noland wrote:

> This is the second release for Apache Flume as a top-level project,
> version 1.3.0. We are voting on release candidate rc2.
> 
> *** Please cast your vote within the next 72 hours ***
> 
> The list of fixed issues:
> http://s.apache.org/A9x
> 
> The tarball (*.tar.gz), signature (*.asc), and checksums (*.md5, *.sha1)
> for the source and binary artifacts can be found at:
> http://people.apache.org/~brock/flume-1.3.0-rc2/
> 
> The tag to be voted on:
> http://s.apache.org/aVa
> 
> The KEYS file can be found here:
> https://dist.apache.org/repos/dist/release/flume/KEYS

[jira] [Commented] (FLUME-1702) HDFSEventSink should write to a hidden file as opposed to a .tmp file

[Mike Percy (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLUME-1702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13496540#comment-13496540
 ] 

Mike Percy commented on FLUME-1702:
---

Whoops, didn't notice you filed this JIRA Brock. Adding description from dup 
ticket:

We should add the capability to the HDFS sink to specify a prefix for the .tmp 
files. I believe this needs to be configurable and disabled by default.
However we should document that we recommend "_" or "." as a prefix for the 
temp files.
This is because Hadoop's default FileInputFormat will skip files beginning with 
"_" or "." (hidden files)

> HDFSEventSink should write to a hidden file as opposed to a .tmp file
> -
>
> Key: FLUME-1702
> URL: https://issues.apache.org/jira/browse/FLUME-1702
> Project: Flume
>  Issue Type: Improvement
>Reporter: Brock Noland
>
> Currently we write to a .tmp file. The problem is that if MR jobs are being 
> run on the directory we are writing to, then it's common for an MR job to 
> list the directory, get a .tmp file and then in the mean time the .tmp file 
> is renamed causing the job to fail when run.
> Using JavaMR you can use a PathFilter to avoid this, however a custom 
> solution is required for Pig, Hive, etc.
> Perhaps we should write to a hidden file so that MR never tries to process 
> data in flight.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: [VOTE] Release Apache Flume version 1.3.0-rc2

[Roshan Naik]

Looks good. Verified build and tests.
+1  (non binding)
-roshan


On Tue, Nov 13, 2012 at 8:41 AM, Brock Noland  wrote:

> +1 verified hashes, sigs, and performed single node sanity test
>
> On Tue, Nov 13, 2012 at 10:39 AM, Brock Noland  wrote:
> > This is the second release for Apache Flume as a top-level project,
> > version 1.3.0. We are voting on release candidate rc2.
> >
> > *** Please cast your vote within the next 72 hours ***
> >
> > The list of fixed issues:
> > http://s.apache.org/A9x
> >
> > The tarball (*.tar.gz), signature (*.asc), and checksums (*.md5, *.sha1)
> > for the source and binary artifacts can be found at:
> > http://people.apache.org/~brock/flume-1.3.0-rc2/
> >
> > The tag to be voted on:
> > http://s.apache.org/aVa
> >
> > The KEYS file can be found here:
> > https://dist.apache.org/repos/dist/release/flume/KEYS
>
>
>
> --
> Apache MRUnit - Unit testing MapReduce -
> http://incubator.apache.org/mrunit/
>

[jira] [Resolved] (FLUME-1704) HDFS sink: Add option to prefix .tmp files with some string like "_" or "."

[Brock Noland (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-1704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brock Noland resolved FLUME-1704.
-

Resolution: Duplicate

Duplicates FLUME-1702

> HDFS sink: Add option to prefix .tmp files with some string like "_" or "."
> ---
>
> Key: FLUME-1704
> URL: https://issues.apache.org/jira/browse/FLUME-1704
> Project: Flume
>  Issue Type: Improvement
>Reporter: Mike Percy
> Fix For: v1.4.0
>
>
> We should add the capability to the HDFS sink to specify a prefix for the 
> .tmp files. I believe this needs to be configurable and disabled by default.
> However we should document that we recommend "_" or "." as a prefix for the 
> temp files.
> This is because Hadoop's default FileInputFormat will skip files beginning 
> with "_" or "."

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (FLUME-1704) HDFS sink: Add option to prefix .tmp files with some string like "_" or "."

[Mike Percy (JIRA)]

Mike Percy created FLUME-1704:
-

 Summary: HDFS sink: Add option to prefix .tmp files with some 
string like "_" or "."
 Key: FLUME-1704
 URL: https://issues.apache.org/jira/browse/FLUME-1704
 Project: Flume
  Issue Type: Improvement
Reporter: Mike Percy
 Fix For: v1.4.0


We should add the capability to the HDFS sink to specify a prefix for the .tmp 
files. I believe this needs to be configurable and disabled by default.

However we should document that we recommend "_" or "." as a prefix for the 
temp files.

This is because Hadoop's default FileInputFormat will skip files beginning with 
"_" or "."

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (FLUME-1703) Update RELEASE-NOTES in trunk to 1.4

[Brock Noland (JIRA)]

Brock Noland created FLUME-1703:
---

 Summary: Update RELEASE-NOTES in trunk to 1.4
 Key: FLUME-1703
 URL: https://issues.apache.org/jira/browse/FLUME-1703
 Project: Flume
  Issue Type: Sub-task
Reporter: Brock Noland
Assignee: Brock Noland




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (FLUME-1702) HDFSEventSink should write to a hidden file as opposed to a .tmp file

[Brock Noland (JIRA)]

Brock Noland created FLUME-1702:
---

 Summary: HDFSEventSink should write to a hidden file as opposed to 
a .tmp file
 Key: FLUME-1702
 URL: https://issues.apache.org/jira/browse/FLUME-1702
 Project: Flume
  Issue Type: Improvement
Reporter: Brock Noland


Currently we write to a .tmp file. The problem is that if MR jobs are being run 
on the directory we are writing to, then it's common for an MR job to list the 
directory, get a .tmp file and then in the mean time the .tmp file is renamed 
causing the job to fail when run.

Using JavaMR you can use a PathFilter to avoid this, however a custom solution 
is required for Pig, Hive, etc.

Perhaps we should write to a hidden file so that MR never tries to process data 
in flight.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: [VOTE] Release Apache Flume version 1.3.0-rc2

[Brock Noland]

+1 verified hashes, sigs, and performed single node sanity test

On Tue, Nov 13, 2012 at 10:39 AM, Brock Noland  wrote:
> This is the second release for Apache Flume as a top-level project,
> version 1.3.0. We are voting on release candidate rc2.
>
> *** Please cast your vote within the next 72 hours ***
>
> The list of fixed issues:
> http://s.apache.org/A9x
>
> The tarball (*.tar.gz), signature (*.asc), and checksums (*.md5, *.sha1)
> for the source and binary artifacts can be found at:
> http://people.apache.org/~brock/flume-1.3.0-rc2/
>
> The tag to be voted on:
> http://s.apache.org/aVa
>
> The KEYS file can be found here:
> https://dist.apache.org/repos/dist/release/flume/KEYS



-- 
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/

[VOTE] Release Apache Flume version 1.3.0-rc2

[Brock Noland]

This is the second release for Apache Flume as a top-level project,
version 1.3.0. We are voting on release candidate rc2.

*** Please cast your vote within the next 72 hours ***

The list of fixed issues:
http://s.apache.org/A9x

The tarball (*.tar.gz), signature (*.asc), and checksums (*.md5, *.sha1)
for the source and binary artifacts can be found at:
http://people.apache.org/~brock/flume-1.3.0-rc2/

The tag to be voted on:
http://s.apache.org/aVa

The KEYS file can be found here:
https://dist.apache.org/repos/dist/release/flume/KEYS

[jira] [Commented] (FLUME-1227) Introduce some sort of SpillableChannel

[Jarek Jarcec Cecho (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLUME-1227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13496300#comment-13496300
 ] 

Jarek Jarcec Cecho commented on FLUME-1227:
---

Hi Roshan,
you're right that some higher general logic like channel overflow would be 
interesting. I'm just worried that such logic would be hard to test and even 
harder to debug on users boxes. 

I would recommend taking a look on linked linked issue FLUME-1045 where we've 
discussed similar concept and the "general" consensus seems to be that 
SpillableChannel is quite limited in functionality when compared to the general 
solutions, but it should solve most of the use cases and still be testable and 
debugable.

Jarcec

> Introduce some sort of SpillableChannel
> ---
>
> Key: FLUME-1227
> URL: https://issues.apache.org/jira/browse/FLUME-1227
> Project: Flume
>  Issue Type: New Feature
>  Components: Channel
>Reporter: Jarek Jarcec Cecho
>
> I would like to introduce new channel that would behave similarly as scribe 
> (https://github.com/facebook/scribe). It would be something between memory 
> and file channel. Input events would be saved directly to the memory (only) 
> and would be served from there. In case that the memory would be full, we 
> would outsource the events to file.
> Let me describe the use case behind this request. We have plenty of frontend 
> servers that are generating events. We want to send all events to just 
> limited number of machines from where we would send the data to HDFS (some 
> sort of staging layer). Reason for this second layer is our need to decouple 
> event aggregation and front end code to separate machines. Using memory 
> channel is fully sufficient as we can survive lost of some portion of the 
> events. However in order to sustain maintenance windows or networking issues 
> we would have to end up with a lot of memory assigned to those "staging" 
> machines. Referenced "scribe" is dealing with this problem by implementing 
> following logic - events are saved in memory similarly as our MemoryChannel. 
> However in case that the memory gets full (because of maintenance, networking 
> issues, ...) it will spill data to disk where they will be sitting until 
> everything start working again.
> I would like to introduce channel that would implement similar logic. It's 
> durability guarantees would be same as MemoryChannel - in case that someone 
> would remove power cord, this channel would lose data. Based on the 
> discussion in FLUME-1201, I would propose to have the implementation 
> completely independent on any other channel internal code.
> Jarcec

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (FLUME-1502) Support for running simple configurations embedded in host process

[Brock Noland (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLUME-1502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13496247#comment-13496247
 ] 

Brock Noland commented on FLUME-1502:
-

Great! It'd be awesome if someone could give me a +1 or feedback on FLUME-1630 
since I will be building a patch based on that. It's not absolutely necessary 
but if there is feedback it will require re-work here.

> Support for running simple configurations embedded in host process
> --
>
> Key: FLUME-1502
> URL: https://issues.apache.org/jira/browse/FLUME-1502
> Project: Flume
>  Issue Type: Improvement
>Affects Versions: v1.2.0
>Reporter: Arvind Prabhakar
>Assignee: Brock Noland
> Attachments: embeeded-agent-1.pdf, embeeded-agent-2.pdf
>
>
> Flume should provide a light-weight embeddable node manager that can be 
> started in process where necessary. This will allow the users to embed 
> light-weight agents within the host process where necessary.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: [ANNOUNCE] New Apache Flume committer - Patrick Wendell

[Mike Percy]

Patrick, welcome! Great to have you on board.

Regards,
Mike

On Mon, Nov 12, 2012 at 1:04 PM, Hari Shreedharan  wrote:

> On behalf of the Apache Flume PMC,  I am excited to welcome Patrick
> Wendell as a committer on Flume! Patrick has contributed significantly to
> the project, by adding new features, fixing bugs and helping users on the
> Flume users list. Here is a list of jiras Patrick has worked on:
>
> http://s.apache.org/6EG
>
> Please join me in congratulating Patrick on his new role!
>
>
> Thanks,
> Hari
>
>
>
>
>

Re: [VOTE] Release Apache Flume version 1.3.0-rc1

[Brock Noland]

Yeah my environment didn't hit this issue... I'll fix it and re-spin
this morning.

On Tue, Nov 13, 2012 at 12:57 AM, Hari Shreedharan
 wrote:
> Looks like the Jenkins build also failed the unit tests because of this.
>
> Hari
>
> --
> Hari Shreedharan
>
>
> On Monday, November 12, 2012 at 4:40 PM, Brock Noland wrote:
>
>> https://issues.apache.org/jira/browse/FLUME-1701
>>
>> On Mon, Nov 12, 2012 at 6:38 PM, Brock Noland > (mailto:br...@cloudera.com)> wrote:
>> > Hi,
>> >
>> > Hmm, my test alias adds that property. Looks like StagedInstall does
>> > hard code the -dist name,
>> >
>> > && name.endsWith("-dist.tar.gz")) {
>> >
>> > Brock
>> >
>> > On Mon, Nov 12, 2012 at 6:33 PM, Roshan Naik > > (mailto:ros...@hortonworks.com)> wrote:
>> > > -1 (non binding)
>> > >
>> > > I see the following Integration tests .. sepcifically (TestFileChannel,
>> > > TestRpcClient, TestRpcClientCommunicationFailure )
>> > >
>> > > The nature of the error is the same...
>> > >
>> > > > > > classname="org.apache.flume.test.agent.TestFileChannel" name="testInOut">
>> > > > > > type="java.lang.Exception">java.lang.Exception: Failed to locate tar-ball
>> > > distribution. Please specify explicitly via system property:
>> > > flume.dist.tarball
>> > > at
>> > > org.apache.flume.test.util.StagedInstall.(StagedInstall.java:207)
>> > > at
>> > > org.apache.flume.test.util.StagedInstall.getInstance(StagedInstall.java:75)
>> > >
>> > > Build Steps:...
>> > >
>> > > - mvn package -DskipTests
>> > > - mvn test
>> > >
>> > > I got the same erors with ... mvn clean install.
>> > >
>> > > See attachment for more details on the error. fyi.. I am building on a
>> > > macbook pro.
>> > > -roshan
>> > >
>> >
>> >
>> >
>> >
>> > --
>> > Apache MRUnit - Unit testing MapReduce - 
>> > http://incubator.apache.org/mrunit/
>> >
>>
>>
>>
>>
>> --
>> Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
>>
>>
>
>



-- 
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/

Re: [ANNOUNCE] New Apache Flume committer - Patrick Wendell

[Arvind Prabhakar]

Congratulations Patrick! Well deserved!

Regards,
Arvind Prabhakar

On Mon, Nov 12, 2012 at 1:04 PM, Hari Shreedharan  wrote:

> On behalf of the Apache Flume PMC,  I am excited to welcome Patrick
> Wendell as a committer on Flume! Patrick has contributed significantly to
> the project, by adding new features, fixing bugs and helping users on the
> Flume users list. Here is a list of jiras Patrick has worked on:
>
> http://s.apache.org/6EG
>
> Please join me in congratulating Patrick on his new role!
>
>
> Thanks,
> Hari
>
>
>
>
>