[jira] [Updated] (FLUME-2538) TestResettableFileInputStream fails on JDK 8

[Johny Rufus (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Johny Rufus updated FLUME-2538:
---
Attachment: FLUME-2538-1.patch

Thanks Hari, attaching the modified patch

> TestResettableFileInputStream fails on JDK 8
> 
>
> Key: FLUME-2538
> URL: https://issues.apache.org/jira/browse/FLUME-2538
> Project: Flume
>  Issue Type: Bug
>Affects Versions: v1.5.0.1
>Reporter: Johny Rufus
>Assignee: Johny Rufus
> Fix For: v1.6.0
>
> Attachments: FLUME-2538-1.patch, FLUME-2538.patch
>
>
> TestResettableFileInputStream.testUtf8DecodeErrorHandlingReplace fails in JDK 
> 8
> "testUtf8DecodeErrorHandlingReplace(org.apache.flume.serialization.TestResettableFileInputStream)
>   Time elapsed: 6 sec  <<< FAILURE!
> org.junit.ComparisonFailure: expected:<...(���)
> NonUnicode: (�[])
> > but was:<...(���)
> NonUnicode: (�[]) "
> Charsetdecoder.decode has changed in its behavior, as to how it handles 
> CodingErrorAction.Replace policy 
> Will submit a patch today.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: [VOTE] Apache Flume 1.5.2 RC1

[Jarek Jarcec Cecho]

+1

* Verified checksums and signature files
* Verified that each jar in binary tarball is in the license
* Checked top level files (NOTICE, ...)
* Run tests

(pretty much the same email I’ve sent for 1.5.1 :))

Jarcec
> On Nov 12, 2014, at 1:15 PM, Hari Shreedharan  
> wrote:
> 
> This is the eighth release for Apache Flume as a top-level project,
> version 1.5.2. We are voting on release candidate RC1.
> 
> This release fixes an incompatibility with Java 6 based clients found
> in Apache Flume 1.5.1 Release.
> 
> It fixes the following
> issues:https://git-wip-us.apache.org/repos/asf?p=flume.git;a=blob;f=CHANGELOG;h=cc7321361d0b702ba870de20d6a3d2106987186a;hb=229442aa6835ee0faa17e3034bcab42754c460f5
> 
> *** Please cast your vote within the next 72 hours ***
> 
> The tarball (*.tar.gz), signature (*.asc), and checksums (*.md5, *.sha1)
> for the source and binary artifacts can be found here:
>  *https://people.apache.org/~hshreedharan/apache-flume-1.5.2-rc1/
> *
> 
> Maven staging repo:
>  *https://repository.apache.org/content/repositories/orgapacheflume-1008/
> *
> 
> The tag to be voted on:
>  
> *https://git-wip-us.apache.org/repos/asf?p=flume.git;a=commit;h=229442aa6835ee0faa17e3034bcab42754c460f5
> *
> 
> Flume's KEYS file containing PGP keys we use to sign the release:
>  http://www.apache.org/dist/flume/KEYS
> 
> 
> Thanks,
> Hari

Re: [VOTE] Apache Flume 1.5.2 RC1

[Ashish]

+1

- Build work fine
- All test cases pass

On Thu, Nov 13, 2014 at 2:45 AM, Hari Shreedharan
 wrote:
> This is the eighth release for Apache Flume as a top-level project,
> version 1.5.2. We are voting on release candidate RC1.
>
> This release fixes an incompatibility with Java 6 based clients found
> in Apache Flume 1.5.1 Release.
>
> It fixes the following
> issues:https://git-wip-us.apache.org/repos/asf?p=flume.git;a=blob;f=CHANGELOG;h=cc7321361d0b702ba870de20d6a3d2106987186a;hb=229442aa6835ee0faa17e3034bcab42754c460f5
>
> *** Please cast your vote within the next 72 hours ***
>
> The tarball (*.tar.gz), signature (*.asc), and checksums (*.md5, *.sha1)
> for the source and binary artifacts can be found here:
>   *https://people.apache.org/~hshreedharan/apache-flume-1.5.2-rc1/
> *
>
> Maven staging repo:
>   *https://repository.apache.org/content/repositories/orgapacheflume-1008/
> *
>
> The tag to be voted on:
>   
> *https://git-wip-us.apache.org/repos/asf?p=flume.git;a=commit;h=229442aa6835ee0faa17e3034bcab42754c460f5
> *
>
> Flume's KEYS file containing PGP keys we use to sign the release:
>   http://www.apache.org/dist/flume/KEYS
>
>
> Thanks,
> Hari



-- 
thanks
ashish

Blog: http://www.ashishpaliwal.com/blog
My Photo Galleries: http://www.pbase.com/ashishpaliwal

Re: Review Request 18544: Hive Streaming sink

[Roshan Naik]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18544/
---

(Updated Nov. 13, 2014, 3:05 a.m.)


Review request for Flume.


Bugs: FLUME-1734
https://issues.apache.org/jira/browse/FLUME-1734


Repository: flume-git


Description
---

Hive streaming sink.


Diffs (updated)
-

  bin/flume-ng 4b323a6 
  conf/log4j.properties 3918511 
  
flume-ng-configuration/src/main/java/org/apache/flume/conf/sink/SinkConfiguration.java
 ac11558 
  flume-ng-configuration/src/main/java/org/apache/flume/conf/sink/SinkType.java 
0a1cd7a 
  flume-ng-dist/pom.xml a5db0c7 
  flume-ng-doc/sphinx/FlumeUserGuide.rst 0ab23fd 
  
flume-ng-legacy-sources/flume-thrift-source/src/main/java/com/cloudera/flume/handlers/thrift/ThriftFlumeEventServer.java
 ff32c45 
  
flume-ng-legacy-sources/flume-thrift-source/src/test/java/org/apache/flume/source/thriftLegacy/TestThriftLegacySource.java
 8e08f22 
  flume-ng-sdk/src/main/java/org/apache/flume/thrift/ThriftSourceProtocol.java 
7f966b0 
  flume-ng-sinks/flume-hive-sink/pom.xml PRE-CREATION 
  
flume-ng-sinks/flume-hive-sink/src/main/java/org/apache/flume/sink/hive/Config.java
 PRE-CREATION 
  
flume-ng-sinks/flume-hive-sink/src/main/java/org/apache/flume/sink/hive/HiveDelimitedTextSerializer.java
 PRE-CREATION 
  
flume-ng-sinks/flume-hive-sink/src/main/java/org/apache/flume/sink/hive/HiveEventSerializer.java
 PRE-CREATION 
  
flume-ng-sinks/flume-hive-sink/src/main/java/org/apache/flume/sink/hive/HiveJsonSerializer.java
 PRE-CREATION 
  
flume-ng-sinks/flume-hive-sink/src/main/java/org/apache/flume/sink/hive/HiveSink.java
 PRE-CREATION 
  
flume-ng-sinks/flume-hive-sink/src/main/java/org/apache/flume/sink/hive/HiveWriter.java
 PRE-CREATION 
  
flume-ng-sinks/flume-hive-sink/src/test/java/org/apache/flume/sink/hive/TestHiveSink.java
 PRE-CREATION 
  
flume-ng-sinks/flume-hive-sink/src/test/java/org/apache/flume/sink/hive/TestHiveWriter.java
 PRE-CREATION 
  
flume-ng-sinks/flume-hive-sink/src/test/java/org/apache/flume/sink/hive/TestUtil.java
 PRE-CREATION 
  flume-ng-sinks/flume-hive-sink/src/test/resources/log4j.properties 
PRE-CREATION 
  flume-ng-sinks/pom.xml 4bac019 
  
flume-ng-sources/flume-scribe-source/src/main/java/org/apache/flume/source/scribe/Scribe.java
 eba8d2e 
  pom.xml 4f550d3 

Diff: https://reviews.apache.org/r/18544/diff/


Testing
---

includes unit tests.


Thanks,

Roshan Naik

Re: Review Request 18544: Hive Streaming sink

[Roshan Naik]

> On Oct. 14, 2014, 11:05 p.m., Brock Noland wrote:
> > flume-ng-sinks/flume-hive-sink/src/test/java/org/apache/flume/sink/hive/TestUtil.java,
> >  line 136
> > 
> >
> > We should log exceptions like this.

loogging not yet been added in for unit tests. this class is used for unit 
tests.


> On Oct. 14, 2014, 11:05 p.m., Brock Noland wrote:
> > flume-ng-legacy-sources/flume-thrift-source/src/main/java/com/cloudera/flume/handlers/thrift/ThriftFlumeEventServer.java,
> >  line 212
> > 
> >
> > are these changes due to the rev in the thrift version?

yes. my notes suggest a race condition in TThreadSeverPool


- Roshan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18544/#review56592
---


On Sept. 3, 2014, 3:19 a.m., Roshan Naik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/18544/
> ---
> 
> (Updated Sept. 3, 2014, 3:19 a.m.)
> 
> 
> Review request for Flume.
> 
> 
> Bugs: FLUME-1734
> https://issues.apache.org/jira/browse/FLUME-1734
> 
> 
> Repository: flume-git
> 
> 
> Description
> ---
> 
> Hive streaming sink.
> 
> 
> Diffs
> -
> 
>   bin/flume-ng e09e26b 
>   conf/log4j.properties 3918511 
>   
> flume-ng-configuration/src/main/java/org/apache/flume/conf/sink/SinkConfiguration.java
>  ac11558 
>   
> flume-ng-configuration/src/main/java/org/apache/flume/conf/sink/SinkType.java 
> 0a1cd7a 
>   flume-ng-dist/pom.xml 8c18af6 
>   flume-ng-doc/sphinx/FlumeUserGuide.rst a718fbf 
>   
> flume-ng-legacy-sources/flume-thrift-source/src/main/java/com/cloudera/flume/handlers/thrift/ThriftFlumeEventServer.java
>  ff32c45 
>   
> flume-ng-legacy-sources/flume-thrift-source/src/test/java/org/apache/flume/source/thriftLegacy/TestThriftLegacySource.java
>  8e08f22 
>   
> flume-ng-sdk/src/main/java/org/apache/flume/thrift/ThriftSourceProtocol.java 
> 7f966b0 
>   flume-ng-sinks/flume-hive-sink/pom.xml PRE-CREATION 
>   
> flume-ng-sinks/flume-hive-sink/src/main/java/org/apache/flume/sink/hive/Config.java
>  PRE-CREATION 
>   
> flume-ng-sinks/flume-hive-sink/src/main/java/org/apache/flume/sink/hive/HiveDelimitedTextSerializer.java
>  PRE-CREATION 
>   
> flume-ng-sinks/flume-hive-sink/src/main/java/org/apache/flume/sink/hive/HiveEventSerializer.java
>  PRE-CREATION 
>   
> flume-ng-sinks/flume-hive-sink/src/main/java/org/apache/flume/sink/hive/HiveJsonSerializer.java
>  PRE-CREATION 
>   
> flume-ng-sinks/flume-hive-sink/src/main/java/org/apache/flume/sink/hive/HiveSink.java
>  PRE-CREATION 
>   
> flume-ng-sinks/flume-hive-sink/src/main/java/org/apache/flume/sink/hive/HiveWriter.java
>  PRE-CREATION 
>   
> flume-ng-sinks/flume-hive-sink/src/test/java/org/apache/flume/sink/hive/TestHiveSink.java
>  PRE-CREATION 
>   
> flume-ng-sinks/flume-hive-sink/src/test/java/org/apache/flume/sink/hive/TestHiveWriter.java
>  PRE-CREATION 
>   
> flume-ng-sinks/flume-hive-sink/src/test/java/org/apache/flume/sink/hive/TestUtil.java
>  PRE-CREATION 
>   flume-ng-sinks/flume-hive-sink/src/test/resources/log4j.properties 
> PRE-CREATION 
>   flume-ng-sinks/pom.xml 3381bde 
>   
> flume-ng-sources/flume-scribe-source/src/main/java/org/apache/flume/source/scribe/Scribe.java
>  eba8d2e 
>   pom.xml 4bdfcac 
> 
> Diff: https://reviews.apache.org/r/18544/diff/
> 
> 
> Testing
> ---
> 
> includes unit tests.
> 
> 
> Thanks,
> 
> Roshan Naik
> 
>

[jira] [Updated] (FLUME-1734) Create a Hive Sink based on the new Hive Streaming support

[Roshan Naik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-1734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Roshan Naik updated FLUME-1734:
---
Attachment: FLUME-1734.v6.patch

Uploading patch which addresses [~brocknoland]'s review comments.

> Create a Hive Sink based on the new Hive Streaming support
> --
>
> Key: FLUME-1734
> URL: https://issues.apache.org/jira/browse/FLUME-1734
> Project: Flume
>  Issue Type: New Feature
>  Components: Sinks+Sources
>Affects Versions: v1.2.0
>Reporter: Roshan Naik
>Assignee: Roshan Naik
>  Labels: features
> Attachments: FLUME-1734.draft.1.patch, FLUME-1734.draft.2.patch, 
> FLUME-1734.v1.patch, FLUME-1734.v2.patch, FLUME-1734.v4.patch, 
> FLUME-1734.v5.patch, FLUME-1734.v6.patch
>
>
> Hive 0.13 has introduced Streaming support which is itself transactional in 
> nature and fits well with Flume's transaction model.
> Short overview of  Hive's  Streaming support on which this sink is based can 
> be found here:
> http://hive.apache.org/javadocs/r0.13.1/api/hcatalog/streaming/index.html
> This jira is for creating a sink that would continuously stream data into 
> Hive tables using the above APIs. The primary goal being that the data 
> streamed by the sink should be instantly queryable (using say Hive or Pig) 
> without requiring additional post-processing steps on behalf of the users. 
> Sink should manage the creation of new partitions periodically if needed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: [VOTE] Apache Flume 1.5.2 RC1

[Johny Rufus]

+1 .  Thanks Hari.

Verified the following:

1. Verified all tests against hbase-98
2. Verified all tests against hadoop-1
3. Verified License, Changelog, Notice, Readme, Release-notes
4. Verified all signatures/checksums


Thanks,
Rufus

On Wed, Nov 12, 2014 at 3:49 PM, Hari Shreedharan  wrote:

> 1.5.2 RC1 does exactly that. Please see:
> https://issues.apache.org/jira/browse/FLUME-2547
>
>
>
>
>
> We brought SSlv2Hello back in. Since 1.5.1 was already released, nothing
> we can do about it - so we are pushing out a new release.
>
>
> Thanks,
> Hari
>
> On Wed, Nov 12, 2014 at 3:25 PM, Roshan Naik 
> wrote:
>
> > Just found out that  .. both Ozzie and Hadoop had blocker jiras to revert
> > their decision to disable  SSLv2Hello.
> > OOZIE-2034 
> > HDFS-7391 
> > -roshan
> > On Wed, Nov 12, 2014 at 3:00 PM, Hari Shreedharan <
> hshreedha...@cloudera.com
> >> wrote:
> >> I tried to address all of the issues discussed in the 1.5.1 release vote
> >> (Except the snappy issue — I didn’t want to upgrade Avro in a
> maintenance
> >> release, lets do it in 1.6?).
> >>
> >>
> >>
> >>
> >> If you find something out of place, let me know - we can spin another
> RC.
> >>
> >>
> >> Thanks,
> >> Hari
> >>
> >> On Wed, Nov 12, 2014 at 1:15 PM, Hari Shreedharan
> >>  wrote:
> >>
> >> > This is the eighth release for Apache Flume as a top-level project,
> >> > version 1.5.2. We are voting on release candidate RC1.
> >> > This release fixes an incompatibility with Java 6 based clients found
> >> > in Apache Flume 1.5.1 Release.
> >> > It fixes the following
> >> > issues:
> >>
> https://git-wip-us.apache.org/repos/asf?p=flume.git;a=blob;f=CHANGELOG;h=cc7321361d0b702ba870de20d6a3d2106987186a;hb=229442aa6835ee0faa17e3034bcab42754c460f5
> >> > *** Please cast your vote within the next 72 hours ***
> >> > The tarball (*.tar.gz), signature (*.asc), and checksums (*.md5,
> *.sha1)
> >> > for the source and binary artifacts can be found here:
> >> >   *https://people.apache.org/~hshreedharan/apache-flume-1.5.2-rc1/
> >> > *
> >> > Maven staging repo:
> >> >   *
> >> https://repository.apache.org/content/repositories/orgapacheflume-1008/
> >> > <
> https://repository.apache.org/content/repositories/orgapacheflume-1008/
> >> >*
> >> > The tag to be voted on:
> >> >   *
> >>
> https://git-wip-us.apache.org/repos/asf?p=flume.git;a=commit;h=229442aa6835ee0faa17e3034bcab42754c460f5
> >> > <
> >>
> https://git-wip-us.apache.org/repos/asf?p=flume.git;a=commit;h=229442aa6835ee0faa17e3034bcab42754c460f5
> >> >*
> >> > Flume's KEYS file containing PGP keys we use to sign the release:
> >> >   http://www.apache.org/dist/flume/KEYS
> >> > Thanks,
> >> > Hari
> >>
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or entity
> to
> > which it is addressed and may contain information that is confidential,
> > privileged and exempt from disclosure under applicable law. If the reader
> > of this message is not the intended recipient, you are hereby notified
> that
> > any printing, copying, dissemination, distribution, disclosure or
> > forwarding of this communication is strictly prohibited. If you have
> > received this communication in error, please contact the sender
> immediately
> > and delete it from your system. Thank You.
>

Re: [VOTE] Apache Flume 1.5.2 RC1

[Hari Shreedharan]

1.5.2 RC1 does exactly that. Please see: 
https://issues.apache.org/jira/browse/FLUME-2547





We brought SSlv2Hello back in. Since 1.5.1 was already released, nothing we can 
do about it - so we are pushing out a new release.


Thanks,
Hari

On Wed, Nov 12, 2014 at 3:25 PM, Roshan Naik 
wrote:

> Just found out that  .. both Ozzie and Hadoop had blocker jiras to revert
> their decision to disable  SSLv2Hello.
> OOZIE-2034 
> HDFS-7391 
> -roshan
> On Wed, Nov 12, 2014 at 3:00 PM, Hari Shreedharan > wrote:
>> I tried to address all of the issues discussed in the 1.5.1 release vote
>> (Except the snappy issue — I didn’t want to upgrade Avro in a maintenance
>> release, lets do it in 1.6?).
>>
>>
>>
>>
>> If you find something out of place, let me know - we can spin another RC.
>>
>>
>> Thanks,
>> Hari
>>
>> On Wed, Nov 12, 2014 at 1:15 PM, Hari Shreedharan
>>  wrote:
>>
>> > This is the eighth release for Apache Flume as a top-level project,
>> > version 1.5.2. We are voting on release candidate RC1.
>> > This release fixes an incompatibility with Java 6 based clients found
>> > in Apache Flume 1.5.1 Release.
>> > It fixes the following
>> > issues:
>> https://git-wip-us.apache.org/repos/asf?p=flume.git;a=blob;f=CHANGELOG;h=cc7321361d0b702ba870de20d6a3d2106987186a;hb=229442aa6835ee0faa17e3034bcab42754c460f5
>> > *** Please cast your vote within the next 72 hours ***
>> > The tarball (*.tar.gz), signature (*.asc), and checksums (*.md5, *.sha1)
>> > for the source and binary artifacts can be found here:
>> >   *https://people.apache.org/~hshreedharan/apache-flume-1.5.2-rc1/
>> > *
>> > Maven staging repo:
>> >   *
>> https://repository.apache.org/content/repositories/orgapacheflume-1008/
>> > > >*
>> > The tag to be voted on:
>> >   *
>> https://git-wip-us.apache.org/repos/asf?p=flume.git;a=commit;h=229442aa6835ee0faa17e3034bcab42754c460f5
>> > <
>> https://git-wip-us.apache.org/repos/asf?p=flume.git;a=commit;h=229442aa6835ee0faa17e3034bcab42754c460f5
>> >*
>> > Flume's KEYS file containing PGP keys we use to sign the release:
>> >   http://www.apache.org/dist/flume/KEYS
>> > Thanks,
>> > Hari
>>
> -- 
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to 
> which it is addressed and may contain information that is confidential, 
> privileged and exempt from disclosure under applicable law. If the reader 
> of this message is not the intended recipient, you are hereby notified that 
> any printing, copying, dissemination, distribution, disclosure or 
> forwarding of this communication is strictly prohibited. If you have 
> received this communication in error, please contact the sender immediately 
> and delete it from your system. Thank You.

Re: [VOTE] Apache Flume 1.5.2 RC1

[Roshan Naik]

Just found out that  .. both Ozzie and Hadoop had blocker jiras to revert
their decision to disable  SSLv2Hello.
OOZIE-2034 
HDFS-7391 

-roshan



On Wed, Nov 12, 2014 at 3:00 PM, Hari Shreedharan  wrote:

> I tried to address all of the issues discussed in the 1.5.1 release vote
> (Except the snappy issue — I didn’t want to upgrade Avro in a maintenance
> release, lets do it in 1.6?).
>
>
>
>
> If you find something out of place, let me know - we can spin another RC.
>
>
> Thanks,
> Hari
>
> On Wed, Nov 12, 2014 at 1:15 PM, Hari Shreedharan
>  wrote:
>
> > This is the eighth release for Apache Flume as a top-level project,
> > version 1.5.2. We are voting on release candidate RC1.
> > This release fixes an incompatibility with Java 6 based clients found
> > in Apache Flume 1.5.1 Release.
> > It fixes the following
> > issues:
> https://git-wip-us.apache.org/repos/asf?p=flume.git;a=blob;f=CHANGELOG;h=cc7321361d0b702ba870de20d6a3d2106987186a;hb=229442aa6835ee0faa17e3034bcab42754c460f5
> > *** Please cast your vote within the next 72 hours ***
> > The tarball (*.tar.gz), signature (*.asc), and checksums (*.md5, *.sha1)
> > for the source and binary artifacts can be found here:
> >   *https://people.apache.org/~hshreedharan/apache-flume-1.5.2-rc1/
> > *
> > Maven staging repo:
> >   *
> https://repository.apache.org/content/repositories/orgapacheflume-1008/
> >  >*
> > The tag to be voted on:
> >   *
> https://git-wip-us.apache.org/repos/asf?p=flume.git;a=commit;h=229442aa6835ee0faa17e3034bcab42754c460f5
> > <
> https://git-wip-us.apache.org/repos/asf?p=flume.git;a=commit;h=229442aa6835ee0faa17e3034bcab42754c460f5
> >*
> > Flume's KEYS file containing PGP keys we use to sign the release:
> >   http://www.apache.org/dist/flume/KEYS
> > Thanks,
> > Hari
>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: [VOTE] Apache Flume 1.5.2 RC1

[Hari Shreedharan]

I tried to address all of the issues discussed in the 1.5.1 release vote 
(Except the snappy issue — I didn’t want to upgrade Avro in a maintenance 
release, lets do it in 1.6?). 




If you find something out of place, let me know - we can spin another RC.


Thanks,
Hari

On Wed, Nov 12, 2014 at 1:15 PM, Hari Shreedharan
 wrote:

> This is the eighth release for Apache Flume as a top-level project,
> version 1.5.2. We are voting on release candidate RC1.
> This release fixes an incompatibility with Java 6 based clients found
> in Apache Flume 1.5.1 Release.
> It fixes the following
> issues:https://git-wip-us.apache.org/repos/asf?p=flume.git;a=blob;f=CHANGELOG;h=cc7321361d0b702ba870de20d6a3d2106987186a;hb=229442aa6835ee0faa17e3034bcab42754c460f5
> *** Please cast your vote within the next 72 hours ***
> The tarball (*.tar.gz), signature (*.asc), and checksums (*.md5, *.sha1)
> for the source and binary artifacts can be found here:
>   *https://people.apache.org/~hshreedharan/apache-flume-1.5.2-rc1/
> *
> Maven staging repo:
>   *https://repository.apache.org/content/repositories/orgapacheflume-1008/
> *
> The tag to be voted on:
>   
> *https://git-wip-us.apache.org/repos/asf?p=flume.git;a=commit;h=229442aa6835ee0faa17e3034bcab42754c460f5
> *
> Flume's KEYS file containing PGP keys we use to sign the release:
>   http://www.apache.org/dist/flume/KEYS
> Thanks,
> Hari

[jira] [Commented] (FLUME-2538) TestResettableFileInputStream fails on JDK 8

[Hari Shreedharan (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLUME-2538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14208848#comment-14208848
 ] 

Hari Shreedharan commented on FLUME-2538:
-

I like the patch, but I'd recommend not accepting both. We should make sure 
that pre-JDK 8 the old string is valid and for JDK 8 it will be the new string. 
You can get the java version from the system property - java.version.

> TestResettableFileInputStream fails on JDK 8
> 
>
> Key: FLUME-2538
> URL: https://issues.apache.org/jira/browse/FLUME-2538
> Project: Flume
>  Issue Type: Bug
>Affects Versions: v1.5.0.1
>Reporter: Johny Rufus
>Assignee: Johny Rufus
> Fix For: v1.6.0
>
> Attachments: FLUME-2538.patch
>
>
> TestResettableFileInputStream.testUtf8DecodeErrorHandlingReplace fails in JDK 
> 8
> "testUtf8DecodeErrorHandlingReplace(org.apache.flume.serialization.TestResettableFileInputStream)
>   Time elapsed: 6 sec  <<< FAILURE!
> org.junit.ComparisonFailure: expected:<...(���)
> NonUnicode: (�[])
> > but was:<...(���)
> NonUnicode: (�[]) "
> Charsetdecoder.decode has changed in its behavior, as to how it handles 
> CodingErrorAction.Replace policy 
> Will submit a patch today.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[VOTE] Apache Flume 1.5.2 RC1

[Hari Shreedharan]

This is the eighth release for Apache Flume as a top-level project,
version 1.5.2. We are voting on release candidate RC1.

This release fixes an incompatibility with Java 6 based clients found
in Apache Flume 1.5.1 Release.

It fixes the following
issues:https://git-wip-us.apache.org/repos/asf?p=flume.git;a=blob;f=CHANGELOG;h=cc7321361d0b702ba870de20d6a3d2106987186a;hb=229442aa6835ee0faa17e3034bcab42754c460f5

*** Please cast your vote within the next 72 hours ***

The tarball (*.tar.gz), signature (*.asc), and checksums (*.md5, *.sha1)
for the source and binary artifacts can be found here:
  *https://people.apache.org/~hshreedharan/apache-flume-1.5.2-rc1/
*

Maven staging repo:
  *https://repository.apache.org/content/repositories/orgapacheflume-1008/
*

The tag to be voted on:
  
*https://git-wip-us.apache.org/repos/asf?p=flume.git;a=commit;h=229442aa6835ee0faa17e3034bcab42754c460f5
*

Flume's KEYS file containing PGP keys we use to sign the release:
  http://www.apache.org/dist/flume/KEYS


Thanks,
Hari

[jira] [Commented] (FLUME-2551) Add dev-support to src tarball

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLUME-2551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14208658#comment-14208658
 ] 

Hudson commented on FLUME-2551:
---

UNSTABLE: Integrated in flume-trunk #696 (See 
[https://builds.apache.org/job/flume-trunk/696/])
FLUME-2551. Add dev-support directory to source tarball. (hshreedharan: 
http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=293a8c0db2b3b2e696eab0e64bd83d140bfe56d8)
* flume-ng-dist/src/main/assembly/src.xml


> Add dev-support to src tarball
> --
>
> Key: FLUME-2551
> URL: https://issues.apache.org/jira/browse/FLUME-2551
> Project: Flume
>  Issue Type: Improvement
>Affects Versions: v1.5.1
>Reporter: Hari Shreedharan
>Assignee: Hari Shreedharan
> Fix For: v1.6.0, v1.5.2
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Jenkins build became unstable: flume-trunk #696

[Apache Jenkins Server]

See 

[jira] [Commented] (FLUME-2551) Add dev-support to src tarball

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLUME-2551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14208646#comment-14208646
 ] 

Hudson commented on FLUME-2551:
---

SUCCESS: Integrated in Flume-trunk-hbase-98 #53 (See 
[https://builds.apache.org/job/Flume-trunk-hbase-98/53/])
FLUME-2551. Add dev-support directory to source tarball. (hshreedharan: 
http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=293a8c0db2b3b2e696eab0e64bd83d140bfe56d8)
* flume-ng-dist/src/main/assembly/src.xml


> Add dev-support to src tarball
> --
>
> Key: FLUME-2551
> URL: https://issues.apache.org/jira/browse/FLUME-2551
> Project: Flume
>  Issue Type: Improvement
>Affects Versions: v1.5.1
>Reporter: Hari Shreedharan
>Assignee: Hari Shreedharan
> Fix For: v1.6.0, v1.5.2
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Jenkins build is back to stable : Flume-trunk-hbase-98 #53

[Apache Jenkins Server]

See 

[jira] [Updated] (FLUME-2551) Add dev-support to src tarball

[Hari Shreedharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hari Shreedharan updated FLUME-2551:

Issue Type: Improvement  (was: Bug)

> Add dev-support to src tarball
> --
>
> Key: FLUME-2551
> URL: https://issues.apache.org/jira/browse/FLUME-2551
> Project: Flume
>  Issue Type: Improvement
>Affects Versions: v1.5.1
>Reporter: Hari Shreedharan
>Assignee: Hari Shreedharan
> Fix For: v1.6.0, v1.5.2
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (FLUME-2547) Removing SSLv2Hello causes Java 6 clients to break

[Hari Shreedharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hari Shreedharan updated FLUME-2547:

Fix Version/s: v1.5.2

> Removing SSLv2Hello causes Java 6 clients to break
> --
>
> Key: FLUME-2547
> URL: https://issues.apache.org/jira/browse/FLUME-2547
> Project: Flume
>  Issue Type: Bug
>Reporter: Hari Shreedharan
>Assignee: Hari Shreedharan
> Fix For: v1.6.0, v1.5.2
>
>
> In Java 6, if the server side does not accept SSLv3, SSLv2Hello is required 
> even if TLS is used. SSLv2Hello itself is not insecure so we should bring it 
> back for compat with older Java versions



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (FLUME-2549) Enable SSLv2Hello for HttpSource

[Hari Shreedharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hari Shreedharan updated FLUME-2549:

Fix Version/s: v1.5.2

> Enable SSLv2Hello for HttpSource
> 
>
> Key: FLUME-2549
> URL: https://issues.apache.org/jira/browse/FLUME-2549
> Project: Flume
>  Issue Type: Sub-task
>Reporter: Hari Shreedharan
>Assignee: Hari Shreedharan
> Fix For: v1.6.0, v1.5.2
>
> Attachments: FLUME-2549.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (FLUME-2548) Enable SSLv2Hello for Avro Source and NettyAvroRpcClient

[Hari Shreedharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hari Shreedharan updated FLUME-2548:

Fix Version/s: v1.5.2

> Enable SSLv2Hello for Avro Source and NettyAvroRpcClient
> 
>
> Key: FLUME-2548
> URL: https://issues.apache.org/jira/browse/FLUME-2548
> Project: Flume
>  Issue Type: Sub-task
>Reporter: Hari Shreedharan
>Assignee: Hari Shreedharan
> Fix For: v1.6.0, v1.5.2
>
> Attachments: FLUME-2548.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLUME-2549) Enable SSLv2Hello for HttpSource

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLUME-2549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14208558#comment-14208558
 ] 

ASF subversion and git services commented on FLUME-2549:


Commit 3aceda6eb8f677e457e3322ac2fce9e2719a8c6d in flume's branch 
refs/heads/flume-1.5 from [~jarcec]
[ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=3aceda6 ]

FLUME-2549: Enable SSLv2Hello for HttpSource

(Hari Shreedharan via Jarek Jarcec Cecho)


> Enable SSLv2Hello for HttpSource
> 
>
> Key: FLUME-2549
> URL: https://issues.apache.org/jira/browse/FLUME-2549
> Project: Flume
>  Issue Type: Sub-task
>Reporter: Hari Shreedharan
>Assignee: Hari Shreedharan
> Fix For: v1.6.0
>
> Attachments: FLUME-2549.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLUME-2548) Enable SSLv2Hello for Avro Source and NettyAvroRpcClient

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLUME-2548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14208557#comment-14208557
 ] 

ASF subversion and git services commented on FLUME-2548:


Commit 2a59111090433a49ef2bad56219c4838277238dc in flume's branch 
refs/heads/flume-1.5 from [~jarcec]
[ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=2a59111 ]

FLUME-2548: Enable SSLv2Hello for Avro Source and NettyAvroRpcClient

(Hari Shreedharan via Jarek Jarcec Cecho)


> Enable SSLv2Hello for Avro Source and NettyAvroRpcClient
> 
>
> Key: FLUME-2548
> URL: https://issues.apache.org/jira/browse/FLUME-2548
> Project: Flume
>  Issue Type: Sub-task
>Reporter: Hari Shreedharan
>Assignee: Hari Shreedharan
> Fix For: v1.6.0
>
> Attachments: FLUME-2548.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLUME-2551) Add dev-support to src tarball

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLUME-2551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14208552#comment-14208552
 ] 

ASF subversion and git services commented on FLUME-2551:


Commit 57356c748bd8fdbc73a24344277132cc3159f8c9 in flume's branch 
refs/heads/flume-1.6 from [~hshreedharan]
[ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=57356c7 ]

FLUME-2551. Add dev-support directory to source tarball.


> Add dev-support to src tarball
> --
>
> Key: FLUME-2551
> URL: https://issues.apache.org/jira/browse/FLUME-2551
> Project: Flume
>  Issue Type: Bug
>Affects Versions: v1.5.1
>Reporter: Hari Shreedharan
>Assignee: Hari Shreedharan
> Fix For: v1.6.0, v1.5.2
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (FLUME-2551) Add dev-support to src tarball

[Hari Shreedharan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hari Shreedharan resolved FLUME-2551.
-
Resolution: Fixed

Committed.

> Add dev-support to src tarball
> --
>
> Key: FLUME-2551
> URL: https://issues.apache.org/jira/browse/FLUME-2551
> Project: Flume
>  Issue Type: Bug
>Affects Versions: v1.5.1
>Reporter: Hari Shreedharan
>Assignee: Hari Shreedharan
> Fix For: v1.6.0, v1.5.2
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLUME-2551) Add dev-support to src tarball

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLUME-2551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14208551#comment-14208551
 ] 

ASF subversion and git services commented on FLUME-2551:


Commit 451fad0271280d18ea7c6a1c7114f7c3f9a32665 in flume's branch 
refs/heads/flume-1.5 from [~hshreedharan]
[ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=451fad0 ]

FLUME-2551. Add dev-support directory to source tarball.


> Add dev-support to src tarball
> --
>
> Key: FLUME-2551
> URL: https://issues.apache.org/jira/browse/FLUME-2551
> Project: Flume
>  Issue Type: Bug
>Affects Versions: v1.5.1
>Reporter: Hari Shreedharan
>Assignee: Hari Shreedharan
> Fix For: v1.6.0, v1.5.2
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLUME-2551) Add dev-support to src tarball

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLUME-2551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14208550#comment-14208550
 ] 

ASF subversion and git services commented on FLUME-2551:


Commit 293a8c0db2b3b2e696eab0e64bd83d140bfe56d8 in flume's branch 
refs/heads/trunk from [~hshreedharan]
[ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=293a8c0 ]

FLUME-2551. Add dev-support directory to source tarball.


> Add dev-support to src tarball
> --
>
> Key: FLUME-2551
> URL: https://issues.apache.org/jira/browse/FLUME-2551
> Project: Flume
>  Issue Type: Bug
>Affects Versions: v1.5.1
>Reporter: Hari Shreedharan
>Assignee: Hari Shreedharan
> Fix For: v1.6.0, v1.5.2
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (FLUME-2551) Add dev-support to src tarball

[Hari Shreedharan (JIRA)]

Hari Shreedharan created FLUME-2551:
---

 Summary: Add dev-support to src tarball
 Key: FLUME-2551
 URL: https://issues.apache.org/jira/browse/FLUME-2551
 Project: Flume
  Issue Type: Bug
Affects Versions: v1.5.1
Reporter: Hari Shreedharan
Assignee: Hari Shreedharan
 Fix For: v1.5.2, v1.6.0






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: SSLv2Hello is required on Java 6

[Hari Shreedharan]

Thanks. I will spin up an RC later today.


Thanks,
Hari

On Wed, Nov 12, 2014 at 7:51 AM, Jarek Jarcec Cecho 
wrote:

> Hi Hari,
> I’ve just reviewed and committed FLUME-2547 and it’s subtasks. As we are 
> supporting JDK6, I would be in favor of doing another quick release.
> Jarcec
>> On Nov 11, 2014, at 2:03 PM, Hari Shreedharan  
>> wrote:
>> 
>> Hi,
>> 
>> 
>> Right after we pushed 1.5.1 out (I have not even sent the announce email), 
>> we discovered that Java 6 requires SSLv2Hello on the server side for 
>> negotiation even if TLS is used (unless the client code is also changed to 
>> disable SSLv2Hello). So:
>> 
>> 
>> - For HTTP Source, any clients running on Java 6 would need code changes to 
>> also disable SSLv2Hello to be able to send data via TLSv1.
>> - For Avro Source, any clients running Flume SDK < 1.5.1 on Java 6 would 
>> break and requires the client application to upgrade to 1.5.1. 
>> 
>> 
>> I filed FLUME-2547 to fix this.
>> 
>> 
>> My question to the community here is whether we want a new release bringing 
>> SSLv2Hello back or if we are willing to just document this and move forward?
>> 
>> 
>> I am willing to put together an RC if required.
>> 
>> Thanks,
>> Hari

[jira] [Commented] (FLUME-2307) Remove Log writetimeout

[Nina Safonova (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLUME-2307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14208414#comment-14208414
 ] 

Nina Safonova commented on FLUME-2307:
--

Flume 1.5.0-cdh5.1.0
Source code repository: https://git-wip-us.apache.org/repos/asf/flume.git
Revision: 14be91ec816bac5a91c321b9e8620ffb04acf04c
Compiled by jenkins on Sat Jul 12 09:17:48 PDT 2014
>From source with checksum bf4451b17198a612fea60ad6f5420bbc

> Remove Log writetimeout
> ---
>
> Key: FLUME-2307
> URL: https://issues.apache.org/jira/browse/FLUME-2307
> Project: Flume
>  Issue Type: Bug
>  Components: Channel
>Affects Versions: v1.4.0
>Reporter: Steve Zesch
>Assignee: Hari Shreedharan
> Fix For: v1.5.0
>
> Attachments: FLUME-2307-1.patch, FLUME-2307.patch
>
>
> I've observed Flume failing to clean up old log data in FileChannels. The 
> amount of old log data can range anywhere from tens to hundreds of GB. I was 
> able to confirm that the channels were in fact empty. This behavior always 
> occurs after lock timeouts when attempting to put, take, rollback, or commit 
> to a FileChannel. Once the timeout occurs, Flume stops cleaning up the old 
> files. I was able to confirm that the Log's writeCheckpoint method was still 
> being called and successfully obtaining a lock from tryLockExclusive(), but I 
> was not able to confirm removeOldLogs being called. The application log did 
> not include "Removing old file: log-xyz" for the old files which the Log 
> class would output if they were correctly being removed. I suspect the lock 
> timeouts were due to high I/O load at the time.
> Some stack traces:
> {code}
> org.apache.flume.ChannelException: Failed to obtain lock for writing to the 
> log. Try increasing the log write timeout value. [channel=fileChannel]
> at 
> org.apache.flume.channel.file.FileChannel$FileBackedTransaction.doPut(FileChannel.java:478)
> at 
> org.apache.flume.channel.BasicTransactionSemantics.put(BasicTransactionSemantics.java:93)
> at 
> org.apache.flume.channel.BasicChannelSemantics.put(BasicChannelSemantics.java:80)
> at 
> org.apache.flume.channel.ChannelProcessor.processEventBatch(ChannelProcessor.java:189)
> org.apache.flume.ChannelException: Failed to obtain lock for writing to the 
> log. Try increasing the log write timeout value. [channel=fileChannel]
> at 
> org.apache.flume.channel.file.FileChannel$FileBackedTransaction.doCommit(FileChannel.java:594)
> at 
> org.apache.flume.channel.BasicTransactionSemantics.commit(BasicTransactionSemantics.java:151)
> at 
> dataxu.flume.plugins.avro.AsyncAvroSink.process(AsyncAvroSink.java:548)
> at 
> dataxu.flume.plugins.ClassLoaderFlumeSink.process(ClassLoaderFlumeSink.java:33)
> at 
> org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
> at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
> at java.lang.Thread.run(Thread.java:619)
> org.apache.flume.ChannelException: Failed to obtain lock for writing to the 
> log. Try increasing the log write timeout value. [channel=fileChannel]
> at 
> org.apache.flume.channel.file.FileChannel$FileBackedTransaction.doRollback(FileChannel.java:621)
> at 
> org.apache.flume.channel.BasicTransactionSemantics.rollback(BasicTransactionSemantics.java:168)
> at 
> org.apache.flume.channel.ChannelProcessor.processEventBatch(ChannelProcessor.java:194)
> at 
> dataxu.flume.plugins.avro.AvroSource.appendBatch(AvroSource.java:209)
> at sun.reflect.GeneratedMethodAccessor19.invoke(Unknown Source)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at 
> org.apache.avro.ipc.specific.SpecificResponder.respond(SpecificResponder.java:91)
> at org.apache.avro.ipc.Responder.respond(Responder.java:151)
> at 
> org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.messageReceived(NettyServer.java:188)
> at 
> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:75)
> at 
> org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.handleUpstream(NettyServer.java:173)
> at 
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> at 
> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:792)
> at 
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
> at 
> org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:321)
> at 
> org.jboss.netty.handler.codec.frame.Fr

[jira] [Comment Edited] (FLUME-2538) TestResettableFileInputStream fails on JDK 8

[Johny Rufus (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLUME-2538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14199523#comment-14199523
 ] 

Johny Rufus edited comment on FLUME-2538 at 11/12/14 5:09 PM:
--

One way to see the difference in the behavior is by running the below code in 
Jdk 7 and 8
@Test
  public void testTest() {
CharsetDecoder decoder = Charsets.UTF_8.newDecoder();

decoder.onMalformedInput(CodingErrorAction.REPLACE);
decoder.onUnmappableCharacter(CodingErrorAction.REPLACE);

ByteBuffer buf = ByteBuffer.allocate(20);
buf.put(new byte[] { (byte)0xf8, (byte)0xa1, (byte)0xa1, (byte)0xa1,
(byte)0xa1 });
buf.flip();

CharBuffer cbuf = CharBuffer.allocate(1);
CoderResult res = decoder.decode(buf, cbuf, false);
System.out.println(decoder.getClass().getName());
System.out.println("Pos --- "+buf.position()+ "  cbuf pos 
--"+cbuf.position());
}

Jdk 7 output -->Pos --- 5  cbuf pos --1
Jdk 8 output -->Pos --- 1  cbuf pos --1

In Jdk7: If there is a invalid byte sequence and CodingErrorAction.Replace is 
specified, then the complete set of invalid bye sequence is treated as one 
malformed character and replaced by one replacement character in the output 
buffer  [Hence the position is advanced by 5 as seen in the output as its a 5 
byte invalid sequence]

In Jdk8: Each invalid byte in the sequence is treated as a malformed character 
and hence we see the buffer being advanced by only one position. So for every 
malformed character, we see the replacement character included in the output 
buffer

Attaching a patch that accommodates the above modified behavior 


was (Author: jrufus):
One way to see the difference in the behavior is by running the below code in 
Jdk 7 and 8
@Test
  public void testTest() {
CharsetDecoder decoder = Charsets.UTF_8.newDecoder();

decoder.onMalformedInput(CodingErrorAction.REPLACE);
decoder.onUnmappableCharacter(CodingErrorAction.REPLACE);

ByteBuffer buf = ByteBuffer.allocate(20);
buf.put(new byte[] { (byte)0xf8, (byte)0xa1, (byte)0xa1, (byte)0xa1,
(byte)0xa1 });
buf.flip();

CharBuffer cbuf = CharBuffer.allocate(1);
CoderResult res = decoder.decode(buf, cbuf, false);
System.out.println(decoder.getClass().getName());
System.out.println("Pos --- "+buf.position()+ "  cbuf pos 
--"+cbuf.position());
}

Jdk 7 output -->Pos --- 5  cbuf pos --1
Jdk 8 output -->Pos --- 1  cbuf pos --1

In Jdk7: If there are a group of malformed characters and 
CodingErrorAction.Replace is specified, then the complete set of adjacent 
malformed characters in the buffer are replaced [Hence the position is advanced 
by 5 as seen in the output as there are 5 malformed chars in the buffer]

In Jdk8: Each malformed character is treated as a separate entity and hence we 
see the buffer being advanced by only one position. So for every malformed 
character, we see the replacement character included in the output buffer

Attaching a patch that accomodates the above modified behavior 

> TestResettableFileInputStream fails on JDK 8
> 
>
> Key: FLUME-2538
> URL: https://issues.apache.org/jira/browse/FLUME-2538
> Project: Flume
>  Issue Type: Bug
>Affects Versions: v1.5.0.1
>Reporter: Johny Rufus
>Assignee: Johny Rufus
> Fix For: v1.6.0
>
> Attachments: FLUME-2538.patch
>
>
> TestResettableFileInputStream.testUtf8DecodeErrorHandlingReplace fails in JDK 
> 8
> "testUtf8DecodeErrorHandlingReplace(org.apache.flume.serialization.TestResettableFileInputStream)
>   Time elapsed: 6 sec  <<< FAILURE!
> org.junit.ComparisonFailure: expected:<...(���)
> NonUnicode: (�[])
> > but was:<...(���)
> NonUnicode: (�[]) "
> Charsetdecoder.decode has changed in its behavior, as to how it handles 
> CodingErrorAction.Replace policy 
> Will submit a patch today.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Jenkins build is back to stable : flume-trunk #695

[Apache Jenkins Server]

See 

Jenkins build became unstable: Flume-trunk-hbase-98 #52

[Apache Jenkins Server]

See 

[jira] [Commented] (FLUME-2549) Enable SSLv2Hello for HttpSource

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLUME-2549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14208232#comment-14208232
 ] 

Hudson commented on FLUME-2549:
---

SUCCESS: Integrated in flume-trunk #695 (See 
[https://builds.apache.org/job/flume-trunk/695/])
FLUME-2549: Enable SSLv2Hello for HttpSource (jarcec: 
http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=94b25aebc6d90480bb00898de4b98257cc7d8cbb)
* flume-ng-core/src/main/java/org/apache/flume/source/http/HTTPSource.java
* flume-ng-core/src/test/java/org/apache/flume/source/http/TestHTTPSource.java
* flume-ng-doc/sphinx/FlumeUserGuide.rst
* 
flume-ng-core/src/main/java/org/apache/flume/source/http/HTTPSourceConfigurationConstants.java


> Enable SSLv2Hello for HttpSource
> 
>
> Key: FLUME-2549
> URL: https://issues.apache.org/jira/browse/FLUME-2549
> Project: Flume
>  Issue Type: Sub-task
>Reporter: Hari Shreedharan
>Assignee: Hari Shreedharan
> Fix For: v1.6.0
>
> Attachments: FLUME-2549.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLUME-2549) Enable SSLv2Hello for HttpSource

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLUME-2549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14208230#comment-14208230
 ] 

Hudson commented on FLUME-2549:
---

UNSTABLE: Integrated in Flume-trunk-hbase-98 #52 (See 
[https://builds.apache.org/job/Flume-trunk-hbase-98/52/])
FLUME-2549: Enable SSLv2Hello for HttpSource (jarcec: 
http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=94b25aebc6d90480bb00898de4b98257cc7d8cbb)
* flume-ng-core/src/test/java/org/apache/flume/source/http/TestHTTPSource.java
* flume-ng-doc/sphinx/FlumeUserGuide.rst
* 
flume-ng-core/src/main/java/org/apache/flume/source/http/HTTPSourceConfigurationConstants.java
* flume-ng-core/src/main/java/org/apache/flume/source/http/HTTPSource.java


> Enable SSLv2Hello for HttpSource
> 
>
> Key: FLUME-2549
> URL: https://issues.apache.org/jira/browse/FLUME-2549
> Project: Flume
>  Issue Type: Sub-task
>Reporter: Hari Shreedharan
>Assignee: Hari Shreedharan
> Fix For: v1.6.0
>
> Attachments: FLUME-2549.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLUME-2538) TestResettableFileInputStream fails on JDK 8

[Johny Rufus (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLUME-2538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14208218#comment-14208218
 ] 

Johny Rufus commented on FLUME-2538:


Atlast I was able to find the bug that caused this change 
https://bugs.openjdk.java.net/browse/JDK-7096080

The relevant portion to this bug taken from - 
http://mail.openjdk.java.net/pipermail/core-libs-dev/2011-September/007722.html 
[which is a link mentioned in this bug report]

"Another corner case is how to deal with the old 5-6 bytes byte sequence, 
such as
"fc 80 80 8f bf bf", we are now treating them as 1 malformed utf-8 byte 
sequence, so any
of these 5-6 bytes "old" formed will be treated one malformed character 
and then replaced
by one "\ufffd". But according to the new "best practice" 
recommendation, it probably should
be replaced by 6 \ufffd."

> TestResettableFileInputStream fails on JDK 8
> 
>
> Key: FLUME-2538
> URL: https://issues.apache.org/jira/browse/FLUME-2538
> Project: Flume
>  Issue Type: Bug
>Affects Versions: v1.5.0.1
>Reporter: Johny Rufus
>Assignee: Johny Rufus
> Fix For: v1.6.0
>
> Attachments: FLUME-2538.patch
>
>
> TestResettableFileInputStream.testUtf8DecodeErrorHandlingReplace fails in JDK 
> 8
> "testUtf8DecodeErrorHandlingReplace(org.apache.flume.serialization.TestResettableFileInputStream)
>   Time elapsed: 6 sec  <<< FAILURE!
> org.junit.ComparisonFailure: expected:<...(���)
> NonUnicode: (�[])
> > but was:<...(���)
> NonUnicode: (�[]) "
> Charsetdecoder.decode has changed in its behavior, as to how it handles 
> CodingErrorAction.Replace policy 
> Will submit a patch today.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: SSLv2Hello is required on Java 6

[Jarek Jarcec Cecho]

Hi Hari,
I’ve just reviewed and committed FLUME-2547 and it’s subtasks. As we are 
supporting JDK6, I would be in favor of doing another quick release.

Jarcec

> On Nov 11, 2014, at 2:03 PM, Hari Shreedharan  
> wrote:
> 
> Hi,
> 
> 
> Right after we pushed 1.5.1 out (I have not even sent the announce email), we 
> discovered that Java 6 requires SSLv2Hello on the server side for negotiation 
> even if TLS is used (unless the client code is also changed to disable 
> SSLv2Hello). So:
> 
> 
> - For HTTP Source, any clients running on Java 6 would need code changes to 
> also disable SSLv2Hello to be able to send data via TLSv1.
> - For Avro Source, any clients running Flume SDK < 1.5.1 on Java 6 would 
> break and requires the client application to upgrade to 1.5.1. 
> 
> 
> I filed FLUME-2547 to fix this.
> 
> 
> My question to the community here is whether we want a new release bringing 
> SSLv2Hello back or if we are willing to just document this and move forward?
> 
> 
> I am willing to put together an RC if required.
> 
> Thanks,
> Hari

[jira] [Commented] (FLUME-2548) Enable SSLv2Hello for Avro Source and NettyAvroRpcClient

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLUME-2548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14208150#comment-14208150
 ] 

Hudson commented on FLUME-2548:
---

SUCCESS: Integrated in Flume-trunk-hbase-98 #51 (See 
[https://builds.apache.org/job/Flume-trunk-hbase-98/51/])
FLUME-2548: Enable SSLv2Hello for Avro Source and NettyAvroRpcClient (jarcec: 
http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=8c7f6936045222026c93f34bda861acec79066cf)
* flume-ng-sdk/src/main/java/org/apache/flume/api/NettyAvroRpcClient.java
* flume-ng-core/src/main/java/org/apache/flume/source/AvroSource.java
* flume-ng-doc/sphinx/FlumeUserGuide.rst


> Enable SSLv2Hello for Avro Source and NettyAvroRpcClient
> 
>
> Key: FLUME-2548
> URL: https://issues.apache.org/jira/browse/FLUME-2548
> Project: Flume
>  Issue Type: Sub-task
>Reporter: Hari Shreedharan
>Assignee: Hari Shreedharan
> Fix For: v1.6.0
>
> Attachments: FLUME-2548.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLUME-2548) Enable SSLv2Hello for Avro Source and NettyAvroRpcClient

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLUME-2548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14208145#comment-14208145
 ] 

Hudson commented on FLUME-2548:
---

UNSTABLE: Integrated in flume-trunk #694 (See 
[https://builds.apache.org/job/flume-trunk/694/])
FLUME-2548: Enable SSLv2Hello for Avro Source and NettyAvroRpcClient (jarcec: 
http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=8c7f6936045222026c93f34bda861acec79066cf)
* flume-ng-doc/sphinx/FlumeUserGuide.rst
* flume-ng-core/src/main/java/org/apache/flume/source/AvroSource.java
* flume-ng-sdk/src/main/java/org/apache/flume/api/NettyAvroRpcClient.java


> Enable SSLv2Hello for Avro Source and NettyAvroRpcClient
> 
>
> Key: FLUME-2548
> URL: https://issues.apache.org/jira/browse/FLUME-2548
> Project: Flume
>  Issue Type: Sub-task
>Reporter: Hari Shreedharan
>Assignee: Hari Shreedharan
> Fix For: v1.6.0
>
> Attachments: FLUME-2548.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Jenkins build is still unstable: flume-trunk #694

[Apache Jenkins Server]

See 

[jira] [Commented] (FLUME-2549) Enable SSLv2Hello for HttpSource

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLUME-2549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14208143#comment-14208143
 ] 

ASF subversion and git services commented on FLUME-2549:


Commit d0a51440149f5c26d483c10203bd6a09752a7072 in flume's branch 
refs/heads/flume-1.6 from [~jarcec]
[ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=d0a5144 ]

FLUME-2549: Enable SSLv2Hello for HttpSource

(Hari Shreedharan via Jarek Jarcec Cecho)


> Enable SSLv2Hello for HttpSource
> 
>
> Key: FLUME-2549
> URL: https://issues.apache.org/jira/browse/FLUME-2549
> Project: Flume
>  Issue Type: Sub-task
>Reporter: Hari Shreedharan
>Assignee: Hari Shreedharan
> Attachments: FLUME-2549.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLUME-2549) Enable SSLv2Hello for HttpSource

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLUME-2549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14208142#comment-14208142
 ] 

ASF subversion and git services commented on FLUME-2549:


Commit 94b25aebc6d90480bb00898de4b98257cc7d8cbb in flume's branch 
refs/heads/trunk from [~jarcec]
[ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=94b25ae ]

FLUME-2549: Enable SSLv2Hello for HttpSource

(Hari Shreedharan via Jarek Jarcec Cecho)


> Enable SSLv2Hello for HttpSource
> 
>
> Key: FLUME-2549
> URL: https://issues.apache.org/jira/browse/FLUME-2549
> Project: Flume
>  Issue Type: Sub-task
>Reporter: Hari Shreedharan
>Assignee: Hari Shreedharan
> Attachments: FLUME-2549.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLUME-2548) Enable SSLv2Hello for Avro Source and NettyAvroRpcClient

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLUME-2548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14208070#comment-14208070
 ] 

ASF subversion and git services commented on FLUME-2548:


Commit d90fa649d49190658e40f80659d1f555b3de047a in flume's branch 
refs/heads/flume-1.6 from [~jarcec]
[ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=d90fa64 ]

FLUME-2548: Enable SSLv2Hello for Avro Source and NettyAvroRpcClient

(Hari Shreedharan via Jarek Jarcec Cecho)


> Enable SSLv2Hello for Avro Source and NettyAvroRpcClient
> 
>
> Key: FLUME-2548
> URL: https://issues.apache.org/jira/browse/FLUME-2548
> Project: Flume
>  Issue Type: Sub-task
>Reporter: Hari Shreedharan
>Assignee: Hari Shreedharan
> Attachments: FLUME-2548.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLUME-2548) Enable SSLv2Hello for Avro Source and NettyAvroRpcClient

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLUME-2548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14208069#comment-14208069
 ] 

ASF subversion and git services commented on FLUME-2548:


Commit 8c7f6936045222026c93f34bda861acec79066cf in flume's branch 
refs/heads/trunk from [~jarcec]
[ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=8c7f693 ]

FLUME-2548: Enable SSLv2Hello for Avro Source and NettyAvroRpcClient

(Hari Shreedharan via Jarek Jarcec Cecho)


> Enable SSLv2Hello for Avro Source and NettyAvroRpcClient
> 
>
> Key: FLUME-2548
> URL: https://issues.apache.org/jira/browse/FLUME-2548
> Project: Flume
>  Issue Type: Sub-task
>Reporter: Hari Shreedharan
>Assignee: Hari Shreedharan
> Attachments: FLUME-2548.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLUME-2548) Enable SSLv2Hello for Avro Source and NettyAvroRpcClient

[Tom White (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLUME-2548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14207938#comment-14207938
 ] 

Tom White commented on FLUME-2548:
--

+1

> Enable SSLv2Hello for Avro Source and NettyAvroRpcClient
> 
>
> Key: FLUME-2548
> URL: https://issues.apache.org/jira/browse/FLUME-2548
> Project: Flume
>  Issue Type: Sub-task
>Reporter: Hari Shreedharan
>Assignee: Hari Shreedharan
> Attachments: FLUME-2548.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (FLUME-2227) Move BlobDeserializer from Morphline Sink to flume-ng-core

[Santiago M. Mola (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLUME-2227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14207890#comment-14207890
 ] 

Santiago M. Mola commented on FLUME-2227:
-

This sounds good. Any chance this gets pulled into 1.6.0?

> Move BlobDeserializer from Morphline Sink to flume-ng-core
> --
>
> Key: FLUME-2227
> URL: https://issues.apache.org/jira/browse/FLUME-2227
> Project: Flume
>  Issue Type: Bug
>  Components: Sinks+Sources
>Affects Versions: v1.4.0
>Reporter: Roshan Naik
>Assignee: Roshan Naik
> Fix For: v1.6.0
>
> Attachments: FLUME-2227.1.patch, FLUME-2227.v2.patch, 
> FLUME-2227.v3.patch
>
>
> This deserializer is more applicable to SpoolDir source and has no 
> dependencies on Morphline sink.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (FLUME-2227) Move BlobDeserializer from Morphline Sink to flume-ng-core

[Santiago M. Mola (JIRA)]

 [ 
https://issues.apache.org/jira/browse/FLUME-2227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Santiago M. Mola updated FLUME-2227:

  Component/s: Sinks+Sources
Fix Version/s: v1.6.0

> Move BlobDeserializer from Morphline Sink to flume-ng-core
> --
>
> Key: FLUME-2227
> URL: https://issues.apache.org/jira/browse/FLUME-2227
> Project: Flume
>  Issue Type: Bug
>  Components: Sinks+Sources
>Affects Versions: v1.4.0
>Reporter: Roshan Naik
>Assignee: Roshan Naik
> Fix For: v1.6.0
>
> Attachments: FLUME-2227.1.patch, FLUME-2227.v2.patch, 
> FLUME-2227.v3.patch
>
>
> This deserializer is more applicable to SpoolDir source and has no 
> dependencies on Morphline sink.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)