Review Request 19716: ACCUMULO-2564 Backport changes to unify Hadoop 1/2

2014-03-27 Thread Mike Drob
: ACCUMULO-2564 https://issues.apache.org/jira/browse/ACCUMULO-2564 Repository: accumulo Description --- ACCUMULO-2564 Backport changes to unify Hadoop 1/2 This is a backport of the changes originally made for 1.5.0 under ACCUMULO-1421 for binary compatability between Hadoop versions 1

Re: Review Request 19716: ACCUMULO-2564 Backport changes to unify Hadoop 1/2

2014-03-27 Thread Mike Drob
, Eric Newton, and Josh Elser. Changes --- Updated decscription with proper attribution. Bugs: ACCUMULO-2564 https://issues.apache.org/jira/browse/ACCUMULO-2564 Repository: accumulo Description (updated) --- ACCUMULO-2564 Backport changes to unify Hadoop 1/2 This is a backport

Re: Review Request 19716: ACCUMULO-2564 Backport changes to unify Hadoop 1/2

2014-03-27 Thread Sean Busbey
of Hadoop fail to run if executed against the libraries of the other? Open issues: * src/server/src/main/java/org/apache/accumulo/server/util/CountRowKeys.java uses Counter directly, issue still exists in 1.5.2-SNAPSHOT * src/server/src/main/java/org/apache/accumulo/server/test/functional

Re: Review Request 19716: ACCUMULO-2564 Backport changes to unify Hadoop 1/2

2014-03-27 Thread Sean Busbey
On March 27, 2014, 4:31 p.m., Sean Busbey wrote: I'm presuming we don't care if unit tests built against one version of Hadoop fail to run if executed against the libraries of the other? Open issues: * src/server/src/main/java/org/apache/accumulo/server/util/CountRowKeys.java

Re: Review Request 19716: ACCUMULO-2564 Backport changes to unify Hadoop 1/2

2014-03-27 Thread Sean Busbey
On March 27, 2014, 4:31 p.m., Sean Busbey wrote: I'm presuming we don't care if unit tests built against one version of Hadoop fail to run if executed against the libraries of the other? Open issues: * src/server/src/main/java/org/apache/accumulo/server/util/CountRowKeys.java

Re: Review Request 19716: ACCUMULO-2564 Backport changes to unify Hadoop 1/2

2014-03-27 Thread Josh Elser
On March 27, 2014, 4:31 p.m., Sean Busbey wrote: I'm presuming we don't care if unit tests built against one version of Hadoop fail to run if executed against the libraries of the other? Open issues: * src/server/src/main/java/org/apache/accumulo/server/util/CountRowKeys.java

Re: Review Request 19716: ACCUMULO-2564 Backport changes to unify Hadoop 1/2

2014-03-27 Thread Mike Drob
On March 27, 2014, 4:31 p.m., Sean Busbey wrote: I'm presuming we don't care if unit tests built against one version of Hadoop fail to run if executed against the libraries of the other? Open issues: * src/server/src/main/java/org/apache/accumulo/server/util/CountRowKeys.java

Re: Review Request 19716: ACCUMULO-2564 Backport changes to unify Hadoop 1/2

2014-03-27 Thread Sean Busbey
On March 27, 2014, 4:31 p.m., Sean Busbey wrote: I'm presuming we don't care if unit tests built against one version of Hadoop fail to run if executed against the libraries of the other? Open issues: * src/server/src/main/java/org/apache/accumulo/server/util/CountRowKeys.java

Review Request 19749: ACCUMULO-2566 Hadoop reflection for TeraSortIngest

2014-03-27 Thread Mike Drob
: ACCUMULO-2566 https://issues.apache.org/jira/browse/ACCUMULO-2566 Repository: accumulo Description --- ACCUMULO-2566 Hadoop reflection for TeraSortIngest ACCUMULO-2566 Add more Counter-based reflection Pull reflection out of ContinuousVerify and apply it to server utils as well

Re: Review Request 19716: ACCUMULO-2564 Replace more hadoop 1/2 incompat

2014-03-27 Thread Mike Drob
, Sean Busbey, Eric Newton, and Josh Elser. Changes --- Addressed ContinuousMoru, AccumuloInputFormat, LogSort, and RangePartitioner. Summary (updated) - ACCUMULO-2564 Replace more hadoop 1/2 incompat Bugs: ACCUMULO-2564 https://issues.apache.org/jira/browse/ACCUMULO

Problems with installing Accumulo 1.4.3 on CDH4 quickstart (Hadoop 2.0) compatability issues.

2014-02-06 Thread Chris Rigano
about the compatibility issues with 1.4.3 and hadoop 2.0, part of CDH 4 quick start. CDH 3 quick start is deprecated, Do I have any clear options or do I need to do something else? I am surprised cloudera didn't check this out! Thanks!

Review Request 17192: ACCUMULO-2225 handle IllegalArgumentExceptions from Hadoop on host lookup failure.

2014-01-22 Thread Sean Busbey
://issues.apache.org/jira/browse/ACCUMULO-2225 Repository: accumulo Description --- ACCUMULO-2225 handle IllegalArgumentExceptions from Hadoop on host lookup failure. Looks for cases where we treat IOExceptions out of Hadoop specially, then attempts to replicate

Re: Review Request 17192: ACCUMULO-2225 handle IllegalArgumentExceptions from Hadoop on host lookup failure.

2014-01-22 Thread Bill Havanki
/ --- (Updated Jan. 22, 2014, 2:13 p.m.) Review request for accumulo and Bill Havanki. Bugs: ACCUMULO-2225 https://issues.apache.org/jira/browse/ACCUMULO-2225 Repository: accumulo Description --- ACCUMULO-2225 handle IllegalArgumentExceptions from Hadoop

Re: Review Request 17192: ACCUMULO-2225 handle IllegalArgumentExceptions from Hadoop on host lookup failure.

2014-01-22 Thread Mike Drob
Description --- ACCUMULO-2225 handle IllegalArgumentExceptions from Hadoop on host lookup failure. Looks for cases where we treat IOExceptions out of Hadoop specially, then attempts to replicate for UnknownHostExceptions that have been wrapped in IllegalArgumentExceptions. Diffs

Re: Review Request 17192: ACCUMULO-2225 handle IllegalArgumentExceptions from Hadoop on host lookup failure.

2014-01-22 Thread Sean Busbey
: ACCUMULO-2225 https://issues.apache.org/jira/browse/ACCUMULO-2225 Repository: accumulo Description --- ACCUMULO-2225 handle IllegalArgumentExceptions from Hadoop on host lookup failure. Looks for cases where we treat IOExceptions out of Hadoop specially

Re: Review Request 17192: ACCUMULO-2225 handle IllegalArgumentExceptions from Hadoop on host lookup failure.

2014-01-22 Thread Sean Busbey
IllegalArgumentExceptions from Hadoop on host lookup failure. Looks for cases where we treat IOExceptions out of Hadoop specially, then attempts to replicate for UnknownHostExceptions that have been wrapped in IllegalArgumentExceptions. Diffs - src/core/src/main/java/org/apache

Re: Review Request 17192: ACCUMULO-2225 handle IllegalArgumentExceptions from Hadoop on host lookup failure.

2014-01-22 Thread Sean Busbey
IllegalArgumentExceptions from Hadoop on host lookup failure. Looks for cases where we treat IOExceptions out of Hadoop specially, then attempts to replicate for UnknownHostExceptions that have been wrapped in IllegalArgumentExceptions. Diffs - src/core/src/main/java/org

Re: Review Request 17192: ACCUMULO-2225 handle IllegalArgumentExceptions from Hadoop on host lookup failure.

2014-01-22 Thread Sean Busbey
for accumulo and Bill Havanki. Bugs: ACCUMULO-2225 https://issues.apache.org/jira/browse/ACCUMULO-2225 Repository: accumulo Description --- ACCUMULO-2225 handle IllegalArgumentExceptions from Hadoop on host lookup failure. Looks for cases where we treat

Re: Review Request 17192: ACCUMULO-2225 handle IllegalArgumentExceptions from Hadoop on host lookup failure.

2014-01-22 Thread Mike Drob
, 2014, 7:13 p.m.) Review request for accumulo and Bill Havanki. Bugs: ACCUMULO-2225 https://issues.apache.org/jira/browse/ACCUMULO-2225 Repository: accumulo Description --- ACCUMULO-2225 handle IllegalArgumentExceptions from Hadoop on host lookup failure

Re: Review Request 17192: ACCUMULO-2225 handle IllegalArgumentExceptions from Hadoop on host lookup failure.

2014-01-22 Thread Sean Busbey
IllegalArgumentExceptions from Hadoop on host lookup failure. Looks for cases where we treat IOExceptions out of Hadoop specially, then attempts to replicate for UnknownHostExceptions that have been wrapped in IllegalArgumentExceptions. Diffs - src/core/src/main/java/org/apache

Re: Review Request 17192: ACCUMULO-2225 handle IllegalArgumentExceptions from Hadoop on host lookup failure.

2014-01-22 Thread Sean Busbey
Havanki. Changes --- first round of modifications based on feedbakc. Bugs: ACCUMULO-2225 https://issues.apache.org/jira/browse/ACCUMULO-2225 Repository: accumulo Description --- ACCUMULO-2225 handle IllegalArgumentExceptions from Hadoop on host lookup failure. Looks

Re: Review Request 17192: ACCUMULO-2225 handle IllegalArgumentExceptions from Hadoop on host lookup failure.

2014-01-22 Thread Bill Havanki
Repository: accumulo Description --- ACCUMULO-2225 handle IllegalArgumentExceptions from Hadoop on host lookup failure. Looks for cases where we treat IOExceptions out of Hadoop specially, then attempts to replicate for UnknownHostExceptions that have been wrapped

Re: Review Request 17192: ACCUMULO-2225 handle IllegalArgumentExceptions from Hadoop on host lookup failure.

2014-01-22 Thread Sean Busbey
Repository: accumulo Description --- ACCUMULO-2225 handle IllegalArgumentExceptions from Hadoop on host lookup failure. Looks for cases where we treat IOExceptions out of Hadoop specially, then attempts to replicate for UnknownHostExceptions that have been wrapped

Re: Review Request 17192: ACCUMULO-2225 handle IllegalArgumentExceptions from Hadoop on host lookup failure.

2014-01-22 Thread Sean Busbey
Havanki. Changes --- updated logs / comments based on feedback from bhavanki Bugs: ACCUMULO-2225 https://issues.apache.org/jira/browse/ACCUMULO-2225 Repository: accumulo Description --- ACCUMULO-2225 handle IllegalArgumentExceptions from Hadoop on host lookup failure

Re: Review Request 17192: ACCUMULO-2225 handle IllegalArgumentExceptions from Hadoop on host lookup failure.

2014-01-22 Thread Bill Havanki
for accumulo and Bill Havanki. Bugs: ACCUMULO-2225 https://issues.apache.org/jira/browse/ACCUMULO-2225 Repository: accumulo Description --- ACCUMULO-2225 handle IllegalArgumentExceptions from Hadoop on host lookup failure. Looks for cases where we treat

Re: documentation on dealing with legacy Hadoop versions

2014-01-17 Thread Steve Loughran
the Hadoop policy here is no guarantees: http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html that said, nobody rushes to update things, especially on the 2.x branch -protobuf was traumatic enough that google guava is on hold, leading to BOOKEEPER 708 Apart

documentation on dealing with legacy Hadoop versions

2014-01-03 Thread Sean Busbey
Heya! Earlier this week we had a user in IRC that was having difficulty running 1.5.0 because their classpath didn't include commons-configuration. In one case, they just needed to fix their accumulo-site to include hadoop 2 paths. In the other, they were using Apache Hadoop 0.20.2, which has

Re: documentation on dealing with legacy Hadoop versions

2014-01-03 Thread Josh Elser
On 1/3/14, 10:55 AM, Sean Busbey wrote: Heya! Earlier this week we had a user in IRC that was having difficulty running 1.5.0 because their classpath didn't include commons-configuration. In one case, they just needed to fix their accumulo-site to include hadoop 2 paths. In the other

Re: documentation on dealing with legacy Hadoop versions

2014-01-03 Thread Josh Elser
On 1/3/14, 11:17 AM, Sean Busbey wrote: On Fri, Jan 3, 2014 at 10:15 AM, Josh Elser josh.el...@gmail.com wrote: 2) Should we document commons-configuration similar to commons-io? The README already has a section about how some older versions of Hadoop don't have commons-io. I think

Re: documentation on dealing with legacy Hadoop versions

2014-01-03 Thread Sean Busbey
On Fri, Jan 3, 2014 at 10:26 AM, Josh Elser josh.el...@gmail.com wrote: Point of reference: HBase-0.96.0 will pull *all* dependencies into $HBASE_HOME/lib. Now, while I don't think I want to re-package all of the Hadoop jars and its dependencies, I don't think it's unreasonable to repackage

Re: documentation on dealing with legacy Hadoop versions

2014-01-03 Thread Christopher
: an ACCUMULO_BOOTSTRAP_CLASSPATH or something that is exposed directly in our configuration, with reasonable defaults, so it's relatively obvious what we need to start (and, so it's more flexible for redistributing with dependency packaging we don't expect... like the Fedora Hadoop RPM, which depends on the commons

Re: Review Request 16087: ACCUMULO-1796 - document Hadoop versions supported by 1.4.x binary

2013-12-13 Thread Eric Newton
for accumulo. Bugs: ACCUMULO-1796 https://issues.apache.org/jira/browse/ACCUMULO-1796 Repository: accumulo Description --- Update to README to document which versions of Hadoop the default binary distribution of Accumulo 1.4.5 works for. This is option #3 of the ticket

Re: Review Request 16087: ACCUMULO-1796 - document Hadoop versions supported by 1.4.x binary

2013-12-13 Thread Mike Drob
for accumulo. Bugs: ACCUMULO-1796 https://issues.apache.org/jira/browse/ACCUMULO-1796 Repository: accumulo Description --- Update to README to document which versions of Hadoop the default binary distribution of Accumulo 1.4.5 works for. This is option #3 of the ticket

Re: Review Request 16087: ACCUMULO-1796 - document Hadoop versions supported by 1.4.x binary

2013-12-13 Thread Christopher Tubbs
request for accumulo. Bugs: ACCUMULO-1796 https://issues.apache.org/jira/browse/ACCUMULO-1796 Repository: accumulo Description --- Update to README to document which versions of Hadoop the default binary distribution of Accumulo 1.4.5 works for. This is option #3

Re: Review Request 16087: ACCUMULO-1796 - document Hadoop versions supported by 1.4.x binary

2013-12-09 Thread Bill Havanki
for accumulo. Bugs: ACCUMULO-1796 https://issues.apache.org/jira/browse/ACCUMULO-1796 Repository: accumulo Description --- Update to README to document which versions of Hadoop the default binary distribution of Accumulo 1.4.5 works for. This is option #3 of the ticket

Re: Review Request 16087: ACCUMULO-1796 - document Hadoop versions supported by 1.4.x binary

2013-12-09 Thread Bill Havanki
--- Referring now to release notes for Hadoop versions tested. Bugs: ACCUMULO-1796 https://issues.apache.org/jira/browse/ACCUMULO-1796 Repository: accumulo Description --- Update to README to document which versions of Hadoop the default binary distribution of Accumulo 1.4.5 works

Re: Review Request 16087: ACCUMULO-1796 - document Hadoop versions supported by 1.4.x binary

2013-12-09 Thread Sean Busbey
This sounds kind of awkward. As a part of our release process Accumulo is tested on a variety of Hadoop versions. For specific details about tested configurations please see the release notes for this version of Accumulo. As an added bonus, this removes a place where we'd have

Re: Review Request 16087: ACCUMULO-1796 - document Hadoop versions supported by 1.4.x binary

2013-12-09 Thread Bill Havanki
--- Prepositions. Bugs: ACCUMULO-1796 https://issues.apache.org/jira/browse/ACCUMULO-1796 Repository: accumulo Description --- Update to README to document which versions of Hadoop the default binary distribution of Accumulo 1.4.5 works for. This is option #3 of the ticket. Diffs

Re: Review Request 16087: ACCUMULO-1796 - document Hadoop versions supported by 1.4.x binary

2013-12-09 Thread Sean Busbey
for accumulo. Bugs: ACCUMULO-1796 https://issues.apache.org/jira/browse/ACCUMULO-1796 Repository: accumulo Description --- Update to README to document which versions of Hadoop the default binary distribution of Accumulo 1.4.5 works for. This is option #3 of the ticket

Review Request 16087: ACCUMULO-1796 - document Hadoop versions supported by 1.4.x binary

2013-12-06 Thread Bill Havanki
/jira/browse/ACCUMULO-1796 Repository: accumulo Description --- Update to README to document which versions of Hadoop the default binary distribution of Accumulo 1.4.5 works for. This is option #3 of the ticket. Diffs - README bc469fc491ba2203837409933a6786e351831aa8 Diff: https

Re: Review Request 16087: ACCUMULO-1796 - document Hadoop versions supported by 1.4.x binary

2013-12-06 Thread Sean Busbey
Description --- Update to README to document which versions of Hadoop the default binary distribution of Accumulo 1.4.5 works for. This is option #3 of the ticket. Diffs - README bc469fc491ba2203837409933a6786e351831aa8 Diff: https://reviews.apache.org/r/16087/diff

Hadoop Summit Europe -- Accumulo Talks

2013-11-26 Thread Josh Elser
Noticed these talks related to Accumulo were recently posted: https://hadoopsummit.uservoice.com/forums/196820-hadoop-deployment-and-operations/suggestions/5059276-managing-apache-accumulo-at-enterprise-scale https://hadoopsummit.uservoice.com/forums/196822-future-of-apache-hadoop/suggestions

Review Request 15604: ACCUMULO-1792 Backport changes for Hadoop 2.0 from 1.5 to 1.4.

2013-11-15 Thread Sean Busbey
to intact as applicable for 1.4.5-SNAPSHOT. It presumes things like the hadoop versions for the profiles (0.20.203.0 and 2.2.0 respectively, will be set in follow on patches). Because there are so many commits, I'm not including all of their message text here. I think applying the patch

Re: Hadoop 2.0 Support for Accumulo 1.4 Branch

2013-11-14 Thread Christopher
about adding profile support to the pom's for Hadoop 2.2.0 for a 1.4.5 and 1.5.1 release, correct? We are not talking about changing the default build profile for these branches are we? for 1.4.5-SNAPSHOT I am only talking about adding support Hadoop 2.2.0. I am not suggesting we change

Re: Hadoop 2.0 Support for Accumulo 1.4 Branch

2013-11-14 Thread Christopher
thing to avoid. Ah. So I see, not sure why I phrased that that way. Since the default build should still be 0.20.203.0, I'm not sure how it'd end up not being binary compatible. I can update the ticket to clarify the language. Any need to compile should be limited to running Hadoop 2.2.0. Sound

Re: Hadoop 2.0 Support for Accumulo 1.4 Branch

2013-11-14 Thread Sean Busbey
On Thu, Nov 14, 2013 at 6:27 PM, Christopher ctubb...@apache.org wrote: The main thing is that I would not want to see an ACCUMULO-1790 *without* ACCUMULO-1795. Having 1792 alone would be insufficient for me. That is precisely the intention of ACCUMULO-1790. All of the subtasks (including

Re: Hadoop 2.0 Support for Accumulo 1.4 Branch

2013-11-14 Thread Christopher
Nope, I think we're on the same page now. -- Christopher L Tubbs II http://gravatar.com/ctubbsii On Thu, Nov 14, 2013 at 7:39 PM, Sean Busbey busbey...@clouderagovt.com wrote: On Thu, Nov 14, 2013 at 6:27 PM, Christopher ctubb...@apache.org wrote: The main thing is that I would not want to

Re: Hadoop 2.0 Support for Accumulo 1.4 Branch

2013-11-12 Thread Josh Elser
Hadoop 2.x support in the 1.4 release line that breaks compatibility with 0.20. I don't think breaking 0.20 and then possibly fixing it again as a second step is acceptable (because that subsequent work may not ever be done, and I don't think we should break the compatibility contract that we've

Re: Hadoop 2.0 Support for Accumulo 1.4 Branch

2013-11-12 Thread William Slacum
on ACCUMULO-1792 and ACCUMULO-1795, I want to resurrect this thread to make sure everyone's concerns are addressed. For context, here's a link to the start of the last thread: http://bit.ly/1aPqKuH From ACCUMULO-1792, ctubbsii: I'd be reluctant to support any Hadoop 2.x support in the 1.4

Re: Hadoop 2.0 Support for Accumulo 1.4 Branch

2013-11-12 Thread Sean Busbey
On Tue, Nov 12, 2013 at 1:12 PM, Josh Elser josh.el...@gmail.com wrote: To me, it seems like the argument may be coming down to whether or not we break 0.20 hadoop compatibility on a bug-fix release and how concerned we are about letting users lag behind the upstream development. I think

Re: Hadoop 2.0 Support for Accumulo 1.4 Branch

2013-11-12 Thread Sean Busbey
On Tue, Nov 12, 2013 at 1:28 PM, William Slacum wilhelm.von.cl...@accumulo.net wrote: A user of 1.4.a should be able to move to 1.4.b without any major infrastructure changes, such as swapping out HDFS or installing extra add-ons. Right, exactly. Hopefully no part of the original plan

Re: Hadoop 2.0 Support for Accumulo 1.4 Branch

2013-11-12 Thread Josh Elser
On 11/12/13, 12:24 PM, Sean Busbey wrote: On Tue, Nov 12, 2013 at 1:12 PM, Josh Elser josh.el...@gmail.com wrote: To me, it seems like the argument may be coming down to whether or not we break 0.20 hadoop compatibility on a bug-fix release and how concerned we are about letting users lag

Re: Hadoop 2.0 Support for Accumulo 1.4 Branch

2013-11-12 Thread William Slacum
The language of ACCUMULO-1795 indicated that an acceptable state was something that wasn't binary compatible. That's my #1 thing to avoid. Maybe expressly only doing a binary convenience package for 0.20.203.0? If we need an extra package, doesn't that mean a user can't just upgrade Accumulo?

Re: Hadoop 2.0 Support for Accumulo 1.4 Branch

2013-11-12 Thread Sean Busbey
On Tue, Nov 12, 2013 at 2:48 PM, Josh Elser josh.el...@gmail.com wrote: What about the other half: encouraging users to lag (soon to be) two major releases behind? I don't think our current user base needs to be encouraged strongly to upgrade. And as I said previously I think this change

Building Apache Accumulo on top of Apache Hadoop 2.2

2013-10-31 Thread Pierre Kande
Is it possible to build and run the current Apache Accumulo 1.6.0-snapshot code base on top of Hadoop 2.2.0 recently released? Does anyone try it? I will appreciate your feedback -- Regards, Pierre

Re: Building Apache Accumulo on top of Apache Hadoop 2.2

2013-10-31 Thread Christopher
The current master branch builds against 2.2.0 by default. -- Christopher L Tubbs II http://gravatar.com/ctubbsii On Thu, Oct 31, 2013 at 4:48 AM, Pierre Kande pierre.ka...@gmail.com wrote: Is it possible to build and run the current Apache Accumulo 1.6.0-snapshot code base on top of Hadoop

Re: [DISCUSS] Hadoop 2 and Accumulo 1.6.0

2013-10-24 Thread David Medinets
Can an application work with parts in Hadoop 1 and parts in YARN? Could we convert one component at a time to work inside YARN? For example, first move monitor then tracer, etc. Where there any architectural tradeoffs made for Hadoop 1 that no longer apply in Hadoop 2? On Thu, Oct 24, 2013 at 12

Re: [DISCUSS] Hadoop 2 and Accumulo 1.6.0

2013-10-24 Thread John Vines
Accumulo isn't going to run in YARN, it's just a matter of replacing MR compatibility with YARN compatibility. On Thu, Oct 24, 2013 at 7:04 AM, David Medinets david.medin...@gmail.comwrote: Can an application work with parts in Hadoop 1 and parts in YARN? Could we convert one component

Re: [DISCUSS] Hadoop 2 and Accumulo 1.6.0

2013-10-24 Thread Josh Elser
that Accumulo needs to do here. The only real concern is the compatibility with Hadoop1 and Hadoop2 APIs (which is already done). There are no architectural tradeoffs that I am aware of between hadoop 1 and 2. On 10/24/13 10:01 AM, John Vines wrote: Accumulo isn't going to run in YARN, it's just

Re: [DISCUSS] Hadoop 2 and Accumulo 1.6.0

2013-10-24 Thread Steve Loughran
On 24 October 2013 18:01, John Vines vi...@apache.org wrote: Accumulo isn't going to run in YARN, Well it can if you want it too - what we are doing with hoya is giving the option to do without changing the core application. Some applications (e.g. Samza) are being written from the ground up

Re: [DISCUSS] Hadoop 2 and Accumulo 1.6.0

2013-10-24 Thread Steve Loughran
at a lower level, Hadoop 2.2 has some other opportunities too - Move up the dependencies (inc things like SLF4J, protobuf) to be consistent. -co-exist with YARN. That shouldn't take any effort, merely that a YARN node manager can run on every server node, so that YARN-scheduled work can

[DISCUSS] Hadoop 2 and Accumulo 1.6.0

2013-10-23 Thread Josh Elser
To ensure that we get broader community interaction than only on a Jira issue [1], I want to get community feedback about the version of Hadoop which the default, deployed Accumulo artifacts will be compiled against. Currently, Accumulo builds against a Hadoop-1 series release (1.5.1-SNAPSHOT

Re: [DISCUSS] Hadoop 2 and Accumulo 1.6.0

2013-10-23 Thread William Slacum
There wasn't any discussions in those tickets as to what Hadoop 2 provides Accumulo. If we're going to still support 1, then any new features only possible with 2 have to become optional until we ditch support for 1. Is there anything people have in mind, feature wise, that Hadoop 2 would help

Re: [DISCUSS] Hadoop 2 and Accumulo 1.6.0

2013-10-23 Thread Josh Elser
Highlights from hadoop.apache.org YARN - A general purpose resource management system for Hadoop to allow MapReduce and other other data processing frameworks and services High Availability for HDFS HDFS Federation HDFS Snapshots NFSv3 access to data in HDFS Support

Re: Hadoop 2.0 Support for Accumulo 1.4 Branch

2013-10-15 Thread dlmarion
Just to be clear, we are talking about adding profile support to the pom's for Hadoop 2.2.0 for a 1.4.5 and 1.5.1 release, correct? We are not talking about changing the default build profile for these branches are we? - Original Message - From: Billie Rinaldi billie.rina...@gmail.com

Re: Hadoop 2.0 Support for Accumulo 1.4 Branch

2013-10-15 Thread Sean Busbey
On Tue, Oct 15, 2013 at 7:16 AM, dlmar...@comcast.net wrote: Just to be clear, we are talking about adding profile support to the pom's for Hadoop 2.2.0 for a 1.4.5 and 1.5.1 release, correct? We are not talking about changing the default build profile for these branches are we? for 1.4.5

Re: Hadoop 2.0 Support for Accumulo 1.4 Branch

2013-10-15 Thread Sean Busbey
On Tue, Oct 15, 2013 at 10:16 AM, Sean Busbey bus...@cloudera.com wrote: On Tue, Oct 15, 2013 at 7:16 AM, dlmar...@comcast.net wrote: Just to be clear, we are talking about adding profile support to the pom's for Hadoop 2.2.0 for a 1.4.5 and 1.5.1 release, correct? We are not talking about

Re: Hadoop 2.0 Support for Accumulo 1.4 Branch

2013-10-15 Thread Joey Echeverria
I think you meant: Ugh, Hadoop versions.[1] [1] http://blog.cloudera.com/blog/2012/04/apache-hadoop-versions-looking-ahead-3/ On Tue, Oct 15, 2013 at 11:20 AM, Sean Busbey bus...@cloudera.com wrote: On Tue, Oct 15, 2013 at 10:16 AM, Sean Busbey bus...@cloudera.com wrote: On Tue, Oct 15

Hadoop 2.0 Support for Accumulo 1.4 Branch

2013-10-14 Thread Sean Busbey
Hey All, I'd like to restart the conversation from end July / start August about Hadoop 2 support on the 1.4 branch. Specifically, I'd like to get some requirements ironed out so I can file one or more jiras. I'd also like to get a plan for application. =requirements Here's the requirements I

Re: Hadoop 2.0 Support for Accumulo 1.4 Branch

2013-10-14 Thread Mike Drob
Responses Inline. - Mike On Mon, Oct 14, 2013 at 12:55 PM, Sean Busbey bus...@cloudera.com wrote: Hey All, I'd like to restart the conversation from end July / start August about Hadoop 2 support on the 1.4 branch. Specifically, I'd like to get some requirements ironed out so I can file

Re: Hadoop 2.0 Support for Accumulo 1.4 Branch

2013-10-14 Thread Josh Elser
For #2, from what I've read, we should definitely bump up the dependency on 1.5.1-SNAPSHOT to 2.1.0-beta, and, given what Ted replied with, to 2.2.0-beta for that hadoop-2 profile. I probably stated this before, but I'd much rather see more effort in testing Accumulo 1.5.x (and 1.6.0

Re: Hadoop 2.0 Support for Accumulo 1.4 Branch

2013-10-14 Thread Sean Busbey
+ agitation Keith mentioned running these against a CDH4 cluster, but I presume that since Apache Releases are our stated compatibilities it would actually be against whatever versions we list. Based on #1 and #2 above, I would expect that to be Apache Hadoop 0.20.203.0 and Apache Hadoop

Re: Hadoop 2.0 Support for Accumulo 1.4 Branch

2013-10-14 Thread Sean Busbey
On Mon, Oct 14, 2013 at 10:02 PM, Josh Elser josh.el...@gmail.com wrote: For #2, from what I've read, we should definitely bump up the dependency on 1.5.1-SNAPSHOT to 2.1.0-beta, and, given what Ted replied with, to 2.2.0-beta for that hadoop-2 profile. so 1.5.1-SNAPSHOT and this proposed

Re: Hadoop 2.0 Support for Accumulo 1.4 Branch

2013-10-14 Thread Billie Rinaldi
Thanks for the note, Ted. That vote is for 2.2.0, not -beta. On Oct 14, 2013 7:30 PM, Ted Yu yuzhih...@gmail.com wrote: w.r.t. hadoop-2 release, see this thread: http://search-hadoop.com/m/YSTny19y1Ha1/hadoop+2.2.0 Looks like 2.2.0-beta would pass votes. Cheers On Mon, Oct 14, 2013 at 7

Re: some test(s) hung on hadoop 2

2013-08-23 Thread Ted Yu
/ctubbsii On Thu, Aug 15, 2013 at 12:54 PM, Ted Yu yuzhih...@gmail.com wrote: Hi, Some tests, e.g. FormatterCommandTest, hung when I ran test suite on hadoop 2 using Linux and JDK 7. Here is the command I used: ~/apache-maven-3.0.4/bin/mvn test -Dhadoop.profile=2.0 Here is related

Re: some test(s) hung on hadoop 2

2013-08-22 Thread Christopher
. FormatterCommandTest, hung when I ran test suite on hadoop 2 using Linux and JDK 7. Here is the command I used: ~/apache-maven-3.0.4/bin/mvn test -Dhadoop.profile=2.0 Here is related portion from jstack: Thread 1895: (state = IN_NATIVE) - java.io.FileInputStream.readBytes(byte[], int, int) @bci=0

Re: some test(s) hung on hadoop 2

2013-08-22 Thread Ted Yu
Tubbs II http://gravatar.com/ctubbsii On Thu, Aug 15, 2013 at 12:54 PM, Ted Yu yuzhih...@gmail.com wrote: Hi, Some tests, e.g. FormatterCommandTest, hung when I ran test suite on hadoop 2 using Linux and JDK 7. Here is the command I used: ~/apache-maven-3.0.4/bin/mvn test

Re: upgrade hadoop version for hadoop-1.0 profile

2013-08-15 Thread Ted Yu
elsewhere. -- Christopher L Tubbs II http://gravatar.com/ctubbsii On Wed, Aug 14, 2013 at 12:12 PM, Ted Yu yuzhih...@gmail.com wrote: Hi, I noticed that, in trunk, the hadoop.version for hadoop-1.0 profile is 1.0.4 The recent stable release was 1.2.1 Should hadoop version be upgraded

Re: upgrade hadoop version for hadoop-1.0 profile

2013-08-15 Thread John Vines
...@gmail.com wrote: Hi, I noticed that, in trunk, the hadoop.version for hadoop-1.0 profile is 1.0.4 The recent stable release was 1.2.1 Should hadoop version be upgraded ? Thanks

Re: upgrade hadoop version for hadoop-1.0 profile

2013-08-15 Thread Josh Elser
14, 2013 at 12:12 PM, Ted Yu yuzhih...@gmail.com wrote: Hi, I noticed that, in trunk, the hadoop.version for hadoop-1.0 profile is 1.0.4 The recent stable release was 1.2.1 Should hadoop version be upgraded ? Thanks

some test(s) hung on hadoop 2

2013-08-15 Thread Ted Yu
Hi, Some tests, e.g. FormatterCommandTest, hung when I ran test suite on hadoop 2 using Linux and JDK 7. Here is the command I used: ~/apache-maven-3.0.4/bin/mvn test -Dhadoop.profile=2.0 Here is related portion from jstack: Thread 1895: (state = IN_NATIVE) - java.io.FileInputStream.readBytes

upgrade hadoop version for hadoop-1.0 profile

2013-08-14 Thread Ted Yu
Hi, I noticed that, in trunk, the hadoop.version for hadoop-1.0 profile is 1.0.4 The recent stable release was 1.2.1 Should hadoop version be upgraded ? Thanks

Re: upgrade hadoop version for hadoop-1.0 profile

2013-08-14 Thread Christopher
, 2013 at 12:12 PM, Ted Yu yuzhih...@gmail.com wrote: Hi, I noticed that, in trunk, the hadoop.version for hadoop-1.0 profile is 1.0.4 The recent stable release was 1.2.1 Should hadoop version be upgraded ? Thanks

Re: upgrade hadoop version for hadoop-1.0 profile

2013-08-14 Thread Ted Yu
or serious considerations to make about losing compatibility elsewhere. -- Christopher L Tubbs II http://gravatar.com/ctubbsii On Wed, Aug 14, 2013 at 12:12 PM, Ted Yu yuzhih...@gmail.com wrote: Hi, I noticed that, in trunk, the hadoop.version for hadoop-1.0 profile is 1.0.4 The recent

Re: Hadoop 2.0 Support for Accumulo 1.4 Branch

2013-08-02 Thread Joey Echeverria
, Dave Marion dlmar...@comcast.net wrote: Any update? -Original Message- From: Joey Echeverria [mailto:j...@cloudera.com] Sent: Monday, July 29, 2013 1:24 PM To: dev@accumulo.apache.org Subject: Re: Hadoop 2.0 Support for Accumulo 1.4 Branch We're testing this today. I'll report

Re: Hadoop 2.0 Support for Accumulo 1.4 Branch

2013-08-02 Thread Christopher
PM, Dave Marion dlmar...@comcast.net wrote: Any update? -Original Message- From: Joey Echeverria [mailto:j...@cloudera.com] Sent: Monday, July 29, 2013 1:24 PM To: dev@accumulo.apache.org Subject: Re: Hadoop 2.0 Support for Accumulo 1.4 Branch We're testing this today. I'll report

Re: Hadoop 2.0 Support for Accumulo 1.4 Branch

2013-08-02 Thread Joey Echeverria
...@cloudera.com] Sent: Monday, July 29, 2013 1:24 PM To: dev@accumulo.apache.org Subject: Re: Hadoop 2.0 Support for Accumulo 1.4 Branch We're testing this today. I'll report back what we find. -Joey — Sent from Mailbox for iPhone On Fri, Jul 26, 2013 at 3:34 PM, null dlmar

Re: Hadoop 2.0 Support for Accumulo 1.4 Branch

2013-08-02 Thread Mike Drob
, 1.0, 2.0). -Joey On Thu, Aug 1, 2013 at 7:33 PM, Dave Marion dlmar...@comcast.net wrote: Any update? -Original Message- From: Joey Echeverria [mailto:j...@cloudera.com] Sent: Monday, July 29, 2013 1:24 PM To: dev@accumulo.apache.org Subject: Re: Hadoop 2.0 Support

RE: Hadoop 2.0 Support for Accumulo 1.4 Branch

2013-08-01 Thread Dave Marion
Any update? -Original Message- From: Joey Echeverria [mailto:j...@cloudera.com] Sent: Monday, July 29, 2013 1:24 PM To: dev@accumulo.apache.org Subject: Re: Hadoop 2.0 Support for Accumulo 1.4 Branch We're testing this today. I'll report back what we find. -Joey — Sent from Mailbox

Re: Hadoop 2.0 Support for Accumulo 1.4 Branch

2013-07-29 Thread Joey Echeverria
billie.rina...@gmail.com To: dev@accumulo.apache.org Sent: Friday, July 26, 2013 3:02:41 PM Subject: Re: Hadoop 2.0 Support for Accumulo 1.4 Branch On Fri, Jul 26, 2013 at 11:33 AM, Joey Echeverria j...@cloudera.com wrote: If these patches are going to be included with 1.4.4 or 1.4.5, I would

Hadoop 2.0 Support for Accumulo 1.4 Branch

2013-07-26 Thread Joey Echeverria
on how we label support for Hadoop 2.0. Thoughts? -Joey

Re: Hadoop 2.0 Support for Accumulo 1.4 Branch

2013-07-26 Thread Eric Newton
My question is if the community would be interested in us pulling those back ports upstream? Yes, please.

Re: Hadoop 2.0 Support for Accumulo 1.4 Branch

2013-07-26 Thread Joey Echeverria
API changes or Accumulo behavior changes? I believe this would violate the previously agreed upon rule of no feature back ports to 1.4.3, depending on how we label support for Hadoop 2.0. Thoughts? -Joey -- Joey Echeverria Director, Federal FTS Cloudera, Inc.

Re: Hadoop 2.0 Support for Accumulo 1.4 Branch

2013-07-26 Thread Keith Turner
release of the server daemons. Great. I think this would be a good patch for 1.4. I assume that if a user stays with Hadoop 1 there are no dependency changes? -Joey On Fri, Jul 26, 2013 at 11:45 AM, Keith Turner ke...@deenlo.com wrote: On Fri, Jul 26, 2013 at 11:02 AM, Joey Echeverria j

Re: Hadoop 2.0 Support for Accumulo 1.4 Branch

2013-07-26 Thread Joey Echeverria
this would be a good patch for 1.4. I assume that if a user stays with Hadoop 1 there are no dependency changes? Yup. It works the same way as 1.5 where all of the dependency changes are in a Hadoop 2.0 profile. -Joey

Re: Hadoop 2.0 Support for Accumulo 1.4 Branch

2013-07-26 Thread Billie Rinaldi
battery of tests? I have a 10 node cluster that I can use for this. Great. I think this would be a good patch for 1.4. I assume that if a user stays with Hadoop 1 there are no dependency changes? Yup. It works the same way as 1.5 where all of the dependency changes are in a Hadoop 2.0

Re: Hadoop 2.0 Support for Accumulo 1.4 Branch

2013-07-26 Thread Keith Turner
think this would be a good patch for 1.4. I assume that if a user stays with Hadoop 1 there are no dependency changes? Yup. It works the same way as 1.5 where all of the dependency changes are in a Hadoop 2.0 profile. -Joey

Re: Hadoop 2.0 Support for Accumulo 1.4 Branch

2013-07-26 Thread dlmarion
Will 1.4 still work with 0.20 with these patches? Great point Billie. - Original Message - From: Billie Rinaldi billie.rina...@gmail.com To: dev@accumulo.apache.org Sent: Friday, July 26, 2013 3:02:41 PM Subject: Re: Hadoop 2.0 Support for Accumulo 1.4 Branch On Fri, Jul

Re: [propsoal] Changing classpath creation for hadoop to just use `hadoop classpath` 's output

2013-07-10 Thread Eric Newton
Why not just use the hadoop classspath generated by running `hadoop classpath` I like it! +1 On Tue, Jul 9, 2013 at 11:33 PM, Jonathan Hsieh j...@cloudera.com wrote: tl;dr Ideally the generation of hadoop+accumulo's classpath should only be done in one place. At least for all versions

hadoop-2.0 incompatibility

2013-05-21 Thread Eric Newton
Ugh. While running the continuous ingest verify, yarn spit this out: Error: Found interface org.apache.hadoop.mapreduce.Counter, but class was expected This is preventing the reduce step from completing. -Eric

<    1   2   3   >