Re: Accumulo Summit Hackathon

[Joey Echeverria]

I'd love to help folks to work on Kite SDK/Accumulo integration:

https://issues.cloudera.org/browse/CDK-220

-Joey


On Mon, Jun 9, 2014 at 2:44 PM, David Medinets 
wrote:

> I've been researching docker in the hopes of understanding enough to get a
> Dockerized Accumulo instance running.
>
>
> On Mon, Jun 9, 2014 at 2:41 PM, Josh Elser  wrote:
>
> > A couple of things that come to mind, mostly things that I have wanted to
> > use/extend/play-with
> >
> > * JIRA is a great source (like Bill H pointed out)
> > * Cascading+Accumulo integration https://github.com/airawat/
> > cascading.accumulo
> > * Harness for deploying an Accumulo "cluster" via Docker and/or Vagrant
> > against Apache source (e.g. deploy a 3node Docker Accumulo cluster using
> > the 'master' Accumulo branch)
> > * Try to get Flume integration updated/committed (
> > https://issues.apache.org/jira/browse/FLUME-2234)
> > * I'd also love to see some work on https://issues.apache.org/
> > jira/browse/ACCUMULO-1013
> >
> > Also, Bill's link didn't take me directly to the search results, you
> might
> > have to click the "Return to search" link the upper-right-hand corner.
> >
> > I'll make a pass today over the open and unassigned tickets to see if
> > anything else jumps out at me that I think should be marked as "newbie".
> >
> > - Josh
> >
> > On 6/8/14, 8:38 PM, Tamburello, Paul [USA] wrote:
> >
> >> Accumulo Development Team -
> >>
> >> As many of you already know, this week during the Accumulo Summit<
> >> http://accumulosummit.com/>, Booz Allen Hamilton is sponsoring a
> >> Hackathon event following
> >> the conference, from 5PM til 11PM.  As part of the agenda, we are
> working
> >> to put together a list of existing JIRA tickets that folks can work on
> >> during the Hackathon, so we want to solicit input from the Accumulo
> >> contributors. So, if you have suggestions for tasks that people can work
> >> on, please respond directly to me and we will add them to our list.
> >>
> >> Thanks in advance, see you all on Thursday!
> >> Paul
> >>
> >> Paul Tamburello
> >> Senior Lead Engineer
> >> Strategic Innovation Group
> >> 301-821-8861 / 919-260-6158
> >> tamburello_p...@bah.com
> >>
> >>
>

Re: Delays in email delivery?

[Joey Echeverria]

Infra noted a 1.5M long email queue for sending to gmail, so I think it's the 
interaction between the two. --
Joey Echeverria
Chief Architect
Cloudera Government Solutions

On Fri, May 16, 2014 at 4:06 PM, Sean Mackrory 
wrote:

> I've been seeing delays, and another user on the Bigtop mailing list
> reported similar problems a short time ago. I also got a notice yesterday
> that emails from the mailing list were bouncing back, and emails from
> several other committers with whom I communicate regularly are getting
> flagged as possible scams. I would suspect gmail, but all of these problems
> seem isolated to emails coming through the ASF mailing lists.
> On Thu, May 15, 2014 at 1:57 PM, Michael Wall  wrote:
>> I had been having trouble tracking some of the threads on both the user and
>> dev Accumulo lists.  I realized this morning it was because emails were
>> being delayed.  For example, David Medinets posted "Accumulo Blog Posts" to
>> the dev list 6 days ago.
>>
>> I got Billie's response on the 12th.
>>
>> Received: from mail.apache.org (hermes.apache.org. [140.211.11.3])
>> by mx.google.com with SMTP id
>> sf3si10324675pac.91.2014.05.12.08.22.12
>> for ;
>> Mon, 12 May 2014 08:22:12 -0700 (PDT)
>>
>>
>>
>> But just got Dave's original email today.
>>
>> Received: from mail.apache.org (hermes.apache.org. [140.211.11.3])
>> by mx.google.com with SMTP id
>> ov9si3134110pbc.41.2014.05.15.11.54.36
>> for ;
>> Thu, 15 May 2014 11:54:37 -0700 (PDT)
>>
>>
>>
>> Anyone else seeing this?
>>
>> Mike
>>

Re: [DISCUSS] Do we want contributors assigning to themselves?

[Joey Echeverria]

> It seems like a simple solution to that> is some documentation on our web

> site encouraging people to contact

> the person a ticket is assigned to if

> they are interested in working on it. 




+1
--
Joey Echeverria
Chief Architect
Cloudera Government Solutions

On Fri, May 16, 2014 at 9:46 AM, Keith Turner  wrote:

> Looking at the ticket you linked it seems it easy for infra to make this
> change.   As long as its not a hassle for infra I would be in favor of
> changing it back.
> The problem w/ issues being assigned to people who are not working on them
> (it seems this is sparks concern) applies to commiters and contributors.
> It seems like a simple solution to that is some documentation on our web
> site encouraging people to contact the person a ticket is assigned to if
> they are interested in working on it.
> On Wed, May 14, 2014 at 4:38 PM, Sean Busbey  wrote:
>> We don't have a formal onboarding process for drawing in new contributors,
>> but a recent ASF Infra change impacts what I've observed historically.
>>
>> Here's what I've seen historically, more or less:
>>
>> 1) Someone expresses interest in a ticket
>>
>> 2) PMC/committers add them to the list of contributors in jira
>>
>> 3) respond to interest informing person of this change and encouraging them
>> to assign the ticket to themselves
>>
>> 4) work happens on ticket
>>
>> 5) review/commit happens eventually
>>
>> 6) If contributor wants, added to website
>>
>> 7) contributor thanked and encouraged to find more tickets to assign to
>> themselves.
>>
>> Due to a request from Spark, the ASF Jira got changed to default to not
>> allow contributors to assign tickets[1].
>>
>> Before I speak for the PMC and file a follow on to change things back, I
>> just wanted a gut check that we like the above as a general approach.
>>
>>
>> [1]: https://issues.apache.org/jira/browse/INFRA-7675
>>
>> --
>> Sean
>>

Re: [DISCUSS] Do we want contributors assigning to themselves?

[Joey Echeverria]

I like that flow.


I'm very much against not letting contributors assign tickets. If you have a 
problem with "old" contributors "stealing" tickets from new contributors, then 
that needs to be handled with best practices documentation and/or explicit 
coaching. Locking down who can assign tickets is a poor way of managing a 
misbehaving community. As we grow the community, we need to give more 
opportunities for contributors to show their judgement on both technical and 
project decisions. Holding back the ability to assign tickets makes that more 
difficult.

On Thu, May 15, 2014 at 9:15 AM, Sean Busbey  wrote:

> We don't have a formal onboarding process for drawing in new contributors,
> but a recent ASF Infra change impacts what I've observed historically.
> Here's what I've seen historically, more or less:
> 1) Someone expresses interest in a ticket
> 2) PMC/committers add them to the list of contributors in jira
> 3) respond to interest informing person of this change and encouraging them
> to assign the ticket to themselves
> 4) work happens on ticket
> 5) review/commit happens eventually
> 6) If contributor wants, added to website
> 7) contributor thanked and encouraged to find more tickets to assign to
> themselves.
> Due to a request from Spark, the ASF Jira got changed to default to not
> allow contributors to assign tickets[1].
> Before I speak for the PMC and file a follow on to change things back, I
> just wanted a gut check that we like the above as a general approach.
> [1]: https://issues.apache.org/jira/browse/INFRA-7675
> -- 
> Sean

Re: [DISCUSS] packaging our dependencies

[Joey Echeverria]

Packaging other jars that had been made available at runtime by virtue of their 
existence in the Hadoop directories. 


I'm only talking about dependencies that were/are provided by Hadoop. 




But since you brought up ZooKeeper, my understanding is that ZK intends for 
dependent projects to only rely on the ZK jar that is in the top level of the 
tarball. If you need other jars, you should package those yourself. WARNING: my 
info about ZK may be out of date as it's been a long time since I spoke to the 
project about how they intend services that rely on it to be consumed.

On Mon, May 12, 2014 at 7:30 PM, Christopher  wrote:

> Does that mean package everything else?
> What about ZooKeeper?
> --
> Christopher L Tubbs II
> http://gravatar.com/ctubbsii
> On Mon, May 12, 2014 at 3:38 PM, Joey Echeverria  
> wrote:
>> +1 to only depending on Hadoop client jars.
>>
>>
>> --
>> Joey Echeverria
>> Chief Architect
>> Cloudera Government Solutions
>>
>>
>> On Sun, May 11, 2014 at 6:07 PM, Christopher  wrote:
>>> In general, I think this is reasonable... especially because Hadoop
>>> Client stabilizes things a bit. On the other hand, things get really
>>> complicated with dependencies in the pom (somewhat complicated), and
>>> packaged dependencies (more complicated), when we're talking about
>>> supporting both Hadoop 1 and Hadoop 2. I know some of us want to drop
>>> Hadoop 1 support in 2.0.0, and I think this is one more good reason to
>>> do that.
>>>
>>> Another data point that I think is going to complicate things a (very)
>>> tiny bit: the work on ACCUMULO-2589 includes things like: drop the
>>> dependencies on Hadoop from the API. But, we're likely to still have a
>>> dependency on guava (there was a suggestion to use guava's @Beta
>>> annotations in the API). Maybe this is fine because the packaging
>>> considerations for the binary tarball are not the same as the API
>>> module dependencies (though they'll have to be compatible), but it's
>>> something to consider.
>>>
>>> --
>>> Christopher L Tubbs II
>>> http://gravatar.com/ctubbsii
>>>
>>>
>>> On Sun, May 11, 2014 at 4:45 PM, Sean Busbey  wrote:
>>>> ACCUMULO-2786 has brought up the issue of what dependencies we bring with
>>>> Accumulo rather than depend on the environment providing[1].
>>>>
>>>> Christopher explains our extant reasoning thus
>>>>
>>>>> The precedent has been: if vanilla Apache Hadoop provides it in its bin
>>>> tarball, we don't need to.
>>>>
>>>> I'd like us to move to packaging any dependencies that aren't brought in by
>>>> Hadoop Client.
>>>>
>>>> 1) Our existing practice developed before Hadoop Client existed, so we
>>>> essentially *had* to have all of the Hadoop related deps on our classpath.
>>>> For versions where we default to Hadoop 2, we can improve things.
>>>>
>>>> 2) We should encourage users to follow good practice by minimizing the
>>>> number of jars added to the classpath.
>>>>
>>>> 3) We have to still include the jars found in Hadoop Client because we use
>>>> hadoop.
>>>>
>>>> 4) Limiting the dependencies we rely on external sources to provide allows
>>>> us to update more of our dependencies to current versions.
>>>>
>>>> 5) Minimizing the number of jars we rely on from external sources reduces
>>>> the chances that they change out from under us (and thus reduces the number
>>>> of external factors we have to remain cognizant of)
>>>>
>>>> 6) Minimizing the classpath reduces the chances of having multiple
>>>> different versions of the same library present.
>>>>
>>>> I'd also like for us to *not* package any of the jars brought in by Hadoop
>>>> Client. Due to the additional work it would take to downgrade our version
>>>> of guava, I'd like to wait to do that.
>>>>
>>>> [1]: https://issues.apache.org/jira/browse/ACCUMULO-2786
>>>>
>>>> --
>>>> Sean

Re: [DISCUSS] packaging our dependencies

[Joey Echeverria]

+1 to only depending on Hadoop client jars.


--
Joey Echeverria
Chief Architect
Cloudera Government Solutions


On Sun, May 11, 2014 at 6:07 PM, Christopher  wrote:
> In general, I think this is reasonable... especially because Hadoop
> Client stabilizes things a bit. On the other hand, things get really
> complicated with dependencies in the pom (somewhat complicated), and
> packaged dependencies (more complicated), when we're talking about
> supporting both Hadoop 1 and Hadoop 2. I know some of us want to drop
> Hadoop 1 support in 2.0.0, and I think this is one more good reason to
> do that.
>
> Another data point that I think is going to complicate things a (very)
> tiny bit: the work on ACCUMULO-2589 includes things like: drop the
> dependencies on Hadoop from the API. But, we're likely to still have a
> dependency on guava (there was a suggestion to use guava's @Beta
> annotations in the API). Maybe this is fine because the packaging
> considerations for the binary tarball are not the same as the API
> module dependencies (though they'll have to be compatible), but it's
> something to consider.
>
> --
> Christopher L Tubbs II
> http://gravatar.com/ctubbsii
>
>
> On Sun, May 11, 2014 at 4:45 PM, Sean Busbey  wrote:
>> ACCUMULO-2786 has brought up the issue of what dependencies we bring with
>> Accumulo rather than depend on the environment providing[1].
>>
>> Christopher explains our extant reasoning thus
>>
>>> The precedent has been: if vanilla Apache Hadoop provides it in its bin
>> tarball, we don't need to.
>>
>> I'd like us to move to packaging any dependencies that aren't brought in by
>> Hadoop Client.
>>
>> 1) Our existing practice developed before Hadoop Client existed, so we
>> essentially *had* to have all of the Hadoop related deps on our classpath.
>> For versions where we default to Hadoop 2, we can improve things.
>>
>> 2) We should encourage users to follow good practice by minimizing the
>> number of jars added to the classpath.
>>
>> 3) We have to still include the jars found in Hadoop Client because we use
>> hadoop.
>>
>> 4) Limiting the dependencies we rely on external sources to provide allows
>> us to update more of our dependencies to current versions.
>>
>> 5) Minimizing the number of jars we rely on from external sources reduces
>> the chances that they change out from under us (and thus reduces the number
>> of external factors we have to remain cognizant of)
>>
>> 6) Minimizing the classpath reduces the chances of having multiple
>> different versions of the same library present.
>>
>> I'd also like for us to *not* package any of the jars brought in by Hadoop
>> Client. Due to the additional work it would take to downgrade our version
>> of guava, I'd like to wait to do that.
>>
>> [1]: https://issues.apache.org/jira/browse/ACCUMULO-2786
>>
>> --
>> Sean

Re: [DISCUSS] Dev branches reflecting minor/major versions, not bugfixes

[Joey Echeverria]

On Mon, May 12, 2014 at 10:08 AM, Alex Moundalexis
 wrote:
> To confirm, a bug fix release will just cut from whatever specific commit
> is selected by the proposer?

My read is that if there were issues fixed in 1.7.0-SNAPSHOT that
would warrant a 1.6.1 release, then a new branch would be made, those
issues would be fixed in the branch, and then a release would be cut.
Basically, this would change from having long-lived bug fix branches
to just-in-time branches specifically for a bug fix release.

-Joey

>
> On Sun, May 11, 2014 at 3:15 PM, Christopher  wrote:
>
>> Accumulo developers:
>>
>> As part of our transition to better versioning standards, and more
>> regular releases, with better release planning, I was thinking that
>> our development branches should generally reflect an anticipated
>> minor/major release version, and not an expected bugfix version. It
>> seems to me that we should focus active development on minor/major
>> releases, and branch for bugfix releases temporarily, only to push out
>> an important bugfix.
>>
>> With that in mind, I'd like to change the current 1.6.1-SNAPSHOT to
>> 1.7.0-SNAPSHOT in expectation for a forthcoming minor release (we
>> would have to discuss what kinds of things we'd want in such a
>> release. Minimally, I want ACCUMULO-1691), and the master branch to
>> 2.0.0 for development on the next major release.
>>
>> If there's any outstanding bugfixes in the 1.6.1-SNAPSHOT branch that
>> would warrant a separate bugfix release, I think we should discuss
>> them and plan for a 1.6.1 within a month or so (along with a 1.5.2).
>>
>> I'd like to discuss this here a bit and see if this makes sense before
>> initiating a vote on it.
>>
>> --
>> Christopher L Tubbs II
>> http://gravatar.com/ctubbsii
>>

Re: [VOTE] end of life plan for 1.4 branch

[Joey Echeverria]

> 1.4-eol
> 1.4-closed
> 1.4-orphaned
> 1.4-closeout
> 1.4-abandoned
> 1.4-unreleased

I'm also +1 to other, similar names. My only sticking point be that it
be prefixed 1.4 and not 1.4.6.

-Joey

On Tue, May 6, 2014 at 12:22 PM, Joey Echeverria
 wrote:
> [X ] +1 I am in favor of announcing End of Life according to the above
> plan with any of the following for the tag name:
>
> 1.4-eol
> 1.4-closed
> 1.4-orphaned
> 1.4-closeout
> 1.4-abandoned
> 1.4-unreleased
>
> -Joey
>
> On Tue, May 6, 2014 at 9:21 AM, Drew Farris  wrote:
>> On Tue, May 6, 2014 at 8:53 AM, Christopher  wrote:
>>
>>> I don't see how that affects removing of the branch for active
>>> development. If an issue
>>> warrants it, that branch can always be reopened. Removing it indicates
>>> that it's not expected to be reopened, and that we've agreed to focus
>>> on new versions.
>>>
>>
>> I don't like removing branches because forces those folks who are
>> maintaining their own 1.4 branches to figure out how to fix things locally
>> when the remote branch they're tracking goes away. Is it sufficient to tell
>> folks to do the following to address this?
>>
>> git rebase --onto 1.4.6-SNAPSHOT-eol 1.4.6-SNAPSHOT 1.4.6-SNAPSHOT-local
>>
>> What happens if the branch is deleted and then is reopened at a later time?
>> Are there further machinations that a developer maintaining a 1.4.x branch
>> much go through to get back on track?
>>
>> Perhaps this is just the way with git, and I'm trapped in the mindset of
>> long-running branches that run parallel to major revision development and
>> aren't targeted at a specific point release. In looking at this I'm
>> reminded that the Accumulo community has chosen the latter path where
>> branches are short-lived and targeted at the next release.
>>
>>
>>> I'm not sure if that means we should archive the 1.4.x
>>> versions in JIRA, so people can mark those versions as affected or
>>> not. Maybe it'd just be useful to just archive 1.4.0-1.4.3, and leave
>>> 1.4.4/1.4.5 unarchived. (I suggest the last two versions of 1.4, only
>>> because the last version introduced a lot of changes that people may
>>> be reluctant to update to, if they aren't transitioning to hadoop 2).
>>>
>>
>> I see JIRA being useful as both a work tracking/planning tool >and< a user
>> support tool / record of project history (like commit history). Would
>> archiving releases prevent historic issues from being findable via google?
>>
>> --
>>> Christopher L Tubbs II
>>> http://gravatar.com/ctubbsii
>>>
>>>
>>> On Tue, May 6, 2014 at 8:34 AM, Drew Farris  wrote:
>>> > Thanks for the response Joey.
>>> >
>>> > It sounds as if there's agreement on a number of points and it sounds
>>> like
>>> > I'm the only person not in favor of deleting the branch and creating a
>>> tag
>>> > a this point. Also, bug management is an interesting issue. Thoughts
>>> > in-line below:
>>> >
>>> > On Mon, May 5, 2014 at 4:07 PM, Joey Echeverria <
>>> joey...@clouderagovt.com>wrote:
>>> >
>>> >>
>>> >> There is also the impact on ticket workflow. When a version is EOLed,
>>> >> I'd not expect the community to provide any additional fixes for that
>>> >> release line. If 1.4 hangs around, then it creates confusion over what
>>> >> will happen to tickets filed against it. It also will confuse users as
>>> >> they may keep filing 1.4 tickets.
>>> >>
>>> >
>>> > If people find ticket-worthy issues in 1.4 after it's end-of-lifed
>>> wouldn't
>>> > we expect them to file a ticket against that version? Shouldn't these
>>> > tickets reflect known issues with a release of software that people use?
>>> > Regardless of the desire of the development community to produce new
>>> > releases of a specific branch, it is a service to the community of users
>>> to
>>> > be able to record known issues (even if these will ultimately result in a
>>> > wontfix resolution). Google does a very good job indexing the Apache
>>> JIRA.
>>> >
>>> > Furthermore, issue reporting activity is a reflection of real-world use
>>> > which should naturally migrate to future versions, and if people aren't
>>> > mig

Re: [VOTE] end of life plan for 1.4 branch

[Joey Echeverria]

[X ] +1 I am in favor of announcing End of Life according to the above
plan with any of the following for the tag name:

1.4-eol
1.4-closed
1.4-orphaned
1.4-closeout
1.4-abandoned
1.4-unreleased

-Joey

On Tue, May 6, 2014 at 9:21 AM, Drew Farris  wrote:
> On Tue, May 6, 2014 at 8:53 AM, Christopher  wrote:
>
>> I don't see how that affects removing of the branch for active
>> development. If an issue
>> warrants it, that branch can always be reopened. Removing it indicates
>> that it's not expected to be reopened, and that we've agreed to focus
>> on new versions.
>>
>
> I don't like removing branches because forces those folks who are
> maintaining their own 1.4 branches to figure out how to fix things locally
> when the remote branch they're tracking goes away. Is it sufficient to tell
> folks to do the following to address this?
>
> git rebase --onto 1.4.6-SNAPSHOT-eol 1.4.6-SNAPSHOT 1.4.6-SNAPSHOT-local
>
> What happens if the branch is deleted and then is reopened at a later time?
> Are there further machinations that a developer maintaining a 1.4.x branch
> much go through to get back on track?
>
> Perhaps this is just the way with git, and I'm trapped in the mindset of
> long-running branches that run parallel to major revision development and
> aren't targeted at a specific point release. In looking at this I'm
> reminded that the Accumulo community has chosen the latter path where
> branches are short-lived and targeted at the next release.
>
>
>> I'm not sure if that means we should archive the 1.4.x
>> versions in JIRA, so people can mark those versions as affected or
>> not. Maybe it'd just be useful to just archive 1.4.0-1.4.3, and leave
>> 1.4.4/1.4.5 unarchived. (I suggest the last two versions of 1.4, only
>> because the last version introduced a lot of changes that people may
>> be reluctant to update to, if they aren't transitioning to hadoop 2).
>>
>
> I see JIRA being useful as both a work tracking/planning tool >and< a user
> support tool / record of project history (like commit history). Would
> archiving releases prevent historic issues from being findable via google?
>
> --
>> Christopher L Tubbs II
>> http://gravatar.com/ctubbsii
>>
>>
>> On Tue, May 6, 2014 at 8:34 AM, Drew Farris  wrote:
>> > Thanks for the response Joey.
>> >
>> > It sounds as if there's agreement on a number of points and it sounds
>> like
>> > I'm the only person not in favor of deleting the branch and creating a
>> tag
>> > a this point. Also, bug management is an interesting issue. Thoughts
>> > in-line below:
>> >
>> > On Mon, May 5, 2014 at 4:07 PM, Joey Echeverria <
>> joey...@clouderagovt.com>wrote:
>> >
>> >>
>> >> There is also the impact on ticket workflow. When a version is EOLed,
>> >> I'd not expect the community to provide any additional fixes for that
>> >> release line. If 1.4 hangs around, then it creates confusion over what
>> >> will happen to tickets filed against it. It also will confuse users as
>> >> they may keep filing 1.4 tickets.
>> >>
>> >
>> > If people find ticket-worthy issues in 1.4 after it's end-of-lifed
>> wouldn't
>> > we expect them to file a ticket against that version? Shouldn't these
>> > tickets reflect known issues with a release of software that people use?
>> > Regardless of the desire of the development community to produce new
>> > releases of a specific branch, it is a service to the community of users
>> to
>> > be able to record known issues (even if these will ultimately result in a
>> > wontfix resolution). Google does a very good job indexing the Apache
>> JIRA.
>> >
>> > Furthermore, issue reporting activity is a reflection of real-world use
>> > which should naturally migrate to future versions, and if people aren't
>> > migrating to future versions, we have bigger fish to fry.
>> >
>> >  > To something else, perhaps:
>> >> >
>> >> > Current Stable Release: 1.5.1
>> >> > Legacy Bugfix Release: 1.4.5
>> >>
>> >> We used to have something like this, but that lead to some arguments
>> >> over which is stable and which legacy. For example, 1.6.0 is out now
>> >> so that means that there would be three releases we need to identify.
>> >>
>> >
>> > Ok, so, we list three releases instead of two. Two of them happen to be
>>

Re: [VOTE] end of life plan for 1.4 branch

[Joey Echeverria]

On Mon, May 5, 2014 at 3:40 PM, Drew Farris  wrote:
> I suppose this will be a problem we will encounter with every major/minor
> revision as they age, so we have a good chance to hash out a general policy
> for this.
>
> I see two categories of actions proposed here:
>
> 1) Content Changes (e.g: website)
> 2) SCM Changes (e.g: git repo)

There is also the impact on ticket workflow. When a version is EOLed,
I'd not expect the community to provide any additional fixes for that
release line. If 1.4 hangs around, then it creates confusion over what
will happen to tickets filed against it. It also will confuse users as
they may keep filing 1.4 tickets.

> As far as 1 is concerned, I have the sense that there's general consensus
> that we should de-emphasize the 1.4 releases in favor of 1.5 at this point.
> We could go far in doing this by changing the wording on the website:

Agreed.

> From:
>
> Latest 1.5 release: 1.5.1
> Latest 1.4 release: 1.4.5
>
> To something else, perhaps:
>
> Current Stable Release: 1.5.1
> Legacy Bugfix Release: 1.4.5

We used to have something like this, but that lead to some arguments
over which is stable and which legacy. For example, 1.6.0 is out now
so that means that there would be three releases we need to identify.

> I think we also agree that removing Maven artifacts is out of bounds
> because that will cause breakage for all sorts of things. Mirrors will
> likely want to drop release tarballs at some point (e.g for old point
> release versions like 1.4.3, 1.4.4) but I'm not sure how they are signaled
> to do so. I'm generally in favor of keeping the documentation for old
> versions around. (E.g: you can still find docs for Lucene 3.0.3 at Apache
> and it's ancient!)

Absolutely. We should never delete released artifacts.

> I don't think it makes sense to tag a 1.4.6 release until someone is
> prepared to follow the release process and produce votable artifacts. At
> this point I'm hearing that a) work isn't done and b) there isn't
> sufficient reason to create 1.4.6. I don't think that there is any onus on
> the Accumulo community to ever produce a 1.4.6 release, but we should not
> do anything that will prevent that from happening or make it difficult to
> do so. There are still folks out there that are using 1.4 actively and who
> knows what darkness lurks at the heart of Accumulo that might need to be
> fixed.
>
> Could someone explain why we would want to ever delete the 1.4.x branch?

I think you want to delete the branch because of our Git workflow[1]
which is to always target a patch for the earliest, non-end-of-lifed
version. You could argue that the documentation and mailing list
announcement are sufficient to declare the branch EOLed, but I don't
think that's strong enough for a casual contributor.

The purpose of the EOL is to say that the community *won't* create a
new 1.4 release. If you need to stick with it for whatever reason,
it's expected that you and/or your support vendor will backport
required patches and cut your own releases. Nothing here will prevent
that. In fact, creating the tag for the work that was in progress on
the 1.4 line post the 1.4.5 release should make that easier.

> So, in summary:
>
> 1) I agree it makes sense to clearly market 1.5 as the current stable
> release and market 1.4 as something old that you only want to start using
> if you're already using.
> 2) I can't see any good reason to do anything with tags or branches at this
> point.
>
> It is not clear to me why we would want to do anything in SCM to eol 1.4,
> as long as we cover the website. I'm interested if there are specific
> mechanical reasons.

-Joey

[1] http://accumulo.apache.org/git.html

Re: [VOTE] end of life plan for 1.4 branch

[Joey Echeverria]

On Mon, May 5, 2014 at 3:38 PM, Christopher  wrote:
> I elaborated above, but in short, all previous tags have indicated
> releases. This is standard to publish tags in SCM to denote a release.
> it's confusing to have a tag that does not denote a release. Further,
> having a version that is greater than the greatest approved release
> may mislead people who build from source to use the "latest", thinking
> it was approved and it wasn't.

I agree that 1.4.6-eol is not a good name since it implies a 1.4.6
release that never happened.

I don't agree with the fact that tags are only used in SCM to denote a
release. I think that's the most common usage, but I've always viewed
a tag as any significant point on a branch that signifies not to
expect further changes. The places where those exist in most projects
is for releases and branches that have been EOLed.

-Joey

Re: [VOTE] end of life plan for 1.4 branch

[Joey Echeverria]

The tag tells users where development on the branch ended. Can you elaborate on 
what is confusing about that?--
Joey Echeverria
Chief Architect
Cloudera Government Solutions

On Mon, May 5, 2014 at 3:01 PM, Keith Turner  wrote:

> -1
> I am opposed to the tag, because I think what it communicates to users is
> confusing.  I'm in favor of what Christopher suggested.
> I was undecided about the general concept of 1.4 EOL,  I am still working
> w/ some users who are still using 1.4 in the short term.   Should they run
> into a serious bug, we will very likely fix it.  I discussed this situation
> w/ Christopher and he suggested if this situation were to occur we could
> simply post a patch jira.   That plan sounds good w/ me and makes me
> comfortable w/ 1.4 EOL now.
> On Sat, May 3, 2014 at 4:41 PM, Sean Busbey  wrote:
>> Accumulo Folks,
>>
>> I would like to declare end of life for the 1.4 branch of development. It's
>> been active for a little over two years and the release of 1.6.0 means we
>> now have three active releases.
>>
>> Declaring end of life would mean
>>
>> * Posting an ANNOUNCE message to the user list
>> * No longer accepting issue fix version targets for future 1.4 releases
>> * Removing fix version targets of 1.4.x for existing open issues
>> * The end of having a long lived 1.4 related branch in git
>> * Removing direct references to 1.4.x releases on our download page
>> * No longer linking to the 1.4 related documentation from the main
>> navigation area
>>
>> Our issue tracker shows that candidate version 1.4.6 currently has:
>>
>> * 9 closed issues, none of which are blockers or critical.
>> * 1 issue in patch available status, marked critical
>> * 18 open issues with a target fix version of 1.4.6, four of which are
>> marked critical.
>>
>> Because there is existing work, but not yet enough to warrant a release, I
>> propose
>> that on successful passing of this vote we create a "1.4.6-eol" tag with
>> the then
>> current state of the development branch.
>>
>> Please vote
>>
>> [ ] +1 I am in favor of announcing End of Life according to the above plan
>> [ ] +-0 I am indifferent
>> [ ] -1 I am opposed to the above End of Life plan because...
>>
>> I'm treating this like a release vote. Thus, it will be handled with
>> Majority Approval:
>> to pass it will need 3 committers to +1 and more committers voting +1 than
>> -1.
>>
>> Vote will remain open for 72 hours, until Tuesday, May 6 2014, 20:40 UTC
>>
>> --
>> Sean
>>

Re: [VOTE] end of life plan for 1.4 branch

[Joey Echeverria]

I like 1.4-eol or maybe 1.4-closed. --
Joey Echeverria
Chief Architect
Cloudera Government Solutions

On Mon, May 5, 2014 at 11:59 AM, Keith Turner  wrote:

> I also think the -eol tag is confusing.
> Have to be careful w/ deleting the 1.4.6-SNAPSHOT branch and storing the
> commit hash externally to git.  If nothing refs (no other commit, tag,
> branch, etc) that commit hash, then it could be gc'ed by git.
> On Mon, May 5, 2014 at 11:42 AM, Christopher  wrote:
>> If the intent is to treat the tagging as a separate action from this
>> plan, then my vote changes to +1 for this plan.
>>
>> I have no objection to just dropping the branch (and mentioning the
>> HEAD commit in the mailing list, in case the branch needs to be
>> resurrected for some reason). My -1 comes from the "-eol" tag, not the
>> rest of the plan. I don't see value in creating that tag, and worry
>> about its potential for confusion.
>>
>> --
>> Christopher L Tubbs II
>> http://gravatar.com/ctubbsii
>>
>>
>> On Sun, May 4, 2014 at 4:04 PM, Sean Busbey  wrote:
>> > Hi Christopher!
>> >
>> > Responses inline
>> >
>> >
>> > On Sun, May 4, 2014 at 12:50 AM, Christopher 
>> wrote:
>> >
>> >> -1
>> >>
>> >> Summary:
>> >>
>> >> Overall, in favor, but...
>> >> 1. Confusing tag name
>> >> 2. Alt. Option 1: just drop the active dev branch, no tag
>> >> 3. Alt. Option 2: just closeout 1.4 with a quiet administrative 1.4.6
>> >> source release
>> >> 4. Voting under "release" rules is invalid without signed release
>> artifacts
>> >>
>> >> Exposition:
>> >>
>> >> Overall, I'm in favor of EOL'ing 1.4.x, but I'm not sure what an
>> >> "1.4.6-eol" tag in SCM would mean to users. The "-eol" suffix couldn't
>> >> really be documented anywhere for users to understand how that would
>> >> differ from an actual release/tagged version, for users browsing the
>> >> SCM tags. I understand a tag is not a release, but to a user, that may
>> >> not be clear. It's also very confusing, because it does look like an
>> >> updated release... it has a 1-up version number over the last release
>> >> (1.4.5), after all. That's very confusing.
>> >>
>> >> To alleviate the confusion, it may be better to call it "1.4-eol" or
>> >> something else to indicate that it's not a newer release than 1.4.5
>> >> (it's not a release at all).
>> >>
>> >> An alternative option to the 1.4.6-eol tag: just drop the
>> >> development/planning branch. (This is the option that was exercised
>> >> when this decision was made for 1.3.x). All the relevant code is
>> >> merged to newer branches anyway, and the outstanding work planned for
>> >> a future 1.4.6 which will never come to pass is not useful to tag
>> >> distinctly. Besides, the HEAD commit of 1.4.6-SNAPSHOT will be around
>> >> indefinitely, as it's merged to master, so we could achieve a similar
>> >> purpose by simply noting its current HEAD commit
>> >> [5bd4465c433860624091b0d97892e02f58154e7a] in a message to the mailing
>> >> lists, for archival purposes.
>> >>
>> >> Another option: do an actual release vote on a signed 1.4.6 source
>> >> artifact. It wouldn't be hard to pass, since 1.4.5 passed, and the
>> >> current state of the branch isn't substantively different. We could
>> >> just call this an administrative release... no need for release
>> >> announcements and such, but it would clear up the name confusion.
>> >> 1.4.6 would be an actual thing at that point, voted on and approved
>> >> for release.
>> >>
>> >>
>> > I would really like to avoid doing a 1.4.6 release unless someone both
>> > feels strongly about the need and is also willing to shepherd through the
>> > testing process. The issues closed against it to date don't add
>> > substantively to the 1.4.5 release enough to justify the time investment
>> in
>> > testing, IMHO.
>> >
>> > I would be fine with either dropping the tag entirely or using something
>> > like 1.4-eol. I am inclined to have a tag we can refer to in any
>> > announcements, because they are easier to deal with for casual
>> developers.
>> >
>> &g

Re: [VOTE] Accumulo 1.6.0-RC5

[Joey Echeverria]

One quick addendum. I had to run a mvn clean in-between my runs of mvn
package. If I didn't, I got test errors:

https://gist.github.com/joey/4eaca3bc517af0a8b484

-Joey

On Fri, May 2, 2014 at 5:10 PM, Joey Echeverria
 wrote:
> +1 (non-binding)
>
> * Verified hashes.
> * Verified signatures.
> * Ran through unit tests with both Hadoop profiles.
>
> Testing done on CentOS 6.5 with JDK 1.7u55 and maven 3.2.1.
>
> On Fri, May 2, 2014 at 2:19 PM, Sean Mackrory  wrote:
>> +1 (non-binding)
>>
>> Verified hashes. I've also been working on Apache Bigtop-style packaging
>> for Accumulo since 1.4.4, and now that Hadoop 2 is supported this will
>> actually work in the Apache Bigtop stack. I've posted a patch to add
>> Accumulo to Bigtop in BIGTOP-1175. I've built those packages from the 1.6.0
>> RC-5 source and run them on multiple Linux distros against multiple
>> versions of Hadoop and ZooKeeper in psuedo-distributed mode. I was able to
>> perform basic CRUD operations like table creation, inserting rows, deleting
>> cells, etc..
>>
>>
>> On Thu, May 1, 2014 at 1:27 PM, Keith Turner  wrote:
>>
>>> +1
>>>
>>>  * Verified sigs and hashes
>>>  * Was able to build Accismus against Staging repo (had to add
>>> pluginRepositories section to my maven config in addition to repositories
>>> section)
>>>  * Was able to build native maps from bin.tgz
>>>  * Ran 24hr CI w/ agitation on 20 node EC2 cluster... 8 billion written &
>>> verified.. found ACCUMULO-2768..
>>>
>>> Used hadoop 2.2.0 (single non-HA NN) and ZK 3.4.5 (3 zookeepers) for test.
>>>
>>> Used 3G native config w/ following changes for CI test.
>>>
>>> tserver.mutation.queue.max=4M
>>> general.rpc.timeout=180s
>>>
>>>
>>>
>>>
>>> On Tue, Apr 29, 2014 at 5:33 PM, Christopher  wrote:
>>>
>>> > Accumulo Developers,
>>> >
>>> > Please consider the following candidate for Accumulo 1.6.0.
>>> >
>>> > Git Commit: 06162580e885f11863d1a6d22f952bce35b78b68
>>> > Branch: 1.6.0-RC5
>>> >
>>> > Staging repo:
>>> >
>>> https://repository.apache.org/content/repositories/orgapacheaccumulo-1011
>>> > Source:
>>> >
>>> https://repository.apache.org/content/repositories/orgapacheaccumulo-1011/org/apache/accumulo/accumulo/1.6.0/accumulo-1.6.0-src.tar.gz
>>> > Binary:
>>> >
>>> https://repository.apache.org/content/repositories/orgapacheaccumulo-1011/org/apache/accumulo/accumulo/1.6.0/accumulo-1.6.0-bin.tar.gz
>>> > (Append ".sha1", ".md5" or ".asc" to download the signature/hash for a
>>> > given artifact.)
>>> >
>>> > All artifacts were built and staged with:
>>> > mvn release:prepare && mvn release:perform
>>> >
>>> > Signing keys available at: https://www.apache.org/dist/accumulo/KEYS
>>> >
>>> > Release notes (in progress):
>>> > http://accumulo.apache.org/release_notes/1.6.0
>>> >
>>> > Changes since RC4 (`git log 448e757..origin/1.6.0-RC5`):
>>> >
>>> > https://issues.apache.org/jira/browse/ACCUMULO-2680
>>> > https://issues.apache.org/jira/browse/ACCUMULO-2740
>>> > https://issues.apache.org/jira/browse/ACCUMULO-2742
>>> > https://issues.apache.org/jira/browse/ACCUMULO-2745
>>> > https://issues.apache.org/jira/browse/ACCUMULO-2748
>>> > https://issues.apache.org/jira/browse/ACCUMULO-2749
>>> > https://issues.apache.org/jira/browse/ACCUMULO-2750
>>> > https://issues.apache.org/jira/browse/ACCUMULO-2751
>>> > https://issues.apache.org/jira/browse/ACCUMULO-2752
>>> > https://issues.apache.org/jira/browse/ACCUMULO-2756
>>> >
>>> > This vote will remain open for 72 hours (3 days), until Fri, 2014 May
>>> > 2 21:30 UTC.
>>> > (That's 5:30pm EDT on Friday.)
>>> >
>>> > [ ] +1 - I have verified and accept...
>>> > [ ] +/-0 - I have reservations, but not strong enough to vote against...
>>> > [ ] -1 - Because..., I do not accept...
>>> > ... these artifacts as the 1.6.0 release of Apache Accumulo.
>>> >
>>> > Thanks.
>>> >
>>> > P.S. Hint: download the whole staging repo with
>>> > wget -erobots=off -r -l inf -np -nH
>>> >
>>> https://repository.apache.org/content/repositories/orgapacheaccumulo-1011/
>>> > # note the trailing slash is needed
>>> >
>>> > --
>>> > Christopher L Tubbs II
>>> > http://gravatar.com/ctubbsii
>>> >
>>>

Re: [VOTE] Accumulo 1.6.0-RC5

[Joey Echeverria]

+1 (non-binding)

* Verified hashes.
* Verified signatures.
* Ran through unit tests with both Hadoop profiles.

Testing done on CentOS 6.5 with JDK 1.7u55 and maven 3.2.1.

On Fri, May 2, 2014 at 2:19 PM, Sean Mackrory  wrote:
> +1 (non-binding)
>
> Verified hashes. I've also been working on Apache Bigtop-style packaging
> for Accumulo since 1.4.4, and now that Hadoop 2 is supported this will
> actually work in the Apache Bigtop stack. I've posted a patch to add
> Accumulo to Bigtop in BIGTOP-1175. I've built those packages from the 1.6.0
> RC-5 source and run them on multiple Linux distros against multiple
> versions of Hadoop and ZooKeeper in psuedo-distributed mode. I was able to
> perform basic CRUD operations like table creation, inserting rows, deleting
> cells, etc..
>
>
> On Thu, May 1, 2014 at 1:27 PM, Keith Turner  wrote:
>
>> +1
>>
>>  * Verified sigs and hashes
>>  * Was able to build Accismus against Staging repo (had to add
>> pluginRepositories section to my maven config in addition to repositories
>> section)
>>  * Was able to build native maps from bin.tgz
>>  * Ran 24hr CI w/ agitation on 20 node EC2 cluster... 8 billion written &
>> verified.. found ACCUMULO-2768..
>>
>> Used hadoop 2.2.0 (single non-HA NN) and ZK 3.4.5 (3 zookeepers) for test.
>>
>> Used 3G native config w/ following changes for CI test.
>>
>> tserver.mutation.queue.max=4M
>> general.rpc.timeout=180s
>>
>>
>>
>>
>> On Tue, Apr 29, 2014 at 5:33 PM, Christopher  wrote:
>>
>> > Accumulo Developers,
>> >
>> > Please consider the following candidate for Accumulo 1.6.0.
>> >
>> > Git Commit: 06162580e885f11863d1a6d22f952bce35b78b68
>> > Branch: 1.6.0-RC5
>> >
>> > Staging repo:
>> >
>> https://repository.apache.org/content/repositories/orgapacheaccumulo-1011
>> > Source:
>> >
>> https://repository.apache.org/content/repositories/orgapacheaccumulo-1011/org/apache/accumulo/accumulo/1.6.0/accumulo-1.6.0-src.tar.gz
>> > Binary:
>> >
>> https://repository.apache.org/content/repositories/orgapacheaccumulo-1011/org/apache/accumulo/accumulo/1.6.0/accumulo-1.6.0-bin.tar.gz
>> > (Append ".sha1", ".md5" or ".asc" to download the signature/hash for a
>> > given artifact.)
>> >
>> > All artifacts were built and staged with:
>> > mvn release:prepare && mvn release:perform
>> >
>> > Signing keys available at: https://www.apache.org/dist/accumulo/KEYS
>> >
>> > Release notes (in progress):
>> > http://accumulo.apache.org/release_notes/1.6.0
>> >
>> > Changes since RC4 (`git log 448e757..origin/1.6.0-RC5`):
>> >
>> > https://issues.apache.org/jira/browse/ACCUMULO-2680
>> > https://issues.apache.org/jira/browse/ACCUMULO-2740
>> > https://issues.apache.org/jira/browse/ACCUMULO-2742
>> > https://issues.apache.org/jira/browse/ACCUMULO-2745
>> > https://issues.apache.org/jira/browse/ACCUMULO-2748
>> > https://issues.apache.org/jira/browse/ACCUMULO-2749
>> > https://issues.apache.org/jira/browse/ACCUMULO-2750
>> > https://issues.apache.org/jira/browse/ACCUMULO-2751
>> > https://issues.apache.org/jira/browse/ACCUMULO-2752
>> > https://issues.apache.org/jira/browse/ACCUMULO-2756
>> >
>> > This vote will remain open for 72 hours (3 days), until Fri, 2014 May
>> > 2 21:30 UTC.
>> > (That's 5:30pm EDT on Friday.)
>> >
>> > [ ] +1 - I have verified and accept...
>> > [ ] +/-0 - I have reservations, but not strong enough to vote against...
>> > [ ] -1 - Because..., I do not accept...
>> > ... these artifacts as the 1.6.0 release of Apache Accumulo.
>> >
>> > Thanks.
>> >
>> > P.S. Hint: download the whole staging repo with
>> > wget -erobots=off -r -l inf -np -nH
>> >
>> https://repository.apache.org/content/repositories/orgapacheaccumulo-1011/
>> > # note the trailing slash is needed
>> >
>> > --
>> > Christopher L Tubbs II
>> > http://gravatar.com/ctubbsii
>> >
>>

Re: [DISCUSS] Accumulo wiki

[Joey Echeverria]

I don't think it's a good idea to include wiki content in a release.

What I've seen other projects use their wiki space for effectively is:

* Contributor guides
* Tutorials
* Design documents/proposals

I think the reason that some wikis experience content rot is they
never build a big enough user base and they're outside of the release
process so there's no forcing function to review that content is
up-to-date.

For communities that are still relatively small, you can tag specific
pages as things that should/need to be periodically reviewed and could
assign JIRAs or some other notification mechanism to kick off a
review.

-Joey

--
Joey Echeverria
Chief Architect
Cloudera Government Solutions


On Wed, Apr 23, 2014 at 11:19 AM, Bill Havanki
 wrote:
> Quick note that if any of the wiki content is intended to be bundled in a
> release, then editing must be restricted to those who've signed a CLA [1].
> The same restriction is surely in place for MoinMoin, but I don't see a ref.
>
> [1]
> https://cwiki.apache.org/confluence/display/INFRA/Cwiki#Cwiki-Butwhatifwewouldlikethecommunityatlargetohelpmaintainthespace
> ?
>
>
> On Wed, Apr 23, 2014 at 11:11 AM, Sean Busbey  wrote:
>
>> I want a place to put contributor self-help materials. The bar for granting
>> someone edit access to a cwiki space is way lower than edit rights on our
>> main site.
>>
>>
>> On Wed, Apr 23, 2014 at 10:00 AM, Billie Rinaldi
>> wrote:
>>
>> > We originally evaluated whether we would have a wiki and decided against.
>> > I don't mind reevaluating now, but it should not be a foregone conclusion
>> > that we should have one.  Often when projects have a website and wiki,
>> it's
>> > confusing where to look for which information.  Also, project
>> documentation
>> > that is on the wiki is often out of date.  If we want to consider setting
>> > up a wiki because it would be better than our website for housing some
>> > specific types of information, I'd like those types to be identified
>> before
>> > we start.
>> >
>> > Billie
>> >
>> >
>> > On Wed, Apr 23, 2014 at 7:47 AM, Bill Havanki > > >wrote:
>> >
>> > > There has been talk about establishing some wiki space for the Accumulo
>> > > project. ASF offers two wiki platforms: MoinMoin at wiki.apache.organd
>> > > Confluence at cwiki.apache.org. Each has pros and cons, and if we
>> > > establish
>> > > a wiki we should just pick one.
>> > >
>> > > I believe the MoinMoin site is more prevalent ASF-wide, and there is
>> > > already a Hadoop wiki [1] established which we would go under. We can
>> > > request members of its AdminGroup [not viewable] to add us to the
>> > > ContributorsGroup [2], and then we would be able to author content. (We
>> > can
>> > > ask infra as well.)
>> > >
>> > > The Confluence site offers some extra features like commenting and
>> > perhaps
>> > > looks nicer (certainly subjective). There is a page [3] touching on
>> some
>> > > differences. To get started there, we would request a "space" from
>> infra.
>> > >
>> > > Related projects on MoinMoin: Hadoop, HBase
>> > > Related projects on Confluence: Hive, Pig, ZooKeeper (all having
>> migrated
>> > > from MoinMoin)
>> > >
>> > > So, questions to be answered:
>> > >
>> > > 1) Shall we establish a wiki? (I can call a vote if needed.)
>> > > 2) Which platform?
>> > > 3) Who is interested in edit rights?
>> > >
>> > > [1] https://wiki.apache.org/hadoop/
>> > > [2] https://wiki.apache.org/hadoop/ContributorsGroup
>> > > [3] https://cwiki.apache.org/confluence/display/INFRA/Cwiki
>> > >
>> > > Thanks
>> > > Bill H
>> > >
>> > > --
>> > > // Bill Havanki
>> > > // Solutions Architect, Cloudera Govt Solutions
>> > > // 443.686.9283
>> > >
>> >
>>
>>
>>
>> --
>> Sean
>>
>
>
>
> --
> // Bill Havanki
> // Solutions Architect, Cloudera Govt Solutions
> // 443.686.9283

Re: Is v1.4.2 downloadable?

[Joey Echeverria]

There's a 1.4.2 tag on github:

https://github.com/apache/accumulo/tree/1.4.2

On Mon, Apr 14, 2014 at 2:43 PM, David Medinets
 wrote:
> http://accumulo.apache.org/downloads/ - I looked on this page and did not
> see v1.4.2. Nor do I see a v1.4.2 branch on this github site. Is it
> available somewhere else?

Re: [VOTE] Accumulo Blog

[Joey Echeverria]

+1 (non-binding)

--
Joey Echeverria
Chief Architect
Cloudera Government Solutions


On Mon, Apr 14, 2014 at 11:16 AM, Josh Elser  wrote:
> +1
>
>
> On 4/13/14, 8:11 PM, dlmar...@comcast.net wrote:
>>
>> I have reviewed the feedback from the proposal thread and consolidated it
>> into a set of guidelines for an Accumulo Blog. In accordance with the
>> bylaws
>> this vote will require Lazy Approval to pass and will remain open for 3
>> business days. I'll tally the votes on Thursday morning.
>>
>>
>>
>> 1.   The blog will be hosted on the Apache Blogs site[1].
>>
>> 2.   The blog will be set up using the instructions at [2] to enable
>> public preview.
>>
>> 3.   Proposed blog content will be posted in full-text or link form to
>> the dev mailing list.
>>
>> 4.   Blog content requires Lazy Approval votes that are open for at
>> least 3 days.
>>
>> 5.   Content may be cross-posted from other sites provided that the
>> content is more than just a link to the other site. The full text of the
>> original article is preferred.
>>
>> 6.   Content may be cross-posted to other sites provided that there is
>> a
>> link back to the Accumulo blog site.
>>
>>
>>
>> [1] http://blogs.apache.org/
>>
>> [2] http://www.apache.org/dev/project-blogs
>>
>>
>>
>>
>>
>>
>

Re: [DISCUSS] Remove hadoop1 support

[Joey Echeverria]

+1 to deprecating hadoop1 support in 1.6.0
+1 to dropping hadoop1 support in 1.7.0 (or similar release)

--
Joey Echeverria
Chief Architect
Cloudera Government Solutions


On Fri, Apr 4, 2014 at 11:28 AM, Josh Elser  wrote:
> +1 ditto on what's been said
>
> Not sure on how "aggressive" it would be to call hadoop1 deprecated for
> 1.6.0, but I wouldn't mind seeing that happen. Can certainly poll the user
> list for input. We've already "encouraged" hadoop2 by switching the build
> profiles for 1.6, perhaps it's not unreasonable to call it deprecated too.
>
>
> On 4/4/14, 2:20 PM, Ravi Mutyala wrote:
>>
>> +1 for dropping support for hadoop1 in future versions. +1 to call it
>> deprecated for Hadoop 1. It might be to good option to get info from user
>> list if anyone is going to use 1.6 on Hadoop1.
>>
>> Thanks
>>
>>
>> On Fri, Apr 4, 2014 at 1:07 PM, Mike Drob  wrote:
>>
>>> https://issues.apache.org/jira/browse/HBASE-10690
>>>
>>> Deprecated in 0.98, dropped in 1.0
>>>
>>>
>>> On Fri, Apr 4, 2014 at 11:02 AM, Sean Busbey  wrote:
>>>
>>>> Mike, as of which version is HBase dropping?
>>>>
>>>> I wouldn't want to do this in 1.6.x. Presumably that would mean if we
>>>
>>> drop
>>>>
>>>> it in the next version, we'd still be supporting Hadoop 1 until both 1.5
>>>> and 1.6 have been EOLd?
>>>>
>>>> -Sean
>>>>
>>>>
>>>> On Fri, Apr 4, 2014 at 12:56 PM, Mike Drob  wrote:
>>>>
>>>>> HBase has started to drop hadoop 1 support already.
>>>>>
>>>>>
>>>>> On Fri, Apr 4, 2014 at 10:54 AM, John Vines  wrote:
>>>>>
>>>>>> This is something that crossed my mind recently. Hadoop 2 is solidly
>>>>>
>>>>> moving
>>>>>>
>>>>>> forward and, as someone who does not actively follow the hadoop
>>>>>
>>>>> community,
>>>>>>
>>>>>> hadoop 1 is slowing down. Adoption for hadoop2 is on the rise and
>>>
>>> with
>>>>>
>>>>> that
>>>>>>
>>>>>> I'm starting to wonder whether it's worth the code complexity to
>>>>
>>>> support
>>>>>>
>>>>>> both versions, particularly attempting to harness new features.
>>>>>>
>>>>>> What is the communities thought on this, maybe for 1.7 or maybe for
>>>>>
>>>>> future
>>>>>>
>>>>>> releases?
>>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Sean
>>>>
>>>
>>
>

Re: [DISCUSS] Accumulo Bylaws

[Joey Echeverria]

"Emeritus" is not an official ASF designation. As far as the ASF is
concerned, you're either a Committer, a PMC member, or both, or not at all.

The reason other projects use the emeritus designation is to avoid
overstating active involvement. An "emeritus" member does not lose any
privileges as far as ASF is concerned. If you want to remove privileges, I
believe that the PMC has to vote to that effect.

-Joey


On Tue, Feb 18, 2014 at 11:06 AM, Sean Busbey wrote:

> If people have substantive questions (as opposed to requests for edits /
> clarification), I'd rather they be here on the list.
>
> My main issue is the automatic transition to emeritus status for committers
> / PMCs at 6 months. That's a significant change. Do we know what the
> current impact of that would be?
>
>
> On Tue, Feb 18, 2014 at 9:04 AM, Bill Havanki  >wrote:
>
> > I have some minor edits and some questions about it, which I'll add as
> > comments in the doc. I also agree that a weather allowance is a good
> idea.
> >
> >
> > On Tue, Feb 18, 2014 at 9:49 AM, Mike Drob  wrote:
> >
> > > Thanks for putting it in a Google Doc, Arshak!
> > >
> > > What issues do y'all see with this document in it's current state?
> > > Personally, I think it looks fine and would be willing to start a vote
> on
> > > it, but I get the impression that east coast weather has prevented some
> > > folk from looking at it, so maybe another couple of days is fine.
> > >
> > > Mike
> > >
> > >
> > > On Sun, Feb 16, 2014 at 7:18 AM, Arshak Navruzyan 
> > > wrote:
> > >
> > > > Oops, yes of course!  It's editable.
> > > >
> > > >
> > > >
> > > >
> > > > On Sat, Feb 15, 2014 at 7:01 PM, Bill Havanki <
> > bhava...@clouderagovt.com
> > > > >wrote:
> > > >
> > > > > Thanks Arshak! Can you either allow editing or commenting?
> > > > >
> > > > >
> > > > > On Fri, Feb 14, 2014 at 6:10 PM, Arshak Navruzyan <
> arsh...@gmail.com
> > >
> > > > > wrote:
> > > > >
> > > > > > Say no more ...
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://docs.google.com/document/d/1uR8vhIQcKGA6IEtbbF5D7UL_e6WGtfXMUQHp8Fwvg_E/edit?usp=sharing
> > > > > >
> > > > > >
> > > > > > On Fri, Feb 14, 2014 at 1:54 PM, Christopher <
> ctubb...@apache.org>
> > > > > wrote:
> > > > > >
> > > > > > > Perhaps some ambitious volunteer could start a collaborative
> > draft
> > > of
> > > > > > > Accumulo's bylaws in Google Docs or something, using ZK as a
> > > starting
> > > > > > > point. After it stabilizes a bit, we could push it to the
> project
> > > > > > > webpage as a draft and vote on it?
> > > > > > >
> > > > > > > --
> > > > > > > Christopher L Tubbs II
> > > > > > > http://gravatar.com/ctubbsii
> > > > > > >
> > > > > > >
> > > > > > > On Fri, Feb 14, 2014 at 2:11 PM, Mike Drob <
> mad...@cloudera.com>
> > > > > wrote:
> > > > > > > > I didn't get that impression from reading their document.
> > While C
> > > > and
> > > > > > PMC
> > > > > > > > are two distinct roles, there is nothing stating that there
> > > cannot
> > > > be
> > > > > > > > overlap, and the fact that there is 100% overlap is entirely
> > > > > > orthogonal.
> > > > > > > >
> > > > > > > >
> > > > > > > > On Fri, Feb 14, 2014 at 10:23 AM, Josh Elser <
> > > josh.el...@gmail.com
> > > > >
> > > > > > > wrote:
> > > > > > > >
> > > > > > > >> This would change the existing Committer == PMC, no?
> > > > > > > >>
> > > > > > > >> That's the biggest thing I noticed scanning over the
> document.
> > > > > > > >>
> > > > > > > >>
> > > > > > > >> On 2/14/14, 1:19 PM, Mike Drob wrote:
> > > > > > > >>
> > > > > > > >>> I think we should have some Bylaws, as that gives us more
> > > > structure
> > > > > > to
> > > > > > > >>> operate under.
> > > > > > > >>>
> > > > > > > >>> I propose that we adopt the ZooKeeper bylaws, replacing all
> > > > > > references
> > > > > > > to
> > > > > > > >>> ZK with Accumulo.
> > > > > > > >>>
> > > > > > > >>> http://zookeeper.apache.org/bylaws.html
> > > > > > > >>>
> > > > > > > >>> What say ye?
> > > > > > > >>>
> > > > > > > >>>
> > > > > > > >>> Mike
> > > > > > > >>>
> > > > > > > >>>
> > > > > > >
> > > > > >
> > > > >
> > > > >
> >
>

Re: Test set: org.apache.accumulo.examples.simple.filedata.ChunkInputFormatTest failed in accumulo-1.4.3

[Joey Echeverria]

I think he's talking about Cloudera's packaging of 1.4.3 which has hadoop2 
support back ported. 


Chris,




I'll respond to your errors on the Cloudera forum. 




-Joey
—
Joey Echeverria
Director, Federal FTS
Cloudera, Inc.

On Tue, Feb 11, 2014 at 5:01 PM, John Vines  wrote:

> 1.4.3 is hadoop0.20/hadoop1 only, while CDH4 is based off of hadoop2.
> On Tue, Feb 11, 2014 at 4:49 PM, Chris Rigano wrote:
>> Greetings to all esteem colleague,
>>
>> Following indicate a build failure for accumulo-examples indicating  the
>> failure of Test set:
>> org.apache.accumulo.examples.simple.filedata.ChunkInputFormatTest.
>>
>> I would very much appreciate advice on actions to take in detail that would
>> resolve this issue.
>>
>> The environment is accumulo-1.4.3, CDH4.3 QuickStart using netbeans to
>> import the maven project  accumulo-examples after installing accumulo from
>> the accumulo-1.4.3cdh4.3.tar.gz.
>>
>> The VM is available upon request.
>>
>> Cloudera has not responded in there news groups.
>>
>> In advance thanks for your assistance.
>>
>> The results of executing the build with dependencies option are listed
>> below:
>>
>> cd /usr/lib/accumulo/src/examples/simple; JAVA_HOME=/usr/java/jdk1.6.0_32
>> /home/cloudera/netbeans-7.4/java/maven/bin/mvn install
>> Scanning for projects...
>>
>> Some problems were encountered while building the effective model for
>> org.apache.accumulo:examples-simple:jar:1.4.3-cdh4.3.0
>> 'build.plugins.plugin.version' for
>> org.apache.maven.plugins:maven-clean-plugin is missing. @
>> org.apache.accumulo:accumulo:1.4.3-cdh4.3.0, /usr/lib/accumulo/pom.xml,
>> line 81, column 15
>> 'build.plugins.plugin.version' for
>> org.apache.maven.plugins:maven-resources-plugin is missing. @
>> org.apache.accumulo:accumulo:1.4.3-cdh4.3.0, /usr/lib/accumulo/pom.xml,
>> line 152, column 15
>>
>> It is highly recommended to fix these problems because they threaten the
>> stability of your build.
>>
>> For this reason, future Maven versions might no longer support building
>> such malformed projects.
>>
>>
>> 
>> Building examples-simple 1.4.3-cdh4.3.0
>> 
>>
>> --- maven-enforcer-plugin:1.0:enforce (enforce-mvn) @ examples-simple ---
>>
>> --- maven-resources-plugin:2.5:resources (default-resources) @
>> examples-simple ---
>> [debug] execute contextualize
>> Using 'UTF-8' encoding to copy filtered resources.
>> skip non existing resourceDirectory
>> /usr/lib/accumulo/src/examples/simple/src/main/resources
>>
>> --- maven-dependency-plugin:2.1:copy-dependencies (copy-dependencies) @
>> examples-simple ---
>> log4j-1.2.16.jar already exists in destination.
>>
>> --- maven-compiler-plugin:2.3.2:compile (default-compile) @ examples-simple
>> ---
>> Compiling 40 source files to
>> /usr/lib/accumulo/src/examples/simple/target/classes
>>
>> --- maven-resources-plugin:2.5:testResources (default-testResources) @
>> examples-simple ---
>> [debug] execute contextualize
>> Using 'UTF-8' encoding to copy filtered resources.
>> skip non existing resourceDirectory
>> /usr/lib/accumulo/src/examples/simple/src/test/resources
>>
>> --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @
>> examples-simple ---
>> Compiling 5 source files to
>> /usr/lib/accumulo/src/examples/simple/target/test-classes
>>
>> --- maven-surefire-plugin:2.9:test (default-test) @ examples-simple ---
>> Surefire report directory:
>> /usr/lib/accumulo/src/examples/simple/target/surefire-reports
>>
>> ---
>>  T E S T S
>> ---
>> Running org.apache.accumulo.examples.simple.filedata.ChunkInputFormatTest
>> Tests run: 3, Failures: 0, Errors: 3, Skipped: 0, Time elapsed: 1.89 sec
>> <<< FAILURE!
>> Running org.apache.accumulo.examples.simple.filedata.ChunkCombinerTest
>> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.021 sec
>> Running org.apache.accumulo.examples.simple.filedata.ChunkInputStreamTest
>> Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.175 sec
>> Running org.apache.accumulo.examples.simple.filedata.KeyUtilTest
>> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.0

Re: Problems Running accumulo 1.4.3 hellowworld example CDH 4.3

[Joey Echeverria]

Netbeans can open a maven project natively, no special import required so it 
should work. —
Joey Echeverria
Director, Federal FTS
Cloudera, Inc.

On Thu, Feb 6, 2014 at 12:17 PM, Keith Turner  wrote:

> On Thu, Feb 6, 2014 at 11:15 AM, cprigano  wrote:
>> Hi Keith, this sounds good. Can I still use netbeans to develop accumulo
>> applications? Could I install on linux mint-15?
>>
> I use eclipse, so I  am not sure.  Eclipse has nice support for importing
> maven projects.   If netbeans has this, then use the command I sent to
> generate the maven project and then import it into netbeans.
> I would expect it work on any recent linux distro.  If does not we would
> like to hear about it.
>>
>>
>> On Thu, Feb 6, 2014 at 9:54 AM, Keith Turner [via Apache Accumulo] <
>> ml-node+s1065345n7422...@n5.nabble.com> wrote:
>>
>> > If you just want to quickly start writing code against Accumulo, one
>> > option
>> > may be to use the new accumulo maven archetype.  This will generate an
>> > accumulo project that builds and runs example code against
>> > MiniAccumuloCluster.   MiniAccumuloCluster is java code that spawns a
>> > local
>> > Accumulo instance for testing.  It should run nicely on a laptop.  Below
>> > is
>> > the command to generate a sample maven project that uses
>> > MiniAccumuloCluster.
>> >
>> > mvn -DarchetypeArtifactId=accumulo-instamo-archetype
>> > -DarchetypeVersion=1.4.4 -DarchetypeGroupId=org.apache.accumulo
>> >  -DinteractiveMode=false archetype:generate
>> >
>> >
>> > On Wed, Feb 5, 2014 at 11:18 PM, cprigano <[hidden email]<
>> http://user/SendEmail.jtp?type=node&node=7422&i=0>>
>> > wrote:
>> >
>> > > I tried CENTOS with CHD5 but it is sooo slow. I only want a single node
>> > to
>> > > start prototyping stuff in accumulo... suggestions
>> > >
>> > >
>> > > On Wed, Feb 5, 2014 at 10:15 PM, Chris Rigano <[hidden email]<
>> http://user/SendEmail.jtp?type=node&node=7422&i=1>
>> > > >wrote:
>> > >
>> > > > Hi Sean R u saying uses CDH4.3 with accumulo 1.5.0?
>> > > >
>> > > > thanks, Chris
>> > > >
>> > > >
>> > > > On Tue, Feb 4, 2014 at 6:31 PM, Sean Busbey-5 [via Apache Accumulo] <
>> > > > [hidden email] <http://user/SendEmail.jtp?type=node&node=7422&i=2>>
>> > wrote:
>> > > >
>> > > >> Hi Chris!
>> > > >>
>> > > >> You're running into a Hadoop 1 v Hadoop 2 problem.
>> > > >>
>> > > >> The Apache Accumulo 1.4.3 release only works on Hadoop 1 and CDH4
>> > uses
>> > > >> Hadoop 2.
>> > > >>
>> > > >> You can either use 1.5.0, wait for the 1.4.5 release, or use a
>> vendor
>> > > >> customized version of Accumulo.
>> > > >>
>> > > >> HTH
>> > > >> On Feb 4, 2014 7:26 PM, "cprigano" <[hidden email]<
>> > > http://user/SendEmail.jtp?type=node&node=7393&i=0>>
>> > > >> wrote:
>> > > >>
>> > > >> > I am trying to put together a accumulo/cloudera quick start as an
>> > > >> education
>> > > >> > package for students. Accumulo is running, but I am having
>> problems
>> > > >> > attempting to execute samples, namely hellowworld.
>> > > >> >
>> > > >> > It appears it is finding hadoop and not accumulo classes?
>> > > >> >
>> > > >> > Following is execution and error messages. I appreciate your
>> > > >> assistance!
>> > > >> >
>> > > >> > ./bin/accumulo
>> > > >> >
>> > org.apache.accumulo.examples.simple.helloworld.InsertWithOutputFormat
>> > > >> > "instance" localhost:2181 "username" "password" hellotable
>> > > >> >
>> > > >> > Thread
>> > > >> >
>> > > "org.apache.accumulo.examples.simple.helloworld.InsertWithOutputFormat"
>> > > >> > died
>> > > >> > nulljava.lang.reflect.InvocationTargetException
>> > > >> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
&g

ACCUMULO-1898

[Joey Echeverria]

From: Josh Elser
> Are you looking for the equivalent of "Security is OFF" for Accumulo as 
> exists on the NN UI?

I do think it'd be helpful for the monitor page to give a status as to
whether or not Kerberos is enabled for Accumulo.

-Joey

Re: [VOTE] Deprecate mock in 1.6.0

[Joey Echeverria]

One word of warning. Hadoop deprecated the original MR API before the
new API had feature parity and we're not stuck with both
implementations, probably for the lifetime of Hadoop's MR bindings.
I'm not in favor of deprecating something that people still have to
use. You should deprecate when you have something for people to
migrate to. I think MAC will fit some of the use cases, but it
certainly isn't a great fit for unit tests that can and should execute
quickly.

-Joey

On Mon, Nov 18, 2013 at 3:02 PM, Josh Elser  wrote:
> I'll put the question out there:
>
> Is it an immediate non-starter to deprecate something that doesn't have an
> immediate replacement?
>
> 1. You can still use it even if it's deprecated (and our usage of deprecated
> typically falls under "we won't remove it before version X if not later")
> 2. We know there are problems with it.
> 3. We know we should be making other tools that better replace it (MAC) or
> just give you a specific piece of functionality (iterator smoke-test
> framework).
>
> Is this just my interpretation of how to interpret @Deprecated? It seems
> completely logical to deprecate something we know isn't where we want to go
> even if we aren't to where we want to go. Then, we start focusing on
> making/improving the tools we want. This advertises to users that maybe they
> might not want to rely on MockAccumulo.
>
>
> On 11/18/13, 2:51 PM, Eric Newton wrote:
>>
>> I had this basic interaction with a user today:
>>
>> "I upgraded my old-style Filter to the Filter-as-iterator-Filter, how do I
>> test it?"
>>
>> And the easiest thing to tell them was "use MockAccumulo".  Without a
>> better answer, I'm not for deprecating Mock.
>>
>> I agree that there is considerable effort in trying to keep Mock
>> up-to-date, to the extent that I've not bothered to fix many of the
>> failings of Mock.
>>
>>
>>
>> On Mon, Nov 18, 2013 at 2:09 PM, Christopher  wrote:
>>
>>> Eric,
>>>
>>> Is there a reason these cannot be done in Mockito or EasyMock?
>>>
>>> --
>>> Christopher L Tubbs II
>>> http://gravatar.com/ctubbsii
>>>
>>>
>>> On Sun, Nov 17, 2013 at 7:22 PM, Eric Newton 
>>> wrote:

 -1

 I'm a little more invested in Mock since I wrote the first instance of
>>>
>>> it.

   I know it does not simulate the accumulo API perfectly.  And I know it
 adds some maintenance overhead for anyone adding new features to the
 API.

 However, adding additional testing requirements for a new API is
>>>
>>> something

 I like.

 Take a counter example: the "file://" hdfs implementation.  It allows
 you
 to use the local file system through the same API you would use for the
 distributed file system.

 Except, it doesn't. It does not behave the same as hdfs.  None of our
 recovery tests can use the local fs implementation because it just
>>>
>>> doesn't

 implement the proper flush semantics.

 Yet dozens of our own tests rely on the speedy availability of the local
>>>
>>> fs

 implementation.

 Having a fast way to test iterators that uses a test harness is not the
 same thing as testing the iterators using the same API they would use
 without Mock.  I have long called for an iterator test harness to stress
 the issues of iterator lifetimes.

 Finally, I would humbly suggest that our software has stabilized to the
 point where we tests at all levels:

 * iterator stress tester
 * Mock API
 * Integration test using MAC
 * System tests that can be run at full scale







 On Sat, Nov 16, 2013 at 1:12 PM, Corey Nolet  wrote:

> +1 for keeping a fast and easy (and well documented) mechanism for
> debugging iterators. Perhaps the SortedMapiterator is the solution..but
>>>
>>> the
>
> key words here are 'well documented'
>
> -1 for continuing support a half implemented mock framework that we
>>>
>>> have to
>
> maintain. It makes code maintenance very hard when you couldnt, for
> instance in the 1.3 series, even create a MockBatchDeleter. As Chris
> stated, I agree that using the mock in the past had users walking the
>>>
>>> line
>
> too closely between unit and integration tests. With the mock, I could
> write a bunch of fully valid tests against an iterator without the
>>>
>>> ability
>
> to verify that compactions didn't negatively affect my results. Except
>>>
>>> for
>
> being fast, the MAC mostly eliminates the need to use the mock for that
> kind of test at all while it makes the tests more valid to an actual
> runtime environment.
>
> +1 for mocking framework to be used in relevant unit tests. There are
>>>
>>> times
>
> when a quick and dirty mock is immensely useful and MAC is slow and way
> overkill for those tasks. Perhaps it would be worth a ticket to
>>>
>>> investigate
>
> replacing the curre

Re: mvn dependency:analyze

[Joey Echeverria]

On Fri, Nov 8, 2013 at 3:16 PM, Christopher  wrote:
> We won't be able to get rid of all the resulting warnings. So,
> automating it would probably just encourage people to ignore these
> warnings instead of address them. I'd rather just make it a point to
> explicitly check during testing before releasing, and to encourage
> people to consider dependencies throughout development.

Do we have a document on how to test a release? In particular, I'm
thinking of a process to follow after an RC is cut to determine if it
should get a +1 to become the release.

-Joey

Re: "Provided" dependencies

[Joey Echeverria]

I'm a little lost here I think.

On Wed, Nov 6, 2013 at 5:43 PM, Michael Berman  wrote:
> As far as the provided question goes, it seems to me that the only reason
> to mark a dep provided is if we think developers will *usually* want to
> compile against different versions.  Initially I thought it would make
> sense if we thought the runtime versions would vary, but Chris makes a good
> point that the deps we include in the distributed package can be selected
> independently of the maven dep scope.

If I depend on Accumulo in my maven project, then I shouldn't need to
depend on Hadoop unless the APIs I'm using leak that dependency or I
have an explicit dependency on Hadoop elsewhere.

> Since you can build accumulo against
> any version of hadoop and it will still run against any other version of
> hadoop, I think it's better to make things easier on us by having it
> compile scoped.

That's not strictly true. If you build against Hadoop1, I don't think
you can run against Hadoop2, but I could be wrong. I do know that
unless you're doing some reflection magic, you have to modify
[In|Out]putFormats as the APIs moved some classes to interfaces and
vice versa.

> If someone depends on the accumulo server, then they may have to exclude
> the transitive dependency if our hadoop is polluting theirs, but I think
> that issue can be mitigated by not requiring client apps to depend on the
> entire server.

I could see the server artifact having Hadoop scoped compile, but I
can't imagine that most users actually build against it. Or are we
just taking about changing it for -server?

-Joey

> On Wed, Nov 6, 2013 at 5:17 PM, Joey Echeverria 
> wrote:
>
>> Do Accumulo users need Hadoop or it's dependencies in order to use the
>> client APIs?
>>
>> The only client API that I could see needing it would be the
>> [In|Out]putFormats, but it'd be cool if that was a separate module and
>> that module had the appropriate Hadoop dependencies with the compile
>> scope.
>>
>> -Joey
>>
>> On Wed, Nov 6, 2013 at 5:05 PM, Christopher  wrote:
>> > What's the latest opinion whether things should be marked "provided" in
>> the pom?
>> > I've changed my mind on this a few times, myself, so I'm curious what
>> > others think.
>> >
>> > The provided scope means that it will not propagate as a transitive
>> > dependency. Other than that, it doesn't do much... though we can
>> > control packaging based on provided or not.
>> >
>> > I'm not sure this gets us much, and it's inconvenient for users. We
>> > can control packaging in other ways (like being more explicit and
>> > carefully considering which dependencies we include in an RPM or
>> > tarball, for instance).
>> >
>> > If we drop its declaration, what this means, is that if users want to
>> > build with Accumulo as a dependency, but against a different version
>> > of Hadoop than what we declare in our POM, they'll have to explicitly
>> >  the hadoop dependencies, and redeclare them, or they will
>> > have to use their  section to force a particular
>> > dependency of hadoop.
>> >
>> > The advantage to users, though, if we drop this, is that they won't
>> > have to constantly re-declare transitive dependencies to get their
>> > projects to build/test/run.
>> >
>> > See http://s.apache.org/maven-dependency-scopes
>> >
>> > Thoughts?
>> >
>> > --
>> > Christopher L Tubbs II
>> > http://gravatar.com/ctubbsii
>>

Re: "Provided" dependencies

[Joey Echeverria]

Do Accumulo users need Hadoop or it's dependencies in order to use the
client APIs?

The only client API that I could see needing it would be the
[In|Out]putFormats, but it'd be cool if that was a separate module and
that module had the appropriate Hadoop dependencies with the compile
scope.

-Joey

On Wed, Nov 6, 2013 at 5:05 PM, Christopher  wrote:
> What's the latest opinion whether things should be marked "provided" in the 
> pom?
> I've changed my mind on this a few times, myself, so I'm curious what
> others think.
>
> The provided scope means that it will not propagate as a transitive
> dependency. Other than that, it doesn't do much... though we can
> control packaging based on provided or not.
>
> I'm not sure this gets us much, and it's inconvenient for users. We
> can control packaging in other ways (like being more explicit and
> carefully considering which dependencies we include in an RPM or
> tarball, for instance).
>
> If we drop its declaration, what this means, is that if users want to
> build with Accumulo as a dependency, but against a different version
> of Hadoop than what we declare in our POM, they'll have to explicitly
>  the hadoop dependencies, and redeclare them, or they will
> have to use their  section to force a particular
> dependency of hadoop.
>
> The advantage to users, though, if we drop this, is that they won't
> have to constantly re-declare transitive dependencies to get their
> projects to build/test/run.
>
> See http://s.apache.org/maven-dependency-scopes
>
> Thoughts?
>
> --
> Christopher L Tubbs II
> http://gravatar.com/ctubbsii

Re: HDFS durable sync

[Joey Echeverria]

It's set in the hdfs-default.xml that comes in the
hadoop-hdfs-.jar. So if the tserver picks it up by creating a
Configuration() object, it should see it.

-Joey

On Wed, Nov 6, 2013 at 4:39 PM, John Vines  wrote:
> Does CDH default to that the config file or just the hardcoded default? If
> it's the latter, it still needs to be put in because older versions had no
> single place to look in to determine defaults so tservers will fail to
> start if it cannot find one of those settings.
>
> On Wed, Nov 6, 2013 at 4:34 PM, Joey Echeverria 
> wrote:
>
>> The setting is the same as the one for Apache Hadoop 2.0.5-alpha and
>> 2.2.0 (dfs.support.append), but it defaults to true in CDH4 anyway, so
>> you shouldn't have to change it.
>>
>> If you have more CDH specific questions, you can use the CDH mailing
>> list or public forum. Ping me off-list if you need a pointer to the
>> location of either resource.
>>
>> -Joey
>>
>> On Wed, Nov 6, 2013 at 4:09 PM, Miguel Pereira
>>  wrote:
>> > Hia guys,
>> >
>> > Whats the recommended setting for HDFS durable sync CDH4.4 or how do I
>> find
>> > out?
>> > 
>> >
>> > Cloudera CDH 3u0-3u3    true
>> > Cloudera CDH   3u4dfs.support.appendtrue
>> >
>> > 
>> >
>> > Cheers,
>> > Miguel
>>

Re: HDFS durable sync

[Joey Echeverria]

The setting is the same as the one for Apache Hadoop 2.0.5-alpha and
2.2.0 (dfs.support.append), but it defaults to true in CDH4 anyway, so
you shouldn't have to change it.

If you have more CDH specific questions, you can use the CDH mailing
list or public forum. Ping me off-list if you need a pointer to the
location of either resource.

-Joey

On Wed, Nov 6, 2013 at 4:09 PM, Miguel Pereira
 wrote:
> Hia guys,
>
> Whats the recommended setting for HDFS durable sync CDH4.4 or how do I find
> out?
> 
>
> Cloudera CDH 3u0-3u3    true
> Cloudera CDH   3u4dfs.support.appendtrue
>
> 
>
> Cheers,
> Miguel

Re: Accumulo-1379

[Joey Echeverria]

I'm with Sean, we should file new JIRAs to backport the fix.

-Joey


On Wed, Nov 6, 2013 at 9:39 AM, Sean Busbey  wrote:
> On Tue, Nov 5, 2013 at 11:56 PM, Aaron Cordova 
> wrote:
>
>> I see that 1379 affects 1.4.3 and it looks like it’s being fixed for 1.6.0.
>>
>> Was this by any chance fixed in 1.4.4 or somehow not apply?
>>
>>
>>
> AFAICT, it only got fixed in master. I don't see anything in the related
> commits that breaks compatibility.
>
> If you can file a ticket and handle a review, I'd be happy to do the
> backporting to 1.4.x and 1.5.x.
>
>
> --
> Sean

Re: RAccumulo's Home (was Code import for Apache Accumulo)

[Joey Echeverria]

On Wed, Oct 30, 2013 at 6:21 PM, Christopher  wrote:
>
> If existing committers are willing to take on the project, and
> maintain it and really own it (so, more than just being a proxy for
> external committers), I'd be on board accepting it as a sub-project.
>
> However, if its just going to be an orphaned project, or we're just
> going to have to act as a proxy for external developers who've hosted
> their code in our project as a notional parent, but aren't committers
> themselves, I'm opposed to importing it and assuming ownership of it
> as a sub-project, and think it would be better served hosted where the
> originating developers can maintain it.
>
> In either case, I think it's a great related project to serve both the
> R and Accumulo communities, and we should link to it, provide
> feedback, and help make it better if we can. I'm just concerned about
> setting a precedent for accepting/hosting everything related to
> Accumulo, causing us to either be spread too thin or to cause projects
> to die because of lack of care.

+1, orphaned code does no one any good.

Re: Review Request 14880: ACCUMULO-1785 Alter config.sh to optionally just verify environment instead of making changes

[Joey Echeverria]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14880/#review27829
---

Ship it!


I think it's best to keep the message about the missing masters file on STDERR 
and that Sean's comments addressed Tubbs's other questions, so I'm +1 on the 
patch once the change that's now in ACCUMULO-1810 is removed from this patch.

- Joey Echeverria


On Oct. 23, 2013, 7:08 p.m., Sean Busbey wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14880/
> ---
> 
> (Updated Oct. 23, 2013, 7:08 p.m.)
> 
> 
> Review request for accumulo.
> 
> 
> Bugs: ACCUMULO-1785
> https://issues.apache.org/jira/browse/ACCUMULO-1785
> 
> 
> Repository: accumulo
> 
> 
> Description
> ---
> 
> Updates config.sh to skip automagic standalone mode and avoid writing to the 
> local filesystem given an optional environment variable. Adds additional 
> error handling since this changes the invariant that a conf/masters file will 
> exist.
> 
> 
> Diffs
> -
> 
>   bin/config.sh 9277406 
> 
> Diff: https://reviews.apache.org/r/14880/diff/
> 
> 
> Testing
> ---
> 
> functional tests
> starting w/o env variable set and no master/slaves correctly started in 
> standalone mode.
> starting w/env variable and no required variables and no master/slaves 
> correctly complained and failed.
> starting w/env variable and required variables and no master/slaves correctly 
> init w/o writing crap, correctly failed to use start-all
> starting with a normal cluster config
> 
> 
> Thanks,
> 
> Sean Busbey
> 
>

Re: Accumulo feature freeze in 1 week

[Joey Echeverria]

On Fri, Oct 25, 2013 at 2:49 PM, John Vines  wrote:
> I'd rather not go down the slippery slope we went down last time. I'll
> gladly be the bad guy and say that things are getting pushed to 1.7.

+1

-Joey

>
> On Fri, Oct 25, 2013 at 2:23 PM, Christopher  wrote:
>
>> It looks like we're 50% in terms of tickets complete.
>>
>> http://s.apache.org/accumulo-1.6.0-issues
>>
>> Extensions are always nice, but I'm not going to be the bad guy and
>> ask for one :)
>>
>> --
>> Christopher L Tubbs II
>> http://gravatar.com/ctubbsii
>>
>>
>> On Fri, Oct 25, 2013 at 12:33 PM, John Vines  wrote:
>> > Alright folks, we're down to the wire. Feature freeze is one week away.
>> If
>> > anyone has any last minute concerns about this schedule, please speak up
>> > now. Otherwise, have a great weekend!
>>

Re: open questions on contrib projects

[Joey Echeverria]

On Tue, Oct 22, 2013 at 12:01 PM, Sean Busbey  wrote:
>
> Is there a reason not to just use sub projects for all contribs? Wikisearch
> is certainly big enough and already has a component jira from when it was
> in the main project.

There's a lot more overhead (INFRA tickets for example) in creating a
sub-project.

For what it's worth, both Hadoop and HBase have migrated away from
having a contrib and sub-projects, other than how Hadoop has all of
it's major components (HDFS, YARN, MR, Common) in sub-projects.

-Joey

Re: Hadoop 2.0 Support for Accumulo 1.4 Branch

[Joey Echeverria]

I think you meant:

"Ugh, Hadoop versions.[1]"

[1]
http://blog.cloudera.com/blog/2012/04/apache-hadoop-versions-looking-ahead-3/


On Tue, Oct 15, 2013 at 11:20 AM, Sean Busbey  wrote:

> On Tue, Oct 15, 2013 at 10:16 AM, Sean Busbey  wrote:
>
> >
> > On Tue, Oct 15, 2013 at 7:16 AM,  wrote:
> >
> >> Just to be clear, we are talking about adding profile support to the
> >> pom's for Hadoop 2.2.0 for a 1.4.5 and 1.5.1 release, correct? We are
> not
> >> talking about changing the default build profile for these branches are
> we?
> >>
> >>
> >>
> > for 1.4.5-SNAPSHOT I am only talking about adding support Hadoop 2.2.0. I
> > am not suggesting we change the default from building against Hadoop
> > 0.23.203.
> >
> >
> >
> I mean 0.20.203.0. Ugh, Hadoop versions.
>
> --
> Sean
>



-- 
Joey Echeverria
Director, Federal FTS
Cloudera, Inc.

Re: [VOTE] 1.6.0 Feature freeze.

[Joey Echeverria]

+1 (non-binding)

On Fri, Sep 27, 2013 at 1:58 PM, Sean Busbey  wrote:
> +1 (non-binding)
>
>
> On Fri, Sep 27, 2013 at 12:57 PM, Sean Busbey  wrote:
>
>> I presume the appropriate links are:
>>
>> [1]:http://www.apache.org/foundation/glossary.html#MajorityApproval
>> [2]:http://www.apache.org/foundation/glossary.html#ConsensusApproval
>>
>>
>> On Fri, Sep 27, 2013 at 12:39 PM, John Vines  wrote:
>>
>>> Please vote on a feature freeze date of Nov 1 23:59 PDT for the master
>>> branch.  Shortly after this time we will branch 1.6.0-SNAPSHOT from master
>>> and increment the version in master.  "Feature Freeze" means only bug
>>> fixes
>>> and documentation updates happen after the date, which implies major code
>>> additions and changes are already in place with appropriate tests.
>>>
>>> If a commiter thinks a new feature in 1.6.0-SNAPSHOT is not ready for
>>> release, they should bring it up on the dev list.  If agreement can not be
>>> reached on the dev list with 72 hours, then the commiter can call for a
>>> vote on reverting the feature from 1.6.0-SNAPSHOT.  The vote must pass
>>> with
>>> majority approval[1].  If the vote passes, any commiter can revert the
>>> feature from 1.6.0-SNAPSHOT.
>>>
>>> This vote will remain open for 72 hours and must have consensus
>>> approval[2]
>>> to pass.
>>>
>>
>>
>>
>> --
>> Sean
>>
>
>
>
> --
> Sean



-- 
Joey Echeverria
Director, Federal FTS
Cloudera, Inc.

Re: Would gerrit help to organize accumulo code reviews?

[Joey Echeverria]

I love code reviews :)




At cloudera, we started using Gerrit for a couple of projects and it's been 
great so far. In particular, it has great git integration.




You can submit a review by just pushing to a special branch and once the review 
is done, pushing to trunk is automated. 

—
Sent from Mailbox for iPhone

On Fri, Sep 27, 2013 at 9:41 AM, Bill Havanki 
wrote:

> Noob question: are code reviews normally done now?
> I've never used gerrit, but I'd be happy to try it. I've used Crucible in
> the past and liked it.
> On Tue, Sep 24, 2013 at 10:29 PM, David Medinets
> wrote:
>> http://code.google.com/p/gerrit/
>>

Re: Hadoop 2.0 Support for Accumulo 1.4 Branch

[Joey Echeverria]

Vanilla.

On Fri, Aug 2, 2013 at 3:58 PM, Mike Drob  wrote:
> Which version of 0.20 are you testing against? Vanilla, or cdh3 flavored?
>
>
> On Fri, Aug 2, 2013 at 2:37 PM, Joey Echeverria  wrote:
>
>> I don't think that's a good idea unless you can come up with very
>> clear version number change.
>>
>> -Joey
>>
>> On Fri, Aug 2, 2013 at 2:31 PM, Christopher  wrote:
>> > Would it be reasonable to consider a version of 1.4 that breaks
>> > compatibility with 0.20? I'm not really a fan of this, personally, but
>> > am curious what others think.
>> >
>> > --
>> > Christopher L Tubbs II
>> > http://gravatar.com/ctubbsii
>> >
>> >
>> > On Fri, Aug 2, 2013 at 2:22 PM, Joey Echeverria 
>> wrote:
>> >> Sorry for the delay, it's been one of those weeks.
>> >>
>> >> The current version would probably not be backwards compatible to
>> >> 0.20.2 just based on changes in dependencies. We're looking right now
>> >> to see how hard it is to have three way compatibility (0.20, 1.0,
>> >> 2.0).
>> >>
>> >> -Joey
>> >>
>> >> On Thu, Aug 1, 2013 at 7:33 PM, Dave Marion 
>> wrote:
>> >>> Any update?
>> >>>
>> >>> -Original Message-
>> >>> From: Joey Echeverria [mailto:j...@cloudera.com]
>> >>> Sent: Monday, July 29, 2013 1:24 PM
>> >>> To: dev@accumulo.apache.org
>> >>> Subject: Re: Hadoop 2.0 Support for Accumulo 1.4 Branch
>> >>>
>> >>> We're testing this today. I'll report back what we find.
>> >>>
>> >>>
>> >>> -Joey
>> >>> —
>> >>> Sent from Mailbox for iPhone
>> >>>
>> >>> On Fri, Jul 26, 2013 at 3:34 PM, null  wrote:
>> >>>
>> >>>> "Will 1.4 still work with 0.20 with these patches?"
>> >>>> Great point Billie.
>> >>>> - Original Message -
>> >>>> From: "Billie Rinaldi" 
>> >>>> To: dev@accumulo.apache.org
>> >>>> Sent: Friday, July 26, 2013 3:02:41 PM
>> >>>> Subject: Re: Hadoop 2.0 Support for Accumulo 1.4 Branch On Fri, Jul
>> >>>> 26, 2013 at 11:33 AM, Joey Echeverria  wrote:
>> >>>>> > If these patches are going to be included with 1.4.4 or 1.4.5, I
>> >>>>> > would
>> >>>>> like
>> >>>>> > to see the following test run using CDH4 on at least a 5 node
>> cluster.
>> >>>>> >  More nodes would be better.
>> >>>>> >
>> >>>>> >   * unit test
>> >>>>> >   * Functional test
>> >>>>> >   * 24 hr Continuous ingest + verification
>> >>>>> >   * 24 hr Continuous ingest + verification + agitation
>> >>>>> >   * 24 hr Random walk
>> >>>>> >   * 24 hr Random walk + agitation
>> >>>>> >
>> >>>>> > I may be able to assist with this, but I can not make any promises.
>> >>>>>
>> >>>>> Sure thing. Is there already a write-up on running this full battery
>> >>>>> of tests? I have a 10 node cluster that I can use for this.
>> >>>>>
>> >>>>>
>> >>>>> > Great.  I think this would be a good patch for 1.4.   I assume that
>> >>>>> > if a user stays with Hadoop 1 there are no dependency changes?
>> >>>>>
>> >>>>> Yup. It works the same way as 1.5 where all of the dependency changes
>> >>>>> are in a Hadoop 2.0 profile.
>> >>>>>
>> >>>> In 1.5.0, we gave up on compatibility with 0.20 (and early versions of
>> >>>> 1.0) to make the compatibility requirements simpler; we ended up
>> >>>> without dependency changes in the hadoop version profiles.  Will 1.4
>> >>>> still work with 0.20 with these patches?  If there are dependency
>> >>>> changes in the profiles, 1.4 would have to be compiled against a
>> >>>> hadoop version compatible with the running version of hadoop, correct?
>> >>>> We had some trouble in the
>> >>>> 1.5 release process with figuring out how to provide multiple binary
>> >>>> artifacts (each compiled against a different version of hadoop) for
>> >>>> the same release.  Just something we should consider before we are in
>> >>>> the midst of releasing 1.4.4.
>> >>>> Billie
>> >>>>> -Joey
>> >>>>>
>> >>>
>> >>
>> >>
>> >>
>> >> --
>> >> Joey Echeverria
>> >> Director, Federal FTS
>> >> Cloudera, Inc.
>>
>>
>>
>> --
>> Joey Echeverria
>> Director, Federal FTS
>> Cloudera, Inc.
>>



-- 
Joey Echeverria
Director, Federal FTS
Cloudera, Inc.

Re: Hadoop 2.0 Support for Accumulo 1.4 Branch

[Joey Echeverria]

I don't think that's a good idea unless you can come up with very
clear version number change.

-Joey

On Fri, Aug 2, 2013 at 2:31 PM, Christopher  wrote:
> Would it be reasonable to consider a version of 1.4 that breaks
> compatibility with 0.20? I'm not really a fan of this, personally, but
> am curious what others think.
>
> --
> Christopher L Tubbs II
> http://gravatar.com/ctubbsii
>
>
> On Fri, Aug 2, 2013 at 2:22 PM, Joey Echeverria  wrote:
>> Sorry for the delay, it's been one of those weeks.
>>
>> The current version would probably not be backwards compatible to
>> 0.20.2 just based on changes in dependencies. We're looking right now
>> to see how hard it is to have three way compatibility (0.20, 1.0,
>> 2.0).
>>
>> -Joey
>>
>> On Thu, Aug 1, 2013 at 7:33 PM, Dave Marion  wrote:
>>> Any update?
>>>
>>> -Original Message-
>>> From: Joey Echeverria [mailto:j...@cloudera.com]
>>> Sent: Monday, July 29, 2013 1:24 PM
>>> To: dev@accumulo.apache.org
>>> Subject: Re: Hadoop 2.0 Support for Accumulo 1.4 Branch
>>>
>>> We're testing this today. I'll report back what we find.
>>>
>>>
>>> -Joey
>>> —
>>> Sent from Mailbox for iPhone
>>>
>>> On Fri, Jul 26, 2013 at 3:34 PM, null  wrote:
>>>
>>>> "Will 1.4 still work with 0.20 with these patches?"
>>>> Great point Billie.
>>>> - Original Message -
>>>> From: "Billie Rinaldi" 
>>>> To: dev@accumulo.apache.org
>>>> Sent: Friday, July 26, 2013 3:02:41 PM
>>>> Subject: Re: Hadoop 2.0 Support for Accumulo 1.4 Branch On Fri, Jul
>>>> 26, 2013 at 11:33 AM, Joey Echeverria  wrote:
>>>>> > If these patches are going to be included with 1.4.4 or 1.4.5, I
>>>>> > would
>>>>> like
>>>>> > to see the following test run using CDH4 on at least a 5 node cluster.
>>>>> >  More nodes would be better.
>>>>> >
>>>>> >   * unit test
>>>>> >   * Functional test
>>>>> >   * 24 hr Continuous ingest + verification
>>>>> >   * 24 hr Continuous ingest + verification + agitation
>>>>> >   * 24 hr Random walk
>>>>> >   * 24 hr Random walk + agitation
>>>>> >
>>>>> > I may be able to assist with this, but I can not make any promises.
>>>>>
>>>>> Sure thing. Is there already a write-up on running this full battery
>>>>> of tests? I have a 10 node cluster that I can use for this.
>>>>>
>>>>>
>>>>> > Great.  I think this would be a good patch for 1.4.   I assume that
>>>>> > if a user stays with Hadoop 1 there are no dependency changes?
>>>>>
>>>>> Yup. It works the same way as 1.5 where all of the dependency changes
>>>>> are in a Hadoop 2.0 profile.
>>>>>
>>>> In 1.5.0, we gave up on compatibility with 0.20 (and early versions of
>>>> 1.0) to make the compatibility requirements simpler; we ended up
>>>> without dependency changes in the hadoop version profiles.  Will 1.4
>>>> still work with 0.20 with these patches?  If there are dependency
>>>> changes in the profiles, 1.4 would have to be compiled against a
>>>> hadoop version compatible with the running version of hadoop, correct?
>>>> We had some trouble in the
>>>> 1.5 release process with figuring out how to provide multiple binary
>>>> artifacts (each compiled against a different version of hadoop) for
>>>> the same release.  Just something we should consider before we are in
>>>> the midst of releasing 1.4.4.
>>>> Billie
>>>>> -Joey
>>>>>
>>>
>>
>>
>>
>> --
>> Joey Echeverria
>> Director, Federal FTS
>> Cloudera, Inc.



-- 
Joey Echeverria
Director, Federal FTS
Cloudera, Inc.

Re: client config files

[Joey Echeverria]

Yeah, I agree. Consistency with Hadoop here is probably not that valuable.

-Joey

On Fri, Aug 2, 2013 at 2:28 PM, Keith Turner  wrote:
> On Thu, Aug 1, 2013 at 4:33 PM, Joey Echeverria  wrote:
>
>> I generally prefer properties files to XML, but there may be a argument
>> for reusing Hadoop's SSL configuration system which is XML based.
>>
>
> I also prefer prefer properties files over XML.   The only reason I can
> think that we might want to use XML is for consistency with Hadoop and
> Accumulo server side config.  But it does not seem like a very compelling
> reason, its not like it prop files are hard to use once you realize you
> need to use them.
>
>
>>
>>
>> -Joey
>> —
>> Sent from Mailbox for iPhone
>>
>> On Thu, Aug 1, 2013 at 3:08 PM, Christopher  wrote:
>>
>> > ^ Another reason I like commons-configuration here is for
>> > property-interpolation with HierarchicalConfiguration.
>> > --
>> > Christopher L Tubbs II
>> > http://gravatar.com/ctubbsii
>> > On Thu, Aug 1, 2013 at 3:07 PM, Christopher  wrote:
>> >> I absolutely DO think they should be combined in a properties file
>> >> located in $HOME/.accumulo/config
>> >> I absolutely DO NOT think this client configuration should be
>> >> exclusive to the shell, and I absolutely DO NOT think it should be
>> >> XML.
>> >>
>> >> I would love to see all our clients/client code use
>> >> commons-configuration to hold properties from the properties file, so
>> >> that only a --config parameter is needed (with reasonable defaults, so
>> >> even that is not absolutely necessary). I also think that every
>> >> property that can exist in the file should be possible to override on
>> >> the command-line. I personally prefer to use system properties, using
>> >> commons-configuration's HierarchicalConfiguration, but jcommander may
>> >> make it easier to do the same thing in a slightly different way.
>> >>
>> >> --
>> >> Christopher L Tubbs II
>> >> http://gravatar.com/ctubbsii
>> >>
>> >>
>> >> On Thu, Aug 1, 2013 at 12:25 PM, Michael Berman 
>> wrote:
>> >>> As part of SSL, we need to introduce configuration so accumulo clients
>> >>> (such as ZooKeeperInstance) can find trust stores.  It seems like this
>> has
>> >>> a lot in common with shell config files in ACCUMULO-1397.  Do people
>> think
>> >>> these should be combined, or should the shell have its own separate
>> config?
>> >>>  I was imagining a simple java .properties-style key=value list.  Does
>> this
>> >>> seem reasonable?  Or should the format be more like the xml of the site
>> >>> config?  I was also imagining looking through a list of files that
>> would
>> >>> each override settings, perhaps in the following order (from lowest to
>> >>> highest priority):
>> >>>
>> >>> /etc/accumulo/client.conf
>> >>> $ACCUMULO_HOME/conf/client.conf
>> >>> $HOME/.accumulo/config
>> >>> --client-config command line switch for shell or explicit parameter
>> passed
>> >>> to ZooKeeperInstance
>> >>>
>> >>> Does this sound good to y'all?  Should the explicit switch/parameter
>> have
>> >>> per-property override semantics, or should it just be used as the
>> exclusive
>> >>> source of properties if specified?
>> >>>
>> >>> Mike Drob, are you actively working on the shell side of this already?
>>  I
>> >>> see that bug is assigned to you...
>> >>>
>> >>> Thanks,
>> >>> Michael
>>



-- 
Joey Echeverria
Director, Federal FTS
Cloudera, Inc.

Re: Hadoop 2.0 Support for Accumulo 1.4 Branch

[Joey Echeverria]

Sorry for the delay, it's been one of those weeks.

The current version would probably not be backwards compatible to
0.20.2 just based on changes in dependencies. We're looking right now
to see how hard it is to have three way compatibility (0.20, 1.0,
2.0).

-Joey

On Thu, Aug 1, 2013 at 7:33 PM, Dave Marion  wrote:
> Any update?
>
> -Original Message-
> From: Joey Echeverria [mailto:j...@cloudera.com]
> Sent: Monday, July 29, 2013 1:24 PM
> To: dev@accumulo.apache.org
> Subject: Re: Hadoop 2.0 Support for Accumulo 1.4 Branch
>
> We're testing this today. I'll report back what we find.
>
>
> -Joey
> —
> Sent from Mailbox for iPhone
>
> On Fri, Jul 26, 2013 at 3:34 PM, null  wrote:
>
>> "Will 1.4 still work with 0.20 with these patches?"
>> Great point Billie.
>> - Original Message -
>> From: "Billie Rinaldi" 
>> To: dev@accumulo.apache.org
>> Sent: Friday, July 26, 2013 3:02:41 PM
>> Subject: Re: Hadoop 2.0 Support for Accumulo 1.4 Branch On Fri, Jul
>> 26, 2013 at 11:33 AM, Joey Echeverria  wrote:
>>> > If these patches are going to be included with 1.4.4 or 1.4.5, I
>>> > would
>>> like
>>> > to see the following test run using CDH4 on at least a 5 node cluster.
>>> >  More nodes would be better.
>>> >
>>> >   * unit test
>>> >   * Functional test
>>> >   * 24 hr Continuous ingest + verification
>>> >   * 24 hr Continuous ingest + verification + agitation
>>> >   * 24 hr Random walk
>>> >   * 24 hr Random walk + agitation
>>> >
>>> > I may be able to assist with this, but I can not make any promises.
>>>
>>> Sure thing. Is there already a write-up on running this full battery
>>> of tests? I have a 10 node cluster that I can use for this.
>>>
>>>
>>> > Great.  I think this would be a good patch for 1.4.   I assume that
>>> > if a user stays with Hadoop 1 there are no dependency changes?
>>>
>>> Yup. It works the same way as 1.5 where all of the dependency changes
>>> are in a Hadoop 2.0 profile.
>>>
>> In 1.5.0, we gave up on compatibility with 0.20 (and early versions of
>> 1.0) to make the compatibility requirements simpler; we ended up
>> without dependency changes in the hadoop version profiles.  Will 1.4
>> still work with 0.20 with these patches?  If there are dependency
>> changes in the profiles, 1.4 would have to be compiled against a
>> hadoop version compatible with the running version of hadoop, correct?
>> We had some trouble in the
>> 1.5 release process with figuring out how to provide multiple binary
>> artifacts (each compiled against a different version of hadoop) for
>> the same release.  Just something we should consider before we are in
>> the midst of releasing 1.4.4.
>> Billie
>>> -Joey
>>>
>



-- 
Joey Echeverria
Director, Federal FTS
Cloudera, Inc.

Re: client config files

[Joey Echeverria]

I generally prefer properties files to XML, but there may be a argument for 
reusing Hadoop's SSL configuration system which is XML based. 


-Joey
—
Sent from Mailbox for iPhone

On Thu, Aug 1, 2013 at 3:08 PM, Christopher  wrote:

> ^ Another reason I like commons-configuration here is for
> property-interpolation with HierarchicalConfiguration.
> --
> Christopher L Tubbs II
> http://gravatar.com/ctubbsii
> On Thu, Aug 1, 2013 at 3:07 PM, Christopher  wrote:
>> I absolutely DO think they should be combined in a properties file
>> located in $HOME/.accumulo/config
>> I absolutely DO NOT think this client configuration should be
>> exclusive to the shell, and I absolutely DO NOT think it should be
>> XML.
>>
>> I would love to see all our clients/client code use
>> commons-configuration to hold properties from the properties file, so
>> that only a --config parameter is needed (with reasonable defaults, so
>> even that is not absolutely necessary). I also think that every
>> property that can exist in the file should be possible to override on
>> the command-line. I personally prefer to use system properties, using
>> commons-configuration's HierarchicalConfiguration, but jcommander may
>> make it easier to do the same thing in a slightly different way.
>>
>> --
>> Christopher L Tubbs II
>> http://gravatar.com/ctubbsii
>>
>>
>> On Thu, Aug 1, 2013 at 12:25 PM, Michael Berman  wrote:
>>> As part of SSL, we need to introduce configuration so accumulo clients
>>> (such as ZooKeeperInstance) can find trust stores.  It seems like this has
>>> a lot in common with shell config files in ACCUMULO-1397.  Do people think
>>> these should be combined, or should the shell have its own separate config?
>>>  I was imagining a simple java .properties-style key=value list.  Does this
>>> seem reasonable?  Or should the format be more like the xml of the site
>>> config?  I was also imagining looking through a list of files that would
>>> each override settings, perhaps in the following order (from lowest to
>>> highest priority):
>>>
>>> /etc/accumulo/client.conf
>>> $ACCUMULO_HOME/conf/client.conf
>>> $HOME/.accumulo/config
>>> --client-config command line switch for shell or explicit parameter passed
>>> to ZooKeeperInstance
>>>
>>> Does this sound good to y'all?  Should the explicit switch/parameter have
>>> per-property override semantics, or should it just be used as the exclusive
>>> source of properties if specified?
>>>
>>> Mike Drob, are you actively working on the shell side of this already?  I
>>> see that bug is assigned to you...
>>>
>>> Thanks,
>>> Michael

Re: Hadoop 2.0 Support for Accumulo 1.4 Branch

[Joey Echeverria]

We're testing this today. I'll report back what we find. 


-Joey
—
Sent from Mailbox for iPhone

On Fri, Jul 26, 2013 at 3:34 PM, null  wrote:

> "Will 1.4 still work with 0.20 with these patches?" 
> Great point Billie. 
> - Original Message -
> From: "Billie Rinaldi"  
> To: dev@accumulo.apache.org 
> Sent: Friday, July 26, 2013 3:02:41 PM 
> Subject: Re: Hadoop 2.0 Support for Accumulo 1.4 Branch 
> On Fri, Jul 26, 2013 at 11:33 AM, Joey Echeverria  wrote: 
>> > If these patches are going to be included with 1.4.4 or 1.4.5, I would 
>> like 
>> > to see the following test run using CDH4 on at least a 5 node cluster. 
>> >  More nodes would be better. 
>> > 
>> >   * unit test 
>> >   * Functional test 
>> >   * 24 hr Continuous ingest + verification 
>> >   * 24 hr Continuous ingest + verification + agitation 
>> >   * 24 hr Random walk 
>> >   * 24 hr Random walk + agitation 
>> > 
>> > I may be able to assist with this, but I can not make any promises. 
>> 
>> Sure thing. Is there already a write-up on running this full battery 
>> of tests? I have a 10 node cluster that I can use for this. 
>> 
>> 
>> > Great.  I think this would be a good patch for 1.4.   I assume that if a 
>> > user stays with Hadoop 1 there are no dependency changes? 
>> 
>> Yup. It works the same way as 1.5 where all of the dependency changes 
>> are in a Hadoop 2.0 profile. 
>> 
> In 1.5.0, we gave up on compatibility with 0.20 (and early versions of 1.0) 
> to make the compatibility requirements simpler; we ended up without 
> dependency changes in the hadoop version profiles.  Will 1.4 still work 
> with 0.20 with these patches?  If there are dependency changes in the 
> profiles, 1.4 would have to be compiled against a hadoop version compatible 
> with the running version of hadoop, correct?  We had some trouble in the 
> 1.5 release process with figuring out how to provide multiple binary 
> artifacts (each compiled against a different version of hadoop) for the 
> same release.  Just something we should consider before we are in the midst 
> of releasing 1.4.4. 
> Billie 
>> -Joey 
>> 

Re: Hadoop 2.0 Support for Accumulo 1.4 Branch

[Joey Echeverria]

> If these patches are going to be included with 1.4.4 or 1.4.5, I would like
> to see the following test run using CDH4 on at least a 5 node cluster.
>  More nodes would be better.
>
>   * unit test
>   * Functional test
>   * 24 hr Continuous ingest + verification
>   * 24 hr Continuous ingest + verification + agitation
>   * 24 hr Random walk
>   * 24 hr Random walk + agitation
>
> I may be able to assist with this, but I can not make any promises.

Sure thing. Is there already a write-up on running this full battery
of tests? I have a 10 node cluster that I can use for this.


> Great.  I think this would be a good patch for 1.4.   I assume that if a
> user stays with Hadoop 1 there are no dependency changes?

Yup. It works the same way as 1.5 where all of the dependency changes
are in a Hadoop 2.0 profile.

-Joey

Re: Hadoop 2.0 Support for Accumulo 1.4 Branch

[Joey Echeverria]

We have both the unit tests and the full system test suite hooked up to a
Jenkins build server.

There are still a couple of tests that fail periodically with the full
system test due to timeouts. We're working on those which is why our
current release is just a beta.

There are no API changes or Accumulo behavior changes. You can use
unmodified 1.4.x clients with our release of the server daemons.

-Joey


On Fri, Jul 26, 2013 at 11:45 AM, Keith Turner  wrote:

> On Fri, Jul 26, 2013 at 11:02 AM, Joey Echeverria 
> wrote:
>
> > Cloudera announced last night our support for Accumulo 1.4.3 on CDH4:
> >
> > http://www.slideshare.net/JoeyEcheverria/apache-accumulo-and-cloudera
> >
> > This required back porting about 11 patches in whole or in part from the
> > 1.5 line on top of 1.4.3. Our release is still in a semi-private beta,
> but
> > when it's fully public it will be downloadable along with all of the
> extra
> > patches that we committed.
> >
> > My question is if the community would be interested in us pulling those
> > back ports upstream?
> >
>
> What testing has been done?  It would be nice to run accumulo's full test
> suite against 1.4.3+CDH4.
>
> Are there any Accumulo API changes or Accumulo behavior changes?
>
>
> > I believe this would violate the previously agreed upon rule of no
> feature
> > back ports to 1.4.3, depending on how we "label" support for Hadoop 2.0.
>
>
> > Thoughts?
> >
> > -Joey
> >
>



-- 
Joey Echeverria
Director, Federal FTS
Cloudera, Inc.

Hadoop 2.0 Support for Accumulo 1.4 Branch

[Joey Echeverria]

Cloudera announced last night our support for Accumulo 1.4.3 on CDH4:

http://www.slideshare.net/JoeyEcheverria/apache-accumulo-and-cloudera

This required back porting about 11 patches in whole or in part from the
1.5 line on top of 1.4.3. Our release is still in a semi-private beta, but
when it's fully public it will be downloadable along with all of the extra
patches that we committed.

My question is if the community would be interested in us pulling those
back ports upstream?

I believe this would violate the previously agreed upon rule of no feature
back ports to 1.4.3, depending on how we "label" support for Hadoop 2.0.

Thoughts?

-Joey