[jira] [Created] (HBASE-14229) Flushing canceled by coprocessor still leads to memstoreSize set down

2015-08-16 Thread sunyerui (JIRA)
sunyerui created HBASE-14229:


 Summary: Flushing canceled by coprocessor still leads to 
memstoreSize set down
 Key: HBASE-14229
 URL: https://issues.apache.org/jira/browse/HBASE-14229
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 1.1.1, 0.98.13, 1.0.2, 1.2.0
Reporter: sunyerui


A Coprocessor override "public InternalScanner preFlush(final Store store, 
final InternalScanner scanner)" and return NULL when calling this method, will 
cancel flush request, leaving snapshot un-flushed, and no new storefile 
created. But the HRegion.internalFlushCache still set down memstoreSize to 0 by 
totalFlushableSize. 
If there's no write requests anymore, the memstoreSize will remaining as 0, and 
no more flush quests will be processed because of the checking of 
memstoreSize.get() <=0 at the beginning of internalFlushCache.
This issue may not cause data loss, but it will confuse coprocessor users. If 
we argree with this, I'll apply a patch later.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Re: [VOTE] First release candidate for HBase 1.1.2 (RC0) is available

2015-08-16 Thread Nick Dimiduk
It now occurs to me this vote period fell over a weekend, which is likely
inconvenient for many folks. Let me extend the period for another 24 hours.
Please provide your evaluation of this candidate by midnight Pacific time
on Tuesday, 2015-08-18.

Thanks for your time.
-n

On Thu, Aug 13, 2015 at 2:21 PM, Nick Dimiduk  wrote:

> I'm happy to announce the first release candidate of HBase 1.1.2
> (HBase-1.1.2RC0) is available for download at
> https://dist.apache.org/repos/dist/dev/hbase/hbase-1.1.2RC0/
>
> Maven artifacts are also available in the staging repository
> https://repository.apache.org/content/repositories/orgapachehbase-1090
>
> Artifacts are signed with my code signing subkey 0xAD9039071C3489BD,
> available in the Apache keys directory
> https://people.apache.org/keys/committer/ndimiduk.asc
>
> There's also a signed tag for this release at
> https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=tag;h=c4b4a91620639fd96e59171369c630dc51e3
>
> The detailed source and binary compatibility report vs 1.1.0 has been
> published for your review, at
> http://people.apache.org/~ndimiduk/1.1.0_1.1.2RC0_compat_report.html
>
> HBase 1.1.2 is the second patch release in the HBase 1.1 line, continuing
> on the theme of bringing a stable, reliable database to the Hadoop and
> NoSQL communities. This release includes over 70 bug fixes since the 1.1.1
> release. Notable correctness fixes include HBASE-14054, HBASE-12865,
> HBASE-13993, HBASE-13337, HBASE-14013, HBASE-13895.
>
> The full list of fixes included in this release is available at
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310753&version=12332793
> and in the CHANGES.txt file included in the distribution.
>
> Please try out this candidate and vote +/-1 by midnight Pacific time on
> Monday, 2015-08-17 as to whether we should release these artifacts as HBase
> 1.1.2.
>
> Thanks,
> Nick
>


Re: [DISCUSSION] Switching from RTC to CTR

2015-08-16 Thread Andrew Purtell
> As Stack mentioned, having the communal norm of reviews means that folks
are more likely to see more of the code.

I don't get it, because that norm can exist equally under CTR as RTC.

Let me repeat a question I had earlier that nobody has responded to: What
would be the practical difference/outcome of a committer immediately
checking in something today without giving time for review versus doing so
if we said "CTR" or "CTR after one week" *AND* "good committer practice is
to get a code review and +1 before commit"? Practically, I mean. In both
cases we'd revert it and have a discussion. Would the lack of attention to
good practice be more actionable by the PMC because in the first case it is
a rule violation and in the latter it's just inattention to community
norms. Is that it? We need rules? (And is that "just because", or is there
really a lack of trust?)

Anyway, I like how in this discussion members of the PMC have expressed
some room for committers to make minor commits on their good judgement
without _necessarily_ getting a review first in order to make progress. My
motivation here is we are a community of busy people and some things just
can't get the level of interest as others. As an RM, things like build
fixes to unblock release candidates of older branches come to mind. I also
like that we have plenty of room for experimental work on branches with
scrutiny coming later at branch commit time with a very reasonable bar of 3
+1s inviting necessary scrutiny. I still think a policy of "one week for
review, then proceed as CTR" would be useful for a variety of reasons but
don't see clear support for that here. After I collect a few instances of
recent anecdotal evidence supporting it I may revive this discussion.


On Fri, Aug 14, 2015 at 9:00 PM, Sean Busbey  wrote:

> My apologies if this thread has run its course and I'm showing up late to
> rehash things.
>
> Here's a short list of the reasons I like RTC:
>
> 1) Number One With A Bullet: It puts committers and
> non-committer-contributors on closer to equal footing. If I'm participating
> in the project and I haven't been blessed with a commit bit, what am I
> supposed to do after my week of having my patch sit around?
>
> 2) Community interaction. As Stack mentioned, having the communal norm of
> reviews means that folks are more likely to see more of the code.
>
> 3) Everyone has a bad day. I totally identify with committership being a
> sign of "I trust you" to project participants. But everyone has one of
> those days where you're in a rush either because of work or life. Having
> even a cursory additional set of eyes on things markedly increases the
> quality of the overall code base over a long enough time line (at least in
> my experience contributing to open source projects). So for me, the trust
> is largely "to follow the rules" and "to provide feedback in reviews".
>
> If we end up with some part of the code that isn't getting reviews, I'd
> rather the PMC fix that problem instead of backslide on the three points
> above.
>
> That we don't have this problem right now is wonderful. I have been in / am
> currently in some projects where the community is very near end-of-life by
> the ASF's definition of 3 folks to approve a release. My observation of
> those communities has the CTR ones much more a loose collection of people
> scratching their own itch. When an ASF project gets to that point, what the
> advantage over just going to the attic and keeping independent forks for
> those few remaining folks?
>
> I kind of view this like the ASF policy on only distributing PMC approved
> releases. The advice from the foundation for folks who don't like the
> limitation of waiting for a release is "make more releases." Similarly, I
> think the problem of reviewer bandwidth is solved with "make more
> committers."
>
> I don't want to hijack this thread, but maybe we could have another one
> about expectations for committership and ways the PMC could help get more
> folks on the path?
>
>
> On Sun, Aug 2, 2015 at 1:59 PM, Andrew Purtell 
> wrote:
>
> > Agreed, committers should be spending more time reviewing others' work.
> We
> > can ask. It may happen. It may work for a short while. It may not. Shrug.
> > People will do what they want.
> >
> > I'm looking to make one substantial change that will allow committers to
> > make progress even if there's nobody else around or interested for one
> > week. It happens sometimes. I've already talked about my concerns on
> > assuming a certain level of available volunteer bandwidth. Let me just
> say
> > that things are great now, it's fantastic.
> >
> > We are pretty much on the honor system already. I don't buy the argument
> > we can't trust that CTR or CTR after one week can work because
> committers,
> > even if asked to customarily get a review before commit, may decide to
> > check in unreviewed untested destabilizing changes. At least, In that
> case
> > I'd argue we have a different problem.