Nominating http://gerrit.cloudera.org:8080/#/c/2702 to be backported. Updates schema design guide for latest partition pruning changes.
- Dan On Thu, Mar 31, 2016 at 4:56 PM, Jean-Daniel Cryans <[email protected]> wrote: > On Thu, Mar 31, 2016 at 4:54 PM, Dan Burkert <[email protected]> wrote: > > > Nominating http://gerrit.cloudera.org:8080/#/c/2692/ to be backported. > > Fixes a build issue on the latest OS X Xcode version. Low risk. > > > > +1 > > > > > > - Dan > > > > On Tue, Mar 29, 2016 at 9:48 AM, Jean-Daniel Cryans <[email protected] > > > > wrote: > > > > > I wrote a first draft of the release notes, am about to post. > > > > > > On Tue, Mar 29, 2016 at 9:33 AM, Misty Stanley-Jones < > > > [email protected]> wrote: > > > > > > > I created https://issues.apache.org/jira/browse/KUDU-1382 for the > > > release > > > > notes. JD, please add it to the blocker list, and @all, please add > any > > > > issues I've missed or let me know if we have nixed any. I'll try to > > get a > > > > first draft of release notes and upgrade instructions up by tomorrow > > > > afternoon. > > > > > > > > On Mon, Mar 28, 2016 at 12:03 PM, Mike Percy <[email protected]> > > wrote: > > > > > > > > > On Mon, Mar 28, 2016 at 11:12 AM, Jean-Daniel Cryans < > > > > [email protected]> > > > > > wrote: > > > > > > > > > > > Hey Mike, > > > > > > > > > > > > If you fix 1380, I'll let you get 1377 in :) > > > > > > > > > > > > > > > > Sweet. > > > > > > > > > > More seriously, when do think 1377 can get in? Also, looks like > Adar > > > > > > already took a look, but do you need more eyes on it? What about > > > > cluster > > > > > > testing, are the unit tests enough? > > > > > > > > > > > > > > > > I spoke w/ Adar about this bug fix on Slack, as well as on the WIP > > > patch > > > > on > > > > > Gerrit @ http://gerrit.cloudera.org/#/c/2595/. What we think needs > > to > > > > > happen there is we truncate the metadata file if the last record is > > > > corrupt > > > > > *and* if it is the last record in the metadata file. > > > > > > > > > > I think the patch will be fine as-is, assuming it gets reviewed and > > has > > > > > tests. However it exposes a hole in our testing, since we don't > > > currently > > > > > simulate failing disks or full disks. But I think that's something > > that > > > > > results in relatively rare bugs such as this one and doesn't > require > > a > > > > > solution for this patch. The reason I'd like to get this in is that > > we > > > > had > > > > > one user hit this issue mutliple times (due to full disks) and I'd > > like > > > > to > > > > > remove that pain point for them. > > > > > > > > > > For Flume, I'm not an expert so can't comment on the patch, but do > > you > > > > > > think what's there offers the right interfaces/idioms? Also, > should > > > we > > > > > mark > > > > > > it as "experimental"? > > > > > > > > > > > > > > > > I think it's starting to shape up, but yeah we could mark it as > > > > > experimental for the first release. It seems to be starting to > shape > > > up. > > > > It > > > > > would be even better if we had UPSERT support, since right now if > > there > > > > is > > > > > a duplicate key error (possibly due to a retry on an insert) we > plan > > to > > > > > just drop the event (this will be configurable). > > > > > > > > > > Mike > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > J-D > > > > > > > > > > > > On Mon, Mar 28, 2016 at 11:01 AM, Mike Percy <[email protected]> > > > > wrote: > > > > > > > > > > > > > Hey JD, > > > > > > > I'll start taking a look at KUDU-1380. I'd also like to get the > > > Flume > > > > > > sink > > > > > > > committed (it's under review but almost there) and get > KUDU-1377 > > > into > > > > > 0.8 > > > > > > > as well (I almost have a fix ready but it's not quite done). > > > > > > > > > > > > > > Mike > > > > > > > > > > > > > > On Mon, Mar 28, 2016 at 10:31 AM, Jean-Daniel Cryans < > > > > > > [email protected]> > > > > > > > wrote: > > > > > > > > > > > > > > > Hey Todd, > > > > > > > > > > > > > > > > Thanks for all the testing! > > > > > > > > > > > > > > > > @Dan B, you handling KUDU-1379? > > > > > > > > > > > > > > > > @Mike P, since you were in that part of the code recently, > can > > > you > > > > > make > > > > > > > > some time for KUDU-1380? > > > > > > > > > > > > > > > > And regarding the list of things I said I wanted to pull in, > > > > > partition > > > > > > > > pruning made it in and the Java client's new KuduPredicate > > looks > > > > > almost > > > > > > > > ready. The Get API didn't see any action since the 11th, so > I'm > > > > > > inclined > > > > > > > to > > > > > > > > drop it from 0.8.0. > > > > > > > > > > > > > > > > I'm writing the release notes, should be up soon. > > > > > > > > > > > > > > > > J-D > > > > > > > > > > > > > > > > On Mon, Mar 28, 2016 at 8:57 AM, Todd Lipcon < > > [email protected]> > > > > > > wrote: > > > > > > > > > > > > > > > > > I discovered a couple bad issues over the weekend that I > > think > > > we > > > > > > > should > > > > > > > > > prioritize to get in for 0.8: > > > > > > > > > > > > > > > > > > - KUDU-1379 - this is a regression blocker where column > range > > > > > > > predicates > > > > > > > > on > > > > > > > > > float/double/bool columns no longer work. We should either > > fix > > > > this > > > > > > or > > > > > > > > > revert the predicate changes that Dan committed last week > > > > > (preferably > > > > > > > > just > > > > > > > > > fix it, since I dont think it's too hard) > > > > > > > > > - KUDU-1380 - this is not a regression but it's a case > where > > we > > > > can > > > > > > get > > > > > > > > > silently wrong results (duplicated rows) from the C++ > > scanner. > > > > > Since > > > > > > > > it's a > > > > > > > > > correctness issue I marked it as blocker. Does anyone have > > time > > > > to > > > > > > look > > > > > > > > at > > > > > > > > > this today/tomorrow? If not I will try to take a look. > > > > > > > > > > > > > > > > > > -Todd > > > > > > > > > > > > > > > > > > On Fri, Mar 25, 2016 at 2:12 PM, Jean-Daniel Cryans < > > > > > > > [email protected] > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > I just went through the Blocker and Critical jiras > targeted > > > for > > > > > > 0.8.0 > > > > > > > > and > > > > > > > > > > moved a bunch that I knew wouldn't get done in time. > Once I > > > > > branch > > > > > > > I'll > > > > > > > > > > bulk move the rest. In most cases it's fine to bring them > > > back > > > > > into > > > > > > > > > 0.8.0, > > > > > > > > > > as long as it can be done by midweek next week. In doubt, > > > poke > > > > me > > > > > > in > > > > > > > > the > > > > > > > > > > jira. > > > > > > > > > > > > > > > > > > > > J-D > > > > > > > > > > > > > > > > > > > > On Fri, Mar 25, 2016 at 8:29 AM, Jean-Daniel Cryans < > > > > > > > > [email protected] > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > Hey devs, > > > > > > > > > > > > > > > > > > > > > > It's almost time for 0.8.0, I'd like to get a first RC > > out > > > > in a > > > > > > > week > > > > > > > > on > > > > > > > > > > > April 1st (but more realistically it might happen the > > > 4th). I > > > > > > will > > > > > > > > > create > > > > > > > > > > > the branch on March 30th. > > > > > > > > > > > > > > > > > > > > > > The release note-worthy items that have been committed > > > since > > > > > > 0.7.0 > > > > > > > > that > > > > > > > > > > > aren't in 0.7.1: > > > > > > > > > > > - RPC and application feature flags > > > > > > > > > > > - KUDU-1337 Avoid spurious remote bootstraps on > > > > DeleteTablet() > > > > > > > > > > > - Fixed the Spark shutdown bug > > > > > > > > > > > - KUDU-1322 failover writes on tablet not found in the > > C++ > > > > > > client > > > > > > > > > > > - KUDU-969. Fix handling of crashes just before tablet > > > > > metadata > > > > > > > > flush > > > > > > > > > > > - KUDU-1354. Writes should not release locks before > > > > committing > > > > > > > MVCC > > > > > > > > > > > - master: don't expose intermediate CreateTable() > state > > to > > > > > > > consumers > > > > > > > > > > > > > > > > > > > > > > The things that are still in flight that I really want > to > > > get > > > > > in: > > > > > > > > > > > - Partition pruning in the C++ client and improvements > > on > > > > the > > > > > > > > > > > server-side. http://gerrit.cloudera.org:8080/#/c/2413/ > > > > > > > > > > > - [java-client] implement ColumnPredicate API > > > > > > > > > > > http://gerrit.cloudera.org:8080/#/c/2591/ > > > > > > > > > > > - KUDU-1235. Add Get API > > > > > > > http://gerrit.cloudera.org:8080/#/c/2519/ > > > > > > > > > > > > > > > > > > > > > > Anything else that comes in is gravy, but I think the > > scan > > > > > token > > > > > > > API > > > > > > > > > > > integration in the Java client needs more thinking. I > > think > > > > Dan > > > > > > is > > > > > > > > > > supposed > > > > > > > > > > > to send an email to dev@ about it. > > > > > > > > > > > > > > > > > > > > > > What do you devs think? > > > > > > > > > > > > > > > > > > > > > > J-D > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > Todd Lipcon > > > > > > > > > Software Engineer, Cloudera > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
