Re: Release notes for 1.8.0 (was: [VOTE] Accumulo 1.8.0-rc2)

2016-08-19 Thread Michael Wall
I have started the release notes, I'll get a draft up today. Does it need to say draft somewhere? On Thu, Aug 18, 2016 at 3:59 PM, Josh Elser wrote: > Have you started this process, Mike Wall? > > I seem to recall you said you were making a list, but might not have > gotten to updating the webs

Re: Release notes for 1.8.0 (was: [VOTE] Accumulo 1.8.0-rc2)

2016-08-19 Thread Josh Elser
Wouldn't hurt to have that prominent. I would recommend not updating the nav bar links as well. We can access it directly which will cut back on search engines crawling it. Thanks Mike! Michael Wall wrote: I have started the release notes, I'll get a draft up today. Does it need to say draft

Re: Release notes for 1.8.0 (was: [VOTE] Accumulo 1.8.0-rc2)

2016-08-19 Thread Michael Wall
Cool, thanks. I was considering only pushing it to the gh-pages branch until the final release to further reduce public consumption. That would mean the link in the email wouldn't work, instead it would be at http://apache.github.io/accumulo/release_notes/1.8.0. Any one have concerns with that?

Re: Release notes for 1.8.0 (was: [VOTE] Accumulo 1.8.0-rc2)

2016-08-19 Thread Josh Elser
That would probably also be fine. There's already a disclaimer banner on there that it's not official (thank you, Mister Tubbs). Michael Wall wrote: Cool, thanks. I was considering only pushing it to the gh-pages branch until the final release to further reduce public consumption. That would

Re: [VOTE] Accumulo 1.8.0-rc2

2016-08-19 Thread Michael Wall
This vote fails with three +1's (1 binding - 2 non binding) one +0 one -1 Based on the results of the Java 8 discussion started by Christopher, I will make new artifacts and start a new vote early next week. Mike On Thu, Aug 18, 2016 at 9:44 PM, Christopher wrote: > It looks like these got in

Re: [DISCUSS] Java 8

2016-08-19 Thread Christopher
Re-reading the old thread, I'm reminded that we actually can't bump without either disabling the modernizer plugin or making some minimal breaking changes in mock (which is already deprecated). That's not necessarily a problem for a major version bump, but it means that it wouldn't necessarily be a

BatchWriter Auth/Audit Question

2016-08-19 Thread Mike Drob
Devs, I was recently running a 1.7.2 cluster under a heavy write workload and noticed a _lot_ of audit messages showing up in the logs. After some digging, I found that one of the causes is the following call chain: TabletServerBatchWriter::sendMutationsToTabletServer calls TabletServer.ThriftCli

Re: BatchWriter Auth/Audit Question

2016-08-19 Thread Marc P.
Funny, exactly the same thing I mentioned to Josh last night. Are you watching us? In something I"m doing now I'm making the thrift calls and modeled it after what the client code does; however, once I removed the authenticateUser my throughput increased by 25 per cent. In my trace table it was b

Re: BatchWriter Auth/Audit Question

2016-08-19 Thread Marc P.
Sorryby out of band, It could be done once per writer or even in the connector once...but I think we're in agreement Drob, Mike. On Fri, Aug 19, 2016 at 2:30 PM, Marc P. wrote: > Funny, exactly the same thing I mentioned to Josh last night. Are you > watching us? > > In something I"m doing

Re: [VOTE] Accumulo 1.8.0-rc2

2016-08-19 Thread Mike Drob
Did this fix your count discrepancy? On Thu, Aug 18, 2016, 10:53 AM Michael Wall wrote: > Ok, the run-verify MR took over 5 hours. I think muchos needs some tweaks > in yarn settings. > > On Thu, Aug 18, 2016 at 11:50 AM, Josh Elser wrote: > > > Uhh, woah. That's a rather big discrepancy. Yes,

Re: [VOTE] Accumulo 1.8.0-rc2

2016-08-19 Thread Michael Wall
Did my tweaks fix the count? Short answer is not yet. After giving mappers and reducers 2048m, setting mapreduce.task.io.sort.mb to 1024 and reducing the number of mappers and reducers from 264 to 140, I reran with agitation yesterday. Josh had a good run without agitation and I had removed the

Re: BatchWriter Auth/Audit Question

2016-08-19 Thread Christopher
If you note the comment in the parent class, the implementation of canAskAboutUser is relying on the authentication being done in the call to canPerformSystemActions. If you reverse the order here, you lose that authentication check, and the action will be allowed simply if the user is equal to its

Re: BatchWriter Auth/Audit Question

2016-08-19 Thread Mike Drob
I tried out what Marc suggested, and after I removed the authenticateUser line from my tservers and my write throughput DOUBLED. This does not seem like a minor detail here. In the tserver logs, I still see a bunch of "2016-08-19 12:28:11,621 [Audit ] INFO : operation: permitted; user: mdrob; cl

Re: [DISCUSS] Java 8

2016-08-19 Thread Christopher
There seem to be differences of opinion about what action to take (or not take), but I think everybody understands the relevant concerns which have been expressed so far. I'm going to let this thread sit for the weekend so folks can think and comment some more, and then call a vote on Monday to dec

Re: BatchWriter Auth/Audit Question

2016-08-19 Thread Christopher
Correct. That code is not executed in the write path. It should only be executed for the APIs where a user (typically an admin) is explicitly checking another user's permissions/authorizations/etc. On Fri, Aug 19, 2016 at 3:59 PM Mike Drob wrote: > I tried out what Marc suggested, and after I re

Re: BatchWriter Auth/Audit Question

2016-08-19 Thread Christopher
Sorry, I meant, that it wouldn't be executed in the write path if you switch the order. The two credentials should always be the same in that case. On Fri, Aug 19, 2016 at 4:37 PM Christopher wrote: > Correct. That code is not executed in the write path. It should only be > executed for the APIs

Re: [VOTE] Accumulo 1.8.0-rc2

2016-08-19 Thread Josh Elser
24hr CI with agitation also just finished successfully for me: org.apache.accumulo.test.continuous.ContinuousVerify$Counts REFERENCED=6598144114 UNREFERENCED=2000263 Michael Wall wrote: Did my tweaks fix the count? Short answer is not yet. After giving

Re: BatchWriter Auth/Audit Question

2016-08-19 Thread Mike Drob
I got a bit confused here, so I hopped on IRC and tried to reason this out with Christopher. The highlights are thus: * The fail fast relies on canPerformSystemActions->authenticate->throw Exception when it is not a valid user. * If we want to switch the order attributes are checked in the canAskA

Re: BatchWriter Auth/Audit Question

2016-08-19 Thread Christopher
It may be the case that removing authenticateUser call is fine from a correctness perspective, but it's relying on a very tenuous fact that that the UpdateSession is initialized with a null KeyExtent for the currentTablet. The authentication only happens in the applyUpdate method is called with a K