If you want to setup a User for a series of requests to and from the Blur cluster, use UserContext.setUser(...) and UserContext.reset() and the calling thread. Between those 2 calls all requests to the controllers (and shard servers) will be performed as that User.
Also something else that's new is the trace function. To see it in action fire up the blur shell and run the trace command followed by any other command like a query command (in the shell). Then go a status page on a Blur process (http://<hostname>:40090 for shard servers 40080 for controllers). Click on the trace tab and see what the trace call actually did. It's a nice feature for debugging performance problems. Aaron On Fri, Apr 25, 2014 at 7:20 AM, Ravikumar Govindarajan < [email protected]> wrote: > Thanks Aaron for the clarification... Will try as you suggest. > > BTW, > > I find a setUser() method in Iface class [thrift interface] > > We need this "User" object passed from BlurClient to ControllerServer and > then to ShardServer for every incoming query. > > Will this happen automatically in current code via thrift > marshal/unmarshal? > > -- > Ravi > > > On Wed, Apr 23, 2014 at 4:34 PM, Aaron McCurry <[email protected]> wrote: > > > On Wed, Apr 23, 2014 at 1:53 AM, Ravikumar Govindarajan < > > [email protected]> wrote: > > > > > Thanks Aaron, will definitely look at moving to 0.2.2 version... > > > > > > There is one problem when performing commit on every mutate call, > which I > > > need your help in understanding > > > > > > Our NRT reader re-opens every 10 seconds... Approx... 10 seconds worth > of > > > data is buffered in IndexWriter RAM and then flushed to disk. With > 0.2.2, > > > every mutate is auto flushed to disk because of commit... This will > > create > > > a lot of tiny segments and problem gets exacerbated when MergeScheduler > > > starts accumulating backlog... > > > > > > In the case of deletes/updates, the calling thread previously will > return > > > immediately, adding the operation to IW RAM-Buffer. A background NRT > > thread > > > will apply these deletes. But now, the calling thread is stalled until > > the > > > delete/update is successfully performed... > > > > > > One option is to accumulate change-sets and push it to Blur for every 5 > > > seconds etc..., instead of pushing immediately to blur... > > > > > > > There is a call in 0.2.2 that does this by default (or close). It's > called > > enqueueMutate. Feature wise it gives you NRT updates with a buffer in > > front of the writer. It delays the visibility of data but allows for a > > much higher volume of mutates without blocking. > > > > Aaron > > > > > > > > > > -- > > > Ravi > > > > > > > > > On Tue, Apr 22, 2014 at 10:25 PM, Aaron McCurry <[email protected]> > > > wrote: > > > > > > > I think that you are using 0.2.0. The DirectoryReferenceCounter was > a > > > > flawed piece of code and has been completely removed in 0.2.2. A > much > > > > simpler and more reliable piece of code now exists that handles the > > same > > > > function. I can point you to the new way it's all implemented if you > > > would > > > > like. However I would suggest that you not use 0.2.0 because of > these > > > > issues as well as many other stability problems. > > > > > > > > Aaron > > > > > > > > > > > > On Tue, Apr 22, 2014 at 4:21 AM, Ravikumar Govindarajan < > > > > [email protected]> wrote: > > > > > > > > > DirectoryReferenceCounter wraps an IndexInput with an AtomicInteger > > for > > > > > ref-count. > > > > > > > > > > I believe this file exists, because lucene does not correctly > handle > > > > > clones. Is this right? > > > > > > > > > > Rarely for me, an ongoing search hits FileNotFoundException > because a > > > > > just-now completed merge has deleted the file... > > > > > > > > > > I am yet to pinpoint the problem, as it is quite difficult to > > reproduce > > > > > it... > > > > > > > > > > I tried a sysout in deleteFile() method and was quite surprised to > > find > > > > > that refcounts are in negative for some-files.... Is this normally > > > > expected > > > > > behavior? > > > > > > > > > > 14/04/22 13:32:35 INFO refcounter.DirectoryReferenceCounter: Delete > > > file > > > > > [_2y_Lucene41_0.tip] with count [-14] > > > > > > > > > > 14/04/22 13:32:35 INFO refcounter.DirectoryReferenceCounter: Delete > > > file > > > > [_ > > > > > 2s.si] with count [0] > > > > > > > > > > 14/04/22 13:32:35 INFO refcounter.DirectoryReferenceCounter: Delete > > > file > > > > > [segments_4] with count [0] > > > > > > > > > > 14/04/22 13:32:35 INFO refcounter.DirectoryReferenceCounter: Delete > > > file > > > > > [_1z_Lucene41_0.doc] with count [-12] > > > > > > > > > > 14/04/22 13:32:35 INFO refcounter.DirectoryReferenceCounter: Delete > > > file > > > > [_ > > > > > 2y.si] with count [0] > > > > > > > > > > 14/04/22 13:32:35 INFO refcounter.DirectoryReferenceCounter: Delete > > > file > > > > > [_2t.fnm] with count [-1] > > > > > > > > > > 14/04/22 13:32:35 INFO refcounter.DirectoryReferenceCounter: Delete > > > file > > > > > [_24_Lucene41_0.tip] with count [-14] > > > > > > > > > > 14/04/22 13:32:35 INFO refcounter.DirectoryReferenceCounter: Delete > > > file > > > > > [_24_Lucene41_0.tim] with count [-12] > > > > > > > > > > > > > > > -- > > > > > > > > > > Ravi > > > > > > > > > > > > > > >
