> > If you want to setup a User for a series of requests to and from the Blur > cluster, use UserContext.setUser(...) and UserContext.reset() and the > calling thread. Between those 2 calls all requests to the controllers (and > shard servers) will be performed as that User.
I am not sure I understood your explanation. Every call for our mutation/query holds a different User and calls happen in parallel.. In other words, Every request is allocated a thread in controller & shard-servers. This thread must unmarshall the User object and set it in it's ThreadLocal. Is this current behavior of blur? -- Ravi On Fri, Apr 25, 2014 at 5:20 PM, Aaron McCurry <[email protected]> wrote: > If you want to setup a User for a series of requests to and from the Blur > cluster, use UserContext.setUser(...) and UserContext.reset() and the > calling thread. Between those 2 calls all requests to the controllers (and > shard servers) will be performed as that User. > > Also something else that's new is the trace function. > > To see it in action fire up the blur shell and run the trace command > followed by any other command like a query command (in the shell). Then go > a status page on a Blur process (http://<hostname>:40090 for shard servers > 40080 for controllers). Click on the trace tab and see what the trace call > actually did. It's a nice feature for debugging performance problems. > > Aaron > > > On Fri, Apr 25, 2014 at 7:20 AM, Ravikumar Govindarajan < > [email protected]> wrote: > > > Thanks Aaron for the clarification... Will try as you suggest. > > > > BTW, > > > > I find a setUser() method in Iface class [thrift interface] > > > > We need this "User" object passed from BlurClient to ControllerServer and > > then to ShardServer for every incoming query. > > > > Will this happen automatically in current code via thrift > > marshal/unmarshal? > > > > -- > > Ravi > > > > > > On Wed, Apr 23, 2014 at 4:34 PM, Aaron McCurry <[email protected]> > wrote: > > > > > On Wed, Apr 23, 2014 at 1:53 AM, Ravikumar Govindarajan < > > > [email protected]> wrote: > > > > > > > Thanks Aaron, will definitely look at moving to 0.2.2 version... > > > > > > > > There is one problem when performing commit on every mutate call, > > which I > > > > need your help in understanding > > > > > > > > Our NRT reader re-opens every 10 seconds... Approx... 10 seconds > worth > > of > > > > data is buffered in IndexWriter RAM and then flushed to disk. With > > 0.2.2, > > > > every mutate is auto flushed to disk because of commit... This will > > > create > > > > a lot of tiny segments and problem gets exacerbated when > MergeScheduler > > > > starts accumulating backlog... > > > > > > > > In the case of deletes/updates, the calling thread previously will > > return > > > > immediately, adding the operation to IW RAM-Buffer. A background NRT > > > thread > > > > will apply these deletes. But now, the calling thread is stalled > until > > > the > > > > delete/update is successfully performed... > > > > > > > > One option is to accumulate change-sets and push it to Blur for > every 5 > > > > seconds etc..., instead of pushing immediately to blur... > > > > > > > > > > There is a call in 0.2.2 that does this by default (or close). It's > > called > > > enqueueMutate. Feature wise it gives you NRT updates with a buffer in > > > front of the writer. It delays the visibility of data but allows for a > > > much higher volume of mutates without blocking. > > > > > > Aaron > > > > > > > > > > > > > > -- > > > > Ravi > > > > > > > > > > > > On Tue, Apr 22, 2014 at 10:25 PM, Aaron McCurry <[email protected]> > > > > wrote: > > > > > > > > > I think that you are using 0.2.0. The DirectoryReferenceCounter > was > > a > > > > > flawed piece of code and has been completely removed in 0.2.2. A > > much > > > > > simpler and more reliable piece of code now exists that handles the > > > same > > > > > function. I can point you to the new way it's all implemented if > you > > > > would > > > > > like. However I would suggest that you not use 0.2.0 because of > > these > > > > > issues as well as many other stability problems. > > > > > > > > > > Aaron > > > > > > > > > > > > > > > On Tue, Apr 22, 2014 at 4:21 AM, Ravikumar Govindarajan < > > > > > [email protected]> wrote: > > > > > > > > > > > DirectoryReferenceCounter wraps an IndexInput with an > AtomicInteger > > > for > > > > > > ref-count. > > > > > > > > > > > > I believe this file exists, because lucene does not correctly > > handle > > > > > > clones. Is this right? > > > > > > > > > > > > Rarely for me, an ongoing search hits FileNotFoundException > > because a > > > > > > just-now completed merge has deleted the file... > > > > > > > > > > > > I am yet to pinpoint the problem, as it is quite difficult to > > > reproduce > > > > > > it... > > > > > > > > > > > > I tried a sysout in deleteFile() method and was quite surprised > to > > > find > > > > > > that refcounts are in negative for some-files.... Is this > normally > > > > > expected > > > > > > behavior? > > > > > > > > > > > > 14/04/22 13:32:35 INFO refcounter.DirectoryReferenceCounter: > Delete > > > > file > > > > > > [_2y_Lucene41_0.tip] with count [-14] > > > > > > > > > > > > 14/04/22 13:32:35 INFO refcounter.DirectoryReferenceCounter: > Delete > > > > file > > > > > [_ > > > > > > 2s.si] with count [0] > > > > > > > > > > > > 14/04/22 13:32:35 INFO refcounter.DirectoryReferenceCounter: > Delete > > > > file > > > > > > [segments_4] with count [0] > > > > > > > > > > > > 14/04/22 13:32:35 INFO refcounter.DirectoryReferenceCounter: > Delete > > > > file > > > > > > [_1z_Lucene41_0.doc] with count [-12] > > > > > > > > > > > > 14/04/22 13:32:35 INFO refcounter.DirectoryReferenceCounter: > Delete > > > > file > > > > > [_ > > > > > > 2y.si] with count [0] > > > > > > > > > > > > 14/04/22 13:32:35 INFO refcounter.DirectoryReferenceCounter: > Delete > > > > file > > > > > > [_2t.fnm] with count [-1] > > > > > > > > > > > > 14/04/22 13:32:35 INFO refcounter.DirectoryReferenceCounter: > Delete > > > > file > > > > > > [_24_Lucene41_0.tip] with count [-14] > > > > > > > > > > > > 14/04/22 13:32:35 INFO refcounter.DirectoryReferenceCounter: > Delete > > > > file > > > > > > [_24_Lucene41_0.tim] with count [-12] > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > Ravi > > > > > > > > > > > > > > > > > > > > >
