Gary has provided nice summary of things to watch out for.
One more thing I want to mention is that care should be taken w.r.t.
coordinating the progress of the thread pool and normal region operations.
There're already many threads running in the region server JVM. Adding one
more thread pool may make resource consumption more complex.
What if the custom processing cannot keep up with the rate at which
asynchronous requests are queued ?
w.r.t. thread pool, you can refer to the following code in HRegion:
ThreadPoolExecutor storeOpenerThreadPool =
getStoreOpenAndCloseThreadPool(
"StoreOpenerThread-" + this.getRegionNameAsString());
CompletionService<HStore> completionService =
new ExecutorCompletionService<HStore>(storeOpenerThreadPool);
Cheers
On Tue, Apr 9, 2013 at 11:57 AM, Gary Helmling <ghelml...@gmail.com> wrote:
> Hi Rami,
>
> One thing to note for RegionObservers, is that each table region gets its
> own instance of each configured coprocessor. So if your cluster has N
> regions per region server, with your RegionObserver loaded on all tables,
> then each region server will have N instances of your coprocessor. You
> should just be aware of this in case you, say, create a thread pool in your
> coprocessor constructor. An alternative in this case is to use a singleton
> class per region server (aka per jvm) to manage the resources.
>
> You do want to be sure that all threads are daemon threads, so that they
> don't block region server shutdown. Or else you'll need to ensure you
> properly stop/join all the threads you've spawned on shutdown.
> RegionServerObserver.preStopRegionServer() may help there.
>
> --gh
>
>
>
> On Tue, Apr 9, 2013 at 11:40 AM, Ted Yu <yuzhih...@gmail.com> wrote:
>
> > Rami:
> > Can you tell us what coprocessor hook you plan to use ?
> >
> > Thanks
> >
> > On Tue, Apr 9, 2013 at 10:51 AM, Rami Mankevich <ra...@amdocs.com>
> wrote:
> >
> > > First of all - thanks for the quick response.
> > >
> > > Basically threads I want to open are for my own internal structure
> > > updates and I guess have no relations to HBase internal structures.
> > > All I want is initiations for some asynchronous structure updates as
> part
> > > of coprocessor execution in order not to block user reponse.
> > >
> > > The only reason I was asking is to be sure Hbase will not kill those
> > > threads.
> > > As I understand - shouldn't be any issue with that. Am I correct?
> > >
> > > In addition - Is there any Hbase Thread pool I can use?
> > >
> > >
> > > Thanks
> > > From: Andrew Purtell [mailto:apurt...@apache.org]
> > > Sent: Tuesday, April 09, 2013 6:53 PM
> > > To: Rami Mankevich
> > > Cc: apurt...@apache.org
> > > Subject: Re: Hbase question
> > >
> > > Hi Rami,
> > >
> > > It is no problem to create threads in a coprocessor as a generic
> answer.
> > > More specifically there could be issues depending on exactly what you
> > want
> > > to do, since coprocessor code changes HBase internals. Perhaps you
> could
> > > say a bit more. I also encourage you to ask this question on
> > > user@hbase.apache.org<mailto:user@hbase.apache.org> so other
> > contributors
> > > can chime in too.
> > >
> > > On Tuesday, April 9, 2013, Rami Mankevich wrote:
> > > Hey
> > > According to the Hbase documentation you are one of contrinuters to the
> > > HBase project
> > > I would like to raise some question when nobody can basically advice
> me:
> > >
> > > In context of coprocessors I want to raise some threads.
> > > Do you see any problems with that?
> > >
> > > Thanks
> > > This message and the information contained herein is proprietary and
> > > confidential and subject to the Amdocs policy statement, you may review
> > at
> > > http://www.amdocs.com/email_disclaimer.asp
> > >
> > >
> > > --
> > > Best regards,
> > >
> > > - Andy
> > >
> > > Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> > > (via Tom White)
> > >
> >
>
