Re: What if I want to write another table in MasterObserver.preModifyColumnFamily ?
Zookeeper is supposed to store transient data.
Plus you need to secure the audit log in zookeeper so that unauthorized
user cannot temper the audit.
On Thu, Mar 9, 2017 at 9:24 AM, Xi Yang wrote:
> Thank you Anoop. After thinking, I give up the idea about write records to
> RS, I will using other ways like zk, or just a text file. I appreciate your
> help!
>
> 2017-03-07 23:36 GMT-08:00 Anoop John :
>
> > It will be cross server call.. The pre/postModifyColumn is in master
> > side. (MasterObserver) You want to write the info to a table means
> > this will be in some RS. So an RPC request will be needed. But still
> > it is not a case like one RS to another RS cross server call where
> > there is a remote chance of all handlers getting stuck and possible
> > deadlock. Need to carefully done though!
> >
> > -Anoop-
> >
> > On Wed, Mar 8, 2017 at 8:26 AM, Xi Yang wrote:
> > > Got it. I appreciate your help!
> > >
> > > 2017-03-07 17:05 GMT-08:00 Ted Yu :
> > >
> > >> It seems the following hook is better for your use case:
> > >>
> > >> default void postModifyColumn(final
> > >> ObserverContext ctx,
> > >> TableName tableName, HColumnDescriptor columnFamily) throws
> > >> IOException {}
> > >>
> > >> since there is no guarantee that column family is modified at
> > >> time preModifyColumnFamily() is called.
> > >>
> > >> Cheers
> > >>
> > >> On Tue, Mar 7, 2017 at 4:43 PM, Xi Yang
> wrote:
> > >>
> > >> > Requirement:
> > >> >
> > >> > I want to record every change of modify columnFamily by using
> > >> > preModifyColumnFamily().
> > >> > Now I have a table "my_ddl_log" which used to record the change of
> > >> > columnFamily. For example:
> > >> >
> > >> > If jack change the TTL of columnFamily "primary" in table
> "employee".
> > >> Then
> > >> > we should add a put to "my_ddl_log" like this record:
> > >> > log:name= 'jack'
> > >> > log:updateTime= '2017-03-07 12:12 GMT-08:00'
> > >> > log:change= ''Change TTL of Table: employee ColumnFamily: primary'
> > >> >
> > >> > I try to use preModifyColumnFamily to do this stuff.
> > >> >
> > >> > Thanks,
> > >> > Alex
> > >> >
> > >> >
> > >> > 2017-03-07 12:12 GMT-08:00 Ted Yu :
> > >> >
> > >> > > Describing your use case would allow people to give better answer.
> > >> > >
> > >> > > What kind of data do you write to other table in
> > >> preModifyColumnFamily()
> > >> > ?
> > >> > >
> > >> > > Cross server call within observer is not good idea.
> > >> > >
> > >> > > Take a look at ConnectionUtils.createShortCircuitConnection().
> > >> > >
> > >> > > Cheers
> > >> > >
> > >> > > On Tue, Mar 7, 2017 at 11:42 AM, Xi Yang
> > >> wrote:
> > >> > >
> > >> > > > All the articles I've ever seen are talking about add
> > increment
> > >> or
> > >> > > > change put/get status or pinrt out logs. what if I want to write
> > some
> > >> > > data
> > >> > > > to another table in Observer? For
> > >> > > > example, MasterObserver.preModifyColumnFamily()? Seems Observer
> > is
> > >> > > runing
> > >> > > > at server side, so use connection is unneccessary and might
> raise
> > >> some
> > >> > > > problem.
> > >> > > >I know this might be a stupid question, so if you can just
> give
> > >> some
> > >> > > > links let me to learn without explain in email, I will be
> > grateful to
> > >> > you
> > >> > > > for your help
> > >> > > >
> > >> > > > Thanks,
> > >> > > > Alex
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
>
Re: What if I want to write another table in MasterObserver.preModifyColumnFamily ?
Thank you Anoop. After thinking, I give up the idea about write records to
RS, I will using other ways like zk, or just a text file. I appreciate your
help!
2017-03-07 23:36 GMT-08:00 Anoop John :
> It will be cross server call.. The pre/postModifyColumn is in master
> side. (MasterObserver) You want to write the info to a table means
> this will be in some RS. So an RPC request will be needed. But still
> it is not a case like one RS to another RS cross server call where
> there is a remote chance of all handlers getting stuck and possible
> deadlock. Need to carefully done though!
>
> -Anoop-
>
> On Wed, Mar 8, 2017 at 8:26 AM, Xi Yang wrote:
> > Got it. I appreciate your help!
> >
> > 2017-03-07 17:05 GMT-08:00 Ted Yu :
> >
> >> It seems the following hook is better for your use case:
> >>
> >> default void postModifyColumn(final
> >> ObserverContext ctx,
> >> TableName tableName, HColumnDescriptor columnFamily) throws
> >> IOException {}
> >>
> >> since there is no guarantee that column family is modified at
> >> time preModifyColumnFamily() is called.
> >>
> >> Cheers
> >>
> >> On Tue, Mar 7, 2017 at 4:43 PM, Xi Yang wrote:
> >>
> >> > Requirement:
> >> >
> >> > I want to record every change of modify columnFamily by using
> >> > preModifyColumnFamily().
> >> > Now I have a table "my_ddl_log" which used to record the change of
> >> > columnFamily. For example:
> >> >
> >> > If jack change the TTL of columnFamily "primary" in table "employee".
> >> Then
> >> > we should add a put to "my_ddl_log" like this record:
> >> > log:name= 'jack'
> >> > log:updateTime= '2017-03-07 12:12 GMT-08:00'
> >> > log:change= ''Change TTL of Table: employee ColumnFamily: primary'
> >> >
> >> > I try to use preModifyColumnFamily to do this stuff.
> >> >
> >> > Thanks,
> >> > Alex
> >> >
> >> >
> >> > 2017-03-07 12:12 GMT-08:00 Ted Yu :
> >> >
> >> > > Describing your use case would allow people to give better answer.
> >> > >
> >> > > What kind of data do you write to other table in
> >> preModifyColumnFamily()
> >> > ?
> >> > >
> >> > > Cross server call within observer is not good idea.
> >> > >
> >> > > Take a look at ConnectionUtils.createShortCircuitConnection().
> >> > >
> >> > > Cheers
> >> > >
> >> > > On Tue, Mar 7, 2017 at 11:42 AM, Xi Yang
> >> wrote:
> >> > >
> >> > > > All the articles I've ever seen are talking about add
> increment
> >> or
> >> > > > change put/get status or pinrt out logs. what if I want to write
> some
> >> > > data
> >> > > > to another table in Observer? For
> >> > > > example, MasterObserver.preModifyColumnFamily()? Seems Observer
> is
> >> > > runing
> >> > > > at server side, so use connection is unneccessary and might raise
> >> some
> >> > > > problem.
> >> > > >I know this might be a stupid question, so if you can just give
> >> some
> >> > > > links let me to learn without explain in email, I will be
> grateful to
> >> > you
> >> > > > for your help
> >> > > >
> >> > > > Thanks,
> >> > > > Alex
> >> > > >
> >> > >
> >> >
> >>
>
Re: What if I want to write another table in MasterObserver.preModifyColumnFamily ?
It will be cross server call.. The pre/postModifyColumn is in master
side. (MasterObserver) You want to write the info to a table means
this will be in some RS. So an RPC request will be needed. But still
it is not a case like one RS to another RS cross server call where
there is a remote chance of all handlers getting stuck and possible
deadlock. Need to carefully done though!
-Anoop-
On Wed, Mar 8, 2017 at 8:26 AM, Xi Yang wrote:
> Got it. I appreciate your help!
>
> 2017-03-07 17:05 GMT-08:00 Ted Yu :
>
>> It seems the following hook is better for your use case:
>>
>> default void postModifyColumn(final
>> ObserverContext ctx,
>> TableName tableName, HColumnDescriptor columnFamily) throws
>> IOException {}
>>
>> since there is no guarantee that column family is modified at
>> time preModifyColumnFamily() is called.
>>
>> Cheers
>>
>> On Tue, Mar 7, 2017 at 4:43 PM, Xi Yang wrote:
>>
>> > Requirement:
>> >
>> > I want to record every change of modify columnFamily by using
>> > preModifyColumnFamily().
>> > Now I have a table "my_ddl_log" which used to record the change of
>> > columnFamily. For example:
>> >
>> > If jack change the TTL of columnFamily "primary" in table "employee".
>> Then
>> > we should add a put to "my_ddl_log" like this record:
>> > log:name= 'jack'
>> > log:updateTime= '2017-03-07 12:12 GMT-08:00'
>> > log:change= ''Change TTL of Table: employee ColumnFamily: primary'
>> >
>> > I try to use preModifyColumnFamily to do this stuff.
>> >
>> > Thanks,
>> > Alex
>> >
>> >
>> > 2017-03-07 12:12 GMT-08:00 Ted Yu :
>> >
>> > > Describing your use case would allow people to give better answer.
>> > >
>> > > What kind of data do you write to other table in
>> preModifyColumnFamily()
>> > ?
>> > >
>> > > Cross server call within observer is not good idea.
>> > >
>> > > Take a look at ConnectionUtils.createShortCircuitConnection().
>> > >
>> > > Cheers
>> > >
>> > > On Tue, Mar 7, 2017 at 11:42 AM, Xi Yang
>> wrote:
>> > >
>> > > > All the articles I've ever seen are talking about add increment
>> or
>> > > > change put/get status or pinrt out logs. what if I want to write some
>> > > data
>> > > > to another table in Observer? For
>> > > > example, MasterObserver.preModifyColumnFamily()? Seems Observer is
>> > > runing
>> > > > at server side, so use connection is unneccessary and might raise
>> some
>> > > > problem.
>> > > >I know this might be a stupid question, so if you can just give
>> some
>> > > > links let me to learn without explain in email, I will be grateful to
>> > you
>> > > > for your help
>> > > >
>> > > > Thanks,
>> > > > Alex
>> > > >
>> > >
>> >
>>
Re: What if I want to write another table in MasterObserver.preModifyColumnFamily ?
Got it. I appreciate your help!
2017-03-07 17:05 GMT-08:00 Ted Yu :
> It seems the following hook is better for your use case:
>
> default void postModifyColumn(final
> ObserverContext ctx,
> TableName tableName, HColumnDescriptor columnFamily) throws
> IOException {}
>
> since there is no guarantee that column family is modified at
> time preModifyColumnFamily() is called.
>
> Cheers
>
> On Tue, Mar 7, 2017 at 4:43 PM, Xi Yang wrote:
>
> > Requirement:
> >
> > I want to record every change of modify columnFamily by using
> > preModifyColumnFamily().
> > Now I have a table "my_ddl_log" which used to record the change of
> > columnFamily. For example:
> >
> > If jack change the TTL of columnFamily "primary" in table "employee".
> Then
> > we should add a put to "my_ddl_log" like this record:
> > log:name= 'jack'
> > log:updateTime= '2017-03-07 12:12 GMT-08:00'
> > log:change= ''Change TTL of Table: employee ColumnFamily: primary'
> >
> > I try to use preModifyColumnFamily to do this stuff.
> >
> > Thanks,
> > Alex
> >
> >
> > 2017-03-07 12:12 GMT-08:00 Ted Yu :
> >
> > > Describing your use case would allow people to give better answer.
> > >
> > > What kind of data do you write to other table in
> preModifyColumnFamily()
> > ?
> > >
> > > Cross server call within observer is not good idea.
> > >
> > > Take a look at ConnectionUtils.createShortCircuitConnection().
> > >
> > > Cheers
> > >
> > > On Tue, Mar 7, 2017 at 11:42 AM, Xi Yang
> wrote:
> > >
> > > > All the articles I've ever seen are talking about add increment
> or
> > > > change put/get status or pinrt out logs. what if I want to write some
> > > data
> > > > to another table in Observer? For
> > > > example, MasterObserver.preModifyColumnFamily()? Seems Observer is
> > > runing
> > > > at server side, so use connection is unneccessary and might raise
> some
> > > > problem.
> > > >I know this might be a stupid question, so if you can just give
> some
> > > > links let me to learn without explain in email, I will be grateful to
> > you
> > > > for your help
> > > >
> > > > Thanks,
> > > > Alex
> > > >
> > >
> >
>
Re: What if I want to write another table in MasterObserver.preModifyColumnFamily ?
It seems the following hook is better for your use case:
default void postModifyColumn(final
ObserverContext ctx,
TableName tableName, HColumnDescriptor columnFamily) throws
IOException {}
since there is no guarantee that column family is modified at
time preModifyColumnFamily() is called.
Cheers
On Tue, Mar 7, 2017 at 4:43 PM, Xi Yang wrote:
> Requirement:
>
> I want to record every change of modify columnFamily by using
> preModifyColumnFamily().
> Now I have a table "my_ddl_log" which used to record the change of
> columnFamily. For example:
>
> If jack change the TTL of columnFamily "primary" in table "employee". Then
> we should add a put to "my_ddl_log" like this record:
> log:name= 'jack'
> log:updateTime= '2017-03-07 12:12 GMT-08:00'
> log:change= ''Change TTL of Table: employee ColumnFamily: primary'
>
> I try to use preModifyColumnFamily to do this stuff.
>
> Thanks,
> Alex
>
>
> 2017-03-07 12:12 GMT-08:00 Ted Yu :
>
> > Describing your use case would allow people to give better answer.
> >
> > What kind of data do you write to other table in preModifyColumnFamily()
> ?
> >
> > Cross server call within observer is not good idea.
> >
> > Take a look at ConnectionUtils.createShortCircuitConnection().
> >
> > Cheers
> >
> > On Tue, Mar 7, 2017 at 11:42 AM, Xi Yang wrote:
> >
> > > All the articles I've ever seen are talking about add increment or
> > > change put/get status or pinrt out logs. what if I want to write some
> > data
> > > to another table in Observer? For
> > > example, MasterObserver.preModifyColumnFamily()? Seems Observer is
> > runing
> > > at server side, so use connection is unneccessary and might raise some
> > > problem.
> > >I know this might be a stupid question, so if you can just give some
> > > links let me to learn without explain in email, I will be grateful to
> you
> > > for your help
> > >
> > > Thanks,
> > > Alex
> > >
> >
>
Re: What if I want to write another table in MasterObserver.preModifyColumnFamily ?
Requirement: I want to record every change of modify columnFamily by using preModifyColumnFamily(). Now I have a table "my_ddl_log" which used to record the change of columnFamily. For example: If jack change the TTL of columnFamily "primary" in table "employee". Then we should add a put to "my_ddl_log" like this record: log:name= 'jack' log:updateTime= '2017-03-07 12:12 GMT-08:00' log:change= ''Change TTL of Table: employee ColumnFamily: primary' I try to use preModifyColumnFamily to do this stuff. Thanks, Alex 2017-03-07 12:12 GMT-08:00 Ted Yu : > Describing your use case would allow people to give better answer. > > What kind of data do you write to other table in preModifyColumnFamily() ? > > Cross server call within observer is not good idea. > > Take a look at ConnectionUtils.createShortCircuitConnection(). > > Cheers > > On Tue, Mar 7, 2017 at 11:42 AM, Xi Yang wrote: > > > All the articles I've ever seen are talking about add increment or > > change put/get status or pinrt out logs. what if I want to write some > data > > to another table in Observer? For > > example, MasterObserver.preModifyColumnFamily()? Seems Observer is > runing > > at server side, so use connection is unneccessary and might raise some > > problem. > >I know this might be a stupid question, so if you can just give some > > links let me to learn without explain in email, I will be grateful to you > > for your help > > > > Thanks, > > Alex > > >
Re: What if I want to write another table in MasterObserver.preModifyColumnFamily ?
Describing your use case would allow people to give better answer. What kind of data do you write to other table in preModifyColumnFamily() ? Cross server call within observer is not good idea. Take a look at ConnectionUtils.createShortCircuitConnection(). Cheers On Tue, Mar 7, 2017 at 11:42 AM, Xi Yang wrote: > All the articles I've ever seen are talking about add increment or > change put/get status or pinrt out logs. what if I want to write some data > to another table in Observer? For > example, MasterObserver.preModifyColumnFamily()? Seems Observer is runing > at server side, so use connection is unneccessary and might raise some > problem. >I know this might be a stupid question, so if you can just give some > links let me to learn without explain in email, I will be grateful to you > for your help > > Thanks, > Alex >
What if I want to write another table in MasterObserver.preModifyColumnFamily ?
All the articles I've ever seen are talking about add increment or change put/get status or pinrt out logs. what if I want to write some data to another table in Observer? For example, MasterObserver.preModifyColumnFamily()? Seems Observer is runing at server side, so use connection is unneccessary and might raise some problem. I know this might be a stupid question, so if you can just give some links let me to learn without explain in email, I will be grateful to you for your help Thanks, Alex
