Yes Arun, it seems fine to me. Can you do it for BASE_COLUMN_COUNT too so that the dependent features work correctly.(Remember use correct INTEGER byte representation for DATA_TYPE column).
And, can you also please share output of > scan 'SYSTEM.SEQUENCE' Regards, Ankit On Fri, Apr 22, 2016 at 9:14 PM, Arun Kumaran Sabtharishi < [email protected]> wrote: > Ankit, > > Though the ALTER in pheonix failed, a bunch of puts in HBase did the job. > Is it okay to do the following? These puts in HBase intends to do what the > ALTER command in phoenix supposed to do. > > put 'SYSTEM.CATALOG', > "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:DATA_TYPE',"\x80\x00\x00\x10",9 > > put 'SYSTEM.CATALOG', > "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:IS_VIEW_REFERENCED',"\x00",9 > > put 'SYSTEM.CATALOG', > "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:NULLABLE',"\x80\x00\x00\x01",9 > > put 'SYSTEM.CATALOG', > "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:ORDINAL_POSITION',"\x80\x00\x001",9 > > put 'SYSTEM.CATALOG', > "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:SORT_ORDER',"\x80\x00\x00\x02",9 > > put 'SYSTEM.CATALOG', > "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:_0','',9 > > > Thanks, > > Arun > > On Fri, Apr 22, 2016 at 10:22 AM, Arun Kumaran Sabtharishi < > [email protected]> wrote: > >> Ankit, >> >> When tried executing the query, the following exception is thrown. >> >> Error: ERROR 301 (23000): Concurrent modification to table. >> tableName=SYSTEM.CATALOG (state=23000,code=301) >> org.apache.phoenix.schema.ConcurrentTableMutationException: ERROR 301 >> (23000): Concurrent modification to table. tableName=SYSTEM.CATALOG >> >> at >> org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2339) >> at >> org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641) >> at >> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableAddColumnStatement$1.execute(PhoenixStatement.java:1106) >> at >> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:322) >> at >> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:314) >> at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53) >> at >> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:312) >> at >> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1435) >> at sqlline.Commands.execute(Commands.java:822) >> at sqlline.Commands.sql(Commands.java:732) >> at sqlline.SqlLine.dispatch(SqlLine.java:808) >> at sqlline.SqlLine.begin(SqlLine.java:681) >> at sqlline.SqlLine.start(SqlLine.java:398) >> >> >> Thank, >> Arun >> >> On Fri, Apr 22, 2016 at 8:59 AM, Ankit Singhal <[email protected]> >> wrote: >> >>> Arun, >>> As per the system.catalog ,there seems no way that upgrade code(adding >>> new columns and view upgrade for version < v4.5) doesn't execute. >>> >>> But anyways, let's add the required columns manually by following below >>> step; >>> >>> > ./sqlline.py localhost;CurrentSCN=9 >>> > ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER, >>> IS_ROW_TIMESTAMP BOOLEAN; >>> >!quit >>> >>> Quit the shell and start new session without CurrentSCN. >>> > ./sqlline.py localhost >>> > !describe system.catalog >>> >>> this should resolve the issue of missing column. >>> >>> Regards, >>> Ankit Singhal >>> >>> >>> On Fri, Apr 22, 2016 at 3:02 AM, Arun Kumaran Sabtharishi < >>> [email protected]> wrote: >>> >>>> Ankit,James, >>>> >>>> Please find the results you have asked for in the attachment text >>>> document. >>>> >>>> Thanks, >>>> Arun >>>> >>>> On Thu, Apr 21, 2016 at 3:12 AM, Ankit Singhal < >>>> [email protected]> wrote: >>>> >>>>> Arun, >>>>> the observations you had is understandable. >>>>> So, that's why we again requested the output of >>>>> >>>>> *echo "scan 'SYSTEM.CATALOG',{RAW=>true}"|bin/hbase shell|grep CATALOG* >>>>> >>>>> So that we can help you in adding ROW_TIMESTAMP column in >>>>> system.catalog table so that your phoenix setup works properly and you can >>>>> also see it in !describe command. >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On Thu, Apr 21, 2016 at 4:20 AM, Arun Kumaran Sabtharishi < >>>>> [email protected]> wrote: >>>>> >>>>>> Another observation:(After upgrading from Phoenix 4.4 to 4.6.1) >>>>>> >>>>>> In a new SYSTEM.CATALOG table , when connected from phoenix 4.6.1 >>>>>> client, >>>>>> >>>>>> !describe SYSTEM.CATALOG does not show IS_ROW_TIMESTAMP >>>>>> >>>>>> But, select * from SYSTEM.CATALOG shows the IS_ROW_TIMESTAMP column. >>>>>> >>>>>> Is this an expected behavior? >>>>>> >>>>>> If I have to see IS_ROW_TIMESTAMP when doing describe, what should I >>>>>> do? This issue is blocking us in the production environment. Any help to >>>>>> resolve or workaround is highly appreciated. >>>>>> >>>>>> Thanks, >>>>>> Arun >>>>>> >>>>>> >>>>>> On Wed, Apr 20, 2016 at 12:01 PM, Ankit Singhal < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> It's ok if you can just post after grep for CATALOG in a command >>>>>>> output (scan 'SYSTEM.CATALOG', {RAW=>true}). >>>>>>> >>>>>>> On Wed, Apr 20, 2016 at 10:07 PM, Arun Kumaran Sabtharishi < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> One more question to add, >>>>>>>> Do we need to have 1000 versions, and KEEP_DELETED_CELL=true? >>>>>>>> >>>>>>>> I have limited the scan in HBase and here is the following data. >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, >>>>>>>> type=DeleteFamily >>>>>>>> >>>>>>>> _0_1460354090089 >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:BASE_COLUMN_COUNT, >>>>>>>> timestamp=1460354090655, value=\x7F\xFF\xFF\xFF >>>>>>>> >>>>>>>> _0_1460354090089 >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_COUNT, >>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x05 >>>>>>>> >>>>>>>> _0_1460354090089 >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, >>>>>>>> timestamp=1460354090655, type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089 >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:DEFAULT_COLUMN_FAMILY, >>>>>>>> timestamp=1460354090655, value=default >>>>>>>> >>>>>>>> _0_1460354090089 >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:DISABLE_WAL, timestamp=1460354090655, >>>>>>>> value=\x00 >>>>>>>> >>>>>>>> _0_1460354090089 >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:IMMUTABLE_ROWS, >>>>>>>> timestamp=1460354090655, value=\x00 >>>>>>>> >>>>>>>> _0_1460354090089 >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:INDEX_STATE, timestamp=1460354090655, >>>>>>>> type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089 >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:INDEX_TYPE, timestamp=1460354090655, >>>>>>>> type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089 >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:MULTI_TENANT, >>>>>>>> timestamp=1460354090655, value=\x00 >>>>>>>> >>>>>>>> _0_1460354090089 >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, >>>>>>>> type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089 >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:ROW_KEY_ORDER_OPTIMIZABLE, >>>>>>>> timestamp=1460354090655, value=\x01 >>>>>>>> >>>>>>>> _0_1460354090089 >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:SALT_BUCKETS, >>>>>>>> timestamp=1460354090655, type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089 >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:STORE_NULLS, timestamp=1460354090655, >>>>>>>> value=\x00 >>>>>>>> >>>>>>>> _0_1460354090089 >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:TABLE_SEQ_NUM, >>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x00\x00\x00\x00\x00 >>>>>>>> >>>>>>>> _0_1460354090089 >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:TABLE_TYPE, timestamp=1460354090655, >>>>>>>> value=u >>>>>>>> >>>>>>>> _0_1460354090089 >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_INDEX_ID, >>>>>>>> timestamp=1460354090655, type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089 >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_STATEMENT, >>>>>>>> timestamp=1460354090655, type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089 >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_TYPE, timestamp=1460354090655, >>>>>>>> type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089 >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, >>>>>>>> type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089 >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, >>>>>>>> type=DeleteFamily >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\ >>>>>>>> >>>>>>>> x00default >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655, >>>>>>>> type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\ >>>>>>>> >>>>>>>> x00default >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655, >>>>>>>> type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\ >>>>>>>> >>>>>>>> x00default >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655, >>>>>>>> type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\ >>>>>>>> >>>>>>>> x00default >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, >>>>>>>> timestamp=1460354090655, type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\ >>>>>>>> >>>>>>>> x00default >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655, >>>>>>>> value=\x80\x00\x00\x0C >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\ >>>>>>>> >>>>>>>> x00default >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, >>>>>>>> timestamp=1460354090655, type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\ >>>>>>>> >>>>>>>> x00default >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, >>>>>>>> timestamp=1460354090655, value=\x00 >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\ >>>>>>>> >>>>>>>> x00default >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, >>>>>>>> timestamp=1460354090655, value=\x00 >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\ >>>>>>>> >>>>>>>> x00default >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655, >>>>>>>> type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\ >>>>>>>> >>>>>>>> x00default >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655, >>>>>>>> value=\x80\x00\x00\x01 >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\ >>>>>>>> >>>>>>>> x00default >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, >>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x04 >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\ >>>>>>>> >>>>>>>> x00default >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, >>>>>>>> type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\ >>>>>>>> >>>>>>>> x00default >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655, >>>>>>>> value=\x80\x00\x00\x02 >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\ >>>>>>>> >>>>>>>> x00default >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, >>>>>>>> timestamp=1460354090655, type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\ >>>>>>>> >>>>>>>> x00default >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, >>>>>>>> type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\ >>>>>>>> >>>>>>>> x00default >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, >>>>>>>> type=DeleteFamily >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_PK >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655, >>>>>>>> type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_PK >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655, >>>>>>>> type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_PK >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655, >>>>>>>> type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_PK >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, >>>>>>>> timestamp=1460354090655, type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_PK >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655, >>>>>>>> value=\x80\x00\x00\x0C >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_PK >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, >>>>>>>> timestamp=1460354090655, type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_PK >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, >>>>>>>> timestamp=1460354090655, value=\x00 >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_PK >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, >>>>>>>> timestamp=1460354090655, value=\x00 >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_PK >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655, >>>>>>>> value=\x80\x01 >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_PK >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655, >>>>>>>> value=\x80\x00\x00\x00 >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_PK >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, >>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x05 >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_PK >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, >>>>>>>> type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_PK >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655, >>>>>>>> value=\x80\x00\x00\x02 >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_PK >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, >>>>>>>> timestamp=1460354090655, type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_PK >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, >>>>>>>> type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_PK >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, >>>>>>>> type=DeleteFamily >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655, >>>>>>>> type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655, >>>>>>>> type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655, >>>>>>>> type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, >>>>>>>> timestamp=1460354090655, type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655, >>>>>>>> value=\x80\x00\x00\x0C >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, >>>>>>>> timestamp=1460354090655, type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, >>>>>>>> timestamp=1460354090655, value=\x00 >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, >>>>>>>> timestamp=1460354090655, value=\x00 >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655, >>>>>>>> type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655, >>>>>>>> value=\x80\x00\x00\x01 >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, >>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x01 >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, >>>>>>>> type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655, >>>>>>>> value=\x80\x00\x00\x02 >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, >>>>>>>> timestamp=1460354090655, type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, >>>>>>>> type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, >>>>>>>> type=DeleteFamily >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655, >>>>>>>> type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655, >>>>>>>> type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655, >>>>>>>> type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, >>>>>>>> timestamp=1460354090655, type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655, >>>>>>>> value=\x80\x00\x00\x0C >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, >>>>>>>> timestamp=1460354090655, type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, >>>>>>>> timestamp=1460354090655, value=\x00 >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, >>>>>>>> timestamp=1460354090655, value=\x00 >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655, >>>>>>>> type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655, >>>>>>>> value=\x80\x00\x00\x01 >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, >>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x03 >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, >>>>>>>> type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655, >>>>>>>> value=\x80\x00\x00\x02 >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, >>>>>>>> timestamp=1460354090655, type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, >>>>>>>> type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, >>>>>>>> type=DeleteFamily >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau >>>>>>>> >>>>>>>> lt >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655, >>>>>>>> type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau >>>>>>>> >>>>>>>> lt >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655, >>>>>>>> type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau >>>>>>>> >>>>>>>> lt >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655, >>>>>>>> type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau >>>>>>>> >>>>>>>> lt >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, >>>>>>>> timestamp=1460354090655, type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau >>>>>>>> >>>>>>>> lt >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655, >>>>>>>> value=\x80\x00\x00\x0C >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau >>>>>>>> >>>>>>>> lt >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, >>>>>>>> timestamp=1460354090655, type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau >>>>>>>> >>>>>>>> lt >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, >>>>>>>> timestamp=1460354090655, value=\x00 >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau >>>>>>>> >>>>>>>> lt >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, >>>>>>>> timestamp=1460354090655, value=\x00 >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau >>>>>>>> >>>>>>>> lt >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655, >>>>>>>> type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau >>>>>>>> >>>>>>>> lt >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655, >>>>>>>> value=\x80\x00\x00\x01 >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau >>>>>>>> >>>>>>>> lt >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, >>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x02 >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau >>>>>>>> >>>>>>>> lt >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, >>>>>>>> type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau >>>>>>>> >>>>>>>> lt >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655, >>>>>>>> value=\x80\x00\x00\x02 >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau >>>>>>>> >>>>>>>> lt >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, >>>>>>>> timestamp=1460354090655, type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau >>>>>>>> >>>>>>>> lt >>>>>>>> >>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, >>>>>>>> type=DeleteColumn >>>>>>>> >>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau >>>>>>>> >>>>>>>> lt >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Arun >>>>>>>> >>>>>>>> >>>>>>>> On Wed, Apr 20, 2016 at 11:31 AM, Arun Kumaran Sabtharishi < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> James, >>>>>>>>> >>>>>>>>> Table SYSTEM.CATALOG is ENABLED >>>>>>>>> SYSTEM.CATALOG, {TABLE_ATTRIBUTES => {coprocessor$1 => >>>>>>>>> '|org.apache.phoenix.coprocessor.ScanRegionObserver|1|', >>>>>>>>> coprocessor$2 => >>>>>>>>> '|org.apache >>>>>>>>> >>>>>>>>> .phoenix.coprocessor.UngroupedAggregateRegionObserver|1|', >>>>>>>>> coprocessor$3 => >>>>>>>>> '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1 >>>>>>>>> >>>>>>>>> |', coprocessor$4 => >>>>>>>>> '|org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|', >>>>>>>>> coprocessor$5 => '|org.apache.phoenix.coprocessor.MetaDa >>>>>>>>> >>>>>>>>> taEndpointImpl|1|', coprocessor$6 => >>>>>>>>> '|org.apache.phoenix.coprocessor.MetaDataRegionObserver|2|', METADATA >>>>>>>>> => >>>>>>>>> {'SPLIT_POLICY' => 'org.apache. >>>>>>>>> >>>>>>>>> phoenix.schema.MetaDataSplitPolicy', 'UpgradeTo30' => 'true'}} >>>>>>>>> COLUMN FAMILIES DESCRIPTION >>>>>>>>> {NAME => '0', DATA_BLOCK_ENCODING => 'FAST_DIFF', BLOOMFILTER => >>>>>>>>> 'ROW', REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => >>>>>>>>> '1000', >>>>>>>>> >>>>>>>>> MIN_VERSIONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS => >>>>>>>>> 'true', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => >>>>>>>>> 'true'} >>>>>>>>> >>>>>>>>> 1 row in 0.6060 seconds >>>>>>>>> >>>>>>>>> The above is for describe SYSTEM.CATALOG. The output for scan >>>>>>>>> 'SYSTEM.CATALOG', {RAW=>true} is too huge. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Arun >>>>>>>>> >>>>>>>>> >>>>>>>>> On Wed, Apr 20, 2016 at 11:19 AM, James Taylor < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> Arun, >>>>>>>>>> Please run the command Ankit mentioned in an HBase shell and post >>>>>>>>>> the output back here. >>>>>>>>>> Thanks, >>>>>>>>>> James >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Wednesday, April 20, 2016, Arun Kumaran Sabtharishi < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> !describe SYSTEM.CATALOG is not returning IS_ROW_TIMESTAMP >>>>>>>>>>> column. >>>>>>>>>>> >>>>>>>>>>> But we do see this column from select statement: >>>>>>>>>>> >>>>>>>>>>> select * from SYSTEM.CATALOG where TABLE_NAME=’TEST_TABLE_1’ AND >>>>>>>>>>> TABLE_SCHEM IS NULL AND TENANT_ID IS NULL ; >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Arun >>>>>>>>>>> >>>>>>>>>>> On Wed, Apr 20, 2016 at 1:37 AM, Ankit Singhal < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi Arun, >>>>>>>>>>>> >>>>>>>>>>>> Do you see 'IS_ROW_TIMESTAMP' column in SYSTEM.CATALOG, by >>>>>>>>>>>> doing !describe on system.catalog. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> if not, >>>>>>>>>>>> can you share the output of below command. As it seems >>>>>>>>>>>> SYSTEM.CATALOG was updated with timestamp greater v4.6 timestamp , >>>>>>>>>>>> and >>>>>>>>>>>> which stopping upgrade code to add a new column. >>>>>>>>>>>> >>>>>>>>>>>> scan 'SYSTEM.CATALOG', {RAW=>true} >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Regards, >>>>>>>>>>>> Ankit Singhal >>>>>>>>>>>> >>>>>>>>>>>> On Wed, Apr 20, 2016 at 4:25 AM, Arun Kumaran Sabtharishi < >>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> After further investigation, we found that Phoenix Upsert >>>>>>>>>>>>> query SYSTEM.CATALOG has IS_ROW_TIMESTAMP column, but >>>>>>>>>>>>> PTableImpl.getColumn() is failing with error:"Undefined column. >>>>>>>>>>>>> columnName=IS_ROW_TIMESTAMP" . Does this mean that PTableImpl is >>>>>>>>>>>>> reading >>>>>>>>>>>>> from cached entity of SYSTEM.CATALOG before 4.6 upgrade?" >>>>>>>>>>>>> >>>>>>>>>>>>> We do see that clearCache() is being called for 4.7, and 4.7 >>>>>>>>>>>>> upgrades from ConnectionQueryServicesImpl class, but not for 4.6 >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> Arun >>>>>>>>>>>>> >>>>>>>>>>>>> On Tue, Apr 19, 2016 at 10:22 AM, Arun Kumaran Sabtharishi < >>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> James, >>>>>>>>>>>>>> >>>>>>>>>>>>>> To add more information on this issue, this happens in new >>>>>>>>>>>>>> phoenix views associated with brand new tables as well. So, this >>>>>>>>>>>>>> cannot be >>>>>>>>>>>>>> an upgrade/migration issue. Not figured out a specific way to >>>>>>>>>>>>>> reproduce >>>>>>>>>>>>>> this issue yet. Could you throw some ideas on what direction >>>>>>>>>>>>>> this problem >>>>>>>>>>>>>> could be approached from this point? >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> Arun >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> >
