[jira] [Commented] (HBASE-6894) Adding metadata to a table in the shell is both arcane and painful
[ https://issues.apache.org/jira/browse/HBASE-6894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13492694#comment-13492694 ] stack commented on HBASE-6894: -- bq. Fixed TABLE_ATTRIBUTES (which is not technically a constant, but I guess this doesn't have to be valid Ruby)... Right. Probably should support that one day... being able to reference the TABLE_ATTRIBUTES dictionary or should it be called HTableDescriptor or HTABLE_DESCRIPTOR? And so on. Thanks for filing new issue on changing the CONFIG option. Nice cleanup Sergey. Let me commit. Adding metadata to a table in the shell is both arcane and painful -- Key: HBASE-6894 URL: https://issues.apache.org/jira/browse/HBASE-6894 Project: HBase Issue Type: Bug Components: shell Affects Versions: 0.96.0 Reporter: stack Assignee: Sergey Shelukhin Labels: noob Attachments: HBASE-6894.patch, HBASE-6894.patch, HBASE-6894.patch, HBASE-6894-v2.patch, HBASE-6894-v3.1-squashed.patch, HBASE-6894-v3-squashed.patch, HBASE-6894-v4-squashed.patch, HBASE-6894-v5-squashed.patch In production we have hundreds of tables w/ whack names like 'aliaserv', 'ashish_bulk', 'age_gender_topics', etc. It be grand if you could look in master UI and see stuff like owner, eng group responsible, miscellaneous description, etc. Now, HTD has support for this; each carries a dictionary. Whats a PITA though is adding attributes to the dictionary. Here is what seems to work on trunk (though I do not trust it is doing the right thing): {code} hbase create 'SOME_TABLENAME', {NAME = 'd', VERSION = 1, COMPRESSION = 'LZO'} hbase # Here is how I added metadata hbase disable 'SOME_TABLENAME' hbase alter 'SOME_TABLENAME', METHOD = 'table_att', OWNER = 'SOMEON', CONFIG = {'ENVIRONMENT' = 'BLAH BLAH', 'SIZING' = 'The size should be between 0-10K most of the time with new URLs coming in and getting removed as they are processed unless the pipeline has fallen behind', 'MISCELLANEOUS' = 'Holds the list of URLs waiting to be processed in the parked page detection analyzer in ingestion pipeline.'} ... describe... enable... {code} The above doesn't work in 0.94. Complains about the CONFIG, the keyword we are using for the HTD dictionary. It works in 0.96 though I'd have to poke around some more to ensure it is doing the right thing. But this METHOD = 'table_att' stuff is really ugly can we fix it? And I can't add table attributes on table create seemingly. A little bit of thought and a bit of ruby could clean this all up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6894) Adding metadata to a table in the shell is both arcane and painful
[ https://issues.apache.org/jira/browse/HBASE-6894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13492745#comment-13492745 ] Hudson commented on HBASE-6894: --- Integrated in HBase-TRUNK #3519 (See [https://builds.apache.org/job/HBase-TRUNK/3519/]) HBASE-6894 Adding metadata to a table in the shell is both arcane and painful (Revision 1406800) Result = FAILURE stack : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java * /hbase/trunk/hbase-server/src/main/ruby/hbase/admin.rb * /hbase/trunk/hbase-server/src/main/ruby/shell/commands/alter.rb * /hbase/trunk/hbase-server/src/main/ruby/shell/commands/create.rb * /hbase/trunk/hbase-server/src/test/ruby/hbase/admin_test.rb Adding metadata to a table in the shell is both arcane and painful -- Key: HBASE-6894 URL: https://issues.apache.org/jira/browse/HBASE-6894 Project: HBase Issue Type: Bug Components: shell Affects Versions: 0.96.0 Reporter: stack Assignee: Sergey Shelukhin Labels: noob Fix For: 0.96.0 Attachments: HBASE-6894.patch, HBASE-6894.patch, HBASE-6894.patch, HBASE-6894-v2.patch, HBASE-6894-v3.1-squashed.patch, HBASE-6894-v3-squashed.patch, HBASE-6894-v4-squashed.patch, HBASE-6894-v5-squashed.patch In production we have hundreds of tables w/ whack names like 'aliaserv', 'ashish_bulk', 'age_gender_topics', etc. It be grand if you could look in master UI and see stuff like owner, eng group responsible, miscellaneous description, etc. Now, HTD has support for this; each carries a dictionary. Whats a PITA though is adding attributes to the dictionary. Here is what seems to work on trunk (though I do not trust it is doing the right thing): {code} hbase create 'SOME_TABLENAME', {NAME = 'd', VERSION = 1, COMPRESSION = 'LZO'} hbase # Here is how I added metadata hbase disable 'SOME_TABLENAME' hbase alter 'SOME_TABLENAME', METHOD = 'table_att', OWNER = 'SOMEON', CONFIG = {'ENVIRONMENT' = 'BLAH BLAH', 'SIZING' = 'The size should be between 0-10K most of the time with new URLs coming in and getting removed as they are processed unless the pipeline has fallen behind', 'MISCELLANEOUS' = 'Holds the list of URLs waiting to be processed in the parked page detection analyzer in ingestion pipeline.'} ... describe... enable... {code} The above doesn't work in 0.94. Complains about the CONFIG, the keyword we are using for the HTD dictionary. It works in 0.96 though I'd have to poke around some more to ensure it is doing the right thing. But this METHOD = 'table_att' stuff is really ugly can we fix it? And I can't add table attributes on table create seemingly. A little bit of thought and a bit of ruby could clean this all up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6894) Adding metadata to a table in the shell is both arcane and painful
[ https://issues.apache.org/jira/browse/HBASE-6894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13492847#comment-13492847 ] Hudson commented on HBASE-6894: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #252 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/252/]) HBASE-6894 Adding metadata to a table in the shell is both arcane and painful (Revision 1406800) Result = FAILURE stack : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java * /hbase/trunk/hbase-server/src/main/ruby/hbase/admin.rb * /hbase/trunk/hbase-server/src/main/ruby/shell/commands/alter.rb * /hbase/trunk/hbase-server/src/main/ruby/shell/commands/create.rb * /hbase/trunk/hbase-server/src/test/ruby/hbase/admin_test.rb Adding metadata to a table in the shell is both arcane and painful -- Key: HBASE-6894 URL: https://issues.apache.org/jira/browse/HBASE-6894 Project: HBase Issue Type: Bug Components: shell Affects Versions: 0.96.0 Reporter: stack Assignee: Sergey Shelukhin Labels: noob Fix For: 0.96.0 Attachments: HBASE-6894.patch, HBASE-6894.patch, HBASE-6894.patch, HBASE-6894-v2.patch, HBASE-6894-v3.1-squashed.patch, HBASE-6894-v3-squashed.patch, HBASE-6894-v4-squashed.patch, HBASE-6894-v5-squashed.patch In production we have hundreds of tables w/ whack names like 'aliaserv', 'ashish_bulk', 'age_gender_topics', etc. It be grand if you could look in master UI and see stuff like owner, eng group responsible, miscellaneous description, etc. Now, HTD has support for this; each carries a dictionary. Whats a PITA though is adding attributes to the dictionary. Here is what seems to work on trunk (though I do not trust it is doing the right thing): {code} hbase create 'SOME_TABLENAME', {NAME = 'd', VERSION = 1, COMPRESSION = 'LZO'} hbase # Here is how I added metadata hbase disable 'SOME_TABLENAME' hbase alter 'SOME_TABLENAME', METHOD = 'table_att', OWNER = 'SOMEON', CONFIG = {'ENVIRONMENT' = 'BLAH BLAH', 'SIZING' = 'The size should be between 0-10K most of the time with new URLs coming in and getting removed as they are processed unless the pipeline has fallen behind', 'MISCELLANEOUS' = 'Holds the list of URLs waiting to be processed in the parked page detection analyzer in ingestion pipeline.'} ... describe... enable... {code} The above doesn't work in 0.94. Complains about the CONFIG, the keyword we are using for the HTD dictionary. It works in 0.96 though I'd have to poke around some more to ensure it is doing the right thing. But this METHOD = 'table_att' stuff is really ugly can we fix it? And I can't add table attributes on table create seemingly. A little bit of thought and a bit of ruby could clean this all up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6894) Adding metadata to a table in the shell is both arcane and painful
[ https://issues.apache.org/jira/browse/HBASE-6894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491711#comment-13491711 ] stack commented on HBASE-6894: -- MS Ill fix the below comments on help screens on commit (They are not your issues -- I just saw them reviewing this): In alter help, this example does not illustrate the point that you can alter multiple familes at once: hbase alter 't1', 'f1', {NAME = 'f2', IN_MEMORY = true}, 'f3' Lets not have two ways to do something: hbase alter 't1', NAME = 'f1', METHOD = 'delete' hbase alter 't1', 'delete' = 'f1' Should we explain CONFIG? Its not explained in create command. Question for you Sergey: How do I remove something from CONFIG? Maybe you can't just now. For another issue. Table attributes are currently anonymous... They look the same as a CF only w/o a NAME = attribute. Should they be named as in TABLE_ATTRIBUTES = {CONFIG {'x' = 'y', 'OWNER' = 'SOMEOWNER'}} METHOD = 'table_att' as analog for CF NAME = 'name' was ugly. I'm glad you removed that... just wondering if we should add something back to say what they are. I'm up for commit. This fixes a bunch of stuff. It unearths the fact that there is a load more work to do in here but that is not your fault. Adding metadata to a table in the shell is both arcane and painful -- Key: HBASE-6894 URL: https://issues.apache.org/jira/browse/HBASE-6894 Project: HBase Issue Type: Bug Components: shell Affects Versions: 0.96.0 Reporter: stack Assignee: Sergey Shelukhin Labels: noob Attachments: HBASE-6894.patch, HBASE-6894.patch, HBASE-6894.patch, HBASE-6894-v2.patch, HBASE-6894-v3.1-squashed.patch, HBASE-6894-v3-squashed.patch, HBASE-6894-v4-squashed.patch In production we have hundreds of tables w/ whack names like 'aliaserv', 'ashish_bulk', 'age_gender_topics', etc. It be grand if you could look in master UI and see stuff like owner, eng group responsible, miscellaneous description, etc. Now, HTD has support for this; each carries a dictionary. Whats a PITA though is adding attributes to the dictionary. Here is what seems to work on trunk (though I do not trust it is doing the right thing): {code} hbase create 'SOME_TABLENAME', {NAME = 'd', VERSION = 1, COMPRESSION = 'LZO'} hbase # Here is how I added metadata hbase disable 'SOME_TABLENAME' hbase alter 'SOME_TABLENAME', METHOD = 'table_att', OWNER = 'SOMEON', CONFIG = {'ENVIRONMENT' = 'BLAH BLAH', 'SIZING' = 'The size should be between 0-10K most of the time with new URLs coming in and getting removed as they are processed unless the pipeline has fallen behind', 'MISCELLANEOUS' = 'Holds the list of URLs waiting to be processed in the parked page detection analyzer in ingestion pipeline.'} ... describe... enable... {code} The above doesn't work in 0.94. Complains about the CONFIG, the keyword we are using for the HTD dictionary. It works in 0.96 though I'd have to poke around some more to ensure it is doing the right thing. But this METHOD = 'table_att' stuff is really ugly can we fix it? And I can't add table attributes on table create seemingly. A little bit of thought and a bit of ruby could clean this all up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6894) Adding metadata to a table in the shell is both arcane and painful
[ https://issues.apache.org/jira/browse/HBASE-6894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13492025#comment-13492025 ] Sergey Shelukhin commented on HBASE-6894: - Fixed TABLE_ATTRIBUTES (which is not technically a constant, but I guess this doesn't have to be valid Ruby), as well as alter help. The delete CF stuff is legacy, I'm not changing that. I created HBASE-7064 for explaining CONFIG (as well as other things). You can erase from config by setting key to nil, which I guess is not the most obvious way... Adding metadata to a table in the shell is both arcane and painful -- Key: HBASE-6894 URL: https://issues.apache.org/jira/browse/HBASE-6894 Project: HBase Issue Type: Bug Components: shell Affects Versions: 0.96.0 Reporter: stack Assignee: Sergey Shelukhin Labels: noob Attachments: HBASE-6894.patch, HBASE-6894.patch, HBASE-6894.patch, HBASE-6894-v2.patch, HBASE-6894-v3.1-squashed.patch, HBASE-6894-v3-squashed.patch, HBASE-6894-v4-squashed.patch, HBASE-6894-v5-squashed.patch In production we have hundreds of tables w/ whack names like 'aliaserv', 'ashish_bulk', 'age_gender_topics', etc. It be grand if you could look in master UI and see stuff like owner, eng group responsible, miscellaneous description, etc. Now, HTD has support for this; each carries a dictionary. Whats a PITA though is adding attributes to the dictionary. Here is what seems to work on trunk (though I do not trust it is doing the right thing): {code} hbase create 'SOME_TABLENAME', {NAME = 'd', VERSION = 1, COMPRESSION = 'LZO'} hbase # Here is how I added metadata hbase disable 'SOME_TABLENAME' hbase alter 'SOME_TABLENAME', METHOD = 'table_att', OWNER = 'SOMEON', CONFIG = {'ENVIRONMENT' = 'BLAH BLAH', 'SIZING' = 'The size should be between 0-10K most of the time with new URLs coming in and getting removed as they are processed unless the pipeline has fallen behind', 'MISCELLANEOUS' = 'Holds the list of URLs waiting to be processed in the parked page detection analyzer in ingestion pipeline.'} ... describe... enable... {code} The above doesn't work in 0.94. Complains about the CONFIG, the keyword we are using for the HTD dictionary. It works in 0.96 though I'd have to poke around some more to ensure it is doing the right thing. But this METHOD = 'table_att' stuff is really ugly can we fix it? And I can't add table attributes on table create seemingly. A little bit of thought and a bit of ruby could clean this all up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6894) Adding metadata to a table in the shell is both arcane and painful
[ https://issues.apache.org/jira/browse/HBASE-6894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13492067#comment-13492067 ] Hadoop QA commented on HBASE-6894: -- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12552385/HBASE-6894-v5-squashed.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 3 new or modified tests. {color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile. {color:red}-1 javadoc{color}. The javadoc tool appears to have generated 87 warning messages. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:red}-1 findbugs{color}. The patch appears to introduce 15 new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 core tests{color}. The patch failed these unit tests: org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3246//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3246//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3246//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3246//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3246//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3246//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3246//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3246//console This message is automatically generated. Adding metadata to a table in the shell is both arcane and painful -- Key: HBASE-6894 URL: https://issues.apache.org/jira/browse/HBASE-6894 Project: HBase Issue Type: Bug Components: shell Affects Versions: 0.96.0 Reporter: stack Assignee: Sergey Shelukhin Labels: noob Attachments: HBASE-6894.patch, HBASE-6894.patch, HBASE-6894.patch, HBASE-6894-v2.patch, HBASE-6894-v3.1-squashed.patch, HBASE-6894-v3-squashed.patch, HBASE-6894-v4-squashed.patch, HBASE-6894-v5-squashed.patch In production we have hundreds of tables w/ whack names like 'aliaserv', 'ashish_bulk', 'age_gender_topics', etc. It be grand if you could look in master UI and see stuff like owner, eng group responsible, miscellaneous description, etc. Now, HTD has support for this; each carries a dictionary. Whats a PITA though is adding attributes to the dictionary. Here is what seems to work on trunk (though I do not trust it is doing the right thing): {code} hbase create 'SOME_TABLENAME', {NAME = 'd', VERSION = 1, COMPRESSION = 'LZO'} hbase # Here is how I added metadata hbase disable 'SOME_TABLENAME' hbase alter 'SOME_TABLENAME', METHOD = 'table_att', OWNER = 'SOMEON', CONFIG = {'ENVIRONMENT' = 'BLAH BLAH', 'SIZING' = 'The size should be between 0-10K most of the time with new URLs coming in and getting removed as they are processed unless the pipeline has fallen behind', 'MISCELLANEOUS' = 'Holds the list of URLs waiting to be processed in the parked page detection analyzer in ingestion pipeline.'} ... describe... enable... {code} The above doesn't work in 0.94. Complains about the CONFIG, the keyword we are using for the HTD dictionary. It works in 0.96 though I'd have to poke around some more to ensure it is doing the right thing. But this METHOD = 'table_att' stuff is really ugly can we fix it? And I can't add table attributes on table create seemingly. A little bit of thought and a bit of ruby could clean this all up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6894) Adding metadata to a table in the shell is both arcane and painful
[ https://issues.apache.org/jira/browse/HBASE-6894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491015#comment-13491015 ] Sergey Shelukhin commented on HBASE-6894: - Did you try v4 patch? That was the latest one and has no feedback. Adding metadata to a table in the shell is both arcane and painful -- Key: HBASE-6894 URL: https://issues.apache.org/jira/browse/HBASE-6894 Project: HBase Issue Type: Bug Components: shell Affects Versions: 0.96.0 Reporter: stack Assignee: Sergey Shelukhin Labels: noob Attachments: HBASE-6894.patch, HBASE-6894.patch, HBASE-6894.patch, HBASE-6894-v2.patch, HBASE-6894-v3.1-squashed.patch, HBASE-6894-v3-squashed.patch, HBASE-6894-v4-squashed.patch In production we have hundreds of tables w/ whack names like 'aliaserv', 'ashish_bulk', 'age_gender_topics', etc. It be grand if you could look in master UI and see stuff like owner, eng group responsible, miscellaneous description, etc. Now, HTD has support for this; each carries a dictionary. Whats a PITA though is adding attributes to the dictionary. Here is what seems to work on trunk (though I do not trust it is doing the right thing): {code} hbase create 'SOME_TABLENAME', {NAME = 'd', VERSION = 1, COMPRESSION = 'LZO'} hbase # Here is how I added metadata hbase disable 'SOME_TABLENAME' hbase alter 'SOME_TABLENAME', METHOD = 'table_att', OWNER = 'SOMEON', CONFIG = {'ENVIRONMENT' = 'BLAH BLAH', 'SIZING' = 'The size should be between 0-10K most of the time with new URLs coming in and getting removed as they are processed unless the pipeline has fallen behind', 'MISCELLANEOUS' = 'Holds the list of URLs waiting to be processed in the parked page detection analyzer in ingestion pipeline.'} ... describe... enable... {code} The above doesn't work in 0.94. Complains about the CONFIG, the keyword we are using for the HTD dictionary. It works in 0.96 though I'd have to poke around some more to ensure it is doing the right thing. But this METHOD = 'table_att' stuff is really ugly can we fix it? And I can't add table attributes on table create seemingly. A little bit of thought and a bit of ruby could clean this all up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6894) Adding metadata to a table in the shell is both arcane and painful
[ https://issues.apache.org/jira/browse/HBASE-6894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13489576#comment-13489576 ] stack commented on HBASE-6894: -- On the CONFIG, its just wrong but am good w/ dealing w/ this in another issue. Other answers are good. Thanks. Post new patch and I'll take it for another spin. This is very nice improvement Sergey. Lets get it in. Adding metadata to a table in the shell is both arcane and painful -- Key: HBASE-6894 URL: https://issues.apache.org/jira/browse/HBASE-6894 Project: HBase Issue Type: Bug Components: shell Affects Versions: 0.96.0 Reporter: stack Assignee: Sergey Shelukhin Labels: noob Attachments: HBASE-6894.patch, HBASE-6894.patch, HBASE-6894.patch, HBASE-6894-v2.patch, HBASE-6894-v3.1-squashed.patch, HBASE-6894-v3-squashed.patch, HBASE-6894-v4-squashed.patch In production we have hundreds of tables w/ whack names like 'aliaserv', 'ashish_bulk', 'age_gender_topics', etc. It be grand if you could look in master UI and see stuff like owner, eng group responsible, miscellaneous description, etc. Now, HTD has support for this; each carries a dictionary. Whats a PITA though is adding attributes to the dictionary. Here is what seems to work on trunk (though I do not trust it is doing the right thing): {code} hbase create 'SOME_TABLENAME', {NAME = 'd', VERSION = 1, COMPRESSION = 'LZO'} hbase # Here is how I added metadata hbase disable 'SOME_TABLENAME' hbase alter 'SOME_TABLENAME', METHOD = 'table_att', OWNER = 'SOMEON', CONFIG = {'ENVIRONMENT' = 'BLAH BLAH', 'SIZING' = 'The size should be between 0-10K most of the time with new URLs coming in and getting removed as they are processed unless the pipeline has fallen behind', 'MISCELLANEOUS' = 'Holds the list of URLs waiting to be processed in the parked page detection analyzer in ingestion pipeline.'} ... describe... enable... {code} The above doesn't work in 0.94. Complains about the CONFIG, the keyword we are using for the HTD dictionary. It works in 0.96 though I'd have to poke around some more to ensure it is doing the right thing. But this METHOD = 'table_att' stuff is really ugly can we fix it? And I can't add table attributes on table create seemingly. A little bit of thought and a bit of ruby could clean this all up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6894) Adding metadata to a table in the shell is both arcane and painful
[ https://issues.apache.org/jira/browse/HBASE-6894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13488067#comment-13488067 ] Sergey Shelukhin commented on HBASE-6894: - Thanks! What does this do in the alter help screen? hbase alter 't1', 'f1', {NAME = 'f2', IN_MEMORY = true}, 'f3' It's just one of the varied syntax examples - creates two CFs, one with some setting (e.g. in memory). Did you add the new delete method? Thats nicer. I'd say don't mention the old method at all in the help: Delete method was already there, as well as the docs for it. Says change max filesize of family but its for the table? Old doc, fixed. We have to keep table_att for the below? It's optional, removed. Do you think CONFIG a good name for the 'dictionary' that can be in HTableDescriptor and HColumnDescriptor? Would DICT or DICTIONARY or MAP be better names than CONFIG? For example, in the below: I'd rather not make a breaking change, or make it separately so that it's easier to revert/not take. I created a table with some table attributes. When I describe it, the METHOD = 'table_att' is a little confusing I'd say? Do we have to have that METHOD = 'table_att' in there? Could be a dictionary named 'table_att'? or 'table _attributes'? Will remove. I was thinking I could add a new table attribute 'METADATA' doing the following but fails. Should I be able too? User attributes are only allowed in the specific place separate from system ones to avoid ambiguity/conflicts if we add new system attributes. Adding metadata to a table in the shell is both arcane and painful -- Key: HBASE-6894 URL: https://issues.apache.org/jira/browse/HBASE-6894 Project: HBase Issue Type: Bug Components: shell Affects Versions: 0.96.0 Reporter: stack Assignee: Sergey Shelukhin Labels: noob Attachments: HBASE-6894.patch, HBASE-6894.patch, HBASE-6894.patch, HBASE-6894-v2.patch, HBASE-6894-v3.1-squashed.patch, HBASE-6894-v3-squashed.patch In production we have hundreds of tables w/ whack names like 'aliaserv', 'ashish_bulk', 'age_gender_topics', etc. It be grand if you could look in master UI and see stuff like owner, eng group responsible, miscellaneous description, etc. Now, HTD has support for this; each carries a dictionary. Whats a PITA though is adding attributes to the dictionary. Here is what seems to work on trunk (though I do not trust it is doing the right thing): {code} hbase create 'SOME_TABLENAME', {NAME = 'd', VERSION = 1, COMPRESSION = 'LZO'} hbase # Here is how I added metadata hbase disable 'SOME_TABLENAME' hbase alter 'SOME_TABLENAME', METHOD = 'table_att', OWNER = 'SOMEON', CONFIG = {'ENVIRONMENT' = 'BLAH BLAH', 'SIZING' = 'The size should be between 0-10K most of the time with new URLs coming in and getting removed as they are processed unless the pipeline has fallen behind', 'MISCELLANEOUS' = 'Holds the list of URLs waiting to be processed in the parked page detection analyzer in ingestion pipeline.'} ... describe... enable... {code} The above doesn't work in 0.94. Complains about the CONFIG, the keyword we are using for the HTD dictionary. It works in 0.96 though I'd have to poke around some more to ensure it is doing the right thing. But this METHOD = 'table_att' stuff is really ugly can we fix it? And I can't add table attributes on table create seemingly. A little bit of thought and a bit of ruby could clean this all up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6894) Adding metadata to a table in the shell is both arcane and painful
[ https://issues.apache.org/jira/browse/HBASE-6894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13488327#comment-13488327 ] Hadoop QA commented on HBASE-6894: -- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12551628/HBASE-6894-v4-squashed.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 3 new or modified tests. {color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile. {color:red}-1 javadoc{color}. The javadoc tool appears to have generated 85 warning messages. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:red}-1 findbugs{color}. The patch appears to introduce 3 new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 core tests{color}. The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3206//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3206//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3206//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3206//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3206//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3206//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3206//console This message is automatically generated. Adding metadata to a table in the shell is both arcane and painful -- Key: HBASE-6894 URL: https://issues.apache.org/jira/browse/HBASE-6894 Project: HBase Issue Type: Bug Components: shell Affects Versions: 0.96.0 Reporter: stack Assignee: Sergey Shelukhin Labels: noob Attachments: HBASE-6894.patch, HBASE-6894.patch, HBASE-6894.patch, HBASE-6894-v2.patch, HBASE-6894-v3.1-squashed.patch, HBASE-6894-v3-squashed.patch, HBASE-6894-v4-squashed.patch In production we have hundreds of tables w/ whack names like 'aliaserv', 'ashish_bulk', 'age_gender_topics', etc. It be grand if you could look in master UI and see stuff like owner, eng group responsible, miscellaneous description, etc. Now, HTD has support for this; each carries a dictionary. Whats a PITA though is adding attributes to the dictionary. Here is what seems to work on trunk (though I do not trust it is doing the right thing): {code} hbase create 'SOME_TABLENAME', {NAME = 'd', VERSION = 1, COMPRESSION = 'LZO'} hbase # Here is how I added metadata hbase disable 'SOME_TABLENAME' hbase alter 'SOME_TABLENAME', METHOD = 'table_att', OWNER = 'SOMEON', CONFIG = {'ENVIRONMENT' = 'BLAH BLAH', 'SIZING' = 'The size should be between 0-10K most of the time with new URLs coming in and getting removed as they are processed unless the pipeline has fallen behind', 'MISCELLANEOUS' = 'Holds the list of URLs waiting to be processed in the parked page detection analyzer in ingestion pipeline.'} ... describe... enable... {code} The above doesn't work in 0.94. Complains about the CONFIG, the keyword we are using for the HTD dictionary. It works in 0.96 though I'd have to poke around some more to ensure it is doing the right thing. But this METHOD = 'table_att' stuff is really ugly can we fix it? And I can't add table attributes on table create seemingly. A little bit of thought and a bit of ruby could clean this all up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6894) Adding metadata to a table in the shell is both arcane and painful
[ https://issues.apache.org/jira/browse/HBASE-6894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13487146#comment-13487146 ] stack commented on HBASE-6894: -- The doc on the alter and create help is nice and clean. Thanks. What does this do in the alter help screen? hbase alter 't1', 'f1', {NAME = 'f2', IN_MEMORY = true}, 'f3' Did you add the new delete method? Thats nicer. I'd say don't mention the old method at all in the help: hbase alter 't1', NAME = 'f1', METHOD = 'delete' hbase alter 't1', 'delete' = 'f1' Is this right? You can also change table-scope attributes like MAX_FILESIZE, READONLY, MEMSTORE_FLUSHSIZE, DEFERRED_LOG_FLUSH, etc. These can be put at the end; for example, to change the max size of a family to 128MB, do: hbase alter 't1', MAX_FILESIZE = '134217728' Says change max filesize of family but its for the table? We have to keep table_att for the below? hbase alter 't1', METHOD = 'table_att', 'coprocessor'='hdfs:///foo.jar|com.foo.FooRegionObserver|1001|arg1=1,arg2=2' Do you think CONFIG a good name for the 'dictionary' that can be in HTableDescriptor and HColumnDescriptor? Would DICT or DICTIONARY or MAP be better names than CONFIG? For example, in the below: hbase alter 't1', { NAME = 'f1', VERSIONS = 3 }, { MAX_FILESIZE = '134217728' }, { METHOD = 'delete', NAME = 'f2' }, OWNER = 'johndoe', CONFIG = { 'mykey' = 'myvalue' } Should the CONFIG on the end be DICT? or MAP? I created a table with some table attributes. When I describe it, the METHOD = 'table_att' is a little confusing I'd say? Do we have to have that METHOD = 'table_att' in there? Could be a dictionary named 'table_att'? or 'table _attributes'? I was thinking I could add a new table attribute 'METADATA' doing the following but fails. Should I be able too? hbase(main):007:0 create 'y', {NAME = 'x', CONFIG = {'xx' = 'yy'}}, OWNER = 'jim', 'METADATA' = 'xyz', CONFIG = {'aa' = 'bb'} An argument ignored (unknown or overridden): METADATA 0 row(s) in 1.0470 seconds This is nice: hbase(main):009:0 create 'z', {NAME = 'x', CONFIG = {'xx' = 'yy'}}, OWNER = 'jim', CONFIG = {'METADATA' = 'xyz', 'USAGE' = 'usage', 'aa' = 'bb'} 0 row(s) in 1.0440 seconds = Hbase::Table - z hbase(main):010:0 describe 'z' DESCRIPTION ENABLED 'z', {METHOD = 'table_att', CONFIG = {'METADATA' = 'xyz', 'OWNER' = 'jim', 'USAGE' = 'usage', 'aa' = 'bb'}}, {NAME = 'x', D true ATA_BLOCK_ENCODING = 'NONE', BLOOMFILTER = 'NONE', TTL = '2147483647', IN_MEMORY = 'false', REPLICATION_SCOPE = '0', VERSIONS = '3', COMPRESSION = 'NONE', MIN_VERSIONS = '0', KEEP_DELETED_CELLS = 'false', BLOCKSIZE = '65536', ENCODE_ON_DISK = 'true' , BLOCKCACHE = 'true', CONFIG = {'xx' = 'yy'}} 1 row(s) in 0.0310 seconds You have fixed the original issue. Thanks. I think it'd be good if could fix how the above is displayed when I do describe. The original CONFIG way of naming the dictionary I think confusing and worth replacing and the tostring which shows table dictionary stuff as a METHOD ='table_att' is a little odd. This is great: hbase(main):011:0 disable 'z' 0 row(s) in 7.0600 seconds hbase(main):012:0 alter 'z', OWNER = 'sergey' Updating all regions with the new schema... 1/1 regions updated. Done. 0 row(s) in 6.0810 seconds hbase(main):013:0 describe 'z' DESCRIPTION ENABLED 'z', {METHOD = 'table_att', CONFIG = {'METADATA' = 'xyz', 'OWNER' = 'sergey', 'USAGE' = 'usage', 'aa' = 'bb'}}, {NAME = 'x' false , DATA_BLOCK_ENCODING = 'NONE', BLOOMFILTER = 'NONE', TTL = '2147483647', IN_MEMORY = 'false', REPLICATION_SCOPE = '0', VERSI ONS = '3', COMPRESSION = 'NONE', MIN_VERSIONS = '0', KEEP_DELETED_CELLS = 'false', BLOCKSIZE = '65536', ENCODE_ON_DISK = 'tr ue', BLOCKCACHE = 'true', CONFIG = {'xx' = 'yy'}} 1 row(s) in 0.0280
[jira] [Commented] (HBASE-6894) Adding metadata to a table in the shell is both arcane and painful
[ https://issues.apache.org/jira/browse/HBASE-6894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13486403#comment-13486403 ] stack commented on HBASE-6894: -- Yes, tests run as part of mvn test. There is a junit wrapper that invokes all the ruby ones. Release notes are generated from the release note field in each JIRA. Edit the JIRA and you should see a box to fill release notes into. If you that, describing your change, I'll try your patch against your release note. Thanks. Adding metadata to a table in the shell is both arcane and painful -- Key: HBASE-6894 URL: https://issues.apache.org/jira/browse/HBASE-6894 Project: HBase Issue Type: Bug Components: shell Affects Versions: 0.96.0 Reporter: stack Assignee: Sergey Shelukhin Labels: noob Attachments: HBASE-6894.patch, HBASE-6894.patch, HBASE-6894.patch, HBASE-6894-v2.patch, HBASE-6894-v3-squashed.patch In production we have hundreds of tables w/ whack names like 'aliaserv', 'ashish_bulk', 'age_gender_topics', etc. It be grand if you could look in master UI and see stuff like owner, eng group responsible, miscellaneous description, etc. Now, HTD has support for this; each carries a dictionary. Whats a PITA though is adding attributes to the dictionary. Here is what seems to work on trunk (though I do not trust it is doing the right thing): {code} hbase create 'SOME_TABLENAME', {NAME = 'd', VERSION = 1, COMPRESSION = 'LZO'} hbase # Here is how I added metadata hbase disable 'SOME_TABLENAME' hbase alter 'SOME_TABLENAME', METHOD = 'table_att', OWNER = 'SOMEON', CONFIG = {'ENVIRONMENT' = 'BLAH BLAH', 'SIZING' = 'The size should be between 0-10K most of the time with new URLs coming in and getting removed as they are processed unless the pipeline has fallen behind', 'MISCELLANEOUS' = 'Holds the list of URLs waiting to be processed in the parked page detection analyzer in ingestion pipeline.'} ... describe... enable... {code} The above doesn't work in 0.94. Complains about the CONFIG, the keyword we are using for the HTD dictionary. It works in 0.96 though I'd have to poke around some more to ensure it is doing the right thing. But this METHOD = 'table_att' stuff is really ugly can we fix it? And I can't add table attributes on table create seemingly. A little bit of thought and a bit of ruby could clean this all up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6894) Adding metadata to a table in the shell is both arcane and painful
[ https://issues.apache.org/jira/browse/HBASE-6894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13486449#comment-13486449 ] Hadoop QA commented on HBASE-6894: -- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12551252/HBASE-6894-v3-squashed.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 3 new or modified tests. {color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile. {color:red}-1 javadoc{color}. The javadoc tool appears to have generated 85 warning messages. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:red}-1 findbugs{color}. The patch appears to introduce 3 new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 core tests{color}. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3174//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3174//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3174//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3174//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3174//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3174//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3174//console This message is automatically generated. Adding metadata to a table in the shell is both arcane and painful -- Key: HBASE-6894 URL: https://issues.apache.org/jira/browse/HBASE-6894 Project: HBase Issue Type: Bug Components: shell Affects Versions: 0.96.0 Reporter: stack Assignee: Sergey Shelukhin Labels: noob Attachments: HBASE-6894.patch, HBASE-6894.patch, HBASE-6894.patch, HBASE-6894-v2.patch, HBASE-6894-v3-squashed.patch In production we have hundreds of tables w/ whack names like 'aliaserv', 'ashish_bulk', 'age_gender_topics', etc. It be grand if you could look in master UI and see stuff like owner, eng group responsible, miscellaneous description, etc. Now, HTD has support for this; each carries a dictionary. Whats a PITA though is adding attributes to the dictionary. Here is what seems to work on trunk (though I do not trust it is doing the right thing): {code} hbase create 'SOME_TABLENAME', {NAME = 'd', VERSION = 1, COMPRESSION = 'LZO'} hbase # Here is how I added metadata hbase disable 'SOME_TABLENAME' hbase alter 'SOME_TABLENAME', METHOD = 'table_att', OWNER = 'SOMEON', CONFIG = {'ENVIRONMENT' = 'BLAH BLAH', 'SIZING' = 'The size should be between 0-10K most of the time with new URLs coming in and getting removed as they are processed unless the pipeline has fallen behind', 'MISCELLANEOUS' = 'Holds the list of URLs waiting to be processed in the parked page detection analyzer in ingestion pipeline.'} ... describe... enable... {code} The above doesn't work in 0.94. Complains about the CONFIG, the keyword we are using for the HTD dictionary. It works in 0.96 though I'd have to poke around some more to ensure it is doing the right thing. But this METHOD = 'table_att' stuff is really ugly can we fix it? And I can't add table attributes on table create seemingly. A little bit of thought and a bit of ruby could clean this all up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6894) Adding metadata to a table in the shell is both arcane and painful
[ https://issues.apache.org/jira/browse/HBASE-6894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13486476#comment-13486476 ] Sergey Shelukhin commented on HBASE-6894: - updated release notes Adding metadata to a table in the shell is both arcane and painful -- Key: HBASE-6894 URL: https://issues.apache.org/jira/browse/HBASE-6894 Project: HBase Issue Type: Bug Components: shell Affects Versions: 0.96.0 Reporter: stack Assignee: Sergey Shelukhin Labels: noob Attachments: HBASE-6894.patch, HBASE-6894.patch, HBASE-6894.patch, HBASE-6894-v2.patch, HBASE-6894-v3-squashed.patch In production we have hundreds of tables w/ whack names like 'aliaserv', 'ashish_bulk', 'age_gender_topics', etc. It be grand if you could look in master UI and see stuff like owner, eng group responsible, miscellaneous description, etc. Now, HTD has support for this; each carries a dictionary. Whats a PITA though is adding attributes to the dictionary. Here is what seems to work on trunk (though I do not trust it is doing the right thing): {code} hbase create 'SOME_TABLENAME', {NAME = 'd', VERSION = 1, COMPRESSION = 'LZO'} hbase # Here is how I added metadata hbase disable 'SOME_TABLENAME' hbase alter 'SOME_TABLENAME', METHOD = 'table_att', OWNER = 'SOMEON', CONFIG = {'ENVIRONMENT' = 'BLAH BLAH', 'SIZING' = 'The size should be between 0-10K most of the time with new URLs coming in and getting removed as they are processed unless the pipeline has fallen behind', 'MISCELLANEOUS' = 'Holds the list of URLs waiting to be processed in the parked page detection analyzer in ingestion pipeline.'} ... describe... enable... {code} The above doesn't work in 0.94. Complains about the CONFIG, the keyword we are using for the HTD dictionary. It works in 0.96 though I'd have to poke around some more to ensure it is doing the right thing. But this METHOD = 'table_att' stuff is really ugly can we fix it? And I can't add table attributes on table create seemingly. A little bit of thought and a bit of ruby could clean this all up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6894) Adding metadata to a table in the shell is both arcane and painful
[ https://issues.apache.org/jira/browse/HBASE-6894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13486545#comment-13486545 ] Hadoop QA commented on HBASE-6894: -- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12551275/HBASE-6894-v3.1-squashed.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 3 new or modified tests. {color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile. {color:red}-1 javadoc{color}. The javadoc tool appears to have generated 85 warning messages. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:red}-1 findbugs{color}. The patch appears to introduce 3 new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 core tests{color}. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3175//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3175//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3175//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3175//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3175//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3175//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3175//console This message is automatically generated. Adding metadata to a table in the shell is both arcane and painful -- Key: HBASE-6894 URL: https://issues.apache.org/jira/browse/HBASE-6894 Project: HBase Issue Type: Bug Components: shell Affects Versions: 0.96.0 Reporter: stack Assignee: Sergey Shelukhin Labels: noob Attachments: HBASE-6894.patch, HBASE-6894.patch, HBASE-6894.patch, HBASE-6894-v2.patch, HBASE-6894-v3.1-squashed.patch, HBASE-6894-v3-squashed.patch In production we have hundreds of tables w/ whack names like 'aliaserv', 'ashish_bulk', 'age_gender_topics', etc. It be grand if you could look in master UI and see stuff like owner, eng group responsible, miscellaneous description, etc. Now, HTD has support for this; each carries a dictionary. Whats a PITA though is adding attributes to the dictionary. Here is what seems to work on trunk (though I do not trust it is doing the right thing): {code} hbase create 'SOME_TABLENAME', {NAME = 'd', VERSION = 1, COMPRESSION = 'LZO'} hbase # Here is how I added metadata hbase disable 'SOME_TABLENAME' hbase alter 'SOME_TABLENAME', METHOD = 'table_att', OWNER = 'SOMEON', CONFIG = {'ENVIRONMENT' = 'BLAH BLAH', 'SIZING' = 'The size should be between 0-10K most of the time with new URLs coming in and getting removed as they are processed unless the pipeline has fallen behind', 'MISCELLANEOUS' = 'Holds the list of URLs waiting to be processed in the parked page detection analyzer in ingestion pipeline.'} ... describe... enable... {code} The above doesn't work in 0.94. Complains about the CONFIG, the keyword we are using for the HTD dictionary. It works in 0.96 though I'd have to poke around some more to ensure it is doing the right thing. But this METHOD = 'table_att' stuff is really ugly can we fix it? And I can't add table attributes on table create seemingly. A little bit of thought and a bit of ruby could clean this all up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6894) Adding metadata to a table in the shell is both arcane and painful
[ https://issues.apache.org/jira/browse/HBASE-6894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13485284#comment-13485284 ] stack commented on HBASE-6894: -- [~sershe] Pardon my not getting to this earlier (I think it important patch). Patch looks good. I need to try it. Does the help need to change to match your changes? Say, the help around 'create' and for 'alter'? Would you mind listing in the release notes for this issue how the user's view on shell operations has changed? Once that is there I will have something to test against. Thanks for doing this Sergey. Adding metadata to a table in the shell is both arcane and painful -- Key: HBASE-6894 URL: https://issues.apache.org/jira/browse/HBASE-6894 Project: HBase Issue Type: Bug Components: shell Affects Versions: 0.96.0 Reporter: stack Assignee: Sergey Shelukhin Labels: noob Attachments: HBASE-6894.patch, HBASE-6894.patch, HBASE-6894.patch, HBASE-6894-v2.patch In production we have hundreds of tables w/ whack names like 'aliaserv', 'ashish_bulk', 'age_gender_topics', etc. It be grand if you could look in master UI and see stuff like owner, eng group responsible, miscellaneous description, etc. Now, HTD has support for this; each carries a dictionary. Whats a PITA though is adding attributes to the dictionary. Here is what seems to work on trunk (though I do not trust it is doing the right thing): {code} hbase create 'SOME_TABLENAME', {NAME = 'd', VERSION = 1, COMPRESSION = 'LZO'} hbase # Here is how I added metadata hbase disable 'SOME_TABLENAME' hbase alter 'SOME_TABLENAME', METHOD = 'table_att', OWNER = 'SOMEON', CONFIG = {'ENVIRONMENT' = 'BLAH BLAH', 'SIZING' = 'The size should be between 0-10K most of the time with new URLs coming in and getting removed as they are processed unless the pipeline has fallen behind', 'MISCELLANEOUS' = 'Holds the list of URLs waiting to be processed in the parked page detection analyzer in ingestion pipeline.'} ... describe... enable... {code} The above doesn't work in 0.94. Complains about the CONFIG, the keyword we are using for the HTD dictionary. It works in 0.96 though I'd have to poke around some more to ensure it is doing the right thing. But this METHOD = 'table_att' stuff is really ugly can we fix it? And I can't add table attributes on table create seemingly. A little bit of thought and a bit of ruby could clean this all up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6894) Adding metadata to a table in the shell is both arcane and painful
[ https://issues.apache.org/jira/browse/HBASE-6894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13482850#comment-13482850 ] Sergey Shelukhin commented on HBASE-6894: - [~stack] Hi, will you eventually be able to review the new patch? Thanks. Adding metadata to a table in the shell is both arcane and painful -- Key: HBASE-6894 URL: https://issues.apache.org/jira/browse/HBASE-6894 Project: HBase Issue Type: Bug Components: shell Affects Versions: 0.96.0 Reporter: stack Assignee: Sergey Shelukhin Labels: noob Attachments: HBASE-6894.patch, HBASE-6894.patch, HBASE-6894.patch, HBASE-6894-v2.patch In production we have hundreds of tables w/ whack names like 'aliaserv', 'ashish_bulk', 'age_gender_topics', etc. It be grand if you could look in master UI and see stuff like owner, eng group responsible, miscellaneous description, etc. Now, HTD has support for this; each carries a dictionary. Whats a PITA though is adding attributes to the dictionary. Here is what seems to work on trunk (though I do not trust it is doing the right thing): {code} hbase create 'SOME_TABLENAME', {NAME = 'd', VERSION = 1, COMPRESSION = 'LZO'} hbase # Here is how I added metadata hbase disable 'SOME_TABLENAME' hbase alter 'SOME_TABLENAME', METHOD = 'table_att', OWNER = 'SOMEON', CONFIG = {'ENVIRONMENT' = 'BLAH BLAH', 'SIZING' = 'The size should be between 0-10K most of the time with new URLs coming in and getting removed as they are processed unless the pipeline has fallen behind', 'MISCELLANEOUS' = 'Holds the list of URLs waiting to be processed in the parked page detection analyzer in ingestion pipeline.'} ... describe... enable... {code} The above doesn't work in 0.94. Complains about the CONFIG, the keyword we are using for the HTD dictionary. It works in 0.96 though I'd have to poke around some more to ensure it is doing the right thing. But this METHOD = 'table_att' stuff is really ugly can we fix it? And I can't add table attributes on table create seemingly. A little bit of thought and a bit of ruby could clean this all up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6894) Adding metadata to a table in the shell is both arcane and painful
[ https://issues.apache.org/jira/browse/HBASE-6894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13480559#comment-13480559 ] Hadoop QA commented on HBASE-6894: -- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12550109/HBASE-6894-v2.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile. {color:red}-1 javadoc{color}. The javadoc tool appears to have generated 82 warning messages. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:red}-1 findbugs{color}. The patch appears to introduce 4 new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 core tests{color}. The patch failed these unit tests: org.apache.hadoop.hbase.client.TestShell Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3095//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3095//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3095//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3095//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3095//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3095//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3095//console This message is automatically generated. Adding metadata to a table in the shell is both arcane and painful -- Key: HBASE-6894 URL: https://issues.apache.org/jira/browse/HBASE-6894 Project: HBase Issue Type: Bug Components: shell Affects Versions: 0.96.0 Reporter: stack Assignee: Sergey Shelukhin Labels: noob Attachments: HBASE-6894.patch, HBASE-6894.patch, HBASE-6894.patch, HBASE-6894-v2.patch In production we have hundreds of tables w/ whack names like 'aliaserv', 'ashish_bulk', 'age_gender_topics', etc. It be grand if you could look in master UI and see stuff like owner, eng group responsible, miscellaneous description, etc. Now, HTD has support for this; each carries a dictionary. Whats a PITA though is adding attributes to the dictionary. Here is what seems to work on trunk (though I do not trust it is doing the right thing): {code} hbase create 'SOME_TABLENAME', {NAME = 'd', VERSION = 1, COMPRESSION = 'LZO'} hbase # Here is how I added metadata hbase disable 'SOME_TABLENAME' hbase alter 'SOME_TABLENAME', METHOD = 'table_att', OWNER = 'SOMEON', CONFIG = {'ENVIRONMENT' = 'BLAH BLAH', 'SIZING' = 'The size should be between 0-10K most of the time with new URLs coming in and getting removed as they are processed unless the pipeline has fallen behind', 'MISCELLANEOUS' = 'Holds the list of URLs waiting to be processed in the parked page detection analyzer in ingestion pipeline.'} ... describe... enable... {code} The above doesn't work in 0.94. Complains about the CONFIG, the keyword we are using for the HTD dictionary. It works in 0.96 though I'd have to poke around some more to ensure it is doing the right thing. But this METHOD = 'table_att' stuff is really ugly can we fix it? And I can't add table attributes on table create seemingly. A little bit of thought and a bit of ruby could clean this all up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6894) Adding metadata to a table in the shell is both arcane and painful
[ https://issues.apache.org/jira/browse/HBASE-6894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13477352#comment-13477352 ] Sergey Shelukhin commented on HBASE-6894: - ping? :) Adding metadata to a table in the shell is both arcane and painful -- Key: HBASE-6894 URL: https://issues.apache.org/jira/browse/HBASE-6894 Project: HBase Issue Type: Bug Components: shell Affects Versions: 0.96.0 Reporter: stack Assignee: Sergey Shelukhin Labels: noob Attachments: HBASE-6894.patch, HBASE-6894.patch, HBASE-6894.patch In production we have hundreds of tables w/ whack names like 'aliaserv', 'ashish_bulk', 'age_gender_topics', etc. It be grand if you could look in master UI and see stuff like owner, eng group responsible, miscellaneous description, etc. Now, HTD has support for this; each carries a dictionary. Whats a PITA though is adding attributes to the dictionary. Here is what seems to work on trunk (though I do not trust it is doing the right thing): {code} hbase create 'SOME_TABLENAME', {NAME = 'd', VERSION = 1, COMPRESSION = 'LZO'} hbase # Here is how I added metadata hbase disable 'SOME_TABLENAME' hbase alter 'SOME_TABLENAME', METHOD = 'table_att', OWNER = 'SOMEON', CONFIG = {'ENVIRONMENT' = 'BLAH BLAH', 'SIZING' = 'The size should be between 0-10K most of the time with new URLs coming in and getting removed as they are processed unless the pipeline has fallen behind', 'MISCELLANEOUS' = 'Holds the list of URLs waiting to be processed in the parked page detection analyzer in ingestion pipeline.'} ... describe... enable... {code} The above doesn't work in 0.94. Complains about the CONFIG, the keyword we are using for the HTD dictionary. It works in 0.96 though I'd have to poke around some more to ensure it is doing the right thing. But this METHOD = 'table_att' stuff is really ugly can we fix it? And I can't add table attributes on table create seemingly. A little bit of thought and a bit of ruby could clean this all up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6894) Adding metadata to a table in the shell is both arcane and painful
[ https://issues.apache.org/jira/browse/HBASE-6894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13476532#comment-13476532 ] Hadoop QA commented on HBASE-6894: -- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12549216/HBASE-6894.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile. {color:red}-1 javadoc{color}. The javadoc tool appears to have generated 82 warning messages. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:red}-1 findbugs{color}. The patch appears to introduce 5 new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 core tests{color}. The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.TestSplitTransaction Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3054//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3054//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3054//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3054//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3054//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3054//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3054//console This message is automatically generated. Adding metadata to a table in the shell is both arcane and painful -- Key: HBASE-6894 URL: https://issues.apache.org/jira/browse/HBASE-6894 Project: HBase Issue Type: Bug Components: shell Affects Versions: 0.96.0 Reporter: stack Labels: noob Attachments: HBASE-6894.patch, HBASE-6894.patch, HBASE-6894.patch In production we have hundreds of tables w/ whack names like 'aliaserv', 'ashish_bulk', 'age_gender_topics', etc. It be grand if you could look in master UI and see stuff like owner, eng group responsible, miscellaneous description, etc. Now, HTD has support for this; each carries a dictionary. Whats a PITA though is adding attributes to the dictionary. Here is what seems to work on trunk (though I do not trust it is doing the right thing): {code} hbase create 'SOME_TABLENAME', {NAME = 'd', VERSION = 1, COMPRESSION = 'LZO'} hbase # Here is how I added metadata hbase disable 'SOME_TABLENAME' hbase alter 'SOME_TABLENAME', METHOD = 'table_att', OWNER = 'SOMEON', CONFIG = {'ENVIRONMENT' = 'BLAH BLAH', 'SIZING' = 'The size should be between 0-10K most of the time with new URLs coming in and getting removed as they are processed unless the pipeline has fallen behind', 'MISCELLANEOUS' = 'Holds the list of URLs waiting to be processed in the parked page detection analyzer in ingestion pipeline.'} ... describe... enable... {code} The above doesn't work in 0.94. Complains about the CONFIG, the keyword we are using for the HTD dictionary. It works in 0.96 though I'd have to poke around some more to ensure it is doing the right thing. But this METHOD = 'table_att' stuff is really ugly can we fix it? And I can't add table attributes on table create seemingly. A little bit of thought and a bit of ruby could clean this all up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6894) Adding metadata to a table in the shell is both arcane and painful
[ https://issues.apache.org/jira/browse/HBASE-6894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13475183#comment-13475183 ] Sergey Shelukhin commented on HBASE-6894: - 1) MAX_FILESIZE/etc. are not actually part of CONFIG. Config is only the random key/values, which is one of table /attributes/ set via table_att; the other attributes are readonly, max filesize, owner, etc. So it's not a flat structure right now (it's probably a good idea to keep it that way just in case users use some random keys in their config that conflict with newly added system keys in future versions). 2) If you specify A = B, C = D, they will all be a part of one arg. Right now that would mean that you can use either METHOD = table_att, CONFIG = (random k/v), MAX_FILESIZE = ..., or TABLE_CONFIG = (random k/v). And you cannot add any column families after, in the same command. You can enclose parts in {} to make them separate args. I can rewrite parameter handling to intelligently (or hackily :)) figure out from one big hash what user wants, but that probably won't be backward compatible. And as per (1) TABLE_CONFIG probably shouldn't be flat for system and user config, as in your example. 3) As far as I see SPLITs are not handled in alter. Where did you find it in help? Adding metadata to a table in the shell is both arcane and painful -- Key: HBASE-6894 URL: https://issues.apache.org/jira/browse/HBASE-6894 Project: HBase Issue Type: Bug Components: shell Affects Versions: 0.96.0 Reporter: stack Labels: noob Attachments: HBASE-6894.patch, HBASE-6894.patch In production we have hundreds of tables w/ whack names like 'aliaserv', 'ashish_bulk', 'age_gender_topics', etc. It be grand if you could look in master UI and see stuff like owner, eng group responsible, miscellaneous description, etc. Now, HTD has support for this; each carries a dictionary. Whats a PITA though is adding attributes to the dictionary. Here is what seems to work on trunk (though I do not trust it is doing the right thing): {code} hbase create 'SOME_TABLENAME', {NAME = 'd', VERSION = 1, COMPRESSION = 'LZO'} hbase # Here is how I added metadata hbase disable 'SOME_TABLENAME' hbase alter 'SOME_TABLENAME', METHOD = 'table_att', OWNER = 'SOMEON', CONFIG = {'ENVIRONMENT' = 'BLAH BLAH', 'SIZING' = 'The size should be between 0-10K most of the time with new URLs coming in and getting removed as they are processed unless the pipeline has fallen behind', 'MISCELLANEOUS' = 'Holds the list of URLs waiting to be processed in the parked page detection analyzer in ingestion pipeline.'} ... describe... enable... {code} The above doesn't work in 0.94. Complains about the CONFIG, the keyword we are using for the HTD dictionary. It works in 0.96 though I'd have to poke around some more to ensure it is doing the right thing. But this METHOD = 'table_att' stuff is really ugly can we fix it? And I can't add table attributes on table create seemingly. A little bit of thought and a bit of ruby could clean this all up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6894) Adding metadata to a table in the shell is both arcane and painful
[ https://issues.apache.org/jira/browse/HBASE-6894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13475308#comment-13475308 ] stack commented on HBASE-6894: -- On 3., see help 'create'. On 2., do what you think best. I'd be fine w/ CONFIG (or DICTIONARY? or DICT) for the arbitrary kvs and explicit, predefined constants like MAX_FILESIZE being individually called out. It'd be grand if we did not have to have TABLE_CONFIG to refer to the dictionary in HTableDescriptor and then CONFIG for referring to it in the column family; these are not well named. If we can't have the same key for table arbitrary attributes and column family attributes, then I'd say lets at least name them better even if its not backward compatible. I don't think this table attribute stuff has to be backward compatible. I'd be surprised any one even knew it existed never mind using it. Good on you Sergey. Thanks for looking at this. Adding metadata to a table in the shell is both arcane and painful -- Key: HBASE-6894 URL: https://issues.apache.org/jira/browse/HBASE-6894 Project: HBase Issue Type: Bug Components: shell Affects Versions: 0.96.0 Reporter: stack Labels: noob Attachments: HBASE-6894.patch, HBASE-6894.patch In production we have hundreds of tables w/ whack names like 'aliaserv', 'ashish_bulk', 'age_gender_topics', etc. It be grand if you could look in master UI and see stuff like owner, eng group responsible, miscellaneous description, etc. Now, HTD has support for this; each carries a dictionary. Whats a PITA though is adding attributes to the dictionary. Here is what seems to work on trunk (though I do not trust it is doing the right thing): {code} hbase create 'SOME_TABLENAME', {NAME = 'd', VERSION = 1, COMPRESSION = 'LZO'} hbase # Here is how I added metadata hbase disable 'SOME_TABLENAME' hbase alter 'SOME_TABLENAME', METHOD = 'table_att', OWNER = 'SOMEON', CONFIG = {'ENVIRONMENT' = 'BLAH BLAH', 'SIZING' = 'The size should be between 0-10K most of the time with new URLs coming in and getting removed as they are processed unless the pipeline has fallen behind', 'MISCELLANEOUS' = 'Holds the list of URLs waiting to be processed in the parked page detection analyzer in ingestion pipeline.'} ... describe... enable... {code} The above doesn't work in 0.94. Complains about the CONFIG, the keyword we are using for the HTD dictionary. It works in 0.96 though I'd have to poke around some more to ensure it is doing the right thing. But this METHOD = 'table_att' stuff is really ugly can we fix it? And I can't add table attributes on table create seemingly. A little bit of thought and a bit of ruby could clean this all up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6894) Adding metadata to a table in the shell is both arcane and painful
[ https://issues.apache.org/jira/browse/HBASE-6894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13474425#comment-13474425 ] Sergey Shelukhin commented on HBASE-6894: - For create, create 'test', 'cf', {METHOD = 'table_att', CONFIG = {'Key' = 'Value'}} seems to work on trunk, it's not on 0.94 though. CONFIG is also not handled in 0.94 in alter as far as I see. Given the variation of create args in a flat list, I don't think anything can be done with create in backward-compatible manner, other than maybe adding another method such as 'table_config' to flatten it a bit. For altering attributes/CONFIG, do you want separate command(s)? Adding metadata to a table in the shell is both arcane and painful -- Key: HBASE-6894 URL: https://issues.apache.org/jira/browse/HBASE-6894 Project: HBase Issue Type: Bug Components: shell Affects Versions: 0.94.1 Reporter: stack Labels: noob In production we have hundreds of tables w/ whack names like 'aliaserv', 'ashish_bulk', 'age_gender_topics', etc. It be grand if you could look in master UI and see stuff like owner, eng group responsible, miscellaneous description, etc. Now, HTD has support for this; each carries a dictionary. Whats a PITA though is adding attributes to the dictionary. Here is what seems to work on trunk (though I do not trust it is doing the right thing): {code} hbase create 'SOME_TABLENAME', {NAME = 'd', VERSION = 1, COMPRESSION = 'LZO'} hbase # Here is how I added metadata hbase disable 'SOME_TABLENAME' hbase alter 'SOME_TABLENAME', METHOD = 'table_att', OWNER = 'SOMEON', CONFIG = {'ENVIRONMENT' = 'BLAH BLAH', 'SIZING' = 'The size should be between 0-10K most of the time with new URLs coming in and getting removed as they are processed unless the pipeline has fallen behind', 'MISCELLANEOUS' = 'Holds the list of URLs waiting to be processed in the parked page detection analyzer in ingestion pipeline.'} ... describe... enable... {code} The above doesn't work in 0.94. Complains about the CONFIG, the keyword we are using for the HTD dictionary. It works in 0.96 though I'd have to poke around some more to ensure it is doing the right thing. But this METHOD = 'table_att' stuff is really ugly can we fix it? And I can't add table attributes on table create seemingly. A little bit of thought and a bit of ruby could clean this all up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6894) Adding metadata to a table in the shell is both arcane and painful
[ https://issues.apache.org/jira/browse/HBASE-6894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13474437#comment-13474437 ] stack commented on HBASE-6894: -- [~sershe] Yeah, works on trunk. If what was in trunk worked setting multiple attributes even though its ugly with its METHOD = 'table_att', that should be ok? Otherwise, whatever you think best. Would be good if we didn't have to add new commands. Thanks for looking at this Sergey. Adding metadata to a table in the shell is both arcane and painful -- Key: HBASE-6894 URL: https://issues.apache.org/jira/browse/HBASE-6894 Project: HBase Issue Type: Bug Components: shell Affects Versions: 0.94.1 Reporter: stack Labels: noob In production we have hundreds of tables w/ whack names like 'aliaserv', 'ashish_bulk', 'age_gender_topics', etc. It be grand if you could look in master UI and see stuff like owner, eng group responsible, miscellaneous description, etc. Now, HTD has support for this; each carries a dictionary. Whats a PITA though is adding attributes to the dictionary. Here is what seems to work on trunk (though I do not trust it is doing the right thing): {code} hbase create 'SOME_TABLENAME', {NAME = 'd', VERSION = 1, COMPRESSION = 'LZO'} hbase # Here is how I added metadata hbase disable 'SOME_TABLENAME' hbase alter 'SOME_TABLENAME', METHOD = 'table_att', OWNER = 'SOMEON', CONFIG = {'ENVIRONMENT' = 'BLAH BLAH', 'SIZING' = 'The size should be between 0-10K most of the time with new URLs coming in and getting removed as they are processed unless the pipeline has fallen behind', 'MISCELLANEOUS' = 'Holds the list of URLs waiting to be processed in the parked page detection analyzer in ingestion pipeline.'} ... describe... enable... {code} The above doesn't work in 0.94. Complains about the CONFIG, the keyword we are using for the HTD dictionary. It works in 0.96 though I'd have to poke around some more to ensure it is doing the right thing. But this METHOD = 'table_att' stuff is really ugly can we fix it? And I can't add table attributes on table create seemingly. A little bit of thought and a bit of ruby could clean this all up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6894) Adding metadata to a table in the shell is both arcane and painful
[ https://issues.apache.org/jira/browse/HBASE-6894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13474567#comment-13474567 ] Sergey Shelukhin commented on HBASE-6894: - What do you mean by multiple attributes? It seems to work with these (if you mean in CONFIG), and also work with multiple args if the whole table_att is enclosed in {}. At any rate, I am adding the following syntax to both create alter (just CONFIG cannot be used because Hash column descriptor can also have CONFIG so there'd be confusion). create 'table', 'cf', TABLE_CONFIG = {'Key' = 'Value'} or { TABLE_CONFIG = ... } This is already supported for OWNER btw, e.g. you don't have to specify table_att to set owner. Other than that, there's a glitch with type validation of CONFIG value; fixed that. Do you want it to be backported in 0.94? Adding metadata to a table in the shell is both arcane and painful -- Key: HBASE-6894 URL: https://issues.apache.org/jira/browse/HBASE-6894 Project: HBase Issue Type: Bug Components: shell Affects Versions: 0.94.1 Reporter: stack Labels: noob Attachments: HBASE-6894.patch In production we have hundreds of tables w/ whack names like 'aliaserv', 'ashish_bulk', 'age_gender_topics', etc. It be grand if you could look in master UI and see stuff like owner, eng group responsible, miscellaneous description, etc. Now, HTD has support for this; each carries a dictionary. Whats a PITA though is adding attributes to the dictionary. Here is what seems to work on trunk (though I do not trust it is doing the right thing): {code} hbase create 'SOME_TABLENAME', {NAME = 'd', VERSION = 1, COMPRESSION = 'LZO'} hbase # Here is how I added metadata hbase disable 'SOME_TABLENAME' hbase alter 'SOME_TABLENAME', METHOD = 'table_att', OWNER = 'SOMEON', CONFIG = {'ENVIRONMENT' = 'BLAH BLAH', 'SIZING' = 'The size should be between 0-10K most of the time with new URLs coming in and getting removed as they are processed unless the pipeline has fallen behind', 'MISCELLANEOUS' = 'Holds the list of URLs waiting to be processed in the parked page detection analyzer in ingestion pipeline.'} ... describe... enable... {code} The above doesn't work in 0.94. Complains about the CONFIG, the keyword we are using for the HTD dictionary. It works in 0.96 though I'd have to poke around some more to ensure it is doing the right thing. But this METHOD = 'table_att' stuff is really ugly can we fix it? And I can't add table attributes on table create seemingly. A little bit of thought and a bit of ruby could clean this all up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6894) Adding metadata to a table in the shell is both arcane and painful
[ https://issues.apache.org/jira/browse/HBASE-6894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13474570#comment-13474570 ] Sergey Shelukhin commented on HBASE-6894: - Actually it may make sense to change return to next in alter in the attached patch, likewise for OWNER. Not sure why it's returning. Adding metadata to a table in the shell is both arcane and painful -- Key: HBASE-6894 URL: https://issues.apache.org/jira/browse/HBASE-6894 Project: HBase Issue Type: Bug Components: shell Affects Versions: 0.94.1 Reporter: stack Labels: noob Attachments: HBASE-6894.patch In production we have hundreds of tables w/ whack names like 'aliaserv', 'ashish_bulk', 'age_gender_topics', etc. It be grand if you could look in master UI and see stuff like owner, eng group responsible, miscellaneous description, etc. Now, HTD has support for this; each carries a dictionary. Whats a PITA though is adding attributes to the dictionary. Here is what seems to work on trunk (though I do not trust it is doing the right thing): {code} hbase create 'SOME_TABLENAME', {NAME = 'd', VERSION = 1, COMPRESSION = 'LZO'} hbase # Here is how I added metadata hbase disable 'SOME_TABLENAME' hbase alter 'SOME_TABLENAME', METHOD = 'table_att', OWNER = 'SOMEON', CONFIG = {'ENVIRONMENT' = 'BLAH BLAH', 'SIZING' = 'The size should be between 0-10K most of the time with new URLs coming in and getting removed as they are processed unless the pipeline has fallen behind', 'MISCELLANEOUS' = 'Holds the list of URLs waiting to be processed in the parked page detection analyzer in ingestion pipeline.'} ... describe... enable... {code} The above doesn't work in 0.94. Complains about the CONFIG, the keyword we are using for the HTD dictionary. It works in 0.96 though I'd have to poke around some more to ensure it is doing the right thing. But this METHOD = 'table_att' stuff is really ugly can we fix it? And I can't add table attributes on table create seemingly. A little bit of thought and a bit of ruby could clean this all up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6894) Adding metadata to a table in the shell is both arcane and painful
[ https://issues.apache.org/jira/browse/HBASE-6894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13474690#comment-13474690 ] Sergey Shelukhin commented on HBASE-6894: - For now changing to 0.96, the entire CONFIG feature, incl. previous commits, would need to be backported to 0.94 if that is needed. Adding metadata to a table in the shell is both arcane and painful -- Key: HBASE-6894 URL: https://issues.apache.org/jira/browse/HBASE-6894 Project: HBase Issue Type: Bug Components: shell Affects Versions: 0.96.0 Reporter: stack Labels: noob Attachments: HBASE-6894.patch, HBASE-6894.patch In production we have hundreds of tables w/ whack names like 'aliaserv', 'ashish_bulk', 'age_gender_topics', etc. It be grand if you could look in master UI and see stuff like owner, eng group responsible, miscellaneous description, etc. Now, HTD has support for this; each carries a dictionary. Whats a PITA though is adding attributes to the dictionary. Here is what seems to work on trunk (though I do not trust it is doing the right thing): {code} hbase create 'SOME_TABLENAME', {NAME = 'd', VERSION = 1, COMPRESSION = 'LZO'} hbase # Here is how I added metadata hbase disable 'SOME_TABLENAME' hbase alter 'SOME_TABLENAME', METHOD = 'table_att', OWNER = 'SOMEON', CONFIG = {'ENVIRONMENT' = 'BLAH BLAH', 'SIZING' = 'The size should be between 0-10K most of the time with new URLs coming in and getting removed as they are processed unless the pipeline has fallen behind', 'MISCELLANEOUS' = 'Holds the list of URLs waiting to be processed in the parked page detection analyzer in ingestion pipeline.'} ... describe... enable... {code} The above doesn't work in 0.94. Complains about the CONFIG, the keyword we are using for the HTD dictionary. It works in 0.96 though I'd have to poke around some more to ensure it is doing the right thing. But this METHOD = 'table_att' stuff is really ugly can we fix it? And I can't add table attributes on table create seemingly. A little bit of thought and a bit of ruby could clean this all up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6894) Adding metadata to a table in the shell is both arcane and painful
[ https://issues.apache.org/jira/browse/HBASE-6894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13474728#comment-13474728 ] Hadoop QA commented on HBASE-6894: -- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12548834/HBASE-6894.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile. {color:red}-1 javadoc{color}. The javadoc tool appears to have generated 81 warning messages. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:red}-1 findbugs{color}. The patch appears to introduce 5 new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 core tests{color}. The patch failed these unit tests: org.apache.hadoop.hbase.master.TestSplitLogManager Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3038//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3038//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3038//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3038//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3038//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3038//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3038//console This message is automatically generated. Adding metadata to a table in the shell is both arcane and painful -- Key: HBASE-6894 URL: https://issues.apache.org/jira/browse/HBASE-6894 Project: HBase Issue Type: Bug Components: shell Affects Versions: 0.96.0 Reporter: stack Labels: noob Attachments: HBASE-6894.patch, HBASE-6894.patch In production we have hundreds of tables w/ whack names like 'aliaserv', 'ashish_bulk', 'age_gender_topics', etc. It be grand if you could look in master UI and see stuff like owner, eng group responsible, miscellaneous description, etc. Now, HTD has support for this; each carries a dictionary. Whats a PITA though is adding attributes to the dictionary. Here is what seems to work on trunk (though I do not trust it is doing the right thing): {code} hbase create 'SOME_TABLENAME', {NAME = 'd', VERSION = 1, COMPRESSION = 'LZO'} hbase # Here is how I added metadata hbase disable 'SOME_TABLENAME' hbase alter 'SOME_TABLENAME', METHOD = 'table_att', OWNER = 'SOMEON', CONFIG = {'ENVIRONMENT' = 'BLAH BLAH', 'SIZING' = 'The size should be between 0-10K most of the time with new URLs coming in and getting removed as they are processed unless the pipeline has fallen behind', 'MISCELLANEOUS' = 'Holds the list of URLs waiting to be processed in the parked page detection analyzer in ingestion pipeline.'} ... describe... enable... {code} The above doesn't work in 0.94. Complains about the CONFIG, the keyword we are using for the HTD dictionary. It works in 0.96 though I'd have to poke around some more to ensure it is doing the right thing. But this METHOD = 'table_att' stuff is really ugly can we fix it? And I can't add table attributes on table create seemingly. A little bit of thought and a bit of ruby could clean this all up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6894) Adding metadata to a table in the shell is both arcane and painful
[ https://issues.apache.org/jira/browse/HBASE-6894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13474767#comment-13474767 ] Lars Hofhansl commented on HBASE-6894: -- If it's small and easy to grok and there is need I would not mind to have this in 0.94. Adding metadata to a table in the shell is both arcane and painful -- Key: HBASE-6894 URL: https://issues.apache.org/jira/browse/HBASE-6894 Project: HBase Issue Type: Bug Components: shell Affects Versions: 0.96.0 Reporter: stack Labels: noob Attachments: HBASE-6894.patch, HBASE-6894.patch In production we have hundreds of tables w/ whack names like 'aliaserv', 'ashish_bulk', 'age_gender_topics', etc. It be grand if you could look in master UI and see stuff like owner, eng group responsible, miscellaneous description, etc. Now, HTD has support for this; each carries a dictionary. Whats a PITA though is adding attributes to the dictionary. Here is what seems to work on trunk (though I do not trust it is doing the right thing): {code} hbase create 'SOME_TABLENAME', {NAME = 'd', VERSION = 1, COMPRESSION = 'LZO'} hbase # Here is how I added metadata hbase disable 'SOME_TABLENAME' hbase alter 'SOME_TABLENAME', METHOD = 'table_att', OWNER = 'SOMEON', CONFIG = {'ENVIRONMENT' = 'BLAH BLAH', 'SIZING' = 'The size should be between 0-10K most of the time with new URLs coming in and getting removed as they are processed unless the pipeline has fallen behind', 'MISCELLANEOUS' = 'Holds the list of URLs waiting to be processed in the parked page detection analyzer in ingestion pipeline.'} ... describe... enable... {code} The above doesn't work in 0.94. Complains about the CONFIG, the keyword we are using for the HTD dictionary. It works in 0.96 though I'd have to poke around some more to ensure it is doing the right thing. But this METHOD = 'table_att' stuff is really ugly can we fix it? And I can't add table attributes on table create seemingly. A little bit of thought and a bit of ruby could clean this all up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6894) Adding metadata to a table in the shell is both arcane and painful
[ https://issues.apache.org/jira/browse/HBASE-6894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13474803#comment-13474803 ] stack commented on HBASE-6894: -- What I meant by 'multiple attributes' is being able to specify multiple attributes when setting TABLE_CONFIG. TABLE_CONFIG is a big improvement over METHOD + table_att. Its a pity that Can I write TABLE_CONFIG = {MAX_FILESIZE = '123456', READONLY = true, SOME_VALUE_A = 'a', SOME_VALUE_B = 'b'} and set the maximum file size, the table to read only, and then add two random key/values to the HTableDescriptor dictionary? You might need to have some special handling in apply_config? Or do I have to write that as MAX_FILESIZE = '123456', READONLY = true, TABLE_CONFIG = {SOME_VALUE_A = 'a', SOME_VALUE_B = 'b'}? If I look at the create help, it looks like you can pass split info after column families. These only make sense at create time I suppose, these split specifications.. they are not table attributes. Poking around the shell help, I don't see mention of CF CONFIG. This is used to add arbitrary key/values to the HColumnDescriptor dictionary? There is no way we could tell from context whether the CONFIG is for HTableDescriptor or for HColumnDescriptor? Is CONFIG a good name? Should it be DICT or KEYVALUE or ATTRIBUTES? Thanks for looking into this Sergey. I think it'd be great if this made more 'sense'. Adding metadata to a table in the shell is both arcane and painful -- Key: HBASE-6894 URL: https://issues.apache.org/jira/browse/HBASE-6894 Project: HBase Issue Type: Bug Components: shell Affects Versions: 0.96.0 Reporter: stack Labels: noob Attachments: HBASE-6894.patch, HBASE-6894.patch In production we have hundreds of tables w/ whack names like 'aliaserv', 'ashish_bulk', 'age_gender_topics', etc. It be grand if you could look in master UI and see stuff like owner, eng group responsible, miscellaneous description, etc. Now, HTD has support for this; each carries a dictionary. Whats a PITA though is adding attributes to the dictionary. Here is what seems to work on trunk (though I do not trust it is doing the right thing): {code} hbase create 'SOME_TABLENAME', {NAME = 'd', VERSION = 1, COMPRESSION = 'LZO'} hbase # Here is how I added metadata hbase disable 'SOME_TABLENAME' hbase alter 'SOME_TABLENAME', METHOD = 'table_att', OWNER = 'SOMEON', CONFIG = {'ENVIRONMENT' = 'BLAH BLAH', 'SIZING' = 'The size should be between 0-10K most of the time with new URLs coming in and getting removed as they are processed unless the pipeline has fallen behind', 'MISCELLANEOUS' = 'Holds the list of URLs waiting to be processed in the parked page detection analyzer in ingestion pipeline.'} ... describe... enable... {code} The above doesn't work in 0.94. Complains about the CONFIG, the keyword we are using for the HTD dictionary. It works in 0.96 though I'd have to poke around some more to ensure it is doing the right thing. But this METHOD = 'table_att' stuff is really ugly can we fix it? And I can't add table attributes on table create seemingly. A little bit of thought and a bit of ruby could clean this all up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira