[jira] [Commented] (HIVE-6936) Provide table properties to InputFormats
[ https://issues.apache.org/jira/browse/HIVE-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14139512#comment-14139512 ] Ashutosh Chauhan commented on HIVE-6936: +1 > Provide table properties to InputFormats > > > Key: HIVE-6936 > URL: https://issues.apache.org/jira/browse/HIVE-6936 > Project: Hive > Issue Type: Bug > Components: File Formats >Reporter: Owen O'Malley >Assignee: Owen O'Malley > Fix For: 0.14.0 > > Attachments: HIVE-6936.patch, HIVE-6936.patch, HIVE-6936.patch, > HIVE-6936.patch, HIVE-6936.patch, HIVE-6936.patch, HIVE-6936.patch, > HIVE-6936.patch, HIVE-6936.patch, HIVE-6936.patch > > > Some advanced file formats need the table properties made available to them. > Additionally, it would be convenient to provide a unique id for fetch > operators and the complete list of directories. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-6936) Provide table properties to InputFormats
[ https://issues.apache.org/jira/browse/HIVE-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14139069#comment-14139069 ] Hive QA commented on HIVE-6936: --- {color:red}Overall{color}: -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12669527/HIVE-6936.patch {color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 6281 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_schemeAuthority org.apache.hive.hcatalog.pig.TestOrcHCatLoader.testReadDataPrimitiveTypes {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/860/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/860/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-860/ Messages: {noformat} Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 2 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12669527 > Provide table properties to InputFormats > > > Key: HIVE-6936 > URL: https://issues.apache.org/jira/browse/HIVE-6936 > Project: Hive > Issue Type: Bug > Components: File Formats >Reporter: Owen O'Malley >Assignee: Owen O'Malley > Fix For: 0.14.0 > > Attachments: HIVE-6936.patch, HIVE-6936.patch, HIVE-6936.patch, > HIVE-6936.patch, HIVE-6936.patch, HIVE-6936.patch, HIVE-6936.patch, > HIVE-6936.patch, HIVE-6936.patch, HIVE-6936.patch > > > Some advanced file formats need the table properties made available to them. > Additionally, it would be convenient to provide a unique id for fetch > operators and the complete list of directories. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-6936) Provide table properties to InputFormats
[ https://issues.apache.org/jira/browse/HIVE-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14137102#comment-14137102 ] Hive QA commented on HIVE-6936: --- {color:green}Overall{color}: +1 all checks pass Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12669225/HIVE-6936.patch {color:green}SUCCESS:{color} +1 6280 tests passed Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/838/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/838/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-838/ Messages: {noformat} Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase {noformat} This message is automatically generated. ATTACHMENT ID: 12669225 > Provide table properties to InputFormats > > > Key: HIVE-6936 > URL: https://issues.apache.org/jira/browse/HIVE-6936 > Project: Hive > Issue Type: Bug > Components: File Formats >Reporter: Owen O'Malley >Assignee: Owen O'Malley > Fix For: 0.14.0 > > Attachments: HIVE-6936.patch, HIVE-6936.patch, HIVE-6936.patch, > HIVE-6936.patch, HIVE-6936.patch, HIVE-6936.patch, HIVE-6936.patch, > HIVE-6936.patch, HIVE-6936.patch > > > Some advanced file formats need the table properties made available to them. > Additionally, it would be convenient to provide a unique id for fetch > operators and the complete list of directories. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-6936) Provide table properties to InputFormats
[ https://issues.apache.org/jira/browse/HIVE-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14011338#comment-14011338 ] Ashutosh Chauhan commented on HIVE-6936: Failures look unrelated +1 for latest patch > Provide table properties to InputFormats > > > Key: HIVE-6936 > URL: https://issues.apache.org/jira/browse/HIVE-6936 > Project: Hive > Issue Type: Bug > Components: File Formats >Reporter: Owen O'Malley >Assignee: Owen O'Malley > Fix For: 0.14.0 > > Attachments: HIVE-6936.patch, HIVE-6936.patch, HIVE-6936.patch, > HIVE-6936.patch, HIVE-6936.patch, HIVE-6936.patch, HIVE-6936.patch, > HIVE-6936.patch > > > Some advanced file formats need the table properties made available to them. > Additionally, it would be convenient to provide a unique id for fetch > operators and the complete list of directories. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6936) Provide table properties to InputFormats
[ https://issues.apache.org/jira/browse/HIVE-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14010968#comment-14010968 ] Hive QA commented on HIVE-6936: --- {color:red}Overall{color}: -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12646920/HIVE-6936.patch {color:red}ERROR:{color} -1 due to 13 failed/errored test(s), 5543 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_parquet_decimal1 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_dynpart_sort_optimization org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_load_dyn_part1 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_scriptfile1 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_dml org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_ctas org.apache.hadoop.hive.metastore.TestMetastoreVersion.testDefaults org.apache.hadoop.hive.ql.exec.tez.TestTezTask.testSubmit org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimal org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimalX org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimalXY org.apache.hive.jdbc.miniHS2.TestHiveServer2.testConnection {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/317/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/317/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-317/ Messages: {noformat} Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 13 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12646920 > Provide table properties to InputFormats > > > Key: HIVE-6936 > URL: https://issues.apache.org/jira/browse/HIVE-6936 > Project: Hive > Issue Type: Bug > Components: File Formats >Reporter: Owen O'Malley >Assignee: Owen O'Malley > Fix For: 0.14.0 > > Attachments: HIVE-6936.patch, HIVE-6936.patch, HIVE-6936.patch, > HIVE-6936.patch, HIVE-6936.patch, HIVE-6936.patch, HIVE-6936.patch, > HIVE-6936.patch > > > Some advanced file formats need the table properties made available to them. > Additionally, it would be convenient to provide a unique id for fetch > operators and the complete list of directories. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6936) Provide table properties to InputFormats
[ https://issues.apache.org/jira/browse/HIVE-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14008262#comment-14008262 ] Hive QA commented on HIVE-6936: --- {color:red}Overall{color}: -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12646677/HIVE-6936.patch {color:red}ERROR:{color} -1 due to 9 failed/errored test(s), 5539 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_multi_single_reducer3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_parquet_decimal1 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_dynpart_sort_optimization org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_load_dyn_part1 org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table org.apache.hadoop.hive.common.metrics.TestMetrics.testScopeConcurrency org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimal org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimalX org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimalXY {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/284/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/284/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-284/ Messages: {noformat} Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 9 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12646677 > Provide table properties to InputFormats > > > Key: HIVE-6936 > URL: https://issues.apache.org/jira/browse/HIVE-6936 > Project: Hive > Issue Type: Bug > Components: File Formats >Reporter: Owen O'Malley >Assignee: Owen O'Malley > Fix For: 0.14.0 > > Attachments: HIVE-6936.patch, HIVE-6936.patch, HIVE-6936.patch, > HIVE-6936.patch, HIVE-6936.patch, HIVE-6936.patch, HIVE-6936.patch > > > Some advanced file formats need the table properties made available to them. > Additionally, it would be convenient to provide a unique id for fetch > operators and the complete list of directories. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6936) Provide table properties to InputFormats
[ https://issues.apache.org/jira/browse/HIVE-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14007417#comment-14007417 ] Hive QA commented on HIVE-6936: --- {color:red}Overall{color}: -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12646394/HIVE-6936.patch {color:red}ERROR:{color} -1 due to 703 failed/errored test(s), 5458 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_add_part_multiple org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_char1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_numbuckets_partitioned_table2_h23 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_numbuckets_partitioned_table_h23 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_partition_coltype org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_rename_partition_authorization org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_table_serde2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_varchar1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_filter org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_groupby org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_join org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_limit org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_part org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_select org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_table org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_union org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_archive_excludeHadoop20 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_archive_multi org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_6 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_index org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join14 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join17 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join19 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join24 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join25 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join26 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join5 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join6 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join7 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join8 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join9 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join_reordering_values org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_smb_mapjoin_14 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_10 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_13 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_14 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_15 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_6 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_9 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_avro_decimal org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ba_table1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ba_table2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ba_table3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ba_table_udfs org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ba_table_union org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_binary_table_bincolserde org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_binary_table_colserde org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucket1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucket2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucket3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucket_groupby org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin13 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriv
[jira] [Commented] (HIVE-6936) Provide table properties to InputFormats
[ https://issues.apache.org/jira/browse/HIVE-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14006523#comment-14006523 ] Ashutosh Chauhan commented on HIVE-6936: +1 > Provide table properties to InputFormats > > > Key: HIVE-6936 > URL: https://issues.apache.org/jira/browse/HIVE-6936 > Project: Hive > Issue Type: Bug > Components: File Formats >Reporter: Owen O'Malley >Assignee: Owen O'Malley > Fix For: 0.14.0 > > Attachments: HIVE-6936.patch, HIVE-6936.patch, HIVE-6936.patch, > HIVE-6936.patch, HIVE-6936.patch > > > Some advanced file formats need the table properties made available to them. > Additionally, it would be convenient to provide a unique id for fetch > operators and the complete list of directories. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6936) Provide table properties to InputFormats
[ https://issues.apache.org/jira/browse/HIVE-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14006512#comment-14006512 ] Ashutosh Chauhan commented on HIVE-6936: Sounds good to me. > Provide table properties to InputFormats > > > Key: HIVE-6936 > URL: https://issues.apache.org/jira/browse/HIVE-6936 > Project: Hive > Issue Type: Bug > Components: File Formats >Reporter: Owen O'Malley >Assignee: Owen O'Malley > Fix For: 0.14.0 > > Attachments: HIVE-6936.patch, HIVE-6936.patch, HIVE-6936.patch, > HIVE-6936.patch > > > Some advanced file formats need the table properties made available to them. > Additionally, it would be convenient to provide a unique id for fetch > operators and the complete list of directories. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6936) Provide table properties to InputFormats
[ https://issues.apache.org/jira/browse/HIVE-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14006505#comment-14006505 ] Owen O'Malley commented on HIVE-6936: - My concern with removing the "table." prefix is that it becomes possible to clobber security settings. For example, creating a table with the "hive.security.authorization.manager" table property shouldn't allow the query to override the setting. For a compromise, how about removing the table prefix, but we check to see if that key has a value in the configuration and won't override such a value. Does that make sense? > Provide table properties to InputFormats > > > Key: HIVE-6936 > URL: https://issues.apache.org/jira/browse/HIVE-6936 > Project: Hive > Issue Type: Bug > Components: File Formats >Reporter: Owen O'Malley >Assignee: Owen O'Malley > Fix For: 0.14.0 > > Attachments: HIVE-6936.patch, HIVE-6936.patch, HIVE-6936.patch, > HIVE-6936.patch > > > Some advanced file formats need the table properties made available to them. > Additionally, it would be convenient to provide a unique id for fetch > operators and the complete list of directories. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6936) Provide table properties to InputFormats
[ https://issues.apache.org/jira/browse/HIVE-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14006218#comment-14006218 ] Xuefu Zhang commented on HIVE-6936: --- {quote} I am in double mind about adding table prefix, although its useful from namespacing point of view, but than users also need to deal with this extra prefix while accessing properties in their IF. {quote} I sympathize with [~ashutoshc] as to the prefix approach. Even we do, the prefix currently chosen (table.) seems so commonly used that it may be in fight with user's existing properties, causing unexpected result. > Provide table properties to InputFormats > > > Key: HIVE-6936 > URL: https://issues.apache.org/jira/browse/HIVE-6936 > Project: Hive > Issue Type: Bug > Components: File Formats >Reporter: Owen O'Malley >Assignee: Owen O'Malley > Fix For: 0.14.0 > > Attachments: HIVE-6936.patch, HIVE-6936.patch, HIVE-6936.patch > > > Some advanced file formats need the table properties made available to them. > Additionally, it would be convenient to provide a unique id for fetch > operators and the complete list of directories. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6936) Provide table properties to InputFormats
[ https://issues.apache.org/jira/browse/HIVE-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14006180#comment-14006180 ] Owen O'Malley commented on HIVE-6936: - I've also created a review board entry: https://reviews.apache.org/r/21818/ > Provide table properties to InputFormats > > > Key: HIVE-6936 > URL: https://issues.apache.org/jira/browse/HIVE-6936 > Project: Hive > Issue Type: Bug > Components: File Formats >Reporter: Owen O'Malley >Assignee: Owen O'Malley > Fix For: 0.14.0 > > Attachments: HIVE-6936.patch, HIVE-6936.patch, HIVE-6936.patch > > > Some advanced file formats need the table properties made available to them. > Additionally, it would be convenient to provide a unique id for fetch > operators and the complete list of directories. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6936) Provide table properties to InputFormats
[ https://issues.apache.org/jira/browse/HIVE-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13980317#comment-13980317 ] Sushanth Sowmyan commented on HIVE-6936: Looking into it. Will provide a quick description of how it's handled in HCat: In HCat, we pass along tableProperties using HiveStorageHandler.configureInputJobProperties and HiveStorageHandler.configureOutputJobProperties. HBaseSH sees only this. To standardize usage across all IF/OF cases, we have a storage handler called FosterStorageHandler that acts as a wrapper around the generic IF/OF cases that do not have storage handlers, so that the rest of the HCat code can still stick to using StorageHandlers as-is and not worry about the underlying system. To make FosterStorageHandler work with orc, we had to change orc so it accepted table property overrides from jobconf as well (since we have to conform to mapreduce interfaces only, and that has no knowledge of tables), and had FosterStorageHandler set up jobconf appropriately with any overrides from table properties. > Provide table properties to InputFormats > > > Key: HIVE-6936 > URL: https://issues.apache.org/jira/browse/HIVE-6936 > Project: Hive > Issue Type: Bug > Components: File Formats >Reporter: Owen O'Malley >Assignee: Owen O'Malley > Fix For: 0.14.0 > > Attachments: HIVE-6936.patch, HIVE-6936.patch > > > Some advanced file formats need the table properties made available to them. > Additionally, it would be convenient to provide a unique id for fetch > operators and the complete list of directories. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6936) Provide table properties to InputFormats
[ https://issues.apache.org/jira/browse/HIVE-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13979919#comment-13979919 ] Ashutosh Chauhan commented on HIVE-6936: [~owen.omalley] Can you create RB request for this? Some high level comments: * It will be useful to list use-case for table properties in IF, UUID in FetchOp and dir lists for FetchOp. * I am in double mind about adding table prefix, although its useful from namespacing point of view, but than users also need to deal with this extra prefix while accessing properties in their IF. * [~sushanth] I believe we also play same game for HBaseIF & HCatIF (of supplying table properties to them). Can you take a look at the patch and see if we are doing this in a common & consistent way? * Also, consider to use one of test data files in data/files/ instead of adding a new one. > Provide table properties to InputFormats > > > Key: HIVE-6936 > URL: https://issues.apache.org/jira/browse/HIVE-6936 > Project: Hive > Issue Type: Bug > Components: File Formats >Reporter: Owen O'Malley >Assignee: Owen O'Malley > Fix For: 0.14.0 > > Attachments: HIVE-6936.patch, HIVE-6936.patch > > > Some advanced file formats need the table properties made available to them. > Additionally, it would be convenient to provide a unique id for fetch > operators and the complete list of directories. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6936) Provide table properties to InputFormats
[ https://issues.apache.org/jira/browse/HIVE-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13975629#comment-13975629 ] Owen O'Malley commented on HIVE-6936: - Sorry, I closed the wrong bug! > Provide table properties to InputFormats > > > Key: HIVE-6936 > URL: https://issues.apache.org/jira/browse/HIVE-6936 > Project: Hive > Issue Type: Bug > Components: File Formats >Reporter: Owen O'Malley >Assignee: Owen O'Malley > Fix For: 0.14.0 > > Attachments: HIVE-6936.patch > > > Some advanced file formats need the table properties made available to them. > Additionally, it would be convenient to provide a unique id for fetch > operators and the complete list of directories. -- This message was sent by Atlassian JIRA (v6.2#6252)