[jira] [Commented] (HIVE-4044) Add URL type
[ https://issues.apache.org/jira/browse/HIVE-4044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13681573#comment-13681573 ] Samuel Yuan commented on HIVE-4044: --- I tried breaking the URL into parts and encoding them as individual columns; the dictionary shrunk, but the overhead of the other ORC columns introduced (mostly the column of indices) made a bigger impact, so compression was actually worse overall. I also tried storing the query string as a map and putting common keys into separate columns; this improved compression somewhat, but still not enough to offset the overhead of new columns for the query string. > Add URL type > > > Key: HIVE-4044 > URL: https://issues.apache.org/jira/browse/HIVE-4044 > Project: Hive > Issue Type: Improvement >Reporter: Samuel Yuan >Assignee: Samuel Yuan > Attachments: HIVE-4044.HIVE-4044.HIVE-4044.D8799.1.patch > > > Having a separate type for URLs would enable improvements in storage > efficiency based on breaking up a URL into its components. The new type will > be named "URL" and made a non-reserved keyword (see HIVE-701). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4492) Revert HIVE-4322
[ https://issues.apache.org/jira/browse/HIVE-4492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samuel Yuan updated HIVE-4492: -- Status: Patch Available (was: Open) > Revert HIVE-4322 > > > Key: HIVE-4492 > URL: https://issues.apache.org/jira/browse/HIVE-4492 > Project: Hive > Issue Type: Bug > Components: Metastore, Thrift API >Reporter: Samuel Yuan >Assignee: Samuel Yuan > Attachments: HIVE-4492.1.patch.txt > > > See HIVE-4432 and HIVE-4433. It's possible to work around these issues but a > better solution is probably to roll back the "fix" and change the API to use > a primitive type as the map key (in a backwards-compatible manner). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4492) Revert HIVE-4322
[ https://issues.apache.org/jira/browse/HIVE-4492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samuel Yuan updated HIVE-4492: -- Attachment: HIVE-4492.1.patch.txt > Revert HIVE-4322 > > > Key: HIVE-4492 > URL: https://issues.apache.org/jira/browse/HIVE-4492 > Project: Hive > Issue Type: Bug > Components: Metastore, Thrift API >Reporter: Samuel Yuan >Assignee: Samuel Yuan > Attachments: HIVE-4492.1.patch.txt > > > See HIVE-4432 and HIVE-4433. It's possible to work around these issues but a > better solution is probably to roll back the "fix" and change the API to use > a primitive type as the map key (in a backwards-compatible manner). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4433) Fix C++ Thrift bindings broken in HIVE-4322
[ https://issues.apache.org/jira/browse/HIVE-4433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13648683#comment-13648683 ] Samuel Yuan commented on HIVE-4433: --- I'm thinking it's possible to work around this by defining '<' since it's present in the auto-generated header file. Given that other language bindings might also have been broken by HIVE-4322 though it's probably better to change the map key to a primitive type instead. I have filed HIVE-4492 to revert the original change. > Fix C++ Thrift bindings broken in HIVE-4322 > --- > > Key: HIVE-4433 > URL: https://issues.apache.org/jira/browse/HIVE-4433 > Project: Hive > Issue Type: Bug > Components: Metastore, Thrift API >Affects Versions: 0.12.0 >Reporter: Carl Steinbach >Assignee: Samuel Yuan >Priority: Blocker > Fix For: 0.12.0 > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4492) Revert HIVE-4322
Samuel Yuan created HIVE-4492: - Summary: Revert HIVE-4322 Key: HIVE-4492 URL: https://issues.apache.org/jira/browse/HIVE-4492 Project: Hive Issue Type: Bug Components: Metastore, Thrift API Reporter: Samuel Yuan Assignee: Samuel Yuan See HIVE-4432 and HIVE-4433. It's possible to work around these issues but a better solution is probably to roll back the "fix" and change the API to use a primitive type as the map key (in a backwards-compatible manner). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4474) Column access not tracked properly for partitioned tables
[ https://issues.apache.org/jira/browse/HIVE-4474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samuel Yuan updated HIVE-4474: -- Status: Patch Available (was: Open) > Column access not tracked properly for partitioned tables > - > > Key: HIVE-4474 > URL: https://issues.apache.org/jira/browse/HIVE-4474 > Project: Hive > Issue Type: Bug > Components: Query Processor >Reporter: Samuel Yuan >Assignee: Samuel Yuan > Attachments: HIVE-4474.1.patch.txt > > > The columns recorded as being accessed is incorrect for partitioned tables. > The index of accessed columns is a position in the list of non-partition > columns, but a list of all columns is being used right now to do the lookup. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4474) Column access not tracked properly for partitioned tables
[ https://issues.apache.org/jira/browse/HIVE-4474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samuel Yuan updated HIVE-4474: -- Attachment: HIVE-4474.1.patch.txt > Column access not tracked properly for partitioned tables > - > > Key: HIVE-4474 > URL: https://issues.apache.org/jira/browse/HIVE-4474 > Project: Hive > Issue Type: Bug > Components: Query Processor >Reporter: Samuel Yuan >Assignee: Samuel Yuan > Attachments: HIVE-4474.1.patch.txt > > > The columns recorded as being accessed is incorrect for partitioned tables. > The index of accessed columns is a position in the list of non-partition > columns, but a list of all columns is being used right now to do the lookup. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4474) Column access not tracked properly for partitioned tables
Samuel Yuan created HIVE-4474: - Summary: Column access not tracked properly for partitioned tables Key: HIVE-4474 URL: https://issues.apache.org/jira/browse/HIVE-4474 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Samuel Yuan Assignee: Samuel Yuan The columns recorded as being accessed is incorrect for partitioned tables. The index of accessed columns is a position in the list of non-partition columns, but a list of all columns is being used right now to do the lookup. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4221) Stripe-level merge for ORC files
[ https://issues.apache.org/jira/browse/HIVE-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samuel Yuan updated HIVE-4221: -- Status: Patch Available (was: Open) > Stripe-level merge for ORC files > > > Key: HIVE-4221 > URL: https://issues.apache.org/jira/browse/HIVE-4221 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Reporter: Samuel Yuan >Assignee: Samuel Yuan > Attachments: HIVE-4221.HIVE-4221.HIVE-4221.HIVE-4221.D9759.1.patch, > HIVE-4221.HIVE-4221.HIVE-4221.HIVE-4221.D9759.2.patch > > > As with RC files, we would like to be able to merge ORC files efficiently by > reading/writing stripes without decompressing/recompressing them. This will > be similar to the RC file merge, except that footers will have to be updated > with the stripe positions in the new file. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4433) Fix C++ Thrift bindings broken in HIVE-4322
[ https://issues.apache.org/jira/browse/HIVE-4433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13643497#comment-13643497 ] Samuel Yuan commented on HIVE-4433: --- The problem is that we were originally using a list as the map key, which is worse (at least for Python). Other alternatives were considered, but since the list could contain arbitrary strings, replacing it with a struct wrapper seemed to be the best solution. Would you mind posting a log of what's breaking in C++? > Fix C++ Thrift bindings broken in HIVE-4322 > --- > > Key: HIVE-4433 > URL: https://issues.apache.org/jira/browse/HIVE-4433 > Project: Hive > Issue Type: Bug > Components: Metastore, Thrift API >Affects Versions: 0.11.0 >Reporter: Carl Steinbach >Assignee: Samuel Yuan >Priority: Blocker > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4432) Follow-up to HIVE-4322 - make metastore API changes backwards compatible
[ https://issues.apache.org/jira/browse/HIVE-4432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samuel Yuan updated HIVE-4432: -- Description: Right now the fix for HIVE-4322 makes different versions of the metastore server and client incompatible with each other. This can make deployment very painful. > Follow-up to HIVE-4322 - make metastore API changes backwards compatible > > > Key: HIVE-4432 > URL: https://issues.apache.org/jira/browse/HIVE-4432 > Project: Hive > Issue Type: Bug > Components: Metastore, Thrift API >Reporter: Samuel Yuan >Assignee: Samuel Yuan > > Right now the fix for HIVE-4322 makes different versions of the metastore > server and client incompatible with each other. This can make deployment very > painful. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4432) Follow-up to HIVE-4322 - make metastore API changes backwards compatible
Samuel Yuan created HIVE-4432: - Summary: Follow-up to HIVE-4322 - make metastore API changes backwards compatible Key: HIVE-4432 URL: https://issues.apache.org/jira/browse/HIVE-4432 Project: Hive Issue Type: Bug Components: Metastore, Thrift API Reporter: Samuel Yuan Assignee: Samuel Yuan -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4344) CREATE VIEW fails when redundant casts are rewritten
[ https://issues.apache.org/jira/browse/HIVE-4344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samuel Yuan updated HIVE-4344: -- Status: Patch Available (was: Open) > CREATE VIEW fails when redundant casts are rewritten > > > Key: HIVE-4344 > URL: https://issues.apache.org/jira/browse/HIVE-4344 > Project: Hive > Issue Type: Bug > Components: Query Processor >Affects Versions: 0.11.0 >Reporter: Samuel Yuan >Assignee: Samuel Yuan > Attachments: HIVE-4344.HIVE-4344.HIVE-4344.HIVE-4344.D10221.1.patch > > > e.g. create view v as select cast(key as string) from src; > The rewriter tries to replace both cast(key as string) and key as > `src`.`key`, because cast(key as string) is a no-op. > There may be other cases like this one. > See HIVE-2439 for context. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4344) CREATE VIEW fails when redundant casts are rewritten
[ https://issues.apache.org/jira/browse/HIVE-4344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samuel Yuan updated HIVE-4344: -- Description: e.g. create view v as select cast(key as string) from src; The rewriter tries to replace both cast(key as string) and key as `src`.`key`, because cast(key as string) is a no-op. There may be other cases like this one. See HIVE-2439 for context. was: e.g. create view v as select cast(key as string) from src; The rewriter tries to replace both cast(key as string) and key as `src`.`key`, because cast(key as string) is a no-op. There may be other cases like this one. > CREATE VIEW fails when redundant casts are rewritten > > > Key: HIVE-4344 > URL: https://issues.apache.org/jira/browse/HIVE-4344 > Project: Hive > Issue Type: Bug > Components: Query Processor >Affects Versions: 0.11.0 >Reporter: Samuel Yuan >Assignee: Samuel Yuan > > e.g. create view v as select cast(key as string) from src; > The rewriter tries to replace both cast(key as string) and key as > `src`.`key`, because cast(key as string) is a no-op. > There may be other cases like this one. > See HIVE-2439 for context. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4322) SkewedInfo in Metastore Thrift API cannot be deserialized in Python
[ https://issues.apache.org/jira/browse/HIVE-4322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samuel Yuan updated HIVE-4322: -- Status: Patch Available (was: Open) > SkewedInfo in Metastore Thrift API cannot be deserialized in Python > --- > > Key: HIVE-4322 > URL: https://issues.apache.org/jira/browse/HIVE-4322 > Project: Hive > Issue Type: Bug > Components: Metastore, Thrift API >Affects Versions: 0.11.0 >Reporter: Samuel Yuan >Assignee: Samuel Yuan >Priority: Minor > Attachments: HIVE-4322.HIVE-4322.HIVE-4322.HIVE-4322.D10203.1.patch > > > The Thrift-generated Python code that deserializes Thrift objects fails > whenever a complex type is used as a map key, because by default mutable > Python objects such as lists do not have a hash function. See > https://issues.apache.org/jira/browse/THRIFT-162 for related discussion. > The SkewedInfo struct contains a map which uses a list as a key, breaking the > Python Thrift interface. It is not possible to specify the mapping from > Thrift types to Python types, or otherwise we could map Thrift lists to > Python tuples. Instead, the proposed workaround wraps the list inside a new > struct. This alone does not accomplish anything, but allows Python clients to > define a hash function for the struct class, e.g.: > def f(object): > return hash(tuple(object.skewedValueList)) > SkewedValueList.__hash__ = f > In practice a more efficient hash might be defined that does not involve > copying the list. The advantage of wrapping the list inside a struct is that > the client does not have to define the hash on the list itself, which would > change the behaviour of lists everywhere else in the code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4344) CREATE VIEW fails when redundant casts are rewritten
Samuel Yuan created HIVE-4344: - Summary: CREATE VIEW fails when redundant casts are rewritten Key: HIVE-4344 URL: https://issues.apache.org/jira/browse/HIVE-4344 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.11.0 Reporter: Samuel Yuan Assignee: Samuel Yuan e.g. create view v as select cast(key as string) from src; The rewriter tries to replace both cast(key as string) and key as `src`.`key`, because cast(key as string) is a no-op. There may be other cases like this one. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4337) Update list bucketing test results
[ https://issues.apache.org/jira/browse/HIVE-4337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samuel Yuan updated HIVE-4337: -- Status: Patch Available (was: Open) > Update list bucketing test results > -- > > Key: HIVE-4337 > URL: https://issues.apache.org/jira/browse/HIVE-4337 > Project: Hive > Issue Type: Test > Components: Tests >Affects Versions: 0.11.0 >Reporter: Samuel Yuan >Assignee: Samuel Yuan >Priority: Trivial > Attachments: HIVE-4337.HIVE-4337.HIVE-4337.D10131.1.patch > > > A recent change resulted in different output for the list bucketing tests, > which run for Hadoop23. The output files were not updated to reflect this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4337) Update list bucketing test results
[ https://issues.apache.org/jira/browse/HIVE-4337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samuel Yuan updated HIVE-4337: -- Status: Open (was: Patch Available) > Update list bucketing test results > -- > > Key: HIVE-4337 > URL: https://issues.apache.org/jira/browse/HIVE-4337 > Project: Hive > Issue Type: Test > Components: Tests >Affects Versions: 0.11.0 >Reporter: Samuel Yuan >Assignee: Samuel Yuan >Priority: Trivial > Attachments: HIVE-4337.HIVE-4337.HIVE-4337.D10131.1.patch > > > A recent change resulted in different output for the list bucketing tests, > which run for Hadoop23. The output files were not updated to reflect this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4337) Update list bucketing test results
Samuel Yuan created HIVE-4337: - Summary: Update list bucketing test results Key: HIVE-4337 URL: https://issues.apache.org/jira/browse/HIVE-4337 Project: Hive Issue Type: Test Components: Tests Affects Versions: 0.11.0 Reporter: Samuel Yuan Assignee: Samuel Yuan Priority: Trivial A recent change resulted in different output for the list bucketing tests, which run for Hadoop23. The output files were not updated to reflect this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4322) SkewedInfo in Metastore Thrift API cannot be deserialized in Python
Samuel Yuan created HIVE-4322: - Summary: SkewedInfo in Metastore Thrift API cannot be deserialized in Python Key: HIVE-4322 URL: https://issues.apache.org/jira/browse/HIVE-4322 Project: Hive Issue Type: Bug Components: Metastore, Thrift API Affects Versions: 0.11.0 Reporter: Samuel Yuan Assignee: Samuel Yuan Priority: Minor The Thrift-generated Python code that deserializes Thrift objects fails whenever a complex type is used as a map key, because by default mutable Python objects such as lists do not have a hash function. See https://issues.apache.org/jira/browse/THRIFT-162 for related discussion. The SkewedInfo struct contains a map which uses a list as a key, breaking the Python Thrift interface. It is not possible to specify the mapping from Thrift types to Python types, or otherwise we could map Thrift lists to Python tuples. Instead, the proposed workaround wraps the list inside a new struct. This alone does not accomplish anything, but allows Python clients to define a hash function for the struct class, e.g.: def f(object): return hash(tuple(object.skewedValueList)) SkewedValueList.__hash__ = f In practice a more efficient hash might be defined that does not involve copying the list. The advantage of wrapping the list inside a struct is that the client does not have to define the hash on the list itself, which would change the behaviour of lists everywhere else in the code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4067) Followup to HIVE-701: reduce ambiguity in grammar
[ https://issues.apache.org/jira/browse/HIVE-4067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samuel Yuan updated HIVE-4067: -- Status: Patch Available (was: Open) > Followup to HIVE-701: reduce ambiguity in grammar > - > > Key: HIVE-4067 > URL: https://issues.apache.org/jira/browse/HIVE-4067 > Project: Hive > Issue Type: Bug > Components: Query Processor >Affects Versions: 0.11.0 >Reporter: Samuel Yuan >Assignee: Samuel Yuan >Priority: Minor > Attachments: HIVE-4067.D8883.1.patch, > HIVE-4067.HIVE-4067.HIVE-4067.HIVE-4067.D8883.2.patch > > > After HIVE-701 the grammar has become much more ambiguous, and the > compilation generates a large number of warnings. Making FROM, DISTINCT, > PRESERVE, COLUMN, ALL, AND, OR, and NOT reserved keywords again reduces the > number of warnings to 134, up from the original 81 warnings but down from the > 565 after HIVE-701. Most of the remaining ambiguity is trivial, an example > being "KW_ELEM_TYPE | KW_KEY_TYPE | KW_VALUE_TYPE | identifier", and they are > all correctly handled by ANTLR. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4289) HCatalog build fails when behind a firewall
[ https://issues.apache.org/jira/browse/HIVE-4289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samuel Yuan updated HIVE-4289: -- Status: Patch Available (was: Open) > HCatalog build fails when behind a firewall > --- > > Key: HIVE-4289 > URL: https://issues.apache.org/jira/browse/HIVE-4289 > Project: Hive > Issue Type: Bug > Components: Build Infrastructure, HCatalog >Affects Versions: 0.11.0 >Reporter: Samuel Yuan >Assignee: Samuel Yuan > Attachments: HIVE-4289.HIVE-4289.HIVE-4289.HIVE-4289.D9921.1.patch > > > A bug in Maven makes it impossible to set a proxy for a Maven Ant POM task > (see https://jira.codehaus.org/browse/MANTTASKS-216). Building behind a > firewall results in the following error: > [artifact:pom] Downloading: org/apache/apache/11/apache-11.pom from > repository central at http://repo1.maven.org/maven2 > [artifact:pom] Transferring 14K from central > [artifact:pom] [WARNING] Unable to get resource 'org.apache:apache:pom:11' > from repository central (http://repo1.maven.org/maven2): Error transferring > file: No route to host > [artifact:pom] An error has occurred while processing the Maven artifact > tasks. > [artifact:pom] Diagnosis: > [artifact:pom] > [artifact:pom] Unable to initialize POM pom.xml: Cannot find parent: > org.apache:apache for project: > org.apache.hcatalog:hcatalog:pom:0.11.0-SNAPSHOT for project > org.apache.hcatalog:hcatalog:pom:0.11.0-SNAPSHOT > [artifact:pom] Unable to download the artifact from any repository > Despite the error message, Ant/Maven is actually able to retrieve the POM > file by using the proxy set for Ant. However, it mysteriously fails when > trying to retrieve the checksum, which causes the entire operation to fail. > Regardless, a proxy should be set through Maven's settings.xml file. Since > this is not possible, the only way to build HCat behind a firewall right now > is to manually fetch the POM file and have Maven read it from the cache. > Ideally we would fix this in Maven, but given that this issue has been > reported for a long time in a number of separate places I think it is more > practical to modify the HCatalog build to specify the POM as a dependency, > fetching it into the cache so that the artifact:pom task can succeed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4289) HCatalog build fails when behind a firewall
Samuel Yuan created HIVE-4289: - Summary: HCatalog build fails when behind a firewall Key: HIVE-4289 URL: https://issues.apache.org/jira/browse/HIVE-4289 Project: Hive Issue Type: Bug Components: Build Infrastructure, HCatalog Affects Versions: 0.11.0 Reporter: Samuel Yuan Assignee: Samuel Yuan A bug in Maven makes it impossible to set a proxy for a Maven Ant POM task (see https://jira.codehaus.org/browse/MANTTASKS-216). Building behind a firewall results in the following error: [artifact:pom] Downloading: org/apache/apache/11/apache-11.pom from repository central at http://repo1.maven.org/maven2 [artifact:pom] Transferring 14K from central [artifact:pom] [WARNING] Unable to get resource 'org.apache:apache:pom:11' from repository central (http://repo1.maven.org/maven2): Error transferring file: No route to host [artifact:pom] An error has occurred while processing the Maven artifact tasks. [artifact:pom] Diagnosis: [artifact:pom] [artifact:pom] Unable to initialize POM pom.xml: Cannot find parent: org.apache:apache for project: org.apache.hcatalog:hcatalog:pom:0.11.0-SNAPSHOT for project org.apache.hcatalog:hcatalog:pom:0.11.0-SNAPSHOT [artifact:pom] Unable to download the artifact from any repository Despite the error message, Ant/Maven is actually able to retrieve the POM file by using the proxy set for Ant. However, it mysteriously fails when trying to retrieve the checksum, which causes the entire operation to fail. Regardless, a proxy should be set through Maven's settings.xml file. Since this is not possible, the only way to build HCat behind a firewall right now is to manually fetch the POM file and have Maven read it from the cache. Ideally we would fix this in Maven, but given that this issue has been reported for a long time in a number of separate places I think it is more practical to modify the HCatalog build to specify the POM as a dependency, fetching it into the cache so that the artifact:pom task can succeed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4221) Stripe-level merge for ORC files
[ https://issues.apache.org/jira/browse/HIVE-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samuel Yuan updated HIVE-4221: -- Status: Patch Available (was: Open) > Stripe-level merge for ORC files > > > Key: HIVE-4221 > URL: https://issues.apache.org/jira/browse/HIVE-4221 > Project: Hive > Issue Type: Improvement > Components: Query Processor >Reporter: Samuel Yuan >Assignee: Samuel Yuan > Attachments: HIVE-4221.HIVE-4221.HIVE-4221.HIVE-4221.D9759.1.patch > > > As with RC files, we would like to be able to merge ORC files efficiently by > reading/writing stripes without decompressing/recompressing them. This will > be similar to the RC file merge, except that footers will have to be updated > with the stripe positions in the new file. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4221) Stripe-level merge for ORC files
Samuel Yuan created HIVE-4221: - Summary: Stripe-level merge for ORC files Key: HIVE-4221 URL: https://issues.apache.org/jira/browse/HIVE-4221 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Samuel Yuan Assignee: Samuel Yuan As with RC files, we would like to be able to merge ORC files efficiently by reading/writing stripes without decompressing/recompressing them. This will be similar to the RC file merge, except that footers will have to be updated with the stripe positions in the new file. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4199) ORC writer doesn't handle non-UTF8 encoded Text properly
[ https://issues.apache.org/jira/browse/HIVE-4199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samuel Yuan updated HIVE-4199: -- Status: Patch Available (was: Open) > ORC writer doesn't handle non-UTF8 encoded Text properly > > > Key: HIVE-4199 > URL: https://issues.apache.org/jira/browse/HIVE-4199 > Project: Hive > Issue Type: Bug > Components: Serializers/Deserializers >Reporter: Samuel Yuan >Assignee: Samuel Yuan >Priority: Minor > Attachments: HIVE-4199.HIVE-4199.HIVE-4199.D9501.1.patch, > HIVE-4199.HIVE-4199.HIVE-4199.D9501.2.patch, > HIVE-4199.HIVE-4199.HIVE-4199.D9501.3.patch > > > StringTreeWriter currently converts fields stored as Text objects into > Strings. This can lose information (see > http://en.wikipedia.org/wiki/Replacement_character#Replacement_character), > and is also unnecessary since the dictionary stores Text objects. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4199) ORC writer doesn't handle non-UTF8 encoded Text properly
Samuel Yuan created HIVE-4199: - Summary: ORC writer doesn't handle non-UTF8 encoded Text properly Key: HIVE-4199 URL: https://issues.apache.org/jira/browse/HIVE-4199 Project: Hive Issue Type: Bug Components: Serializers/Deserializers Reporter: Samuel Yuan Assignee: Samuel Yuan Priority: Minor StringTreeWriter currently converts fields stored as Text objects into Strings. This can lose information (see http://en.wikipedia.org/wiki/Replacement_character#Replacement_character), and is also unnecessary since the dictionary stores Text objects. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4067) Followup to HIVE-701: reduce ambiguity in grammar
[ https://issues.apache.org/jira/browse/HIVE-4067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13604068#comment-13604068 ] Samuel Yuan commented on HIVE-4067: --- The primary motivation was that it was tricky to add new keywords, which are almost always reserved words by default in Hive, because doing so could easily break existing queries. The changes for HIVE-701 make it easy to add non-reserved keywords in the future. HIVE-701 also removes the reserved status of most keywords, to prevent recently introduced keywords from breaking queries. I can undo the changes for the Hive keywords which are reserved in SQL 2003, but would there be any reason to do so besides to adhere to the standard, given that the grammar can support leaving them non-reserved? > Followup to HIVE-701: reduce ambiguity in grammar > - > > Key: HIVE-4067 > URL: https://issues.apache.org/jira/browse/HIVE-4067 > Project: Hive > Issue Type: Bug > Components: Query Processor >Affects Versions: 0.11.0 >Reporter: Samuel Yuan >Assignee: Samuel Yuan >Priority: Minor > Attachments: HIVE-4067.D8883.1.patch > > > After HIVE-701 the grammar has become much more ambiguous, and the > compilation generates a large number of warnings. Making FROM, DISTINCT, > PRESERVE, COLUMN, ALL, AND, OR, and NOT reserved keywords again reduces the > number of warnings to 134, up from the original 81 warnings but down from the > 565 after HIVE-701. Most of the remaining ambiguity is trivial, an example > being "KW_ELEM_TYPE | KW_KEY_TYPE | KW_VALUE_TYPE | identifier", and they are > all correctly handled by ANTLR. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4125) Expose metastore JMX metrics
[ https://issues.apache.org/jira/browse/HIVE-4125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samuel Yuan updated HIVE-4125: -- Status: Patch Available (was: Open) > Expose metastore JMX metrics > > > Key: HIVE-4125 > URL: https://issues.apache.org/jira/browse/HIVE-4125 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 0.11.0 >Reporter: Samuel Yuan >Assignee: Samuel Yuan >Priority: Trivial > Attachments: HIVE-4125.HIVE-4125.HIVE-4125.D9123.1.patch, > HIVE-4125.HIVE-4125.HIVE-4125.D9123.2.patch > > > Add a safe way to access the metrics stored for each MetricsScope, so that > they can be used outside of JMX. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4067) Followup to HIVE-701: reduce ambiguity in grammar
[ https://issues.apache.org/jira/browse/HIVE-4067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13598566#comment-13598566 ] Samuel Yuan commented on HIVE-4067: --- This is referring to the Java code in the @members section. The section is still being properly inserted into the generated HiveParser.java file, which then compiles fine, so I don't know why ANTLR is complaining. I'll look into it. > Followup to HIVE-701: reduce ambiguity in grammar > - > > Key: HIVE-4067 > URL: https://issues.apache.org/jira/browse/HIVE-4067 > Project: Hive > Issue Type: Bug > Components: Query Processor >Affects Versions: 0.11.0 >Reporter: Samuel Yuan >Assignee: Samuel Yuan >Priority: Minor > Attachments: HIVE-4067.D8883.1.patch > > > After HIVE-701 the grammar has become much more ambiguous, and the > compilation generates a large number of warnings. Making FROM, DISTINCT, > PRESERVE, COLUMN, ALL, AND, OR, and NOT reserved keywords again reduces the > number of warnings to 134, up from the original 81 warnings but down from the > 565 after HIVE-701. Most of the remaining ambiguity is trivial, an example > being "KW_ELEM_TYPE | KW_KEY_TYPE | KW_VALUE_TYPE | identifier", and they are > all correctly handled by ANTLR. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4067) Followup to HIVE-701: reduce ambiguity in grammar
[ https://issues.apache.org/jira/browse/HIVE-4067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samuel Yuan updated HIVE-4067: -- Affects Version/s: 0.11.0 Status: Patch Available (was: Open) > Followup to HIVE-701: reduce ambiguity in grammar > - > > Key: HIVE-4067 > URL: https://issues.apache.org/jira/browse/HIVE-4067 > Project: Hive > Issue Type: Bug > Components: Query Processor >Affects Versions: 0.11.0 >Reporter: Samuel Yuan >Assignee: Samuel Yuan >Priority: Minor > > After HIVE-701 the grammar has become much more ambiguous, and the > compilation generates a large number of warnings. Making FROM, DISTINCT, > PRESERVE, COLUMN, ALL, AND, OR, and NOT reserved keywords again reduces the > number of warnings to 134, up from the original 81 warnings but down from the > 565 after HIVE-701. Most of the remaining ambiguity is trivial, an example > being "KW_ELEM_TYPE | KW_KEY_TYPE | KW_VALUE_TYPE | identifier", and they are > all correctly handled by ANTLR. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4125) Expose metastore JMX metrics
Samuel Yuan created HIVE-4125: - Summary: Expose metastore JMX metrics Key: HIVE-4125 URL: https://issues.apache.org/jira/browse/HIVE-4125 Project: Hive Issue Type: Improvement Components: Metastore Affects Versions: 0.11.0 Reporter: Samuel Yuan Assignee: Samuel Yuan Priority: Trivial Add a safe way to access the metrics stored for each MetricsScope, so that they can be used outside of JMX. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4092) Store complete names of tables in column access analyzer
[ https://issues.apache.org/jira/browse/HIVE-4092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samuel Yuan updated HIVE-4092: -- Status: Patch Available (was: Open) > Store complete names of tables in column access analyzer > > > Key: HIVE-4092 > URL: https://issues.apache.org/jira/browse/HIVE-4092 > Project: Hive > Issue Type: Bug > Components: Query Processor >Affects Versions: 0.11.0 >Reporter: Samuel Yuan >Assignee: Samuel Yuan >Priority: Trivial > Fix For: 0.11.0 > > Attachments: HIVE-4092.HIVE-4092.HIVE-4092.D8985.1.patch > > > Right now the db name is not being stored. We should store the complete name, > which includes the db name, as the table access analyzer does. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4092) Store complete names of tables in column access analyzer
[ https://issues.apache.org/jira/browse/HIVE-4092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samuel Yuan updated HIVE-4092: -- Description: Right now the db name is not being stored. We should store the complete name, which includes the db name, as the table access analyzer does. > Store complete names of tables in column access analyzer > > > Key: HIVE-4092 > URL: https://issues.apache.org/jira/browse/HIVE-4092 > Project: Hive > Issue Type: Bug > Components: Query Processor >Affects Versions: 0.11.0 >Reporter: Samuel Yuan >Assignee: Samuel Yuan >Priority: Trivial > Fix For: 0.11.0 > > > Right now the db name is not being stored. We should store the complete name, > which includes the db name, as the table access analyzer does. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4092) Store complete names of tables in column access analyzer
Samuel Yuan created HIVE-4092: - Summary: Store complete names of tables in column access analyzer Key: HIVE-4092 URL: https://issues.apache.org/jira/browse/HIVE-4092 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.11.0 Reporter: Samuel Yuan Assignee: Samuel Yuan Priority: Trivial Fix For: 0.11.0 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-4044) Add URL type
[ https://issues.apache.org/jira/browse/HIVE-4044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1357#comment-1357 ] Samuel Yuan commented on HIVE-4044: --- You're right, the idea is that it will enable better encoding of URLs. Kevin found that breaking up the URL into its components and storing them as separate columns results in significant space savings. The original plan was to implement this idea with RCFile, but with the new ORC file format I decided to wait for that instead, and to submit this part separately. However, it looks like the improvements of the ORC file have erased any gains we would have gotten by breaking up URLs into the individual components, so this won't be needed any more. > Add URL type > > > Key: HIVE-4044 > URL: https://issues.apache.org/jira/browse/HIVE-4044 > Project: Hive > Issue Type: Improvement >Reporter: Samuel Yuan >Assignee: Samuel Yuan > Attachments: HIVE-4044.HIVE-4044.HIVE-4044.D8799.1.patch > > > Having a separate type for URLs would enable improvements in storage > efficiency based on breaking up a URL into its components. The new type will > be named "URL" and made a non-reserved keyword (see HIVE-701). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-701) lots of reserved keywords in hive
[ https://issues.apache.org/jira/browse/HIVE-701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13584786#comment-13584786 ] Samuel Yuan commented on HIVE-701: -- Removing FROM, DISTINCT, PRESERVE, COLUMN, ALL, AND, OR, and NOT brings the number of warnings down to 134. Before this change it was at 81. Most of these other cases are trivial, an example being "KW_ELEM_TYPE | KW_KEY_TYPE | KW_VALUE_TYPE | identifier", and they are all correctly handled by ANTLR. I have created a followup JIRA: HIVE-4067. > lots of reserved keywords in hive > - > > Key: HIVE-701 > URL: https://issues.apache.org/jira/browse/HIVE-701 > Project: Hive > Issue Type: New Feature > Components: Query Processor >Reporter: Namit Jain >Assignee: Samuel Yuan > Fix For: 0.11.0 > > Attachments: HIVE-701.1.patch.txt, HIVE-701.2.patch.txt, > HIVE-701.D8397.1.patch, HIVE-701.HIVE-701.D8397.2.patch, > HIVE-701.HIVE-701.D8397.3.patch > > > There is a problem if we want to use some reserved keywords: > for example, creating a function of name left/right ? left/right is already a > reserved keyword. > The other way around should also be possible - if we want to add a 'show > tables status' and some applications already use status as a column name, > they should not break -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4067) Followup to HIVE-701: reduce ambiguity in grammar
Samuel Yuan created HIVE-4067: - Summary: Followup to HIVE-701: reduce ambiguity in grammar Key: HIVE-4067 URL: https://issues.apache.org/jira/browse/HIVE-4067 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Samuel Yuan Assignee: Samuel Yuan Priority: Minor After HIVE-701 the grammar has become much more ambiguous, and the compilation generates a large number of warnings. Making FROM, DISTINCT, PRESERVE, COLUMN, ALL, AND, OR, and NOT reserved keywords again reduces the number of warnings to 134, up from the original 81 warnings but down from the 565 after HIVE-701. Most of the remaining ambiguity is trivial, an example being "KW_ELEM_TYPE | KW_KEY_TYPE | KW_VALUE_TYPE | identifier", and they are all correctly handled by ANTLR. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-701) lots of reserved keywords in hive
[ https://issues.apache.org/jira/browse/HIVE-701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13584073#comment-13584073 ] Samuel Yuan commented on HIVE-701: -- Taking out DISTINCT and ALL takes the lines of warnings down from about 2900 to around 2300. We were originally at around 1000. I can try taking out more keywords to reduce the ambiguity. By the way, what is the concern about the ambiguity? There is no impact to the user, and I don't think there's much of a performance hit. Is it just that the warnings are clogging up the build log, or is there something else? > lots of reserved keywords in hive > - > > Key: HIVE-701 > URL: https://issues.apache.org/jira/browse/HIVE-701 > Project: Hive > Issue Type: New Feature > Components: Query Processor >Reporter: Namit Jain >Assignee: Samuel Yuan > Fix For: 0.11.0 > > Attachments: HIVE-701.1.patch.txt, HIVE-701.2.patch.txt, > HIVE-701.D8397.1.patch, HIVE-701.HIVE-701.D8397.2.patch, > HIVE-701.HIVE-701.D8397.3.patch > > > There is a problem if we want to use some reserved keywords: > for example, creating a function of name left/right ? left/right is already a > reserved keyword. > The other way around should also be possible - if we want to add a 'show > tables status' and some applications already use status as a column name, > they should not break -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4044) Add URL type
[ https://issues.apache.org/jira/browse/HIVE-4044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samuel Yuan updated HIVE-4044: -- Status: Patch Available (was: Open) > Add URL type > > > Key: HIVE-4044 > URL: https://issues.apache.org/jira/browse/HIVE-4044 > Project: Hive > Issue Type: Improvement >Reporter: Samuel Yuan >Assignee: Samuel Yuan > Attachments: HIVE-4044.HIVE-4044.HIVE-4044.D8799.1.patch > > > Having a separate type for URLs would enable improvements in storage > efficiency based on breaking up a URL into its components. The new type will > be named "URL" and made a non-reserved keyword (see HIVE-701). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4046) Column masking
Samuel Yuan created HIVE-4046: - Summary: Column masking Key: HIVE-4046 URL: https://issues.apache.org/jira/browse/HIVE-4046 Project: Hive Issue Type: New Feature Components: CLI, Metastore, Query Processor Affects Versions: 0.11.0 Reporter: Samuel Yuan Assignee: Samuel Yuan Sometimes data in a table needs to be kept around but made inaccessible. Right now it is possible to offline a table or a partition, but not a specific column of a partition. Also, accessing an offlined table results in an error. With this change, it will be possible to mask a column at the partition level, causing all further queries to that column to return null. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-4044) Add URL type
Samuel Yuan created HIVE-4044: - Summary: Add URL type Key: HIVE-4044 URL: https://issues.apache.org/jira/browse/HIVE-4044 Project: Hive Issue Type: Improvement Reporter: Samuel Yuan Assignee: Samuel Yuan Having a separate type for URLs would enable improvements in storage efficiency based on breaking up a URL into its components. The new type will be named "URL" and made a non-reserved keyword (see HIVE-701). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-701) lots of reserved keywords in hive
[ https://issues.apache.org/jira/browse/HIVE-701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samuel Yuan updated HIVE-701: - Attachment: HIVE-701.2.patch.txt Updated, see Phabricator. > lots of reserved keywords in hive > - > > Key: HIVE-701 > URL: https://issues.apache.org/jira/browse/HIVE-701 > Project: Hive > Issue Type: New Feature > Components: Query Processor >Reporter: Namit Jain >Assignee: Samuel Yuan > Attachments: HIVE-701.1.patch.txt, HIVE-701.2.patch.txt, > HIVE-701.D8397.1.patch, HIVE-701.HIVE-701.D8397.2.patch > > > There is a problem if we want to use some reserved keywords: > for example, creating a function of name left/right ? left/right is already a > reserved keyword. > The other way around should also be possible - if we want to add a 'show > tables status' and some applications already use status as a column name, > they should not break -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-701) lots of reserved keywords in hive
[ https://issues.apache.org/jira/browse/HIVE-701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samuel Yuan updated HIVE-701: - Status: Patch Available (was: Open) > lots of reserved keywords in hive > - > > Key: HIVE-701 > URL: https://issues.apache.org/jira/browse/HIVE-701 > Project: Hive > Issue Type: New Feature > Components: Query Processor >Reporter: Namit Jain >Assignee: Samuel Yuan > Attachments: HIVE-701.1.patch.txt, HIVE-701.D8397.1.patch > > > There is a problem if we want to use some reserved keywords: > for example, creating a function of name left/right ? left/right is already a > reserved keyword. > The other way around should also be possible - if we want to add a 'show > tables status' and some applications already use status as a column name, > they should not break -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-701) lots of reserved keywords in hive
[ https://issues.apache.org/jira/browse/HIVE-701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samuel Yuan updated HIVE-701: - Attachment: HIVE-701.1.patch.txt https://reviews.facebook.net/D8397 > lots of reserved keywords in hive > - > > Key: HIVE-701 > URL: https://issues.apache.org/jira/browse/HIVE-701 > Project: Hive > Issue Type: New Feature > Components: Query Processor >Reporter: Namit Jain >Assignee: Samuel Yuan > Attachments: HIVE-701.1.patch.txt, HIVE-701.D8397.1.patch > > > There is a problem if we want to use some reserved keywords: > for example, creating a function of name left/right ? left/right is already a > reserved keyword. > The other way around should also be possible - if we want to add a 'show > tables status' and some applications already use status as a column name, > they should not break -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-701) lots of reserved keywords in hive
[ https://issues.apache.org/jira/browse/HIVE-701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samuel Yuan reassigned HIVE-701: Assignee: Samuel Yuan > lots of reserved keywords in hive > - > > Key: HIVE-701 > URL: https://issues.apache.org/jira/browse/HIVE-701 > Project: Hive > Issue Type: New Feature > Components: Query Processor >Reporter: Namit Jain >Assignee: Samuel Yuan > > There is a problem if we want to use some reserved keywords: > for example, creating a function of name left/right ? left/right is already a > reserved keyword. > The other way around should also be possible - if we want to add a 'show > tables status' and some applications already use status as a column name, > they should not break -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3940) Track columns accessed in each table in a query
[ https://issues.apache.org/jira/browse/HIVE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samuel Yuan updated HIVE-3940: -- Attachment: HIVE-3940.3.patch.txt Updated. > Track columns accessed in each table in a query > --- > > Key: HIVE-3940 > URL: https://issues.apache.org/jira/browse/HIVE-3940 > Project: Hive > Issue Type: Task > Components: Query Processor >Reporter: Samuel Yuan >Assignee: Samuel Yuan >Priority: Minor > Attachments: HIVE-3940.1.patch.txt, HIVE-3940.2.patch.txt, > HIVE-3940.3.patch.txt > > > Similar to partition access logs, we need to have columns access logs, so > later we can build tools/reports to inform users if there are wasted columns > in a table to be trimmed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3252) Add environment context to metastore Thrift calls
[ https://issues.apache.org/jira/browse/HIVE-3252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samuel Yuan updated HIVE-3252: -- Attachment: HIVE-3252.2.patch.txt Updated, see comments in Phabricator > Add environment context to metastore Thrift calls > - > > Key: HIVE-3252 > URL: https://issues.apache.org/jira/browse/HIVE-3252 > Project: Hive > Issue Type: Improvement > Components: Metastore >Reporter: John Reese >Assignee: John Reese >Priority: Minor > Attachments: HIVE-3252.1.patch.txt, HIVE-3252.2.patch.txt > > > Currently in the Hive Thrift metastore API create_table, add_partition, > alter_table, alter_partition have with_environment_context analogs. It would > be really useful to add similar methods from drop_partition, drop_table, and > append_partition. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3252) Add environment context to metastore Thrift calls
[ https://issues.apache.org/jira/browse/HIVE-3252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samuel Yuan updated HIVE-3252: -- Attachment: HIVE-3252.1.patch.txt https://reviews.facebook.net/D8241 > Add environment context to metastore Thrift calls > - > > Key: HIVE-3252 > URL: https://issues.apache.org/jira/browse/HIVE-3252 > Project: Hive > Issue Type: Improvement > Components: Metastore >Reporter: John Reese >Assignee: John Reese >Priority: Minor > Attachments: HIVE-3252.1.patch.txt > > > Currently in the Hive Thrift metastore API create_table, add_partition, > alter_table, alter_partition have with_environment_context analogs. It would > be really useful to add similar methods from drop_partition, drop_table, and > append_partition. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3940) Track columns accessed in each table in a query
[ https://issues.apache.org/jira/browse/HIVE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samuel Yuan updated HIVE-3940: -- Attachment: HIVE-3940.2.patch.txt > Track columns accessed in each table in a query > --- > > Key: HIVE-3940 > URL: https://issues.apache.org/jira/browse/HIVE-3940 > Project: Hive > Issue Type: Task > Components: Query Processor >Reporter: Samuel Yuan >Assignee: Samuel Yuan >Priority: Minor > Attachments: HIVE-3940.1.patch.txt, HIVE-3940.2.patch.txt > > > Similar to partition access logs, we need to have columns access logs, so > later we can build tools/reports to inform users if there are wasted columns > in a table to be trimmed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3940) Track columns accessed in each table in a query
[ https://issues.apache.org/jira/browse/HIVE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samuel Yuan updated HIVE-3940: -- Status: Patch Available (was: Open) > Track columns accessed in each table in a query > --- > > Key: HIVE-3940 > URL: https://issues.apache.org/jira/browse/HIVE-3940 > Project: Hive > Issue Type: Task > Components: Query Processor >Reporter: Samuel Yuan >Assignee: Samuel Yuan >Priority: Minor > Attachments: HIVE-3940.1.patch.txt, HIVE-3940.2.patch.txt > > > Similar to partition access logs, we need to have columns access logs, so > later we can build tools/reports to inform users if there are wasted columns > in a table to be trimmed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3940) Track columns accessed in each table in a query
[ https://issues.apache.org/jira/browse/HIVE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samuel Yuan updated HIVE-3940: -- Status: Patch Available (was: Open) > Track columns accessed in each table in a query > --- > > Key: HIVE-3940 > URL: https://issues.apache.org/jira/browse/HIVE-3940 > Project: Hive > Issue Type: Task > Components: Query Processor >Reporter: Samuel Yuan >Assignee: Samuel Yuan >Priority: Minor > Attachments: HIVE-3940.1.patch.txt > > > Similar to partition access logs, we need to have columns access logs, so > later we can build tools/reports to inform users if there are wasted columns > in a table to be trimmed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3940) Track columns accessed in each table in a query
[ https://issues.apache.org/jira/browse/HIVE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13562940#comment-13562940 ] Samuel Yuan commented on HIVE-3940: --- https://reviews.facebook.net/D8193 > Track columns accessed in each table in a query > --- > > Key: HIVE-3940 > URL: https://issues.apache.org/jira/browse/HIVE-3940 > Project: Hive > Issue Type: Task > Components: Query Processor >Reporter: Samuel Yuan >Assignee: Samuel Yuan >Priority: Minor > Attachments: HIVE-3940.1.patch.txt > > > Similar to partition access logs, we need to have columns access logs, so > later we can build tools/reports to inform users if there are wasted columns > in a table to be trimmed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3940) Track columns accessed in each table in a query
[ https://issues.apache.org/jira/browse/HIVE-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samuel Yuan updated HIVE-3940: -- Attachment: HIVE-3940.1.patch.txt > Track columns accessed in each table in a query > --- > > Key: HIVE-3940 > URL: https://issues.apache.org/jira/browse/HIVE-3940 > Project: Hive > Issue Type: Task > Components: Query Processor >Reporter: Samuel Yuan >Assignee: Samuel Yuan >Priority: Minor > Attachments: HIVE-3940.1.patch.txt > > > Similar to partition access logs, we need to have columns access logs, so > later we can build tools/reports to inform users if there are wasted columns > in a table to be trimmed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3940) Track columns accessed in each table in a query
Samuel Yuan created HIVE-3940: - Summary: Track columns accessed in each table in a query Key: HIVE-3940 URL: https://issues.apache.org/jira/browse/HIVE-3940 Project: Hive Issue Type: Task Components: Query Processor Reporter: Samuel Yuan Assignee: Samuel Yuan Priority: Minor Similar to partition access logs, we need to have columns access logs, so later we can build tools/reports to inform users if there are wasted columns in a table to be trimmed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3903) Allow updating bucketing/sorting metadata of a partition through the CLI
[ https://issues.apache.org/jira/browse/HIVE-3903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samuel Yuan updated HIVE-3903: -- Status: Patch Available (was: Open) > Allow updating bucketing/sorting metadata of a partition through the CLI > > > Key: HIVE-3903 > URL: https://issues.apache.org/jira/browse/HIVE-3903 > Project: Hive > Issue Type: New Feature > Components: Query Processor >Affects Versions: 0.11.0 >Reporter: Kevin Wilfong >Assignee: Samuel Yuan > Attachments: HIVE-3903.1.patch.txt, HIVE-3903.2.patch.txt > > > Right now users can update the bucketing/sorting metadata of a table through > the CLI, but not a partition. > Use case: > Need to merge a partition's files, but it's bucketed/sorted, so want to mark > the partition as unbucketed/unsorted. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3903) Allow updating bucketing/sorting metadata of a partition through the CLI
[ https://issues.apache.org/jira/browse/HIVE-3903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samuel Yuan updated HIVE-3903: -- Attachment: HIVE-3903.2.patch.txt > Allow updating bucketing/sorting metadata of a partition through the CLI > > > Key: HIVE-3903 > URL: https://issues.apache.org/jira/browse/HIVE-3903 > Project: Hive > Issue Type: New Feature > Components: Query Processor >Affects Versions: 0.11.0 >Reporter: Kevin Wilfong >Assignee: Samuel Yuan > Attachments: HIVE-3903.1.patch.txt, HIVE-3903.2.patch.txt > > > Right now users can update the bucketing/sorting metadata of a table through > the CLI, but not a partition. > Use case: > Need to merge a partition's files, but it's bucketed/sorted, so want to mark > the partition as unbucketed/unsorted. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3903) Allow updating bucketing/sorting metadata of a partition through the CLI
[ https://issues.apache.org/jira/browse/HIVE-3903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samuel Yuan updated HIVE-3903: -- Status: Patch Available (was: Open) > Allow updating bucketing/sorting metadata of a partition through the CLI > > > Key: HIVE-3903 > URL: https://issues.apache.org/jira/browse/HIVE-3903 > Project: Hive > Issue Type: New Feature > Components: Query Processor >Affects Versions: 0.11.0 >Reporter: Kevin Wilfong >Assignee: Samuel Yuan > Attachments: HIVE-3903.1.patch.txt > > > Right now users can update the bucketing/sorting metadata of a table through > the CLI, but not a partition. > Use case: > Need to merge a partition's files, but it's bucketed/sorted, so want to mark > the partition as unbucketed/unsorted. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3903) Allow updating bucketing/sorting metadata of a partition through the CLI
[ https://issues.apache.org/jira/browse/HIVE-3903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samuel Yuan updated HIVE-3903: -- Attachment: HIVE-3903.1.patch.txt > Allow updating bucketing/sorting metadata of a partition through the CLI > > > Key: HIVE-3903 > URL: https://issues.apache.org/jira/browse/HIVE-3903 > Project: Hive > Issue Type: New Feature > Components: Query Processor >Affects Versions: 0.11.0 >Reporter: Kevin Wilfong >Assignee: Samuel Yuan > Attachments: HIVE-3903.1.patch.txt > > > Right now users can update the bucketing/sorting metadata of a table through > the CLI, but not a partition. > Use case: > Need to merge a partition's files, but it's bucketed/sorted, so want to mark > the partition as unbucketed/unsorted. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3903) Allow updating bucketing/sorting metadata of a partition through the CLI
[ https://issues.apache.org/jira/browse/HIVE-3903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561078#comment-13561078 ] Samuel Yuan commented on HIVE-3903: --- https://reviews.facebook.net/D8127 > Allow updating bucketing/sorting metadata of a partition through the CLI > > > Key: HIVE-3903 > URL: https://issues.apache.org/jira/browse/HIVE-3903 > Project: Hive > Issue Type: New Feature > Components: Query Processor >Affects Versions: 0.11.0 >Reporter: Kevin Wilfong >Assignee: Samuel Yuan > > Right now users can update the bucketing/sorting metadata of a table through > the CLI, but not a partition. > Use case: > Need to merge a partition's files, but it's bucketed/sorted, so want to mark > the partition as unbucketed/unsorted. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-3903) Allow updating bucketing/sorting metadata of a partition through the CLI
[ https://issues.apache.org/jira/browse/HIVE-3903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samuel Yuan reassigned HIVE-3903: - Assignee: Samuel Yuan > Allow updating bucketing/sorting metadata of a partition through the CLI > > > Key: HIVE-3903 > URL: https://issues.apache.org/jira/browse/HIVE-3903 > Project: Hive > Issue Type: New Feature > Components: Query Processor >Affects Versions: 0.11.0 >Reporter: Kevin Wilfong >Assignee: Samuel Yuan > > Right now users can update the bucketing/sorting metadata of a table through > the CLI, but not a partition. > Use case: > Need to merge a partition's files, but it's bucketed/sorted, so want to mark > the partition as unbucketed/unsorted. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
