[jira] [Commented] (HIVE-2612) support hive table/partitions coexistes in more than one clusters
[ https://issues.apache.org/jira/browse/HIVE-2612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13188183#comment-13188183 ] Steven Wong commented on HIVE-2612: --- Yongqiang, please clarify what you mean by option 3. Currently, databases contain tables (db1.foo is unrelated to db2.foo). Is option 3 saying make tables span databases (db1.foo is the same table as db2.foo) instead? That would be a radical change, so maybe I've misunderstood it. > support hive table/partitions coexistes in more than one clusters > - > > Key: HIVE-2612 > URL: https://issues.apache.org/jira/browse/HIVE-2612 > Project: Hive > Issue Type: New Feature >Reporter: He Yongqiang >Assignee: He Yongqiang > > 1) add cluster object into hive metastore > 2) each partition/table has a creation cluster and a list of living clusters, > and also data location in each cluster -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2612) support hive table/partitions coexistes in more than one clusters
[ https://issues.apache.org/jira/browse/HIVE-2612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13188322#comment-13188322 ] Steven Wong commented on HIVE-2612: --- Won't that conflict with authorization? > support hive table/partitions coexistes in more than one clusters > - > > Key: HIVE-2612 > URL: https://issues.apache.org/jira/browse/HIVE-2612 > Project: Hive > Issue Type: New Feature > Components: Metastore >Reporter: He Yongqiang >Assignee: He Yongqiang > > 1) add cluster object into hive metastore > 2) each partition/table has a creation cluster and a list of living clusters, > and also data location in each cluster -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1332) Archiving partitions
[ https://issues.apache.org/jira/browse/HIVE-1332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13133587#comment-13133587 ] Steven Wong commented on HIVE-1332: --- If an archive operation runs when a select query is already running, the select may fail, right? > Archiving partitions > > > Key: HIVE-1332 > URL: https://issues.apache.org/jira/browse/HIVE-1332 > Project: Hive > Issue Type: New Feature > Components: Metastore >Reporter: Paul Yang >Assignee: Paul Yang > Fix For: 0.6.0 > > Attachments: HIVE-1332.1.patch, HIVE-1332.2.patch, HIVE-1332.3.patch, > HIVE-1332.4.patch, HIVE-1332.5.patch, HIVE-1332.6.patch > > > Partitions and tables in Hive typically consist of many files on HDFS. An > issue is that as the number of files increase, there will be higher > memory/load requirements on the namenode. Partitions in bucketed tables are a > particular problem because they consist of many files, one for each of the > buckets. > One way to drastically reduce the number of files is to use hadoop archives: > http://hadoop.apache.org/common/docs/current/hadoop_archives.html > This feature would introduce an ALTER TABLE ARCHIVE PARTITION > that would automatically put the files for the partition into a HAR > file. We would also have an UNARCHIVE option to convert the files in the > partition back to the original files. Archived partitions would be slower to > access, but they would have the same functionality and decrease the number of > files drastically. Typically, only seldom accessed partitions would be > archived. > Hadoop archives are still somewhat new, so we'll only put in support for the > latest released major version (0.20). Here are some bug fixes: > https://issues.apache.org/jira/browse/HADOOP-6591 (Important - could > potentially cause data loss without this fix) > https://issues.apache.org/jira/browse/HADOOP-6645 > https://issues.apache.org/jira/browse/MAPREDUCE-1585 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
