[jira] [Commented] (HIVE-446) Implement TRUNCATE
[ https://issues.apache.org/jira/browse/HIVE-446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13632717#comment-13632717 ] caofangkun commented on HIVE-446: - Thank you [~gangtimliu] Many times I have to remove data from external table to free up storage space of the cluster. So it's necessary for me to have some statement like truncate ... force to remove data. I submited an issue https://issues.apache.org/jira/browse/HIVE-4367 Just in case it may be some helpful for people have a similar need . Implement TRUNCATE -- Key: HIVE-446 URL: https://issues.apache.org/jira/browse/HIVE-446 Project: Hive Issue Type: New Feature Components: Query Processor Reporter: Prasad Chakka Assignee: Navis Fix For: 0.11.0 Attachments: HIVE-446.D7371.1.patch, HIVE-446.D7371.2.patch, HIVE-446.D7371.3.patch, HIVE-446.D7371.4.patch truncate the data but leave the table and metadata intact. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-446) Implement TRUNCATE
[ https://issues.apache.org/jira/browse/HIVE-446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13629960#comment-13629960 ] caofangkun commented on HIVE-446: - Hi ALL： Whether it is necessary to enhance the syntax like TRUNCATE TABLE srcpart_truncate PARTITION (dt='201130412') FORCE; to remove data from EXTERNAL table ? Implement TRUNCATE -- Key: HIVE-446 URL: https://issues.apache.org/jira/browse/HIVE-446 Project: Hive Issue Type: New Feature Components: Query Processor Reporter: Prasad Chakka Assignee: Navis Fix For: 0.11.0 Attachments: HIVE-446.D7371.1.patch, HIVE-446.D7371.2.patch, HIVE-446.D7371.3.patch, HIVE-446.D7371.4.patch truncate the data but leave the table and metadata intact. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-446) Implement TRUNCATE
[ https://issues.apache.org/jira/browse/HIVE-446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13630258#comment-13630258 ] Gang Tim Liu commented on HIVE-446: --- External table is used in the context where data is not fully managed. If it ends up that there is a need to remove data behind external table, a question can be asked why do you define it as external table?. Saying that, possibly the proposed syntax and semantics are not consistent to external table use case. thanks Implement TRUNCATE -- Key: HIVE-446 URL: https://issues.apache.org/jira/browse/HIVE-446 Project: Hive Issue Type: New Feature Components: Query Processor Reporter: Prasad Chakka Assignee: Navis Fix For: 0.11.0 Attachments: HIVE-446.D7371.1.patch, HIVE-446.D7371.2.patch, HIVE-446.D7371.3.patch, HIVE-446.D7371.4.patch truncate the data but leave the table and metadata intact. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-446) Implement TRUNCATE
[ https://issues.apache.org/jira/browse/HIVE-446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13548161#comment-13548161 ] Hudson commented on HIVE-446: - Integrated in Hive-trunk-hadoop2 #54 (See [https://builds.apache.org/job/Hive-trunk-hadoop2/54/]) HIVE-446 Implement TRUNCATE (Navis via namit) (Revision 1427681) Result = ABORTED namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1427681 Files : * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/ErrorMsg.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/DDLWork.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/HiveOperation.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/TruncateTableDesc.java * /hive/trunk/ql/src/test/queries/clientnegative/truncate_table_failure1.q * /hive/trunk/ql/src/test/queries/clientnegative/truncate_table_failure2.q * /hive/trunk/ql/src/test/queries/clientnegative/truncate_table_failure3.q * /hive/trunk/ql/src/test/queries/clientnegative/truncate_table_failure4.q * /hive/trunk/ql/src/test/queries/clientpositive/truncate_table.q * /hive/trunk/ql/src/test/results/clientnegative/truncate_table_failure1.q.out * /hive/trunk/ql/src/test/results/clientnegative/truncate_table_failure2.q.out * /hive/trunk/ql/src/test/results/clientnegative/truncate_table_failure3.q.out * /hive/trunk/ql/src/test/results/clientnegative/truncate_table_failure4.q.out * /hive/trunk/ql/src/test/results/clientpositive/truncate_table.q.out Implement TRUNCATE -- Key: HIVE-446 URL: https://issues.apache.org/jira/browse/HIVE-446 Project: Hive Issue Type: New Feature Components: Query Processor Reporter: Prasad Chakka Assignee: Navis Fix For: 0.11.0 Attachments: HIVE-446.D7371.1.patch, HIVE-446.D7371.2.patch, HIVE-446.D7371.3.patch, HIVE-446.D7371.4.patch truncate the data but leave the table and metadata intact. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-446) Implement TRUNCATE
[ https://issues.apache.org/jira/browse/HIVE-446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13542789#comment-13542789 ] Navis commented on HIVE-446: [~mgrover] Thanks! Implement TRUNCATE -- Key: HIVE-446 URL: https://issues.apache.org/jira/browse/HIVE-446 Project: Hive Issue Type: New Feature Components: Query Processor Reporter: Prasad Chakka Assignee: Navis Fix For: 0.11.0 Attachments: HIVE-446.D7371.1.patch, HIVE-446.D7371.2.patch, HIVE-446.D7371.3.patch, HIVE-446.D7371.4.patch truncate the data but leave the table and metadata intact. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-446) Implement TRUNCATE
[ https://issues.apache.org/jira/browse/HIVE-446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13542139#comment-13542139 ] Hudson commented on HIVE-446: - Integrated in Hive-trunk-h0.21 #1890 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1890/]) HIVE-446 Implement TRUNCATE (Navis via namit) (Revision 1427681) Result = FAILURE namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1427681 Files : * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/ErrorMsg.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/DDLWork.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/HiveOperation.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/TruncateTableDesc.java * /hive/trunk/ql/src/test/queries/clientnegative/truncate_table_failure1.q * /hive/trunk/ql/src/test/queries/clientnegative/truncate_table_failure2.q * /hive/trunk/ql/src/test/queries/clientnegative/truncate_table_failure3.q * /hive/trunk/ql/src/test/queries/clientnegative/truncate_table_failure4.q * /hive/trunk/ql/src/test/queries/clientpositive/truncate_table.q * /hive/trunk/ql/src/test/results/clientnegative/truncate_table_failure1.q.out * /hive/trunk/ql/src/test/results/clientnegative/truncate_table_failure2.q.out * /hive/trunk/ql/src/test/results/clientnegative/truncate_table_failure3.q.out * /hive/trunk/ql/src/test/results/clientnegative/truncate_table_failure4.q.out * /hive/trunk/ql/src/test/results/clientpositive/truncate_table.q.out Implement TRUNCATE -- Key: HIVE-446 URL: https://issues.apache.org/jira/browse/HIVE-446 Project: Hive Issue Type: New Feature Components: Query Processor Reporter: Prasad Chakka Assignee: Navis Fix For: 0.11.0 Attachments: HIVE-446.D7371.1.patch, HIVE-446.D7371.2.patch, HIVE-446.D7371.3.patch, HIVE-446.D7371.4.patch truncate the data but leave the table and metadata intact. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-446) Implement TRUNCATE
[ https://issues.apache.org/jira/browse/HIVE-446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13542688#comment-13542688 ] Phabricator commented on HIVE-446: -- mgrover has commented on the revision HIVE-446 [jira] Implement TRUNCATE. I am happy to take care of all rmr changes in HIVE-3701 REVISION DETAIL https://reviews.facebook.net/D7371 To: JIRA, navis Cc: njain, mgrover Implement TRUNCATE -- Key: HIVE-446 URL: https://issues.apache.org/jira/browse/HIVE-446 Project: Hive Issue Type: New Feature Components: Query Processor Reporter: Prasad Chakka Assignee: Navis Fix For: 0.11.0 Attachments: HIVE-446.D7371.1.patch, HIVE-446.D7371.2.patch, HIVE-446.D7371.3.patch, HIVE-446.D7371.4.patch truncate the data but leave the table and metadata intact. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-446) Implement TRUNCATE
[ https://issues.apache.org/jira/browse/HIVE-446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13541983#comment-13541983 ] Phabricator commented on HIVE-446: -- navis has commented on the revision HIVE-446 [jira] Implement TRUNCATE. INLINE COMMENTS ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java:727 Currently, many of hive DDLs are not supporting specifying database name, which is partly fixed on https://issues.apache.org/jira/browse/HIVE-3589. I'm waiting for the issue committed and expecting to use it for other DDLs. ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java:755 I agree but can't think of an alternative way. It's same situation for all other DDLs. ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java:3912 I admit I've just copied it from Hive.class. Could you do that when updating patch for HIVE-3701? I don't know well of differences among the hadoop versions. ql/src/java/org/apache/hadoop/hive/ql/ErrorMsg.java:249 Agreed. ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java:749 done. REVISION DETAIL https://reviews.facebook.net/D7371 To: JIRA, navis Cc: njain, mgrover Implement TRUNCATE -- Key: HIVE-446 URL: https://issues.apache.org/jira/browse/HIVE-446 Project: Hive Issue Type: New Feature Components: Query Processor Reporter: Prasad Chakka Assignee: Navis Attachments: HIVE-446.D7371.1.patch, HIVE-446.D7371.2.patch truncate the data but leave the table and metadata intact. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-446) Implement TRUNCATE
[ https://issues.apache.org/jira/browse/HIVE-446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13541990#comment-13541990 ] Phabricator commented on HIVE-446: -- njain has commented on the revision HIVE-446 [jira] Implement TRUNCATE. The changes look good to me. Can you file a follow-up jira to add -rmr to hadoop shims ? INLINE COMMENTS ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java:3926 not a big deal - but, the outputs should be already populated. Anyway, it is a set, so it should not matter. REVISION DETAIL https://reviews.facebook.net/D7371 To: JIRA, navis Cc: njain, mgrover Implement TRUNCATE -- Key: HIVE-446 URL: https://issues.apache.org/jira/browse/HIVE-446 Project: Hive Issue Type: New Feature Components: Query Processor Reporter: Prasad Chakka Assignee: Navis Attachments: HIVE-446.D7371.1.patch, HIVE-446.D7371.2.patch, HIVE-446.D7371.3.patch truncate the data but leave the table and metadata intact. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-446) Implement TRUNCATE
[ https://issues.apache.org/jira/browse/HIVE-446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13541993#comment-13541993 ] Phabricator commented on HIVE-446: -- navis has commented on the revision HIVE-446 [jira] Implement TRUNCATE. HIVE-3701 seemed good place for it, fixing -rmr cases altogether. Doesn't it? INLINE COMMENTS ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java:3926 fixed. REVISION DETAIL https://reviews.facebook.net/D7371 To: JIRA, navis Cc: njain, mgrover Implement TRUNCATE -- Key: HIVE-446 URL: https://issues.apache.org/jira/browse/HIVE-446 Project: Hive Issue Type: New Feature Components: Query Processor Reporter: Prasad Chakka Assignee: Navis Attachments: HIVE-446.D7371.1.patch, HIVE-446.D7371.2.patch, HIVE-446.D7371.3.patch, HIVE-446.D7371.4.patch truncate the data but leave the table and metadata intact. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-446) Implement TRUNCATE
[ https://issues.apache.org/jira/browse/HIVE-446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13541998#comment-13541998 ] Namit Jain commented on HIVE-446: - @Navis, can you add documentation for this ? Implement TRUNCATE -- Key: HIVE-446 URL: https://issues.apache.org/jira/browse/HIVE-446 Project: Hive Issue Type: New Feature Components: Query Processor Reporter: Prasad Chakka Assignee: Navis Attachments: HIVE-446.D7371.1.patch, HIVE-446.D7371.2.patch, HIVE-446.D7371.3.patch, HIVE-446.D7371.4.patch truncate the data but leave the table and metadata intact. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-446) Implement TRUNCATE
[ https://issues.apache.org/jira/browse/HIVE-446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13542030#comment-13542030 ] Navis commented on HIVE-446: @Namit Jain, done in https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DML, but I'm not sure it's DDL or DML. Implement TRUNCATE -- Key: HIVE-446 URL: https://issues.apache.org/jira/browse/HIVE-446 Project: Hive Issue Type: New Feature Components: Query Processor Reporter: Prasad Chakka Assignee: Navis Attachments: HIVE-446.D7371.1.patch, HIVE-446.D7371.2.patch, HIVE-446.D7371.3.patch, HIVE-446.D7371.4.patch truncate the data but leave the table and metadata intact. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-446) Implement TRUNCATE
[ https://issues.apache.org/jira/browse/HIVE-446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13542033#comment-13542033 ] Namit Jain commented on HIVE-446: - It should be part of DDL Implement TRUNCATE -- Key: HIVE-446 URL: https://issues.apache.org/jira/browse/HIVE-446 Project: Hive Issue Type: New Feature Components: Query Processor Reporter: Prasad Chakka Assignee: Navis Attachments: HIVE-446.D7371.1.patch, HIVE-446.D7371.2.patch, HIVE-446.D7371.3.patch, HIVE-446.D7371.4.patch truncate the data but leave the table and metadata intact. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-446) Implement TRUNCATE
[ https://issues.apache.org/jira/browse/HIVE-446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13538608#comment-13538608 ] Phabricator commented on HIVE-446: -- mgrover has commented on the revision HIVE-446 [jira] Implement TRUNCATE. INLINE COMMENTS ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java:3912 -rmr is deprecated. I think the new hip thing to do is rm -r. Keep in mind though that if hive is being used with really old versions of hadoop, rm -r may not work, so some version matching needs to happen here. If we leave it the way it is, then users will see a deprecation warning. I would prefer that not to be the case (btw, that's presently the case with insert overwrite but HIVE-3701 plans to change that) ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java:727 Ok, we should add a test that verifies that something like this works: hiveuse default; hivetruncate some_other_db.my_table; ql/src/java/org/apache/hadoop/hive/ql/ErrorMsg.java:249 I am a little confused here. The code seems to indicate that this only works managed, native tables. For example, you can't have a HBase table (using HBase storage handler) being truncated with this command, can you? If so, should we have make error messages for those too? ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java:749 Check for non-native tables here? ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java:3913 Touché! Since we preserve the metadata, we are preserving it for partitions as well. So, it makes sense to create empty directories for partitions that the metastore thinks exist. ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java:755 This could be a bottleneck if the number of partitions is large. Can't it? Is there an alternative to adding each of the partitions individually? Can we still not do outputs.add(new WriteEntity(table)); ? REVISION DETAIL https://reviews.facebook.net/D7371 To: JIRA, navis Cc: njain, mgrover Implement TRUNCATE -- Key: HIVE-446 URL: https://issues.apache.org/jira/browse/HIVE-446 Project: Hive Issue Type: New Feature Components: Query Processor Reporter: Prasad Chakka Assignee: Navis Attachments: HIVE-446.D7371.1.patch, HIVE-446.D7371.2.patch truncate the data but leave the table and metadata intact. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
