[jira] [Commented] (HIVE-3750) JDBCStatsPublisher fails when ID length exceeds length of ID column
[ https://issues.apache.org/jira/browse/HIVE-3750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13944331#comment-13944331 ] Lefty Leverenz commented on HIVE-3750: -- For the record, this adds *hive.stats.key.prefix.max.length* to HiveConf.java and hive-default.xml.template. JDBCStatsPublisher fails when ID length exceeds length of ID column --- Key: HIVE-3750 URL: https://issues.apache.org/jira/browse/HIVE-3750 Project: Hive Issue Type: Bug Components: Statistics Affects Versions: 0.11.0 Reporter: Kevin Wilfong Assignee: Kevin Wilfong Fix For: 0.11.0 Attachments: HIVE-3750.1.patch.txt When the length of the ID field passed to JDBCStatsPublisher exceeds the length of the column in the table (currently 255 characters) stats collection fails. This causes the entire query to fail when hive.stats.reliable is set to true. One way to prevent this would be to calculate a deterministic, very low collision hash of the ID prefix used for aggregation and use that when the length of the ID is too long. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-3750) JDBCStatsPublisher fails when ID length exceeds length of ID column
[ https://issues.apache.org/jira/browse/HIVE-3750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13548332#comment-13548332 ] Hudson commented on HIVE-3750: -- Integrated in Hive-trunk-hadoop2 #54 (See [https://builds.apache.org/job/Hive-trunk-hadoop2/54/]) HIVE-3771 HIVE-3750 broke TestParse (Kevin Wilfong via namit) (Revision 1417294) HIVE-3750 JDBCStatsPublisher fails when ID length exceeds length of ID column (Kevin Wilfong via namit) (Revision 1416854) Result = ABORTED namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1417294 Files : * /hive/trunk/ql/src/test/results/compiler/plan/case_sensitivity.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/groupby1.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/input1.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/input2.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/input3.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/input4.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/input5.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/input6.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/input7.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/input9.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/input_testsequencefile.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/join1.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/join2.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/join3.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/sample2.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/sample3.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/sample4.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/sample5.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/sample6.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/sample7.q.xml namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1416854 Files : * /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java * /hive/trunk/conf/hive-default.xml.template * /hive/trunk/hbase-handler/src/test/queries/positive/hbase_stats3.q * /hive/trunk/hbase-handler/src/test/results/positive/hbase_stats3.q.out * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/StatsTask.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/TableScanOperator.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMRFileSink1.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/FileSinkDesc.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/TableScanDesc.java * /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/stats/KeyVerifyingStatsAggregator.java * /hive/trunk/ql/src/test/queries/clientpositive/stats19.q * /hive/trunk/ql/src/test/results/clientpositive/stats19.q.out JDBCStatsPublisher fails when ID length exceeds length of ID column --- Key: HIVE-3750 URL: https://issues.apache.org/jira/browse/HIVE-3750 Project: Hive Issue Type: Bug Components: Statistics Affects Versions: 0.11.0 Reporter: Kevin Wilfong Assignee: Kevin Wilfong Fix For: 0.11.0 Attachments: HIVE-3750.1.patch.txt When the length of the ID field passed to JDBCStatsPublisher exceeds the length of the column in the table (currently 255 characters) stats collection fails. This causes the entire query to fail when hive.stats.reliable is set to true. One way to prevent this would be to calculate a deterministic, very low collision hash of the ID prefix used for aggregation and use that when the length of the ID is too long. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3750) JDBCStatsPublisher fails when ID length exceeds length of ID column
[ https://issues.apache.org/jira/browse/HIVE-3750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13510876#comment-13510876 ] Hudson commented on HIVE-3750: -- Integrated in Hive-trunk-h0.21 #1834 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1834/]) HIVE-3771 HIVE-3750 broke TestParse (Kevin Wilfong via namit) (Revision 1417294) Result = ABORTED namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1417294 Files : * /hive/trunk/ql/src/test/results/compiler/plan/case_sensitivity.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/groupby1.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/input1.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/input2.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/input3.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/input4.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/input5.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/input6.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/input7.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/input9.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/input_testsequencefile.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/join1.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/join2.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/join3.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/sample2.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/sample3.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/sample4.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/sample5.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/sample6.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/sample7.q.xml JDBCStatsPublisher fails when ID length exceeds length of ID column --- Key: HIVE-3750 URL: https://issues.apache.org/jira/browse/HIVE-3750 Project: Hive Issue Type: Bug Components: Statistics Affects Versions: 0.11 Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-3750.1.patch.txt When the length of the ID field passed to JDBCStatsPublisher exceeds the length of the column in the table (currently 255 characters) stats collection fails. This causes the entire query to fail when hive.stats.reliable is set to true. One way to prevent this would be to calculate a deterministic, very low collision hash of the ID prefix used for aggregation and use that when the length of the ID is too long. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3750) JDBCStatsPublisher fails when ID length exceeds length of ID column
[ https://issues.apache.org/jira/browse/HIVE-3750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13511204#comment-13511204 ] Hudson commented on HIVE-3750: -- Integrated in Hive-trunk-h0.21 #1835 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1835/]) HIVE-3771 HIVE-3750 broke TestParse (Kevin Wilfong via namit) (Revision 1417294) Result = FAILURE namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1417294 Files : * /hive/trunk/ql/src/test/results/compiler/plan/case_sensitivity.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/groupby1.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/input1.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/input2.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/input3.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/input4.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/input5.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/input6.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/input7.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/input9.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/input_testsequencefile.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/join1.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/join2.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/join3.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/sample2.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/sample3.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/sample4.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/sample5.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/sample6.q.xml * /hive/trunk/ql/src/test/results/compiler/plan/sample7.q.xml JDBCStatsPublisher fails when ID length exceeds length of ID column --- Key: HIVE-3750 URL: https://issues.apache.org/jira/browse/HIVE-3750 Project: Hive Issue Type: Bug Components: Statistics Affects Versions: 0.11 Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-3750.1.patch.txt When the length of the ID field passed to JDBCStatsPublisher exceeds the length of the column in the table (currently 255 characters) stats collection fails. This causes the entire query to fail when hive.stats.reliable is set to true. One way to prevent this would be to calculate a deterministic, very low collision hash of the ID prefix used for aggregation and use that when the length of the ID is too long. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3750) JDBCStatsPublisher fails when ID length exceeds length of ID column
[ https://issues.apache.org/jira/browse/HIVE-3750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13510146#comment-13510146 ] Kevin Wilfong commented on HIVE-3750: - Sorry about that. I filed HIVE-3771 to fix it and attached a diff. I only saw failures in TestParse. JDBCStatsPublisher fails when ID length exceeds length of ID column --- Key: HIVE-3750 URL: https://issues.apache.org/jira/browse/HIVE-3750 Project: Hive Issue Type: Bug Components: Statistics Affects Versions: 0.11 Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-3750.1.patch.txt When the length of the ID field passed to JDBCStatsPublisher exceeds the length of the column in the table (currently 255 characters) stats collection fails. This causes the entire query to fail when hive.stats.reliable is set to true. One way to prevent this would be to calculate a deterministic, very low collision hash of the ID prefix used for aggregation and use that when the length of the ID is too long. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3750) JDBCStatsPublisher fails when ID length exceeds length of ID column
[ https://issues.apache.org/jira/browse/HIVE-3750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13508583#comment-13508583 ] Namit Jain commented on HIVE-3750: -- +1 JDBCStatsPublisher fails when ID length exceeds length of ID column --- Key: HIVE-3750 URL: https://issues.apache.org/jira/browse/HIVE-3750 Project: Hive Issue Type: Bug Components: Statistics Affects Versions: 0.11 Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-3750.1.patch.txt When the length of the ID field passed to JDBCStatsPublisher exceeds the length of the column in the table (currently 255 characters) stats collection fails. This causes the entire query to fail when hive.stats.reliable is set to true. One way to prevent this would be to calculate a deterministic, very low collision hash of the ID prefix used for aggregation and use that when the length of the ID is too long. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3750) JDBCStatsPublisher fails when ID length exceeds length of ID column
[ https://issues.apache.org/jira/browse/HIVE-3750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13506830#comment-13506830 ] Kevin Wilfong commented on HIVE-3750: - https://reviews.facebook.net/D7017 JDBCStatsPublisher fails when ID length exceeds length of ID column --- Key: HIVE-3750 URL: https://issues.apache.org/jira/browse/HIVE-3750 Project: Hive Issue Type: Bug Components: Statistics Affects Versions: 0.11 Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-3750.1.patch.txt When the length of the ID field passed to JDBCStatsPublisher exceeds the length of the column in the table (currently 255 characters) stats collection fails. This causes the entire query to fail when hive.stats.reliable is set to true. One way to prevent this would be to calculate a deterministic, very low collision hash of the ID prefix used for aggregation and use that when the length of the ID is too long. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
