RE: DESCRIBE EXTENDED show numRows=0
Hi, I’m using 12.0 version and seems that ANALYZE command doesn’t update the table level statistics while partitions level are just fine. For example (after running the command ANALYZE TABLE T1 partition(date_prt) COMPUTE STATISTICS): I get: . . . Partition db1.t1{date_prt=2014-01-26} stats: [num_files: 16, num_rows: 9030050, total_size: 51413507, raw_data_size: 477332702] Table db1.t1 stats: [num_partitions: 0, num_files: 0, num_rows: 0, total_size: 0, raw_data_size: 0] -- a problem . . . Is there any update on this issue? Thanks Dsass From: Navis류승우 [mailto:navis@nexr.com] Sent: Monday, February 03, 2014 2:44 AM To: user@hive.apache.org Subject: Re: DESCRIBE EXTENDED show numRows=0 Could you check the task log? For stat gathering, hive uses derby by default. If the jdbc driver for derby is not in auxlib, task cannot publish stats. 2014-01-30 Stephen Sprague sprag...@gmail.commailto:sprag...@gmail.com: the answer to this would seemingly be no. i just tried it in hive v0.12. numRows=0 before and numRows=0 after my running of analyze table table compute statistics other values are populated though just not numRows. I wonder why that is. Cheers, Stephen {noformat} parameters:{numPartitions=0, numFiles=420, last_modified_by=dwr, last_modified_time=1390986197, transient_lastDdlTime=1391060001, totalSize=10748060517, numRows=0, rawDataSize=0}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE) {noformat} On Mon, Jan 27, 2014 at 3:52 AM, Lefty Leverenz leftylever...@gmail.commailto:leftylever...@gmail.com wrote: Can the ANALYZE statement be used to gather statistics if hive.stats.autogather was 'false' when the data was loaded? (See the wiki's Statistics in Hive doc: Existing Tableshttps://cwiki.apache.org/confluence/display/Hive/StatsDev#StatsDev-ExistingTables.) -- Lefty On Sun, Jan 26, 2014 at 8:01 PM, Navis류승우 navis@nexr.commailto:navis@nexr.com wrote: If the data is loaded into table with insert clause with hive.stats.autogather=true, it will be updated. If it's not, it's zero (or -1 sometimes). 2014-01-23 Haroon Muhammad muhammad.har...@live.commailto:muhammad.har...@live.com Hi, I have observed that DESCRIBE EXTENDED always shows number of rows to be zero despite the fact that the table has data. Is it a bug? Is it known ? Has anyone else also come across the same ? Thanks,
Re: DESCRIBE EXTENDED show numRows=0
Could you check the task log? For stat gathering, hive uses derby by default. If the jdbc driver for derby is not in auxlib, task cannot publish stats. 2014-01-30 Stephen Sprague sprag...@gmail.com: the answer to this would seemingly be no. i just tried it in hive v0.12. numRows=0 before and numRows=0 after my running of analyze table table compute statistics other values are populated though just not numRows. I wonder why that is. Cheers, Stephen {noformat} parameters:{numPartitions=0, numFiles=420, last_modified_by=dwr, last_modified_time=1390986197, transient_lastDdlTime=1391060001, totalSize=10748060517, *numRows=0,* rawDataSize=0}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE) {noformat} On Mon, Jan 27, 2014 at 3:52 AM, Lefty Leverenz leftylever...@gmail.comwrote: Can the ANALYZE statement be used to gather statistics if hive.stats.autogather was 'false' when the data was loaded? (See the wiki's Statistics in Hive doc: Existing Tableshttps://cwiki.apache.org/confluence/display/Hive/StatsDev#StatsDev-ExistingTables .) -- Lefty On Sun, Jan 26, 2014 at 8:01 PM, Navis류승우 navis@nexr.com wrote: If the data is loaded into table with insert clause with hive.stats.autogather=true, it will be updated. If it's not, it's zero (or -1 sometimes). 2014-01-23 Haroon Muhammad muhammad.har...@live.com Hi, I have observed that DESCRIBE EXTENDED always shows number of rows to be zero despite the fact that the table has data. Is it a bug? Is it known ? Has anyone else also come across the same ? Thanks,
Re: DESCRIBE EXTENDED show numRows=0
the answer to this would seemingly be no. i just tried it in hive v0.12. numRows=0 before and numRows=0 after my running of analyze table table compute statistics other values are populated though just not numRows. I wonder why that is. Cheers, Stephen {noformat} parameters:{numPartitions=0, numFiles=420, last_modified_by=dwr, last_modified_time=1390986197, transient_lastDdlTime=1391060001, totalSize=10748060517, *numRows=0,* rawDataSize=0}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE) {noformat} On Mon, Jan 27, 2014 at 3:52 AM, Lefty Leverenz leftylever...@gmail.comwrote: Can the ANALYZE statement be used to gather statistics if hive.stats.autogather was 'false' when the data was loaded? (See the wiki's Statistics in Hive doc: Existing Tableshttps://cwiki.apache.org/confluence/display/Hive/StatsDev#StatsDev-ExistingTables .) -- Lefty On Sun, Jan 26, 2014 at 8:01 PM, Navis류승우 navis@nexr.com wrote: If the data is loaded into table with insert clause with hive.stats.autogather=true, it will be updated. If it's not, it's zero (or -1 sometimes). 2014-01-23 Haroon Muhammad muhammad.har...@live.com Hi, I have observed that DESCRIBE EXTENDED always shows number of rows to be zero despite the fact that the table has data. Is it a bug? Is it known ? Has anyone else also come across the same ? Thanks,
Re: DESCRIBE EXTENDED show numRows=0
Oops, then I'd better correct what I just put in the wiki -- Describe Table/View/Columnhttps://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-DescribeTable/View/Column . Who wants to open a JIRA ticket? -- Lefty On Wed, Jan 29, 2014 at 9:37 PM, Stephen Sprague sprag...@gmail.com wrote: the answer to this would seemingly be no. i just tried it in hive v0.12. numRows=0 before and numRows=0 after my running of analyze table table compute statistics other values are populated though just not numRows. I wonder why that is. Cheers, Stephen {noformat} parameters:{numPartitions=0, numFiles=420, last_modified_by=dwr, last_modified_time=1390986197, transient_lastDdlTime=1391060001, totalSize=10748060517, *numRows=0,* rawDataSize=0}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE) {noformat} On Mon, Jan 27, 2014 at 3:52 AM, Lefty Leverenz leftylever...@gmail.comwrote: Can the ANALYZE statement be used to gather statistics if hive.stats.autogather was 'false' when the data was loaded? (See the wiki's Statistics in Hive doc: Existing Tableshttps://cwiki.apache.org/confluence/display/Hive/StatsDev#StatsDev-ExistingTables .) -- Lefty On Sun, Jan 26, 2014 at 8:01 PM, Navis류승우 navis@nexr.com wrote: If the data is loaded into table with insert clause with hive.stats.autogather=true, it will be updated. If it's not, it's zero (or -1 sometimes). 2014-01-23 Haroon Muhammad muhammad.har...@live.com Hi, I have observed that DESCRIBE EXTENDED always shows number of rows to be zero despite the fact that the table has data. Is it a bug? Is it known ? Has anyone else also come across the same ? Thanks,
Re: DESCRIBE EXTENDED show numRows=0
I forgot, there's already a ticket: HIVE-6285https://issues.apache.org/jira/browse/HIVE-6285 . -- Lefty On Wed, Jan 29, 2014 at 10:36 PM, Lefty Leverenz leftylever...@gmail.comwrote: Oops, then I'd better correct what I just put in the wiki -- Describe Table/View/Columnhttps://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-DescribeTable/View/Column . Who wants to open a JIRA ticket? -- Lefty On Wed, Jan 29, 2014 at 9:37 PM, Stephen Sprague sprag...@gmail.comwrote: the answer to this would seemingly be no. i just tried it in hive v0.12. numRows=0 before and numRows=0 after my running of analyze table table compute statistics other values are populated though just not numRows. I wonder why that is. Cheers, Stephen {noformat} parameters:{numPartitions=0, numFiles=420, last_modified_by=dwr, last_modified_time=1390986197, transient_lastDdlTime=1391060001, totalSize=10748060517, *numRows=0,* rawDataSize=0}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE) {noformat} On Mon, Jan 27, 2014 at 3:52 AM, Lefty Leverenz leftylever...@gmail.comwrote: Can the ANALYZE statement be used to gather statistics if hive.stats.autogather was 'false' when the data was loaded? (See the wiki's Statistics in Hive doc: Existing Tableshttps://cwiki.apache.org/confluence/display/Hive/StatsDev#StatsDev-ExistingTables .) -- Lefty On Sun, Jan 26, 2014 at 8:01 PM, Navis류승우 navis@nexr.com wrote: If the data is loaded into table with insert clause with hive.stats.autogather=true, it will be updated. If it's not, it's zero (or -1 sometimes). 2014-01-23 Haroon Muhammad muhammad.har...@live.com Hi, I have observed that DESCRIBE EXTENDED always shows number of rows to be zero despite the fact that the table has data. Is it a bug? Is it known ? Has anyone else also come across the same ? Thanks,
Re: DESCRIBE EXTENDED show numRows=0
Can the ANALYZE statement be used to gather statistics if hive.stats.autogather was 'false' when the data was loaded? (See the wiki's Statistics in Hive doc: Existing Tableshttps://cwiki.apache.org/confluence/display/Hive/StatsDev#StatsDev-ExistingTables .) -- Lefty On Sun, Jan 26, 2014 at 8:01 PM, Navis류승우 navis@nexr.com wrote: If the data is loaded into table with insert clause with hive.stats.autogather=true, it will be updated. If it's not, it's zero (or -1 sometimes). 2014-01-23 Haroon Muhammad muhammad.har...@live.com Hi, I have observed that DESCRIBE EXTENDED always shows number of rows to be zero despite the fact that the table has data. Is it a bug? Is it known ? Has anyone else also come across the same ? Thanks,
Re: DESCRIBE EXTENDED show numRows=0
If the data is loaded into table with insert clause with hive.stats.autogather=true, it will be updated. If it's not, it's zero (or -1 sometimes). 2014-01-23 Haroon Muhammad muhammad.har...@live.com Hi, I have observed that DESCRIBE EXTENDED always shows number of rows to be zero despite the fact that the table has data. Is it a bug? Is it known ? Has anyone else also come across the same ? Thanks,
DESCRIBE EXTENDED show numRows=0
Hi, I have observed that DESCRIBE EXTENDED always shows number of rows to be zero despite the fact that the table has data. Is it a bug? Is it known ? Has anyone else also come across the same ? Thanks,