Hudson build is back to normal : Hive-trunk-h0.17 #426
See http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.17/426/changes
Hudson build is back to normal : Hive-trunk-h0.18 #429
See http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.18/429/changes
Hudson build is back to normal : Hive-trunk-h0.19 #428
See http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.19/428/changes
[jira] Commented: (HIVE-610) move all properties from jpox.properties to hive-site.xml
[ https://issues.apache.org/jira/browse/HIVE-610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12862797#action_12862797 ] Edward Capriolo commented on HIVE-610: -- Does this mean the jpox.properties is now ignored ? If so how to we set other JPOX variables? move all properties from jpox.properties to hive-site.xml -- Key: HIVE-610 URL: https://issues.apache.org/jira/browse/HIVE-610 Project: Hadoop Hive Issue Type: Improvement Components: Metastore Affects Versions: 0.4.0 Reporter: Prasad Chakka Assignee: Prasad Chakka Fix For: 0.4.0 Attachments: hive-610.patch there some properties in jpox.properties and some in hive-site.xml. move all to the later file. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-610) move all properties from jpox.properties to hive-site.xml
[ https://issues.apache.org/jira/browse/HIVE-610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12862799#action_12862799 ] Paul Yang commented on HIVE-610: Properties for JPOX (i.e. datanucleus) can be set in hive-site.xml like the following property namedatanucleus.validateTables/name valuefalse/value descriptionvalidates existing schema against code. turn this on if you want to verify existing schema /description /property The metastore will pick up variables with names containing datanucleus or jdo and pass them on to the required objects. move all properties from jpox.properties to hive-site.xml -- Key: HIVE-610 URL: https://issues.apache.org/jira/browse/HIVE-610 Project: Hadoop Hive Issue Type: Improvement Components: Metastore Affects Versions: 0.4.0 Reporter: Prasad Chakka Assignee: Prasad Chakka Fix For: 0.4.0 Attachments: hive-610.patch there some properties in jpox.properties and some in hive-site.xml. move all to the later file. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HIVE-1332) Archiving partitions
Archiving partitions Key: HIVE-1332 URL: https://issues.apache.org/jira/browse/HIVE-1332 Project: Hadoop Hive Issue Type: New Feature Components: Metastore Affects Versions: 0.6.0 Reporter: Paul Yang Assignee: Paul Yang Partitions and tables in Hive typically consist of many files on HDFS. An issue is that as the number of files increase, there will be higher memory/load requirements on the namenode. Partitions in bucketed tables are a particular problem because they consist of many files, one for each of the buckets. One way to drastically reduce the number of files is to use hadoop archives: http://hadoop.apache.org/common/docs/current/hadoop_archives.html This feature would introduce an ALTER TABLE table_name ARCHIVE PARTITION spec that would automatically put the files for the partition into a HAR file. We would also have an UNARCHIVE option to convert the files in the partition back to the original files. Archived partitions would be slower to access, but they would have the same functionality and decrease the number of files drastically. Typically, only seldom accessed partitions would be archived. Hadoop archives are still somewhat new, so we'll only put in support for the latest released major version (0.20). Here are some bug fixes: https://issues.apache.org/jira/browse/HADOOP-6591 (Important - could potentially cause data loss without this fix) https://issues.apache.org/jira/browse/HADOOP-6645 https://issues.apache.org/jira/browse/MAPREDUCE-1585 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HIVE-1333) javax.jdo.option.NonTransactionalRead ignored?
javax.jdo.option.NonTransactionalRead ignored?
--
Key: HIVE-1333
URL: https://issues.apache.org/jira/browse/HIVE-1333
Project: Hadoop Hive
Issue Type: Bug
Components: Metastore, Query Processor
Reporter: Edward Capriolo
{noformat}
property
namejavax.jdo.option.NonTransactionalRead/name
valuetrue/value
descriptionreads outside of transactions/description
/property
{noformat}
hive show tables
{noformat}
100430 14:41:39 1874 Connect hiv...@localhost on
1874 Init DB m6_
1874 Query SHOW SESSION VARIABLES
1874 Query SHOW COLLATION
1874 Query SET character_set_results = NULL
1874 Query SET autocommit=1
1874 Query SET sql_mode='STRICT_TRANS_TABLES'
1874 Query SET autocommit=0
1874 Query SELECT @@session.tx_isolation
1874 Query SET SESSION TRANSACTION ISOLATION LEVEL READ
COMMITTED
1874 Query SELECT `THIS`.`TBL_NAME` FROM `TBLS` `THIS`
LEFT OUTER JOIN `DBS` `THIS_DATABASE_NAME` ON `THIS`.`DB_ID` =
`THIS_DATABASE_NAME`.`DB_ID` WHERE `THIS_DATABASE_NAME`.`NAME` = 'default' AND
(LOWER(`THIS`.`TBL_NAME`) LIKE '_%' ESCAPE '\\' )
1874 Query commit
1874 Query rollback
1874 Quit
{noformat}
now set to false
{noformat}
100430 14:46:59 1889 Connect hiv...@localhost on
1889 Init DB m6_rshive
1889 Query SHOW SESSION VARIABLES
1889 Query SHOW COLLATION
1889 Query SET character_set_results = NULL
1889 Query SET autocommit=1
1889 Query SET sql_mode='STRICT_TRANS_TABLES'
1889 Query SET autocommit=0
1889 Query SELECT @@session.tx_isolation
1889 Query SET SESSION TRANSACTION ISOLATION LEVEL READ
COMMITTED
1889 Query SELECT `THIS`.`TBL_NAME` FROM `TBLS` `THIS`
LEFT OUTER JOIN `DBS` `THIS_DATABASE_NAME` ON `THIS`.`DB_ID` =
`THIS_DATABASE_NAME`.`DB_ID` WHERE `THIS_DATABASE_NAME`.`NAME` = 'default' AND
(LOWER(`THIS`.`TBL_NAME`) LIKE '_%' ESCAPE '\\' )
1889 Query commit
1889 Query rollback
1889 Quit
{noformat}
Unless I misuderstand the property it looks like the reads are still inside a
transaction. Also why does this transaction call commit as well as rollback?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1300) support: alter table touch partition
[ https://issues.apache.org/jira/browse/HIVE-1300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12862839#action_12862839 ] John Sichi commented on HIVE-1300: -- Paul, could you update the wiki for this feature? http://wiki.apache.org/hadoop/Hive/LanguageManual/DDL#Alter_Table_Statements support: alter table touch partition Key: HIVE-1300 URL: https://issues.apache.org/jira/browse/HIVE-1300 Project: Hadoop Hive Issue Type: Improvement Components: Query Processor Reporter: Namit Jain Assignee: Paul Yang Fix For: 0.6.0 Attachments: HIVE-1300.1.patch, HIVE-1300.2.patch In some cases, the user wants to touch a partition, since some other operations might be performed on the hdfs directories. Currently, there is no way to do that. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Hudson build is back to normal : Hive-trunk-h0.20 #251
See http://hudson.zones.apache.org/hudson/job/Hive-trunk-h0.20/251/changes
[jira] Commented: (HIVE-1300) support: alter table touch partition
[ https://issues.apache.org/jira/browse/HIVE-1300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12862861#action_12862861 ] Paul Yang commented on HIVE-1300: - Wiki has been updated. support: alter table touch partition Key: HIVE-1300 URL: https://issues.apache.org/jira/browse/HIVE-1300 Project: Hadoop Hive Issue Type: Improvement Components: Query Processor Reporter: Namit Jain Assignee: Paul Yang Fix For: 0.6.0 Attachments: HIVE-1300.1.patch, HIVE-1300.2.patch In some cases, the user wants to touch a partition, since some other operations might be performed on the hdfs directories. Currently, there is no way to do that. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1331) select * does not work if different partitions contain different formats
[ https://issues.apache.org/jira/browse/HIVE-1331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12862866#action_12862866 ] Ning Zhang commented on HIVE-1331: -- Will commit if tests pass. select * does not work if different partitions contain different formats Key: HIVE-1331 URL: https://issues.apache.org/jira/browse/HIVE-1331 Project: Hadoop Hive Issue Type: Bug Components: Query Processor Reporter: Namit Jain Assignee: Namit Jain Fix For: 0.6.0 Attachments: hive.1331.1.patch Will try to come up with a concrete test - but looks like we are using the table's input format -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1192) Build fails when hadoop.version=0.20.1
[ https://issues.apache.org/jira/browse/HIVE-1192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12862882#action_12862882 ] John Sichi commented on HIVE-1192: -- Checksum file has been corrected; please retry. Build fails when hadoop.version=0.20.1 -- Key: HIVE-1192 URL: https://issues.apache.org/jira/browse/HIVE-1192 Project: Hadoop Hive Issue Type: Bug Components: Build Infrastructure Reporter: Carl Steinbach Attachments: hadoop-0.20.1.tar.gz.md5 Setting hadoop.version=0.20.1 causes the build to fail since mirror.facebook.net/facebook/hive-deps does not have 0.20.1 (only 0.17.2.1, 0.18.3, 0.19.0, 0.20.0). Suggested fix: * remove/ignore the hadoop.version configuration parameter or * Remove the patch numbers from these archives and use only the major.minor numbers specified by the user to locate the appropriate tarball to download, so 0.20.0 and 0.20.1 would both map to hadoop-0.20.tar.gz. * Optionally create new tarballs that only contain the components that are actually needed for the build (Hadoop jars), and remove things that aren't needed (all of the source files). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1332) Archiving partitions
[ https://issues.apache.org/jira/browse/HIVE-1332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Yang updated HIVE-1332: Attachment: HIVE-1332.1.patch Archiving partitions Key: HIVE-1332 URL: https://issues.apache.org/jira/browse/HIVE-1332 Project: Hadoop Hive Issue Type: New Feature Components: Metastore Affects Versions: 0.6.0 Reporter: Paul Yang Assignee: Paul Yang Attachments: HIVE-1332.1.patch Partitions and tables in Hive typically consist of many files on HDFS. An issue is that as the number of files increase, there will be higher memory/load requirements on the namenode. Partitions in bucketed tables are a particular problem because they consist of many files, one for each of the buckets. One way to drastically reduce the number of files is to use hadoop archives: http://hadoop.apache.org/common/docs/current/hadoop_archives.html This feature would introduce an ALTER TABLE table_name ARCHIVE PARTITION spec that would automatically put the files for the partition into a HAR file. We would also have an UNARCHIVE option to convert the files in the partition back to the original files. Archived partitions would be slower to access, but they would have the same functionality and decrease the number of files drastically. Typically, only seldom accessed partitions would be archived. Hadoop archives are still somewhat new, so we'll only put in support for the latest released major version (0.20). Here are some bug fixes: https://issues.apache.org/jira/browse/HADOOP-6591 (Important - could potentially cause data loss without this fix) https://issues.apache.org/jira/browse/HADOOP-6645 https://issues.apache.org/jira/browse/MAPREDUCE-1585 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
