[jira] [Commented] (HIVE-2032) create database does not honour warehouse.dir in dbproperties
[
https://issues.apache.org/jira/browse/HIVE-2032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13017419#comment-13017419
]
Thiruvel Thirumoolan commented on HIVE-2032:
I prefer HIVE-1537 approach. I got confused by the test queries and found 1537
only later.
Thanks for the clarification.
> create database does not honour warehouse.dir in dbproperties
> -
>
> Key: HIVE-2032
> URL: https://issues.apache.org/jira/browse/HIVE-2032
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 0.7.0, 0.8.0
>Reporter: Thiruvel Thirumoolan
>Assignee: Thiruvel Thirumoolan
> Fix For: 0.8.0
>
> Attachments: DatabaseLocation.patch
>
>
> # create database db with dbproperties ('hive.metastore.warehouse.dir' =
> 'loc');
> The above command does not set location of 'db' to 'loc'. It instead creates
> 'db.db' under the warehouse directory configured in hive-site.xml of CLI.
> Looks conflicting with HIVE-1820's expectation. If scratch dir is specified
> here, that is honoured.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2032) create database does not honour warehouse.dir in dbproperties
[
https://issues.apache.org/jira/browse/HIVE-2032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13017363#comment-13017363
]
Amareshwari Sriramadasu commented on HIVE-2032:
---
I agree with Ning. There should not be two piece of information with same
semantics. Location should be added to create database and alter database if it
can be altered. Also, We should document that dbproperties are interpreted at
application level.
> create database does not honour warehouse.dir in dbproperties
> -
>
> Key: HIVE-2032
> URL: https://issues.apache.org/jira/browse/HIVE-2032
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 0.7.0, 0.8.0
>Reporter: Thiruvel Thirumoolan
>Assignee: Thiruvel Thirumoolan
> Fix For: 0.8.0
>
> Attachments: DatabaseLocation.patch
>
>
> # create database db with dbproperties ('hive.metastore.warehouse.dir' =
> 'loc');
> The above command does not set location of 'db' to 'loc'. It instead creates
> 'db.db' under the warehouse directory configured in hive-site.xml of CLI.
> Looks conflicting with HIVE-1820's expectation. If scratch dir is specified
> here, that is honoured.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2032) create database does not honour warehouse.dir in dbproperties
[
https://issues.apache.org/jira/browse/HIVE-2032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13017277#comment-13017277
]
Ning Zhang commented on HIVE-2032:
--
Thiruvel, HIVE-1820 (actually the subtask of it HIVE-1836) is not intended to
define the database LOCATION. The database metastore schema already contains a
LOCATION field, but it was not currently exposed through the CREATE DATABASE
syntax. That's what HIVE-1537 is about to solve. HIVE-1836 is to introduce
dbproperties of generic key-value pairs. The dbproperties can be
used/interpreted by application-level pre-execution hooks. We currently do not
require the warehouse.dir (if it is one of the keys) must be consistent with
the LOCATION field, and I think probably we don't need to keep 2 piece of
information with the same semantics.
If you want to change the database location URL I think probably a better way
is to change the CREATE DATABASE syntax to add location (HIVE-1537) and any
table created under that database should honor the database location property.
What do you think?
> create database does not honour warehouse.dir in dbproperties
> -
>
> Key: HIVE-2032
> URL: https://issues.apache.org/jira/browse/HIVE-2032
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 0.7.0, 0.8.0
>Reporter: Thiruvel Thirumoolan
>Assignee: Thiruvel Thirumoolan
> Fix For: 0.8.0
>
> Attachments: DatabaseLocation.patch
>
>
> # create database db with dbproperties ('hive.metastore.warehouse.dir' =
> 'loc');
> The above command does not set location of 'db' to 'loc'. It instead creates
> 'db.db' under the warehouse directory configured in hive-site.xml of CLI.
> Looks conflicting with HIVE-1820's expectation. If scratch dir is specified
> here, that is honoured.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2032) create database does not honour warehouse.dir in dbproperties
[
https://issues.apache.org/jira/browse/HIVE-2032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13015951#comment-13015951
]
Thiruvel Thirumoolan commented on HIVE-2032:
HIVE-1537 is supporting LOCATION in create database command
HIVE-675 - first comment is consistent with HIVE-1537
HIVE-1820 is about warehouse location through DBPROPERTIES
My approach on this JIRA so far was based on HIVE-1820.
What is the suggested way to specify location during create database?
@Carl, @Ning, @He Yongqiang - suggestions?
> create database does not honour warehouse.dir in dbproperties
> -
>
> Key: HIVE-2032
> URL: https://issues.apache.org/jira/browse/HIVE-2032
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 0.7.0, 0.8.0
>Reporter: Thiruvel Thirumoolan
>Assignee: Thiruvel Thirumoolan
> Fix For: 0.8.0
>
> Attachments: DatabaseLocation.patch
>
>
> # create database db with dbproperties ('hive.metastore.warehouse.dir' =
> 'loc');
> The above command does not set location of 'db' to 'loc'. It instead creates
> 'db.db' under the warehouse directory configured in hive-site.xml of CLI.
> Looks conflicting with HIVE-1820's expectation. If scratch dir is specified
> here, that is honoured.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2032) create database does not honour warehouse.dir in dbproperties
[
https://issues.apache.org/jira/browse/HIVE-2032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13015925#comment-13015925
]
Thiruvel Thirumoolan commented on HIVE-2032:
> Use cases make sense. But, drop database would remove tables only from new
> location.
If I am not wrong, drop db succeeds only if all tables under it are dropped.
> Thiruvel, did you get a chance to test this? Because, your changes in patch
> does not look complete. Changes should be propagated to PersistentManager
> through ObjectStore.
Will take a look.
> create database does not honour warehouse.dir in dbproperties
> -
>
> Key: HIVE-2032
> URL: https://issues.apache.org/jira/browse/HIVE-2032
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 0.7.0, 0.8.0
>Reporter: Thiruvel Thirumoolan
>Assignee: Thiruvel Thirumoolan
> Fix For: 0.8.0
>
> Attachments: DatabaseLocation.patch
>
>
> # create database db with dbproperties ('hive.metastore.warehouse.dir' =
> 'loc');
> The above command does not set location of 'db' to 'loc'. It instead creates
> 'db.db' under the warehouse directory configured in hive-site.xml of CLI.
> Looks conflicting with HIVE-1820's expectation. If scratch dir is specified
> here, that is honoured.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2032) create database does not honour warehouse.dir in dbproperties
[
https://issues.apache.org/jira/browse/HIVE-2032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13015918#comment-13015918
]
Amareshwari Sriramadasu commented on HIVE-2032:
---
Use cases make sense. But, drop database would remove tables only from new
location.
Does anyone have the idea why alter Database currently allows to change only DB
properties? Namit/Ning?
bq.Post altering, new tables will be created under new location. Old tables'
have the fully qualified location in metadata and they should continue to work
as before.
Thiruvel, did you get a chance to test this? Because, your changes in patch
does not look complete. Changes should be propagated to PersistentManager
through ObjectStore.
> create database does not honour warehouse.dir in dbproperties
> -
>
> Key: HIVE-2032
> URL: https://issues.apache.org/jira/browse/HIVE-2032
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 0.7.0, 0.8.0
>Reporter: Thiruvel Thirumoolan
>Assignee: Thiruvel Thirumoolan
> Fix For: 0.8.0
>
> Attachments: DatabaseLocation.patch
>
>
> # create database db with dbproperties ('hive.metastore.warehouse.dir' =
> 'loc');
> The above command does not set location of 'db' to 'loc'. It instead creates
> 'db.db' under the warehouse directory configured in hive-site.xml of CLI.
> Looks conflicting with HIVE-1820's expectation. If scratch dir is specified
> here, that is honoured.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2032) create database does not honour warehouse.dir in dbproperties
[
https://issues.apache.org/jira/browse/HIVE-2032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13015818#comment-13015818
]
Thiruvel Thirumoolan commented on HIVE-2032:
@Amareshwari
Post altering, new tables will be created under new location. Old tables' have
the fully qualified location in metadata and they should continue to work as
before.
The reasons I went with alter location are:
1. Allow migration to happen if one would like to reorganize or if quota runs
out. Not sure how many folks have this situation.
2. One can migrate new tables to another DFS cluster (existing cluster becoming
full).
3. One can migrate between file systems if they have sufficient use cases.
Do you think these are valid use cases?
> create database does not honour warehouse.dir in dbproperties
> -
>
> Key: HIVE-2032
> URL: https://issues.apache.org/jira/browse/HIVE-2032
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 0.7.0, 0.8.0
>Reporter: Thiruvel Thirumoolan
>Assignee: Thiruvel Thirumoolan
> Fix For: 0.8.0
>
> Attachments: DatabaseLocation.patch
>
>
> # create database db with dbproperties ('hive.metastore.warehouse.dir' =
> 'loc');
> The above command does not set location of 'db' to 'loc'. It instead creates
> 'db.db' under the warehouse directory configured in hive-site.xml of CLI.
> Looks conflicting with HIVE-1820's expectation. If scratch dir is specified
> here, that is honoured.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2032) create database does not honour warehouse.dir in dbproperties
[
https://issues.apache.org/jira/browse/HIVE-2032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13014458#comment-13014458
]
Amareshwari Sriramadasu commented on HIVE-2032:
---
I see changes for alter Database also to change the location URI. What are
semantics of alter database for changing location uri? Is it going to create
the tables under the new location after altering? If so, what happens to
earlier tables. I see that location uri is accessed only in create db and drop
db. So, I think we should not allow altering the location uri. Shall we throw
an error if it altered?
Changes for create DB look fine. Can you update
TestHiveMetaStore.testDatabase() to test the same?
> create database does not honour warehouse.dir in dbproperties
> -
>
> Key: HIVE-2032
> URL: https://issues.apache.org/jira/browse/HIVE-2032
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 0.7.0, 0.8.0
>Reporter: Thiruvel Thirumoolan
>Assignee: Thiruvel Thirumoolan
> Fix For: 0.8.0
>
> Attachments: DatabaseLocation.patch
>
>
> # create database db with dbproperties ('hive.metastore.warehouse.dir' =
> 'loc');
> The above command does not set location of 'db' to 'loc'. It instead creates
> 'db.db' under the warehouse directory configured in hive-site.xml of CLI.
> Looks conflicting with HIVE-1820's expectation. If scratch dir is specified
> here, that is honoured.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
