RE: Query regarding Metastore(Derby) SDS table data
Thanks Namit,
Here I'm interested to know like can we use full path excluding the Name
Node IP and Port. If we do like this we can have a portability like later
NameNode IP or Port got changed also we can retrieve the data.
For Example:
Using hive with the default configurations of hadoop created a sample table.
Later changed the NameNode port to some other port and restarted hadoop(9000
to 9090). In this case we are not able to access the tables created before
changing the NameNode Port because for that table path contains old port
(before changing the port).
If we store the location of the table by excluding NameNode IP Port and in
hive construct full HDFS URL Path.
Pls give u r inputs
***
This e-mail and attachments contain confidential information
from HUAWEI, which is intended only for the person or entity whose address
is listed above. Any use of the information contained herein in any way
(including, but not limited to, total or partial disclosure, reproduction,
or dissemination) by persons other than the intended recipient's) is
prohibited. If you receive this e-mail in error, please notify the sender by
phone or email immediately and delete it!
-Original Message-
From: Namit Jain [mailto:nj...@fb.com]
Sent: Friday, February 04, 2011 12:58 AM
To: dev@hive.apache.org; chinna...@huawei.com; hive-...@hadoop.apache.org
Subject: Re: Query regarding Metastore(Derby) SDS table data
Different partitions can have different paths -
A partition's path need not be a sub-direcotry of the table's path.
In facebook, we use this regularly, specially for external tables.
So, it simplifies things if the full path is stored for the partition in
the metastore.
Thanks,
-namit
On 2/3/11 2:22 AM, Chinna chinna...@huawei.com wrote:
Hi all,
While creating hive table using create table command, the code flow will
also insert the HDFS location in SDS table present in Derby
For Example on executing:-
create table sample(rate int) stored as textfile;
The SDS table (meta table present in Derby) contains following entry
corresponding to Hive table sample like
http://{IP}:9001/user/hive/warehouse/sample
say 9001 is the port configured,
and {HDFS-URL}/user/hive/warehouse is the warehouse configured.
Now here I'm interested to know the intention of maintaining full path of
the HDFS location in SDS Table, say why not a relative path, anyhow Hive
is
capable of constructing full HDFS URL Path.
Any design inputs on it, will surely help.
Thanks,
Chinna Rao Lalam
**
**
***
This e-mail and attachments contain confidential information
from HUAWEI, which is intended only for the person or entity whose address
is listed above. Any use of the information contained herein in any way
(including, but not limited to, total or partial disclosure, reproduction,
or dissemination) by persons other than the intended recipient's) is
prohibited. If you receive this e-mail in error, please notify the sender
by
phone or email immediately and delete it!
Query regarding Metastore(Derby) SDS table data
Hi all,
While creating hive table using create table command, the code flow will
also insert the HDFS location in SDS table present in Derby
For Example on executing:-
create table sample(rate int) stored as textfile;
The SDS table (meta table present in Derby) contains following entry
corresponding to Hive table sample like
http://{IP}:9001/user/hive/warehouse/sample
say 9001 is the port configured,
and {HDFS-URL}/user/hive/warehouse is the warehouse configured.
Now here I'm interested to know the intention of maintaining full path of
the HDFS location in SDS Table, say why not a relative path, anyhow Hive is
capable of constructing full HDFS URL Path.
Any design inputs on it, will surely help.
Thanks,
Chinna Rao Lalam
***
This e-mail and attachments contain confidential information
from HUAWEI, which is intended only for the person or entity whose address
is listed above. Any use of the information contained herein in any way
(including, but not limited to, total or partial disclosure, reproduction,
or dissemination) by persons other than the intended recipient's) is
prohibited. If you receive this e-mail in error, please notify the sender by
phone or email immediately and delete it!
Re: Query regarding Metastore(Derby) SDS table data
Different partitions can have different paths -
A partition's path need not be a sub-direcotry of the table's path.
In facebook, we use this regularly, specially for external tables.
So, it simplifies things if the full path is stored for the partition in
the metastore.
Thanks,
-namit
On 2/3/11 2:22 AM, Chinna chinna...@huawei.com wrote:
Hi all,
While creating hive table using create table command, the code flow will
also insert the HDFS location in SDS table present in Derby
For Example on executing:-
create table sample(rate int) stored as textfile;
The SDS table (meta table present in Derby) contains following entry
corresponding to Hive table sample like
http://{IP}:9001/user/hive/warehouse/sample
say 9001 is the port configured,
and {HDFS-URL}/user/hive/warehouse is the warehouse configured.
Now here I'm interested to know the intention of maintaining full path of
the HDFS location in SDS Table, say why not a relative path, anyhow Hive
is
capable of constructing full HDFS URL Path.
Any design inputs on it, will surely help.
Thanks,
Chinna Rao Lalam
**
**
***
This e-mail and attachments contain confidential information
from HUAWEI, which is intended only for the person or entity whose address
is listed above. Any use of the information contained herein in any way
(including, but not limited to, total or partial disclosure, reproduction,
or dissemination) by persons other than the intended recipient's) is
prohibited. If you receive this e-mail in error, please notify the sender
by
phone or email immediately and delete it!
