Hi Chalcy
When you create a table you do specify the format of how the data is stored in
hdfs. This value can be determined using , describe extended or describe
formatted at any later point.
Try out
Describe formatted ;
To ensure the file in hdfs is in SequenceFileFormat, you can check the
Hi Tishan
AFAIK there is no way to specify the delimiter in INSERT OVERWRITE command. The
delimiter could be specified on table creation only.
You can get your requirement done my creating an intermediate table .
1) Define the intermediate table with the required delimiter of your choice.
2) L
Hi Bejoy,
Thanks. Will an external table also do the same thing?
On Tue, Jun 19, 2012 at 12:35 PM, Bejoy KS wrote:
> **
> Hi Tishan
>
> AFAIK there is no way to specify the delimiter in INSERT OVERWRITE
> command. The delimiter could be specified on table creation only.
>
> You can get your requ
Yes, external Tables are no different that managed tables. The main difference
in case of external tables is, if you drop a managed table the data in hdfs is
also deleted where as in case of external tables the data is not deleted from
hdfs only the entry from metastore is removed.
Regards
Be
Describe formatted tablename is a great DDL.
For one table sqoop imported into hive table as sequence file, I see the
metadata starts with "SEQ-!".
I created another table like the one which shows SEQ in the metafile and loaded
data into this table and I do not see SEQ in the meta data. I'
So...I have a table that has thousands of files, and Billions of rows
related it.
Lets make this a simple table:
CREATE TABLE test_table (
ts BIGINT,
exec_time DOUBLE,
domain_id BIGINT,
domain_name STRING,
)
PARTITIONED BY (logdate STRING, source STRING, datacenter STRING,
hostnam
You can not change the partition columns. I would use a dynamic
partition insert to select all the data from the original table into
the new table.
On 6/19/12, Tim Havens wrote:
> So...I have a table that has thousands of files, and Billions of rows
> related it.
>
> Lets make this a simple table
Hello,
I would like to force all mapreduce jobs run from the Hive shell to run as the
hdfs user who ran them instead of as the "hive" user. For instance, I have HDFS
testuser1 logged into the edge node under their unix user with the same name
testuser1. This user begins a hive shell and kicks o
I agree.
It would be best if you wrote a script that iterates through each leaf-level
partition of your existing table (logdate='A', source='B', datacenter='C',
hostname='D')
and populate new leaf-level partitions in the new table (logdate='A',
source='B', datacenter='C', hostname='D', loghour).
Rajan,
On a somewhat related note, I had issues when using Amazon's MySQL server (on
the master node) as metastore on a Hive 0.8.1 cluster.
Related thread:
- Original Message -
From: "Carl Steinbach"
To: user@hive.apache.org
Sent: Monday, June 18, 2012 5:07:12 PM
Subject: Re: Migrating to
Thanks for the 'small files' heads up. The current LOGDATE file is about
7.2GB Gzipped. So I don't think we'll be running into many small files
when that's split into 24 hours. But thanks for the reminder about 'small
files'.
I guess I'd rather just reload the 2500 logs or so...than have to jum
Sorry, missed the link:
https://forums.aws.amazon.com/thread.jspa?threadID=95988&tstart=25
- Original Message -
From: "Mark Grover"
To: user@hive.apache.org
Sent: Tuesday, June 19, 2012 1:31:25 PM
Subject: Re: Migrating to hive 8.1 on EMR
Rajan,
On a somewhat related note, I had issues w
Is it possible, to create external table on a existing hive table which is
partitioned.
I have a existing hive table which is partitioned by dt and group like below
desc page_access;
page string
country string
dt string ( Partitioned column )
group string ( Partitioned column )
dfs
Sai,
Maybe I don't understand your question properly but creating an external table
on a partitioned table is no different than create an external table on a
non-partitioned one.
Your syntax looks right. After table creation, you would have to add all
existing partitions of the table so that th
Thanks Mark,
The reason to create the 2nd table is One of the column is defined as
string in the first table, I wanted to read the string into Map data type.
i.e
Existing table.
{"UY": 2, "BR": 1}
{"LV": 1, "BR": 1}
To
Country Map
Thanks
Gopi
On Tue, Jun 19, 2012 at 1:37 PM, Mark Grover
Hi Sai,
You might want to look into Alter table statements to see the type of the
column can be modified. You may need to upgrade Hive if that functionality is
not available in your version.
Mark
- Original Message -
From: "Sai Naga"
To: user@hive.apache.org
Sent: Tuesday, June 19, 201
In hindsight, I have learned to create external tables so if I get into a
situation like this, I don't have to look up the appropriate alter table
statment (if it exists at all), I can just drop the external table and
re-create it:-)
Mark
- Original Message -
From: "Sai Naga"
To: user
Hi,
I have a data file that is exactly equivalent to a CSV , except that the
field delimiter is a control character specifically '\u0001' . How can I
create an external table in hive for this data ?
For instance .
create table ... blah .blah ...
row format delimited fields terminated b
I have multiple statements in a single .hql file that I am calling via an oozie
action.
Most of these statements can be executed in parallel (they do not depend on
each other). I already have the parallel execution flag set to true (although
I have yet to see multiple Hive MR jobs running at on
Hi,
Statements in an query file are executed serially. When a query is parsed by
Hive, independent stages of the query are executed in parallel when you set the
parallelization flag.
If the queries are completely independent of each other, it may be better to
split them into separate files and
Did you add the word external in create table I.e.
Create external table(...blah...blah...)
Sent from my iPhone
On Jun 19, 2012, at 4:15 PM, Sam William wrote:
> Hi,
> I have a data file that is exactly equivalent to a CSV , except that the
> field delimiter is a control character specifi
Has anyone been success full with integrating HBase ACL with Hive?
Recent versions of HBase (0.92.1 and 0.94.0) come with ACL features.
I tried to integrate Hive 0.9.0 with HBase 0.94.0.
I ran hive cli client like following
hive --auxpath
"$HIVE_HOME/lib/hive-hbase-handler-0.9.0.jar,$HIVE_HOME/lib
22 matches
Mail list logo
