Hi
I am using Oozie Coordinators to schedule and run daily Oozie Workflows that
contain 35-40 actions each (I use shell, java , hive and map reduce oozie
actions)
So if anyone needs help and has questions please fire away…
sanjay
From: Sanjay Subramanian
mailto:sanjay.subraman...@wizecommer
We have a similar situation like this in production…for your case case I would
propose the following steps
1. Design a map reduce job (Job Output format - Text, Lzo, Snappy, your choice)
Inputs to Mapper
-- records from these three feeds
Outputs from Mapper
-- Key =Value =
I believe:
alter table _tablename_ set fileformat orcfile;
will do what you want. All future partitions that are added will be in orcfile
format (assuming you use insert to create the partitions) or assumed to be in
orcfile format if you do alter table add partition.
As to whether orcfile wil
not quite sure. you're not happy with the suggestion above?
does the below example work any better for you? ( mytable == the
tablename, myfield == the map column.)
$ hive < wrote:
> Many thanks to all.
>
> The following command returned records like:
> hive>select map_keys(myfield) from mytab
i like #2.
so you have three, say, external tables representing your three feed files.
After the third and final file is loaded then join 'em all together - maybe
make the table partitioned for one per day.
for example:
alter table final add partition (datekey=MMDD);
insert overwrite table f
Many thanks to all.
The following command returned records like:
hive>select map_keys(myfield) from mytable;
["some_key1","some_key2","some_key3"]
["some_key1","some_key2","some_key3"]
hive>select explode(arr) as arr from (select map_keys(myfield) from mytable );
hive>select explode(arr) as ar
More specifically, we have a table that is currently defined as RCFile, to
do this, I'd like to define all new partitions as ORC. With the advent of
ORC, these types of problems are going to come up for many folks, any
guidance would be appreciated ...
Also, based on the strategic goals of ORC fi
Can you give some examples of how to alter partitions for different input
types? I'd appreciate it :)
On Fri, Jul 26, 2013 at 3:29 PM, Alan Gates wrote:
> A table can definitely have partitions with different input
> formats/serdes. We test this all the time.
>
> Assuming your old data doesn't
Hi all,
Below are some of observations based on the on-going rank function
discussion.
1. I executed below mentioned queries and only the query with "rank"
(lowercase) executed successfully, rest were throwing exceptions "FAILED:
SemanticException Failed to breakup Windowing invocations into Gro
A table can definitely have partitions with different input formats/serdes. We
test this all the time.
Assuming your old data doesn't stay for ever and most of your queries are on
more recent data (which is usually the case) I'd advise you to not reprocess
any data, just alter the table to s
Hi,
I'm getting an java.net.UnknownHostException when trying to run a hive insert
on my local. Any ideas?
Output:
> INSERT OVERWRITE TABLE avro_test3 SELECT * FROM avro;
Total MapReduce jobs = 3
Launching Job 1 out of 3
Number of reduce tasks is set to 0 since there's no reduce operator
j
Any further help on this, otherwise I'll file a jira.
On Wed, Jul 24, 2013 at 11:32 PM, j.barrett Strausser <
j.barrett.straus...@gmail.com> wrote:
> As an example : If I run my query above removing the arg the following is
> thrown.
>
> FAILED: SemanticException Failed to breakup Windowing invo
Hi Rams
Please don't think I am sermonizing or preaching and please don't mind what I
am saying :-)
This community is there is help u and there is no doubt about that.
However I am assuming you tried out a few options by yourself before you
reached out to the community with your question.
Sin
Hi,
I need to run Hive 0.11 on Hadoop 2.0.4-alpha. I already have Hive 0.11
compiled (for hadoop 1). Do I need to recompile it against Hadoop 2 so as
to work with Hadoop 2?
Thanks,
John
Yes dean is right.
We can not create external table with dynamic partition.
Here i share with u how to create external table and dynamic partition.:
*Sample Commands:*
*
*
*
*
external table
===
create external table externaltable(a1 int,a2 string,a
Option 1 ) Use pig or oozie, write a workflow and join the files to a
single file
Option 2 ) Create a temp table for each of the different file and then join
them to a single table and delete temp table
Option 3 ) don't do anything, change your queries to look at three
different files when they que
Hi,
Is it possible to write a UDTF with an optional parameter? I was
intending to write a function that works like explode, but can provide
extended functionality given an optional integer flag set to 1. Since
GenericUDTF's initialize function only uses ObjectInspector[] I
currently don't
Hi Dean,
Thanks for the inputs !
I guess have to go by the tedious/hard way :P
Regards,
Omkar Joshi
From: Dean Wampler [mailto:deanwamp...@gmail.com]
Sent: Friday, July 26, 2013 4:26 PM
To: user@hive.apache.org
Subject: Re: Hive - external (dynamically) partitioned table
Hive doesn't support
Hive doesn't support dynamically-partitioned, external tables, in part
because the rational for external is that the data already exists or will
exist outside the control of Hive. Sqoop can create internal tables
directly as it imports and I believe it handles internal,
dynamically-partitioned tabl
Hi,
Please help in providing solution for the below problem... this scenario is
applicable in Banking atleast...
I have a HIVE table with the below structure...
Hive Table:
Field1
...
Field 10
For the above table, I will get the values for each feed in different file.
You can imagine that thes
I'm having a table in MySQL viz. nas_comps.
select comp_code, count(leg_id) from nas_comps_01012011_31012011 n group by
comp_code;
comp_code count(leg_id)
'J' 20640
'Y' 39680
First, I imported data onto HDFSHadoop version 1.0.2) using Sqoop :
sqoop import --connect jdbc:mys
Hi Rama,
Have a look !!
Both sql and hiveql looks similar..
try this one..
*How to create view??*
*
*
*Ans:*
*
*
create view v1 as
select m.Emp_id,
m.Name,
m.Dob,
a.address1,
a.address2,
a.address3,
a.address4,
a.city,
a.state,
a.zip,
a.country from EMP_MASTER m join EMP_ADDRESS a on (m.Emp_id=a
Hi
Have a look at link below. They have enough examples to clarify usage
https://cwiki.apache.org/confluence/display/Hive/Tutorial#Tutorial-Joins
Richa
On Fri, Jul 26, 2013 at 1:47 PM, Ramasubramanian Narayanan <
ramasubramanian.naraya...@gmail.com> wrote:
> Hi,
>
> Need to create a view in H
Hi,
Need to create a view in HIVE as per the below query (written in Oracle).
Need help to provide the equivalent query in HIVE to create view.
*EMP_ADDRESS*
Emp_Id
Address1
Address2
Address3
Address4
City
State
Pin
Country
Alternate_Address1
Alternate_Address2
Alternate_Address3
Alternate_Addr
24 matches
Mail list logo
