Hi Arvind
With this option is it feasible to accomplish dynamic partitions with a
single SQOOP command?
It does well for populating a static partition with values. It'd be great if
you just brief on this implementation.
ie with the step as mentioned, in Sqoop import we need to provide the where
clause to fetch the relevant data from db and load it to some pre partition
defined hive table, where the partitioned hive column is defined by
'--hive-partition-key' and the value defined by '--hive-partition-value'. With
this approach. In order to implement DYNAMIC partitions we need to analyze and
identify the different values corresponding to the partition column of hive
from db table. If there are n different values identified then n different
SQOOP imports with appropriate where clauses need to be fired . Is my
understanding right?
Thank You
Regards
Bejoy K S
-----Original Message-----
From: "[email protected]" <[email protected]>
Date: Mon, 8 Aug 2011 10:00:20
To: <[email protected]>
Reply-To: [email protected]
Subject: Re: [sqoop-user] Dynamic Partition & Sqoop
[bcc:[email protected] (deprecated), to:
[email protected]. Please continue the discussion on
Apache mailing lists - for more information see [1]].
Hi,
Sqoop does support populating Hive partitions. The relevant command
line arguments you need to supply are --hive-partition-key <key-name>,
--hive-partition-value <value-string>. Please refer to the user-guide
for more details.
Thanks,
Arvind
[1] https://cwiki.apache.org/confluence/display/SQOOP/Mailing+Lists
On Mon, Aug 8, 2011 at 5:17 AM, Cyberlord <[email protected]> wrote:
> I’d like to know if Dynamic Partition is supported by Sqoop.
>
> For eg. If I have a database(RDBMS) Informix and want to import data
> into Hive,
> Will it be possible for me to import the data through sqoop into Hive
> after creating dynamic partitions (supported by Hive) on-the-fly ?
>
> I possibly thought of loading the data into a temporary table (through
> Sqoop)
> and then load it into the required table through Hive interface.
>
> I wanted to know if I could migrate the data directly into Hive
> through Sqoop while creating dynamic partitions in Hive without
> using a temporary table.
>
> If it is possible, please let me know how to do it.
>
> Thanks :)
>
> --
> NOTE: The mailing list [email protected] is deprecated in favor of
> Apache Sqoop mailing list [email protected]. Please subscribe
> to it by sending an email to [email protected].
>
--
NOTE: The mailing list [email protected] is deprecated in favor of Apache
Sqoop mailing list [email protected]. Please subscribe to it by
sending an email to [email protected].
