hello:
Sorry to bother you again, can you give me a simple example of sqoop
export using a CAST?
thinks.
------------------ 原始邮件 ------------------
发件人: "Markus Kemper";<[email protected]>;
发送时间: 2017年12月1日(星期五) 晚上9:24
收件人: "user"<[email protected]>;
主题: Re: 回复： How to limit the number of rows to export data when using sqoopto
export data from hdfs to oracle?
Hey Team,
Not sure a VIEW with export will work. If I recall export with --hcatalog is
not aware of HMS VIEWs, need to test.
Markus Kemper
Customer Operations Engineer
On Fri, Dec 1, 2017 at 8:00 AM, Attila Szabó <[email protected]> wrote:
Hey,
IMHO it's CTAS which stands for "Create Table As Select".
On the front of views :
The question is more than valid if Sqoop supports it or not. I do remember a
problem we've faced 1-1.5 year ago in connection with Hive + Hcatalog + Sqoop,
and that was not supported because of some missing Hive Serde implementation.
I'm not sure if this problem exists with standard Hive views and Sqoop export
command, but you should give a try.
The thing what could be problematic with views:
A View is a result of a select statement by design. But in Hive every HiveQL
command is translated to a map/reduce job. But Sqoop also works with map/reduce
jobs and thus it tries to read files from the HDFS ( because of data locality
and things), so this might be a clashing problem here, but as I've advised you
should give it a try.
With CTAS:
It should definitely work, because in this case Hive will store the filtered
results in a different Hive table ( == HDFS directory) and thus export dir is
your friend. :)
Were I able to clarify everything or do you have further questions?
Cheers,
Attila
On Dec 1, 2017 1:33 PM, "qq" <[email protected]> wrote:
Hello:
First of all, thank you very much for your answer, I just started to
touch sqoop, there are many do not understand, you can explain in detail about
the operation steps of sqoop export work with views and the steps of exact
dataset to sqoop using a CAST?
thinks.
I am looking forward to your reply！
------------------ 原始邮件 ------------------
发件人: "Per Ullberg";<[email protected]>;
发送时间: 2017年12月1日(星期五) 晚上6:21
收件人: "user"<[email protected]>;
主题: Re: How to limit the number of rows to export data when using sqoopto
export data from hdfs to oracle?
Does Sqoop export work with Views? If not, you'll have to materialise the exact
dataset you want to sqoop using a CTAS.
regards
/Pelle
On Fri, Dec 1, 2017 at 11:08 AM, Attila Szabó <[email protected]> wrote:
Hey,
If you're trying to export from Hive into RDBMS I would suggest creating a Hive
view and only export the content of the view. Thus you could directly control
the data quantity by the underlying HiveQL query.
My 2cents,
Attila
On Dec 1, 2017 10:54 AM, "qq" <[email protected]> wrote:
Hello：
I have a question on the export of sqoop need your help, the problem is
as follows：
How to limit the number of rows that need to be exported when exporting
data from hdfs to oracle using sqoop?
For example: hive data stored in the hdfs 100 lines, just want the first
10 lines of data through sqoop exported to the oracle table, how to achieve
through sqoop?
I am looking forward to your reply！
thinks.
--
Per Ullberg
Datavault Tech Lead
Odin (Uppsala)
Klarna Bank AB (publ)
Sveavägen 46, 111 34 Stockholm
Tel: +46 8 120 120 00
Reg no: 556737-0431
klarna.com
