Hi Abraham,
Here's the Sqoop command that works.
sqoop export --connect jdbc:mysql://myhost.rds.amazonaws.com/DB1 --username XXX
-password YYY --table test_table --columns "name,ts,age,salary" --update-key
"name" --export-dir /user/aravind/test/year=2015/month=5/day=7/hour=19
--fields-terminated-by "\t" --update-mode allowinsert -m 1
The following workflow does not work.
<workflow-app xmlns="uri:oozie:workflow:0.2" name="ara_sqoop-wf"> <start
to="ara_move_to_mysql"/> <action name="ara_move_to_mysql"> <sqoop
xmlns="uri:oozie:sqoop-action:0.3"> <job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node> <arg>export</arg> <arg>-libjars</arg>
<arg>${wf_folder}lib/mysql-connector-java.jar</arg> <arg>--connect</arg>
<arg>jdbc:mysql://myhost.rds.amazonaws.com/${mysql_db}</arg>
<arg>--driver</arg> <arg>com.mysql.jdbc.Driver</arg> <arg>--username</arg>
<arg>${mysql_usern}</arg> <arg>-password</arg> <arg>${mysql_pswd}</arg>
<arg>--table</arg> <arg>${table_name}</arg> <arg>--columns</arg>
<arg>"name,ts,age,salary"</arg> <arg>--update-key</arg> <arg>"name"</arg>
<arg>--export-dir</arg>
<arg>/user/aravind/test/year=2015/month=5/day=7/hour=19</arg>
<arg>--fields-terminated-by</arg> <arg>"\t"</arg> <arg>--update-mode</arg>
<arg>allowinsert</arg> <arg>-m</arg> <arg>1</arg> </sqoop> <ok to="end"/>
<error to="fail"/> </action> <kill name="fail"> <message>Sqoop failed,
error message[${wf:errorMessage(wf:lastErrorNode())}]</message> </kill> <end
name="end"/></workflow-app>
Thanks,Aravind
From: Abraham Elmahrek <[email protected]>
To: "[email protected]" <[email protected]>; Aravindakshan Srinivasan
<[email protected]>
Sent: Friday, May 8, 2015 10:36 PM
Subject: Re: Oozie + Sqoop + MySql UPSERT
Can you send over your oozie job or sqoop command?
On Fri, May 8, 2015 at 8:30 AM, Aravindakshan Srinivasan <[email protected]>
wrote:
Friends,
I have an odd problem. I am trying to run an MySql upsert via Sqoop. It works
fine from the edge node that has Sqoop 1.4.5 installed. When I try to run via
Oozie, I get this error.
org.apache.sqoop.tool.ExportTool - Error during export: Mixed update/insert is
not supported against the target database
This feature is available in the newer versions of Sqoop starting with Sqoop
1.4.3, which is what my Oozie sharelib contains. Still, my first reaction was
that the Sqoop jars in Oozie's sharelib are different and hence throwing this
error. As you know, the Sqoop via Oozie runs on the Hadoop compute nodes. I
tried copying the Sqoop jars from my edge node to the workflow/lib dir. It
still didn't work.
Since then, I have tried everything I can think of, but can't get past this
error. I have even rebuilt the Oozie sharelib with Sqoop 1.4.5 and tried both
oozie.libpath and oozie.action.sharelib.for.sqoop pointing to my rebuilt lib.
Still no luck. What am I missing?
Thanks,Aravind
--------------------------------------------------------------------------------------------Aravind
Srinivasan
http://goodcynicsports.blogspot.com
I have decided to stick with love. Hate is too great a burden to bear. - MLK,
Jr.--------------------------------------------------------------------------------------------
