Hi Vineet, Basically i'm using sqoop2(1.99.3), iam not setting any num of mappers here
Please suggest me the configurations in sqoop2 , to do larger tables import fastly to HDFS. Setting database server data retrieval timeout : how to set this in sqoop2? When i'm running small tables, its finishing quickly, But when i'm doing larger tables with millions of rows, reaching mapreduce progress to 86% quickly may b less than a minute. Need suggestion on MAPReduce configurations Since machine configurations are 16GB RAM 500GB Disk Thanks For Quick Reply ---- On Thu, 05 Mar 2015 22:58:22 +0530 Vineet Mishra<[email protected]> wrote ---- Hey syed, Can you mention the number of mappers you are running the job with? It may be the problem with your database server data retrieval timeout limit, try these things. . . Try to set the timeout limit of your respective database to a higher number Or Try to run your job with max mapper possible on your cluster using argument -m [number of mapper] Hope that should help you out. On Mar 5, 2015 4:24 PM, "Syed Akram" <[email protected]> wrote: Hi, Iam importing table with more than million rows using sqoop2 1.99.3 After creating job iam submitting it, and then mapreduce job status is RUNNING and less than 1min PROGRESS reaches to 86% and then stuck for ever, sometimes quit with exception TIMEOUT. please suggest me, what iam doing wrong, and what are the things i hav to check. Every large table with is stuck up at 86%. below is the application status Thanks Akram Syed All Applications Cluster About Nodes ApplicationsNEW NEW_SAVING SUBMITTED ACCEPTED RUNNING FINISHED FAILED KILLED Scheduler Tools Cluster Metrics Apps Submitted Apps Pending Apps Running Apps Completed Containers Running Memory Used Memory Total Memory Reserved VCores Used VCores Total VCores Reserved Active Nodes Decommissioned Nodes Lost Nodes Unhealthy Nodes Rebooted Nodes 1 0 1 0 2 9.84 GB 39.41 GB 0 B 2 40 0 5 0 0 0 0 Show 20406080100 entries Search: ID User Name Application Type Queue StartTime FinishTime State FinalStatus Progress Tracking UI application_1425550787783_0001 sas Sqoop: ImportJob MAPREDUCE default Thu, 05 Mar 2015 10:20:31 GMT N/A RUNNING UNDEFINED ApplicationMaster Showing 1 to 1 of 1 entries FirstPrevious1NextLast
