Hello, Asokan,
Thanks a lot for your reply. I can't test against a mainframe right now.
It is not possible to "emulate" a mainframe with a general ftp server?
Best regards,
El 03/02/15 a las 17:38, Asokan, M escribió:
Hi Dima,
First, please make sure that you are indeed connecting to a
mainframe host. Secondly, you need to specify the fully qualified
name of the dataset like:
user1234.empleados
Hope it helps.
*From:*Dima Fadeyev [mailto:[email protected]]
*Sent:* Tuesday, February 03, 2015 9:39 AM
*To:* [email protected]
*Subject:* "sqoop import-mainframe" fails with " No sequential
datasets retrieved"
Hello, everone,
I'm trying to test "import-mainframe" sqoop command added with this
patch: https://issues.apache.org/jira/browse/SQOOP-1272
<https://issues.apache.org/jira/browse/SQOOP-1272>. It is not yet
available in any sqoop releases, so I've compiled sqoop from trunk.
As far as I understand, I could test import-mainframe using a normal
ftp server. So, I've installed one, created a directory 'empleados'
and copied some text files in it. I was hoping sqoop would download
the files from my ftp server.
Here is the command I launch and the result:
sqoop import-mainframe --connect 192.168.10.210:21 --dataset empleados
--username user1234 --password pass1234 --verbose
5/02/03 15:30:10 INFO mainframe.MainframeDatasetInputFormat: Datasets
to transfer from: empleados
15/02/03 15:30:10 DEBUG db.DBConfiguration: Fetching password from job
credentials store
220 (vsFTPd 2.2.2)
15/02/03 15:30:10 INFO util.MainframeFTPClientUtils: Connected to
192.168.10.210 on 21
USER *******
331 Please specify the password.
PASS *******
230 Login successful.
TYPE A
200 Switching to ASCII mode.
CWD 'empleados'
250 Directory successfully changed.
PASV
227 Entering Passive Mode (192,168,10,210,251,170).
LIST
150 Here comes the directory listing.
226 Directory send OK.
NOOP
200 NOOP ok.
QUIT
221 Goodbye.
15/02/03 15:30:10 INFO mapreduce.JobSubmitter: Cleaning up the staging
area /user/root/.staging/job_1422968174081_0008
15/02/03 15:30:10 WARN security.UserGroupInformation:
PriviledgedActionException as:root (auth:SIMPLE)
cause:java.io.IOException: No sequential datasets retrieved from empleados
15/02/03 15:30:10 DEBUG util.ClassLoaderStack: Restoring classloader:
sun.misc.Launcher$AppClassLoader@530f243b
<mailto:sun.misc.Launcher$AppClassLoader@530f243b>
15/02/03 15:30:10 ERROR tool.ImportTool: Encountered IOException
running import job: java.io.IOException: No sequential datasets
retrieved from empleados
at
org.apache.sqoop.mapreduce.mainframe.MainframeDatasetInputFormat.getSplits(MainframeDatasetInputFormat.java:65)
at
org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:493)
at
org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:510)
at
org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:394)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1295)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1292)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1554)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1292)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1313)
at
org.apache.sqoop.mapreduce.ImportJobBase.doSubmitJob(ImportJobBase.java:186)
at org.apache.sqoop.mapreduce.ImportJobBase.runJob(ImportJobBase.java:159)
at
org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:247)
at
org.apache.sqoop.manager.MainframeManager.importTable(MainframeManager.java:97)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:497)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
I could check with tcpdump that my ftp sends a list of files from
directory 'empleados' to sqoop. However sqoop somehow sees an empty list.
If anyone has sqoop import-mainframe working, please help me resolve this.
Thanks in advance
--
AVISO CONFIDENCIAL
Este correo y la información contenida o adjunta al mismo es privada y
confidencial y va dirigida exclusivamente a su destinatario. Pragsis
informa a quien pueda haber recibido este correo por error que
contiene información confidencial cuyo uso, copia, reproducción o
distribución está expresamente prohibida. Si no es Vd. el destinatario
del mismo y recibe este correo por error, le rogamos lo ponga en
conocimiento del emisor y proceda a su eliminación sin copiarlo,
imprimirlo o utilizarlo de ningún modo.
CONFIDENTIALITY WARNING.
This message and the information contained in or attached to it are
private and confidential and intended exclusively for the addressee.
Pragsis informs to whom it may receive it in error that it contains
privileged information and its use, copy, reproduction or distribution
is prohibited. If you are not an intended recipient of this E-mail,
please notify the sender, delete it and do not read, a ct upon, print,
disclose, copy, retain or redistribute any portion of this E-mail.
------------------------------------------------------------------------
ATTENTION: -----
The information contained in this message (including any files
transmitted with this message) may contain proprietary, trade secret
or other confidential and/or legally privileged information. Any
pricing information contained in this message or in any files
transmitted with this message is always confidential and cannot be
shared with any third parties without prior written approval from
Syncsort. This message is intended to be read only by the individual
or entity to whom it is addressed or by their designee. If the reader
of this message is not the intended recipient, you are on notice that
any use, disclosure, copying or distribution of this message, in any
form, is strictly prohibited. If you have received this message in
error, please immediately notify the sender and/or Syncsort and
destroy all copies of this message in your possession, custody or control.
AVISO CONFIDENCIAL\nEste correo y la información contenida o adjunta al mismo
es privada y confidencial y va dirigida exclusivamente a su destinatario.
Pragsis informa a quien pueda haber recibido este correo por error que contiene
información confidencial cuyo uso, copia, reproducción o distribución está
expresamente prohibida. Si no es Vd. el destinatario del mismo y recibe este
correo por error, le rogamos lo ponga en conocimiento del emisor y proceda a su
eliminación sin copiarlo, imprimirlo o utilizarlo de ningún
modo.\nCONFIDENTIALITY WARNING.\nThis message and the information contained in
or attached to it are private and confidential and intended exclusively for the
addressee. Pragsis informs to whom it may receive it in error that it contains
privileged information and its use, copy, reproduction or distribution is
prohibited. If you are not an intended recipient of this E-mail, please notify
the sender, delete it and do not read, act upon,
print, disclose, copy, retain or redistribute any portion of this E-mail.
