Thanks for hanging with me. Still same result. Postgres says: FATAL: GSSAPI authentication failed for user "dlyle"
Here's the current state of the world: klist -f Ticket cache: FILE:/tmp/krb5cc_115000008 Default principal: dl...@lyleco.com Valid starting Expires Service principal 11/20/14 20:07:20 11/21/14 20:07:19 krbtgt/lyleco....@lyleco.com renew until 11/25/14 20:07:09, Flags: FPRIAT [dlyle@hwx1 ~]$ kinit -R [dlyle@hwx1 ~]$ sqoop list-databases --connect "jdbc:postgresql://postgres.lyleco.com:5432?kerberosServerName=postgres/postgres.lylecom....@lyleco.com" --verbose kinit: Generic preauthentication failure while getting initial credentials kinit: Generic preauthentication failure while getting initial credentials 14/11/20 20:07:59 INFO sqoop.Sqoop: Running Sqoop version: 1.4.4.2.1.7.0-784 14/11/20 20:07:59 DEBUG tool.BaseSqoopTool: Enabled debug logging. 14/11/20 20:07:59 DEBUG sqoop.ConnFactory: Loaded manager factory: com.cloudera.sqoop.manager.DefaultManagerFactory 14/11/20 20:07:59 DEBUG sqoop.ConnFactory: Trying ManagerFactory: com.cloudera.sqoop.manager.DefaultManagerFactory 14/11/20 20:07:59 DEBUG manager.DefaultManagerFactory: Trying with scheme: jdbc:postgresql: 14/11/20 20:07:59 INFO manager.SqlManager: Using default fetchSize of 1000 14/11/20 20:07:59 DEBUG sqoop.ConnFactory: Instantiated ConnManager org.apache.sqoop.manager.PostgresqlManager@5cb42b 14/11/20 20:07:59 DEBUG manager.SqlManager: No connection paramenters specified. Using regular API for making connection. 14/11/20 20:07:59 ERROR manager.CatalogQueryManager: Failed to list databases org.postgresql.util.PSQLException: GSS Authentication failed at org.postgresql.gss.MakeGSS.authenticate(MakeGSS.java:49) at org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:380) at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:108) at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) at org.postgresql.Driver.makeConnection(Driver.java:393) at org.postgresql.Driver.connect(Driver.java:267) at java.sql.DriverManager.getConnection(DriverManager.java:571) at java.sql.DriverManager.getConnection(DriverManager.java:233) at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:824) at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52) at org.apache.sqoop.manager.CatalogQueryManager.listDatabases(CatalogQueryManager.java:57) at org.apache.sqoop.tool.ListDatabasesTool.run(ListDatabasesTool.java:49) at org.apache.sqoop.Sqoop.run(Sqoop.java:147) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:222) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:231) at org.apache.sqoop.Sqoop.main(Sqoop.java:240) Caused by: javax.security.auth.login.LoginException: No LoginModules configured for pgjdbc at javax.security.auth.login.LoginContext.init(LoginContext.java:273) at javax.security.auth.login.LoginContext.<init>(LoginContext.java:418) at org.postgresql.gss.MakeGSS.authenticate(MakeGSS.java:42) ... 22 more 14/11/20 20:07:59 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.RuntimeException: org.postgresql.util.PSQLException: GSS Authentication failed java.lang.RuntimeException: org.postgresql.util.PSQLException: GSS Authentication failed at org.apache.sqoop.manager.CatalogQueryManager.listDatabases(CatalogQueryManager.java:73) at org.apache.sqoop.tool.ListDatabasesTool.run(ListDatabasesTool.java:49) at org.apache.sqoop.Sqoop.run(Sqoop.java:147) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:222) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:231) at org.apache.sqoop.Sqoop.main(Sqoop.java:240) Caused by: org.postgresql.util.PSQLException: GSS Authentication failed at org.postgresql.gss.MakeGSS.authenticate(MakeGSS.java:49) at org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:380) at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:108) at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) at org.postgresql.Driver.makeConnection(Driver.java:393) at org.postgresql.Driver.connect(Driver.java:267) at java.sql.DriverManager.getConnection(DriverManager.java:571) at java.sql.DriverManager.getConnection(DriverManager.java:233) at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:824) at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52) at org.apache.sqoop.manager.CatalogQueryManager.listDatabases(CatalogQueryManager.java:57) ... 7 more Caused by: javax.security.auth.login.LoginException: No LoginModules configured for pgjdbc at javax.security.auth.login.LoginContext.init(LoginContext.java:273) at javax.security.auth.login.LoginContext.<init>(LoginContext.java:418) at org.postgresql.gss.MakeGSS.authenticate(MakeGSS.java:42) ... 22 more On Thu, Nov 20, 2014 at 7:24 PM, Abraham Elmahrek <a...@cloudera.com> wrote: > Hey there, > > My mistake. Apparently the it's a bit more like an HTTP string: > jdbc:postgresql://postgres.lyleco.com:5432?kerberosServerName=postgres/postgres.lyleco....@lyleco.com. > Also, are the JDBC jars in /var/lib/sqoop? Or are they part of Sqoop's class > path somehow? > > The "Generic preauthentication" errors are discomforting. Are your > credentials renewable? I believe hadoop-auth starts a renewer thread that > runs "kinit -R". That could be causing that error. > > -Abe > > On Thu, Nov 20, 2014 at 3:56 PM, David Lyle <dlyle65...@gmail.com> wrote: >> >> Hi Abe, >> >> Thanks for the quick reply. Currently, the postgres instance in all >> that is kerberized. When I add the kerberosServerName I get: >> >> java.sql.SQLException: No suitable driver found for >> >> jdbc:postgresql://postgres.lyleco.com:5432;kerberosServerName=postgres/postgres.lyleco....@lyleco.com >> >> or >> >> java.sql.SQLException: No suitable driver found for >> >> jdbc:postgresql://postgres.lyleco.com:5432&kerberosServerName=postgres/postgres.lyleco....@lyleco.com >> >> (I tried both). I'm also troubled by the repeated kinit: Generic >> preauthentication failure while getting initial credentials at the >> very top. I don't understand how sqoop is delegating, but the kinit is >> surprising. >> >> -D... >> >> >> On Thu, Nov 20, 2014 at 6:15 PM, Abraham Elmahrek <a...@cloudera.com> >> wrote: >> > This is a long shot... but I thought I'd say something. >> > >> > If you have Hadoop configured for Kerberos, then Sqoop should >> > automatically >> > use your kerberos credentials to run a MapReduce job. That MapReduce job >> > will run with its own Kerberos principal (normally yarn or mapred). This >> > means that Hadoop has access to postgresql as long as it requests a >> > session >> > from the KDC, assuming postgresql and Hadoop are part of the same realm. >> > My >> > understanding is the JDBC driver will handle this for you if you provide >> > the >> > postgresql service name via "kerberosServerName" in the JDBC connection >> > string: >> > >> > "jdbc:postgresql://postgres.lyleco.com:5432;kerberosServerName=postgres/<host>@<REALM>". >> > >> > Ref: http://jdbc.postgresql.org/documentation/84/connect.html >> > >> > -Abe >> > >> > On Thu, Nov 20, 2014 at 2:53 PM, David Lyle <dlyle65...@gmail.com> >> > wrote: >> >> >> >> Hi, >> >> >> >> I'm trying to use Sqoop to connect to a kerberized Postgres instance. >> >> At this time, I can connect from psql, but my sqoop session gives me >> >> the log below. Any idea where I'm going wrong? >> >> >> >> Thanks! >> >> >> >> -David... >> >> >> >> >> >> sqoop list-databases -Djavax.security.auth.debug=true --connect >> >> "jdbc:postgresql://postgres.lyleco.com:5432" -verbose >> >> >> >> kinit: Generic preauthentication failure while getting initial >> >> credentials >> >> kinit: Generic preauthentication failure while getting initial >> >> credentials >> >> >> >> 14/11/20 17:26:37 INFO sqoop.Sqoop: Running Sqoop version: >> >> 1.4.4.2.1.7.0-784 >> >> 14/11/20 17:26:37 DEBUG tool.BaseSqoopTool: Enabled debug logging. >> >> 14/11/20 17:26:37 DEBUG sqoop.ConnFactory: Loaded manager factory: >> >> com.cloudera.sqoop.manager.DefaultManagerFactory >> >> 14/11/20 17:26:37 DEBUG sqoop.ConnFactory: Trying ManagerFactory: >> >> com.cloudera.sqoop.manager.DefaultManagerFactory >> >> 14/11/20 17:26:37 DEBUG manager.DefaultManagerFactory: Trying with >> >> scheme: jdbc:postgresql: >> >> 14/11/20 17:26:37 INFO manager.SqlManager: Using default fetchSize of >> >> 1000 >> >> 14/11/20 17:26:37 DEBUG sqoop.ConnFactory: Instantiated ConnManager >> >> org.apache.sqoop.manager.PostgresqlManager@1663108 >> >> 14/11/20 17:26:37 DEBUG manager.SqlManager: No connection paramenters >> >> specified. Using regular API for making connection. >> >> 14/11/20 17:26:37 ERROR manager.CatalogQueryManager: Failed to list >> >> databases >> >> org.postgresql.util.PSQLException: GSS Authentication failed >> >> at org.postgresql.gss.MakeGSS.authenticate(MakeGSS.java:49) >> >> at >> >> >> >> org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:380) >> >> at >> >> >> >> org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:108) >> >> at >> >> >> >> org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) >> >> at >> >> >> >> org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) >> >> at >> >> >> >> org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) >> >> at >> >> >> >> org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) >> >> at >> >> >> >> org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) >> >> at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) >> >> at org.postgresql.Driver.makeConnection(Driver.java:393) >> >> at org.postgresql.Driver.connect(Driver.java:267) >> >> at java.sql.DriverManager.getConnection(DriverManager.java:571) >> >> at java.sql.DriverManager.getConnection(DriverManager.java:233) >> >> at >> >> org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:824) >> >> at >> >> >> >> org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52) >> >> at >> >> >> >> org.apache.sqoop.manager.CatalogQueryManager.listDatabases(CatalogQueryManager.java:57) >> >> at >> >> org.apache.sqoop.tool.ListDatabasesTool.run(ListDatabasesTool.java:49) >> >> at org.apache.sqoop.Sqoop.run(Sqoop.java:147) >> >> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) >> >> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) >> >> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:222) >> >> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:231) >> >> at org.apache.sqoop.Sqoop.main(Sqoop.java:240) >> >> Caused by: javax.security.auth.login.LoginException: No LoginModules >> >> configured for pgjdbc >> >> at javax.security.auth.login.LoginContext.init(LoginContext.java:273) >> >> at javax.security.auth.login.LoginContext.<init>(LoginContext.java:418) >> >> at org.postgresql.gss.MakeGSS.authenticate(MakeGSS.java:42) >> >> ... 22 more >> >> 14/11/20 17:26:37 ERROR sqoop.Sqoop: Got exception running Sqoop: >> >> java.lang.RuntimeException: org.postgresql.util.PSQLException: GSS >> >> Authentication failed >> >> java.lang.RuntimeException: org.postgresql.util.PSQLException: GSS >> >> Authentication failed >> >> at >> >> >> >> org.apache.sqoop.manager.CatalogQueryManager.listDatabases(CatalogQueryManager.java:73) >> >> at >> >> org.apache.sqoop.tool.ListDatabasesTool.run(ListDatabasesTool.java:49) >> >> at org.apache.sqoop.Sqoop.run(Sqoop.java:147) >> >> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) >> >> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) >> >> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:222) >> >> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:231) >> >> at org.apache.sqoop.Sqoop.main(Sqoop.java:240) >> >> Caused by: org.postgresql.util.PSQLException: GSS Authentication failed >> >> at org.postgresql.gss.MakeGSS.authenticate(MakeGSS.java:49) >> >> at >> >> >> >> org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:380) >> >> at >> >> >> >> org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:108) >> >> at >> >> >> >> org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) >> >> at >> >> >> >> org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) >> >> at >> >> >> >> org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) >> >> at >> >> >> >> org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) >> >> at >> >> >> >> org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) >> >> at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) >> >> at org.postgresql.Driver.makeConnection(Driver.java:393) >> >> at org.postgresql.Driver.connect(Driver.java:267) >> >> at java.sql.DriverManager.getConnection(DriverManager.java:571) >> >> at java.sql.DriverManager.getConnection(DriverManager.java:233) >> >> at >> >> org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:824) >> >> at >> >> >> >> org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52) >> >> at >> >> >> >> org.apache.sqoop.manager.CatalogQueryManager.listDatabases(CatalogQueryManager.java:57) >> >> ... 7 more >> >> Caused by: javax.security.auth.login.LoginException: No LoginModules >> >> configured for pgjdbc >> >> at javax.security.auth.login.LoginContext.init(LoginContext.java:273) >> >> at javax.security.auth.login.LoginContext.<init>(LoginContext.java:418) >> >> at org.postgresql.gss.MakeGSS.authenticate(MakeGSS.java:42) >> >> ... 22 more >> > >> > > >
