I'd also check /etc/krb5.keytab, you may be running into this issue: https://bugzilla.redhat.com/show_bug.cgi?id=748528
(It also happens on Ubuntu) On Fri, Nov 21, 2014 at 12:54 AM, Abraham Elmahrek <[email protected]> wrote: > Two things I think might be going on: > > 1. kerberosServerName may need to be just the short name "postgre". > Not sure. > 2. jaasApplicationName should be set to "Krb5LoginModule". As per > > https://docs.oracle.com/javase/7/docs/jre/api/security/jaas/spec/com/sun/security/auth/module/Krb5LoginModule.html > and http://jdbc.postgresql.org/documentation/84/connect.html. > > -Abe > > On Thu, Nov 20, 2014 at 5:10 PM, David Lyle <[email protected]> wrote: > >> Thanks for hanging with me. Still same result. Postgres says: >> >> FATAL: GSSAPI authentication failed for user "dlyle" >> >> Here's the current state of the world: >> >> klist -f >> Ticket cache: FILE:/tmp/krb5cc_115000008 >> Default principal: [email protected] >> >> Valid starting Expires Service principal >> 11/20/14 20:07:20 11/21/14 20:07:19 krbtgt/[email protected] >> renew until 11/25/14 20:07:09, Flags: FPRIAT >> >> [dlyle@hwx1 ~]$ kinit -R >> [dlyle@hwx1 ~]$ sqoop list-databases --connect >> "jdbc:postgresql:// >> postgres.lyleco.com:5432?kerberosServerName=postgres/[email protected] >> " >> --verbose >> kinit: Generic preauthentication failure while getting initial credentials >> kinit: Generic preauthentication failure while getting initial credentials >> 14/11/20 20:07:59 INFO sqoop.Sqoop: Running Sqoop version: >> 1.4.4.2.1.7.0-784 >> 14/11/20 20:07:59 DEBUG tool.BaseSqoopTool: Enabled debug logging. >> 14/11/20 20:07:59 DEBUG sqoop.ConnFactory: Loaded manager factory: >> com.cloudera.sqoop.manager.DefaultManagerFactory >> 14/11/20 20:07:59 DEBUG sqoop.ConnFactory: Trying ManagerFactory: >> com.cloudera.sqoop.manager.DefaultManagerFactory >> 14/11/20 20:07:59 DEBUG manager.DefaultManagerFactory: Trying with >> scheme: jdbc:postgresql: >> 14/11/20 20:07:59 INFO manager.SqlManager: Using default fetchSize of 1000 >> 14/11/20 20:07:59 DEBUG sqoop.ConnFactory: Instantiated ConnManager >> org.apache.sqoop.manager.PostgresqlManager@5cb42b >> 14/11/20 20:07:59 DEBUG manager.SqlManager: No connection paramenters >> specified. Using regular API for making connection. >> 14/11/20 20:07:59 ERROR manager.CatalogQueryManager: Failed to list >> databases >> org.postgresql.util.PSQLException: GSS Authentication failed >> at org.postgresql.gss.MakeGSS.authenticate(MakeGSS.java:49) >> at >> org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:380) >> at >> org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:108) >> at >> org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) >> at >> org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) >> at >> org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) >> at >> org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) >> at >> org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) >> at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) >> at org.postgresql.Driver.makeConnection(Driver.java:393) >> at org.postgresql.Driver.connect(Driver.java:267) >> at java.sql.DriverManager.getConnection(DriverManager.java:571) >> at java.sql.DriverManager.getConnection(DriverManager.java:233) >> at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:824) >> at >> org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52) >> at >> org.apache.sqoop.manager.CatalogQueryManager.listDatabases(CatalogQueryManager.java:57) >> at org.apache.sqoop.tool.ListDatabasesTool.run(ListDatabasesTool.java:49) >> at org.apache.sqoop.Sqoop.run(Sqoop.java:147) >> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) >> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) >> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:222) >> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:231) >> at org.apache.sqoop.Sqoop.main(Sqoop.java:240) >> Caused by: javax.security.auth.login.LoginException: No LoginModules >> configured for pgjdbc >> at javax.security.auth.login.LoginContext.init(LoginContext.java:273) >> at javax.security.auth.login.LoginContext.<init>(LoginContext.java:418) >> at org.postgresql.gss.MakeGSS.authenticate(MakeGSS.java:42) >> ... 22 more >> 14/11/20 20:07:59 ERROR sqoop.Sqoop: Got exception running Sqoop: >> java.lang.RuntimeException: org.postgresql.util.PSQLException: GSS >> Authentication failed >> java.lang.RuntimeException: org.postgresql.util.PSQLException: GSS >> Authentication failed >> at >> org.apache.sqoop.manager.CatalogQueryManager.listDatabases(CatalogQueryManager.java:73) >> at org.apache.sqoop.tool.ListDatabasesTool.run(ListDatabasesTool.java:49) >> at org.apache.sqoop.Sqoop.run(Sqoop.java:147) >> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) >> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) >> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:222) >> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:231) >> at org.apache.sqoop.Sqoop.main(Sqoop.java:240) >> Caused by: org.postgresql.util.PSQLException: GSS Authentication failed >> at org.postgresql.gss.MakeGSS.authenticate(MakeGSS.java:49) >> at >> org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:380) >> at >> org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:108) >> at >> org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) >> at >> org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) >> at >> org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) >> at >> org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) >> at >> org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) >> at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) >> at org.postgresql.Driver.makeConnection(Driver.java:393) >> at org.postgresql.Driver.connect(Driver.java:267) >> at java.sql.DriverManager.getConnection(DriverManager.java:571) >> at java.sql.DriverManager.getConnection(DriverManager.java:233) >> at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:824) >> at >> org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52) >> at >> org.apache.sqoop.manager.CatalogQueryManager.listDatabases(CatalogQueryManager.java:57) >> ... 7 more >> Caused by: javax.security.auth.login.LoginException: No LoginModules >> configured for pgjdbc >> at javax.security.auth.login.LoginContext.init(LoginContext.java:273) >> at javax.security.auth.login.LoginContext.<init>(LoginContext.java:418) >> at org.postgresql.gss.MakeGSS.authenticate(MakeGSS.java:42) >> ... 22 more >> >> On Thu, Nov 20, 2014 at 7:24 PM, Abraham Elmahrek <[email protected]> >> wrote: >> > Hey there, >> > >> > My mistake. Apparently the it's a bit more like an HTTP string: >> > jdbc:postgresql:// >> postgres.lyleco.com:5432?kerberosServerName=postgres/[email protected] >> . >> > Also, are the JDBC jars in /var/lib/sqoop? Or are they part of Sqoop's >> class >> > path somehow? >> > >> > The "Generic preauthentication" errors are discomforting. Are your >> > credentials renewable? I believe hadoop-auth starts a renewer thread >> that >> > runs "kinit -R". That could be causing that error. >> > >> > -Abe >> > >> > On Thu, Nov 20, 2014 at 3:56 PM, David Lyle <[email protected]> >> wrote: >> >> >> >> Hi Abe, >> >> >> >> Thanks for the quick reply. Currently, the postgres instance in all >> >> that is kerberized. When I add the kerberosServerName I get: >> >> >> >> java.sql.SQLException: No suitable driver found for >> >> >> >> jdbc:postgresql://postgres.lyleco.com:5432 >> ;kerberosServerName=postgres/[email protected] >> >> >> >> or >> >> >> >> java.sql.SQLException: No suitable driver found for >> >> >> >> jdbc:postgresql://postgres.lyleco.com:5432 >> &kerberosServerName=postgres/[email protected] >> >> >> >> (I tried both). I'm also troubled by the repeated kinit: Generic >> >> preauthentication failure while getting initial credentials at the >> >> very top. I don't understand how sqoop is delegating, but the kinit is >> >> surprising. >> >> >> >> -D... >> >> >> >> >> >> On Thu, Nov 20, 2014 at 6:15 PM, Abraham Elmahrek <[email protected]> >> >> wrote: >> >> > This is a long shot... but I thought I'd say something. >> >> > >> >> > If you have Hadoop configured for Kerberos, then Sqoop should >> >> > automatically >> >> > use your kerberos credentials to run a MapReduce job. That MapReduce >> job >> >> > will run with its own Kerberos principal (normally yarn or mapred). >> This >> >> > means that Hadoop has access to postgresql as long as it requests a >> >> > session >> >> > from the KDC, assuming postgresql and Hadoop are part of the same >> realm. >> >> > My >> >> > understanding is the JDBC driver will handle this for you if you >> provide >> >> > the >> >> > postgresql service name via "kerberosServerName" in the JDBC >> connection >> >> > string: >> >> > >> >> > "jdbc:postgresql://postgres.lyleco.com:5432 >> ;kerberosServerName=postgres/<host>@<REALM>". >> >> > >> >> > Ref: http://jdbc.postgresql.org/documentation/84/connect.html >> >> > >> >> > -Abe >> >> > >> >> > On Thu, Nov 20, 2014 at 2:53 PM, David Lyle <[email protected]> >> >> > wrote: >> >> >> >> >> >> Hi, >> >> >> >> >> >> I'm trying to use Sqoop to connect to a kerberized Postgres >> instance. >> >> >> At this time, I can connect from psql, but my sqoop session gives me >> >> >> the log below. Any idea where I'm going wrong? >> >> >> >> >> >> Thanks! >> >> >> >> >> >> -David... >> >> >> >> >> >> >> >> >> sqoop list-databases -Djavax.security.auth.debug=true --connect >> >> >> "jdbc:postgresql://postgres.lyleco.com:5432" -verbose >> >> >> >> >> >> kinit: Generic preauthentication failure while getting initial >> >> >> credentials >> >> >> kinit: Generic preauthentication failure while getting initial >> >> >> credentials >> >> >> >> >> >> 14/11/20 17:26:37 INFO sqoop.Sqoop: Running Sqoop version: >> >> >> 1.4.4.2.1.7.0-784 >> >> >> 14/11/20 17:26:37 DEBUG tool.BaseSqoopTool: Enabled debug logging. >> >> >> 14/11/20 17:26:37 DEBUG sqoop.ConnFactory: Loaded manager factory: >> >> >> com.cloudera.sqoop.manager.DefaultManagerFactory >> >> >> 14/11/20 17:26:37 DEBUG sqoop.ConnFactory: Trying ManagerFactory: >> >> >> com.cloudera.sqoop.manager.DefaultManagerFactory >> >> >> 14/11/20 17:26:37 DEBUG manager.DefaultManagerFactory: Trying with >> >> >> scheme: jdbc:postgresql: >> >> >> 14/11/20 17:26:37 INFO manager.SqlManager: Using default fetchSize >> of >> >> >> 1000 >> >> >> 14/11/20 17:26:37 DEBUG sqoop.ConnFactory: Instantiated ConnManager >> >> >> org.apache.sqoop.manager.PostgresqlManager@1663108 >> >> >> 14/11/20 17:26:37 DEBUG manager.SqlManager: No connection >> paramenters >> >> >> specified. Using regular API for making connection. >> >> >> 14/11/20 17:26:37 ERROR manager.CatalogQueryManager: Failed to list >> >> >> databases >> >> >> org.postgresql.util.PSQLException: GSS Authentication failed >> >> >> at org.postgresql.gss.MakeGSS.authenticate(MakeGSS.java:49) >> >> >> at >> >> >> >> >> >> >> org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:380) >> >> >> at >> >> >> >> >> >> >> org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:108) >> >> >> at >> >> >> >> >> >> >> org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) >> >> >> at >> >> >> >> >> >> >> org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) >> >> >> at >> >> >> >> >> >> >> org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) >> >> >> at >> >> >> >> >> >> >> org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) >> >> >> at >> >> >> >> >> >> >> org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) >> >> >> at >> org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) >> >> >> at org.postgresql.Driver.makeConnection(Driver.java:393) >> >> >> at org.postgresql.Driver.connect(Driver.java:267) >> >> >> at java.sql.DriverManager.getConnection(DriverManager.java:571) >> >> >> at java.sql.DriverManager.getConnection(DriverManager.java:233) >> >> >> at >> >> >> >> org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:824) >> >> >> at >> >> >> >> >> >> >> org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52) >> >> >> at >> >> >> >> >> >> >> org.apache.sqoop.manager.CatalogQueryManager.listDatabases(CatalogQueryManager.java:57) >> >> >> at >> >> >> >> org.apache.sqoop.tool.ListDatabasesTool.run(ListDatabasesTool.java:49) >> >> >> at org.apache.sqoop.Sqoop.run(Sqoop.java:147) >> >> >> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) >> >> >> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) >> >> >> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:222) >> >> >> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:231) >> >> >> at org.apache.sqoop.Sqoop.main(Sqoop.java:240) >> >> >> Caused by: javax.security.auth.login.LoginException: No LoginModules >> >> >> configured for pgjdbc >> >> >> at >> javax.security.auth.login.LoginContext.init(LoginContext.java:273) >> >> >> at >> javax.security.auth.login.LoginContext.<init>(LoginContext.java:418) >> >> >> at org.postgresql.gss.MakeGSS.authenticate(MakeGSS.java:42) >> >> >> ... 22 more >> >> >> 14/11/20 17:26:37 ERROR sqoop.Sqoop: Got exception running Sqoop: >> >> >> java.lang.RuntimeException: org.postgresql.util.PSQLException: GSS >> >> >> Authentication failed >> >> >> java.lang.RuntimeException: org.postgresql.util.PSQLException: GSS >> >> >> Authentication failed >> >> >> at >> >> >> >> >> >> >> org.apache.sqoop.manager.CatalogQueryManager.listDatabases(CatalogQueryManager.java:73) >> >> >> at >> >> >> >> org.apache.sqoop.tool.ListDatabasesTool.run(ListDatabasesTool.java:49) >> >> >> at org.apache.sqoop.Sqoop.run(Sqoop.java:147) >> >> >> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) >> >> >> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) >> >> >> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:222) >> >> >> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:231) >> >> >> at org.apache.sqoop.Sqoop.main(Sqoop.java:240) >> >> >> Caused by: org.postgresql.util.PSQLException: GSS Authentication >> failed >> >> >> at org.postgresql.gss.MakeGSS.authenticate(MakeGSS.java:49) >> >> >> at >> >> >> >> >> >> >> org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:380) >> >> >> at >> >> >> >> >> >> >> org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:108) >> >> >> at >> >> >> >> >> >> >> org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) >> >> >> at >> >> >> >> >> >> >> org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) >> >> >> at >> >> >> >> >> >> >> org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) >> >> >> at >> >> >> >> >> >> >> org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) >> >> >> at >> >> >> >> >> >> >> org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) >> >> >> at >> org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) >> >> >> at org.postgresql.Driver.makeConnection(Driver.java:393) >> >> >> at org.postgresql.Driver.connect(Driver.java:267) >> >> >> at java.sql.DriverManager.getConnection(DriverManager.java:571) >> >> >> at java.sql.DriverManager.getConnection(DriverManager.java:233) >> >> >> at >> >> >> >> org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:824) >> >> >> at >> >> >> >> >> >> >> org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52) >> >> >> at >> >> >> >> >> >> >> org.apache.sqoop.manager.CatalogQueryManager.listDatabases(CatalogQueryManager.java:57) >> >> >> ... 7 more >> >> >> Caused by: javax.security.auth.login.LoginException: No LoginModules >> >> >> configured for pgjdbc >> >> >> at >> javax.security.auth.login.LoginContext.init(LoginContext.java:273) >> >> >> at >> javax.security.auth.login.LoginContext.<init>(LoginContext.java:418) >> >> >> at org.postgresql.gss.MakeGSS.authenticate(MakeGSS.java:42) >> >> >> ... 22 more >> >> > >> >> > >> > >> > >> > >
