Two things I think might be going on: 1. kerberosServerName may need to be just the short name "postgre". Not sure. 2. jaasApplicationName should be set to "Krb5LoginModule". As per https://docs.oracle.com/javase/7/docs/jre/api/security/jaas/spec/com/sun/security/auth/module/Krb5LoginModule.html and http://jdbc.postgresql.org/documentation/84/connect.html.
-Abe On Thu, Nov 20, 2014 at 5:10 PM, David Lyle <[email protected]> wrote: > Thanks for hanging with me. Still same result. Postgres says: > > FATAL: GSSAPI authentication failed for user "dlyle" > > Here's the current state of the world: > > klist -f > Ticket cache: FILE:/tmp/krb5cc_115000008 > Default principal: [email protected] > > Valid starting Expires Service principal > 11/20/14 20:07:20 11/21/14 20:07:19 krbtgt/[email protected] > renew until 11/25/14 20:07:09, Flags: FPRIAT > > [dlyle@hwx1 ~]$ kinit -R > [dlyle@hwx1 ~]$ sqoop list-databases --connect > "jdbc:postgresql:// > postgres.lyleco.com:5432?kerberosServerName=postgres/[email protected] > " > --verbose > kinit: Generic preauthentication failure while getting initial credentials > kinit: Generic preauthentication failure while getting initial credentials > 14/11/20 20:07:59 INFO sqoop.Sqoop: Running Sqoop version: > 1.4.4.2.1.7.0-784 > 14/11/20 20:07:59 DEBUG tool.BaseSqoopTool: Enabled debug logging. > 14/11/20 20:07:59 DEBUG sqoop.ConnFactory: Loaded manager factory: > com.cloudera.sqoop.manager.DefaultManagerFactory > 14/11/20 20:07:59 DEBUG sqoop.ConnFactory: Trying ManagerFactory: > com.cloudera.sqoop.manager.DefaultManagerFactory > 14/11/20 20:07:59 DEBUG manager.DefaultManagerFactory: Trying with > scheme: jdbc:postgresql: > 14/11/20 20:07:59 INFO manager.SqlManager: Using default fetchSize of 1000 > 14/11/20 20:07:59 DEBUG sqoop.ConnFactory: Instantiated ConnManager > org.apache.sqoop.manager.PostgresqlManager@5cb42b > 14/11/20 20:07:59 DEBUG manager.SqlManager: No connection paramenters > specified. Using regular API for making connection. > 14/11/20 20:07:59 ERROR manager.CatalogQueryManager: Failed to list > databases > org.postgresql.util.PSQLException: GSS Authentication failed > at org.postgresql.gss.MakeGSS.authenticate(MakeGSS.java:49) > at > org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:380) > at > org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:108) > at > org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at > org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at > org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at > org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at > org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at java.sql.DriverManager.getConnection(DriverManager.java:571) > at java.sql.DriverManager.getConnection(DriverManager.java:233) > at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:824) > at > org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52) > at > org.apache.sqoop.manager.CatalogQueryManager.listDatabases(CatalogQueryManager.java:57) > at org.apache.sqoop.tool.ListDatabasesTool.run(ListDatabasesTool.java:49) > at org.apache.sqoop.Sqoop.run(Sqoop.java:147) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) > at org.apache.sqoop.Sqoop.runTool(Sqoop.java:222) > at org.apache.sqoop.Sqoop.runTool(Sqoop.java:231) > at org.apache.sqoop.Sqoop.main(Sqoop.java:240) > Caused by: javax.security.auth.login.LoginException: No LoginModules > configured for pgjdbc > at javax.security.auth.login.LoginContext.init(LoginContext.java:273) > at javax.security.auth.login.LoginContext.<init>(LoginContext.java:418) > at org.postgresql.gss.MakeGSS.authenticate(MakeGSS.java:42) > ... 22 more > 14/11/20 20:07:59 ERROR sqoop.Sqoop: Got exception running Sqoop: > java.lang.RuntimeException: org.postgresql.util.PSQLException: GSS > Authentication failed > java.lang.RuntimeException: org.postgresql.util.PSQLException: GSS > Authentication failed > at > org.apache.sqoop.manager.CatalogQueryManager.listDatabases(CatalogQueryManager.java:73) > at org.apache.sqoop.tool.ListDatabasesTool.run(ListDatabasesTool.java:49) > at org.apache.sqoop.Sqoop.run(Sqoop.java:147) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) > at org.apache.sqoop.Sqoop.runTool(Sqoop.java:222) > at org.apache.sqoop.Sqoop.runTool(Sqoop.java:231) > at org.apache.sqoop.Sqoop.main(Sqoop.java:240) > Caused by: org.postgresql.util.PSQLException: GSS Authentication failed > at org.postgresql.gss.MakeGSS.authenticate(MakeGSS.java:49) > at > org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:380) > at > org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:108) > at > org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at > org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at > org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at > org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at > org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at java.sql.DriverManager.getConnection(DriverManager.java:571) > at java.sql.DriverManager.getConnection(DriverManager.java:233) > at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:824) > at > org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52) > at > org.apache.sqoop.manager.CatalogQueryManager.listDatabases(CatalogQueryManager.java:57) > ... 7 more > Caused by: javax.security.auth.login.LoginException: No LoginModules > configured for pgjdbc > at javax.security.auth.login.LoginContext.init(LoginContext.java:273) > at javax.security.auth.login.LoginContext.<init>(LoginContext.java:418) > at org.postgresql.gss.MakeGSS.authenticate(MakeGSS.java:42) > ... 22 more > > On Thu, Nov 20, 2014 at 7:24 PM, Abraham Elmahrek <[email protected]> > wrote: > > Hey there, > > > > My mistake. Apparently the it's a bit more like an HTTP string: > > jdbc:postgresql:// > postgres.lyleco.com:5432?kerberosServerName=postgres/[email protected] > . > > Also, are the JDBC jars in /var/lib/sqoop? Or are they part of Sqoop's > class > > path somehow? > > > > The "Generic preauthentication" errors are discomforting. Are your > > credentials renewable? I believe hadoop-auth starts a renewer thread that > > runs "kinit -R". That could be causing that error. > > > > -Abe > > > > On Thu, Nov 20, 2014 at 3:56 PM, David Lyle <[email protected]> > wrote: > >> > >> Hi Abe, > >> > >> Thanks for the quick reply. Currently, the postgres instance in all > >> that is kerberized. When I add the kerberosServerName I get: > >> > >> java.sql.SQLException: No suitable driver found for > >> > >> jdbc:postgresql://postgres.lyleco.com:5432;kerberosServerName=postgres/ > [email protected] > >> > >> or > >> > >> java.sql.SQLException: No suitable driver found for > >> > >> jdbc:postgresql://postgres.lyleco.com:5432&kerberosServerName=postgres/ > [email protected] > >> > >> (I tried both). I'm also troubled by the repeated kinit: Generic > >> preauthentication failure while getting initial credentials at the > >> very top. I don't understand how sqoop is delegating, but the kinit is > >> surprising. > >> > >> -D... > >> > >> > >> On Thu, Nov 20, 2014 at 6:15 PM, Abraham Elmahrek <[email protected]> > >> wrote: > >> > This is a long shot... but I thought I'd say something. > >> > > >> > If you have Hadoop configured for Kerberos, then Sqoop should > >> > automatically > >> > use your kerberos credentials to run a MapReduce job. That MapReduce > job > >> > will run with its own Kerberos principal (normally yarn or mapred). > This > >> > means that Hadoop has access to postgresql as long as it requests a > >> > session > >> > from the KDC, assuming postgresql and Hadoop are part of the same > realm. > >> > My > >> > understanding is the JDBC driver will handle this for you if you > provide > >> > the > >> > postgresql service name via "kerberosServerName" in the JDBC > connection > >> > string: > >> > > >> > "jdbc:postgresql://postgres.lyleco.com:5432 > ;kerberosServerName=postgres/<host>@<REALM>". > >> > > >> > Ref: http://jdbc.postgresql.org/documentation/84/connect.html > >> > > >> > -Abe > >> > > >> > On Thu, Nov 20, 2014 at 2:53 PM, David Lyle <[email protected]> > >> > wrote: > >> >> > >> >> Hi, > >> >> > >> >> I'm trying to use Sqoop to connect to a kerberized Postgres instance. > >> >> At this time, I can connect from psql, but my sqoop session gives me > >> >> the log below. Any idea where I'm going wrong? > >> >> > >> >> Thanks! > >> >> > >> >> -David... > >> >> > >> >> > >> >> sqoop list-databases -Djavax.security.auth.debug=true --connect > >> >> "jdbc:postgresql://postgres.lyleco.com:5432" -verbose > >> >> > >> >> kinit: Generic preauthentication failure while getting initial > >> >> credentials > >> >> kinit: Generic preauthentication failure while getting initial > >> >> credentials > >> >> > >> >> 14/11/20 17:26:37 INFO sqoop.Sqoop: Running Sqoop version: > >> >> 1.4.4.2.1.7.0-784 > >> >> 14/11/20 17:26:37 DEBUG tool.BaseSqoopTool: Enabled debug logging. > >> >> 14/11/20 17:26:37 DEBUG sqoop.ConnFactory: Loaded manager factory: > >> >> com.cloudera.sqoop.manager.DefaultManagerFactory > >> >> 14/11/20 17:26:37 DEBUG sqoop.ConnFactory: Trying ManagerFactory: > >> >> com.cloudera.sqoop.manager.DefaultManagerFactory > >> >> 14/11/20 17:26:37 DEBUG manager.DefaultManagerFactory: Trying with > >> >> scheme: jdbc:postgresql: > >> >> 14/11/20 17:26:37 INFO manager.SqlManager: Using default fetchSize of > >> >> 1000 > >> >> 14/11/20 17:26:37 DEBUG sqoop.ConnFactory: Instantiated ConnManager > >> >> org.apache.sqoop.manager.PostgresqlManager@1663108 > >> >> 14/11/20 17:26:37 DEBUG manager.SqlManager: No connection paramenters > >> >> specified. Using regular API for making connection. > >> >> 14/11/20 17:26:37 ERROR manager.CatalogQueryManager: Failed to list > >> >> databases > >> >> org.postgresql.util.PSQLException: GSS Authentication failed > >> >> at org.postgresql.gss.MakeGSS.authenticate(MakeGSS.java:49) > >> >> at > >> >> > >> >> > org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:380) > >> >> at > >> >> > >> >> > org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:108) > >> >> at > >> >> > >> >> > org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > >> >> at > >> >> > >> >> > org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > >> >> at > >> >> > >> >> > org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > >> >> at > >> >> > >> >> > org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > >> >> at > >> >> > >> >> > org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > >> >> at > org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > >> >> at org.postgresql.Driver.makeConnection(Driver.java:393) > >> >> at org.postgresql.Driver.connect(Driver.java:267) > >> >> at java.sql.DriverManager.getConnection(DriverManager.java:571) > >> >> at java.sql.DriverManager.getConnection(DriverManager.java:233) > >> >> at > >> >> > org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:824) > >> >> at > >> >> > >> >> > org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52) > >> >> at > >> >> > >> >> > org.apache.sqoop.manager.CatalogQueryManager.listDatabases(CatalogQueryManager.java:57) > >> >> at > >> >> > org.apache.sqoop.tool.ListDatabasesTool.run(ListDatabasesTool.java:49) > >> >> at org.apache.sqoop.Sqoop.run(Sqoop.java:147) > >> >> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > >> >> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) > >> >> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:222) > >> >> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:231) > >> >> at org.apache.sqoop.Sqoop.main(Sqoop.java:240) > >> >> Caused by: javax.security.auth.login.LoginException: No LoginModules > >> >> configured for pgjdbc > >> >> at javax.security.auth.login.LoginContext.init(LoginContext.java:273) > >> >> at > javax.security.auth.login.LoginContext.<init>(LoginContext.java:418) > >> >> at org.postgresql.gss.MakeGSS.authenticate(MakeGSS.java:42) > >> >> ... 22 more > >> >> 14/11/20 17:26:37 ERROR sqoop.Sqoop: Got exception running Sqoop: > >> >> java.lang.RuntimeException: org.postgresql.util.PSQLException: GSS > >> >> Authentication failed > >> >> java.lang.RuntimeException: org.postgresql.util.PSQLException: GSS > >> >> Authentication failed > >> >> at > >> >> > >> >> > org.apache.sqoop.manager.CatalogQueryManager.listDatabases(CatalogQueryManager.java:73) > >> >> at > >> >> > org.apache.sqoop.tool.ListDatabasesTool.run(ListDatabasesTool.java:49) > >> >> at org.apache.sqoop.Sqoop.run(Sqoop.java:147) > >> >> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > >> >> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) > >> >> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:222) > >> >> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:231) > >> >> at org.apache.sqoop.Sqoop.main(Sqoop.java:240) > >> >> Caused by: org.postgresql.util.PSQLException: GSS Authentication > failed > >> >> at org.postgresql.gss.MakeGSS.authenticate(MakeGSS.java:49) > >> >> at > >> >> > >> >> > org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:380) > >> >> at > >> >> > >> >> > org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:108) > >> >> at > >> >> > >> >> > org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > >> >> at > >> >> > >> >> > org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > >> >> at > >> >> > >> >> > org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > >> >> at > >> >> > >> >> > org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > >> >> at > >> >> > >> >> > org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > >> >> at > org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > >> >> at org.postgresql.Driver.makeConnection(Driver.java:393) > >> >> at org.postgresql.Driver.connect(Driver.java:267) > >> >> at java.sql.DriverManager.getConnection(DriverManager.java:571) > >> >> at java.sql.DriverManager.getConnection(DriverManager.java:233) > >> >> at > >> >> > org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:824) > >> >> at > >> >> > >> >> > org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52) > >> >> at > >> >> > >> >> > org.apache.sqoop.manager.CatalogQueryManager.listDatabases(CatalogQueryManager.java:57) > >> >> ... 7 more > >> >> Caused by: javax.security.auth.login.LoginException: No LoginModules > >> >> configured for pgjdbc > >> >> at javax.security.auth.login.LoginContext.init(LoginContext.java:273) > >> >> at > javax.security.auth.login.LoginContext.<init>(LoginContext.java:418) > >> >> at org.postgresql.gss.MakeGSS.authenticate(MakeGSS.java:42) > >> >> ... 22 more > >> > > >> > > > > > >
