Hey there, My mistake. Apparently the it's a bit more like an HTTP string: jdbc:postgresql://postgres.lyleco.com:5432?kerberosServerName=postgres/ [email protected]. Also, are the JDBC jars in /var/lib/sqoop? Or are they part of Sqoop's class path somehow?
The "Generic preauthentication" errors are discomforting. Are your credentials renewable? I believe hadoop-auth starts a renewer thread that runs "kinit -R". That could be causing that error. -Abe On Thu, Nov 20, 2014 at 3:56 PM, David Lyle <[email protected]> wrote: > Hi Abe, > > Thanks for the quick reply. Currently, the postgres instance in all > that is kerberized. When I add the kerberosServerName I get: > > java.sql.SQLException: No suitable driver found for > jdbc:postgresql://postgres.lyleco.com:5432;kerberosServerName=postgres/ > [email protected] > > or > > java.sql.SQLException: No suitable driver found for > jdbc:postgresql://postgres.lyleco.com:5432&kerberosServerName=postgres/ > [email protected] > > (I tried both). I'm also troubled by the repeated kinit: Generic > preauthentication failure while getting initial credentials at the > very top. I don't understand how sqoop is delegating, but the kinit is > surprising. > > -D... > > > On Thu, Nov 20, 2014 at 6:15 PM, Abraham Elmahrek <[email protected]> > wrote: > > This is a long shot... but I thought I'd say something. > > > > If you have Hadoop configured for Kerberos, then Sqoop should > automatically > > use your kerberos credentials to run a MapReduce job. That MapReduce job > > will run with its own Kerberos principal (normally yarn or mapred). This > > means that Hadoop has access to postgresql as long as it requests a > session > > from the KDC, assuming postgresql and Hadoop are part of the same realm. > My > > understanding is the JDBC driver will handle this for you if you provide > the > > postgresql service name via "kerberosServerName" in the JDBC connection > > string: > > "jdbc:postgresql://postgres.lyleco.com:5432 > ;kerberosServerName=postgres/<host>@<REALM>". > > > > Ref: http://jdbc.postgresql.org/documentation/84/connect.html > > > > -Abe > > > > On Thu, Nov 20, 2014 at 2:53 PM, David Lyle <[email protected]> > wrote: > >> > >> Hi, > >> > >> I'm trying to use Sqoop to connect to a kerberized Postgres instance. > >> At this time, I can connect from psql, but my sqoop session gives me > >> the log below. Any idea where I'm going wrong? > >> > >> Thanks! > >> > >> -David... > >> > >> > >> sqoop list-databases -Djavax.security.auth.debug=true --connect > >> "jdbc:postgresql://postgres.lyleco.com:5432" -verbose > >> > >> kinit: Generic preauthentication failure while getting initial > credentials > >> kinit: Generic preauthentication failure while getting initial > credentials > >> > >> 14/11/20 17:26:37 INFO sqoop.Sqoop: Running Sqoop version: > >> 1.4.4.2.1.7.0-784 > >> 14/11/20 17:26:37 DEBUG tool.BaseSqoopTool: Enabled debug logging. > >> 14/11/20 17:26:37 DEBUG sqoop.ConnFactory: Loaded manager factory: > >> com.cloudera.sqoop.manager.DefaultManagerFactory > >> 14/11/20 17:26:37 DEBUG sqoop.ConnFactory: Trying ManagerFactory: > >> com.cloudera.sqoop.manager.DefaultManagerFactory > >> 14/11/20 17:26:37 DEBUG manager.DefaultManagerFactory: Trying with > >> scheme: jdbc:postgresql: > >> 14/11/20 17:26:37 INFO manager.SqlManager: Using default fetchSize of > 1000 > >> 14/11/20 17:26:37 DEBUG sqoop.ConnFactory: Instantiated ConnManager > >> org.apache.sqoop.manager.PostgresqlManager@1663108 > >> 14/11/20 17:26:37 DEBUG manager.SqlManager: No connection paramenters > >> specified. Using regular API for making connection. > >> 14/11/20 17:26:37 ERROR manager.CatalogQueryManager: Failed to list > >> databases > >> org.postgresql.util.PSQLException: GSS Authentication failed > >> at org.postgresql.gss.MakeGSS.authenticate(MakeGSS.java:49) > >> at > >> > org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:380) > >> at > >> > org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:108) > >> at > >> > org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > >> at > >> > org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > >> at > >> > org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > >> at > >> > org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > >> at > >> > org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > >> at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > >> at org.postgresql.Driver.makeConnection(Driver.java:393) > >> at org.postgresql.Driver.connect(Driver.java:267) > >> at java.sql.DriverManager.getConnection(DriverManager.java:571) > >> at java.sql.DriverManager.getConnection(DriverManager.java:233) > >> at > org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:824) > >> at > >> > org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52) > >> at > >> > org.apache.sqoop.manager.CatalogQueryManager.listDatabases(CatalogQueryManager.java:57) > >> at > org.apache.sqoop.tool.ListDatabasesTool.run(ListDatabasesTool.java:49) > >> at org.apache.sqoop.Sqoop.run(Sqoop.java:147) > >> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > >> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) > >> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:222) > >> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:231) > >> at org.apache.sqoop.Sqoop.main(Sqoop.java:240) > >> Caused by: javax.security.auth.login.LoginException: No LoginModules > >> configured for pgjdbc > >> at javax.security.auth.login.LoginContext.init(LoginContext.java:273) > >> at javax.security.auth.login.LoginContext.<init>(LoginContext.java:418) > >> at org.postgresql.gss.MakeGSS.authenticate(MakeGSS.java:42) > >> ... 22 more > >> 14/11/20 17:26:37 ERROR sqoop.Sqoop: Got exception running Sqoop: > >> java.lang.RuntimeException: org.postgresql.util.PSQLException: GSS > >> Authentication failed > >> java.lang.RuntimeException: org.postgresql.util.PSQLException: GSS > >> Authentication failed > >> at > >> > org.apache.sqoop.manager.CatalogQueryManager.listDatabases(CatalogQueryManager.java:73) > >> at > org.apache.sqoop.tool.ListDatabasesTool.run(ListDatabasesTool.java:49) > >> at org.apache.sqoop.Sqoop.run(Sqoop.java:147) > >> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > >> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) > >> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:222) > >> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:231) > >> at org.apache.sqoop.Sqoop.main(Sqoop.java:240) > >> Caused by: org.postgresql.util.PSQLException: GSS Authentication failed > >> at org.postgresql.gss.MakeGSS.authenticate(MakeGSS.java:49) > >> at > >> > org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:380) > >> at > >> > org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:108) > >> at > >> > org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > >> at > >> > org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > >> at > >> > org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > >> at > >> > org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > >> at > >> > org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > >> at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > >> at org.postgresql.Driver.makeConnection(Driver.java:393) > >> at org.postgresql.Driver.connect(Driver.java:267) > >> at java.sql.DriverManager.getConnection(DriverManager.java:571) > >> at java.sql.DriverManager.getConnection(DriverManager.java:233) > >> at > org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:824) > >> at > >> > org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52) > >> at > >> > org.apache.sqoop.manager.CatalogQueryManager.listDatabases(CatalogQueryManager.java:57) > >> ... 7 more > >> Caused by: javax.security.auth.login.LoginException: No LoginModules > >> configured for pgjdbc > >> at javax.security.auth.login.LoginContext.init(LoginContext.java:273) > >> at javax.security.auth.login.LoginContext.<init>(LoginContext.java:418) > >> at org.postgresql.gss.MakeGSS.authenticate(MakeGSS.java:42) > >> ... 22 more > > > > >
