Hi David,
Can you try using drill-jdbc as dependency instead of jdbc-all ? Reason
being jdbc-all only contains client side dependencies and also has jdbc
driver class in it. All other dependencies inside jdbc-all are shaded to
have oadd prefix. When you load Driver class it's getting loaded from
jdbc-all and all other classes in that jar are also getting loaded and used
with it. Also since you are trying to run Drill in embedded mode you will
end up pulling all other dependencies/jar published/used by Drill which
server side needs for it's execution.
Thanks,
Sorabh
On Fri, Oct 19, 2018 at 9:20 AM David Cuesta <david.cue...@shapelets.io>
wrote:
> Hi,
>
> My name is David Cuesta and I am very interested on using Drill as a
> Library, in embedded mode. The intention is to launch a drillbit from my
> application without using Zookeeper and use it to change the Datasource
> configurations in runtime, do queries to my local file system, to an
> external Hadoop, etc.
>
> How should I do that?
>
> Right now, what I have done is the next (Kotlin code):
>
>
> import java.sql.DriverManager
>
>
>
>
>
> fun main(args: Array<String>) {
> Class.forName("org.apache.drill.jdbc.Driver")
>
> val connection = DriverManager.getConnection("jdbc:drill:zk=local")
>
> val st = connection.createStatement()
> val rs = st.executeQuery("SELECT FROM
> dfs.`/Users/david.cuesta/Downloads/userdata1.parquet` WHERE
> first_name='Patricia'")
> while (rs.next()) {
> println(rs.getString(1))
> }
> }
>
> But there is an error when the library is going to do the bit.run in
> DrillConnectionImpl.java: 134:
>
> try {
> try {
> String connect = null;
> DrillConfig dConfig;
> if (this.config.isLocal()) {
> try {
> Class.forName("org.eclipse.jetty.server.Handler");
> } catch (ClassNotFoundException var11) {
> throw new SQLNonTransientConnectionException("Running
> Drill in embedded mode using Drill's jdbc-all JDBC driver Jar file alone is
> not supported.", var11);
> }
>
> dConfig = DrillConfig.create(info);
> this.allocator = RootAllocatorFactory.newRoot(dConfig);
> RemoteServiceSet set =
> (RemoteServiceSet)GlobalServiceSetReference.SETS.get();
> if (set == null) {
> this.serviceSet =
> RemoteServiceSet.getLocalServiceSet();
> set = this.serviceSet;
>
> try {
> this.bit = new Drillbit(dConfig, this.serviceSet);
> this.bit.run();
> } catch (UserException var9) {
> throw new SQLException("Failure in starting
> embedded Drillbit: " + var9.getMessage(), var9);
> } catch (Exception var10) {
> throw new SQLException("Failure in starting
> embedded Drillbit: " + var10, var10);
> }
> } else {
> this.serviceSet = null;
> this.bit = null;
> }
>
>
> makeTmpSchemaLocationsUnique(this.bit.getContext().getStorage(), info);
> this.client = new DrillClient(dConfig,
> set.getCoordinator());
> } else if (this.config.isDirect()) {
> dConfig = DrillConfig.forClient();
> this.allocator = RootAllocatorFactory.newRoot(dConfig);
> this.client = new DrillClient(dConfig, true);
> connect = this.config.getZookeeperConnectionString();
> } else {
> dConfig = DrillConfig.forClient();
> this.allocator = RootAllocatorFactory.newRoot(dConfig);
> this.client = new DrillClient();
> connect = this.config.getZookeeperConnectionString();
> }
>
> this.client.setClientName("Apache Drill JDBC Driver");
> this.client.connect(connect, info);
> } catch (OutOfMemoryException var12) {
> throw new SQLNonTransientConnectionException("Failure creating
> root allocator", var12);
> } catch (InvalidConnectionInfoException var13) {
> throw new SQLNonTransientConnectionException("Invalid
> parameter in connection string: " + var13.getMessage(), var13);
> } catch (RpcException var14) {
> throw new SQLNonTransientConnectionException("Failure in
> connecting to Drill: " + var14, var14);
> } catch (SQLException var15) {
> throw var15;
> } catch (Exception var16) {
> throw new SQLException("Failure in creating
> DrillConnectionImpl: " + var16, var16);
> }
> } catch (Throwable var17) {
> this.cleanup();
> throw var17;
> }
> }
>
>
> The tracktrace error is the next one:
>
> -----------------------------------------------------------------------------------------------------
> log4j:WARN No appenders could be found for logger
> (oadd.org.apache.drill.common.scanner.ClassPathScanner).
> log4j:WARN Please initialize the log4j system properly.
> log4j:WARN See
> https://urldefense.proofpoint.com/v2/url?u=http-3A__logging.apache.org_log4j_1.2_faq.html-23noconfig&d=DwIGaQ&c=cskdkSMqhcnjZxdQVpwTXg&r=gRpEl0WzXE3EMrwj0KFbZXGXRyadOthF2jlYxvhTlQg&m=zIOXhMwcNW48YH5_jWrk78SX-o0lZxi6rE_CgwB-D3Q&s=b54_H7-eSdKyd0JGJcL_43--C9DjdjVoq1tdfOlGTNc&e=
> for more info.
> Exception in thread "main" java.lang.NoClassDefFoundError:
> oadd/org/apache/drill/exec/store/sys/PersistentStoreProvider
> at
> org.apache.drill.jdbc.impl.DrillConnectionImpl.<init>(DrillConnectionImpl.java:134)
> at
> org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:73)
> at
> org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
> at
> oadd.org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:138)
> at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
> at
> java.sql.DriverManager.getConnection(DriverManager.java:664)
> at
> java.sql.DriverManager.getConnection(DriverManager.java:270)
> at AppKt.main(app.kt:18)
> Caused by: java.lang.ClassNotFoundException:
> oadd.org.apache.drill.exec.store.sys.PersistentStoreProvider
> at
> java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at
> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 8 more
>
> So after a intense research, I have seen that the class
> “oadd/org/apache/drill/exec/store/sys/PersistentStoreProvider” is not
> exposed by Drill, so when the classLoad is done, it is not able to load
> this class.
>
> Apart of that, I am not sure If will be able to change the Drill
> dataSources configuration at runtime.
>
> The dependencies that I have used are the next ones:
>
>
> compile "org.jetbrains.kotlin:kotlin-stdlib-jdk8"
> compile group: 'org.apache.drill.exec', name: 'drill-jdbc-all', version:
> '1.13.0'
> compile group: 'org.eclipse.jetty', name: 'jetty-server', version:
> '9.4.12.v20180830'
> compile group: 'org.apache.drill.exec', name: 'drill-java-exec', version:
> '1.13.0'
>
> compile group: 'org.apache.calcite', name: 'calcite-core', version:
> '1.17.0'
>
> Could you help me?
>
> Please, it is very important as we would like to use Drill in our
> OpenSource project.
>
> Best regards,
>
> David.
>
