Drillbit endpoint not found

[Sebastian Hamann]

Hi,
First: Thanks for all the help for my last question.
Now I got another problem:
I was running drill in embedded mode and my autosetup(download 
everything from java to drill and set it up and run queries) works 
perfectly for it. The next step is to bring this to a cluster.
For testing purposes I want to run zookeeper, drillbit and the 
drill-client all on my main node for node. If that works the next step 
would be to start drillbits on all nodes and find a good configuration 
for the zookeeper, but Iam not so far yet. (just mentioning here if 
someone has a better idea I am always welcoming new ideas).

So now to the actual problem:
So the zookeeper is running on the localhost/127.0.0.1:2181 and I can 
connect to it/check if its running

(cfg consists only of tickTime=2000 dataDir=/tmp/zookeeper clientPort=2181).
Then I start the drillbit with the drillbit.sh on the same node and when 
I check the status it says that its running.
When I know try to start the drill client, either via drill-localhost, 
drill-conf (drill.exec: {cluster-id: "drillbits1",zk.connect: 
"127.0.0.1:2181"}) or sqlline -u jdbc:drill:zk:127.0.0.1 (which if I 
understood it right should have all the same effect in my 
configuration), I get the error message at the bottom (currently drill 
is always supposed to execute that test query).

Again, thanks in advance!
Error:
No DrillbitEndpoint can be found
apache drill 1.6.0
"drill baby drill"
0: jdbc:drill:> select count(*) from (values(1));
0: jdbc:drill:> No current connection
java.lang.IllegalStateException: No DrillbitEndpoint can be found
at 
com.google.common.base.Preconditions.checkState(Preconditions.java:173)
at 
org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:205)
at 
org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:151)
at 
org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:64)
at 
org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
at 
net.hydromatic.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:126)

at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
at sqlline.DatabaseConnection.connect(DatabaseConnection.java:167)
at 
sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:213)

at sqlline.Commands.close(Commands.java:925)
at sqlline.Commands.closeall(Commands.java:899)
at sqlline.SqlLine.begin(SqlLine.java:649)
at sqlline.SqlLine.start(SqlLine.java:375)
at sqlline.SqlLine.main(SqlLine.java:268)

Re: Drillbit endpoint not found

[Andries Engelbrecht]

You have to specify the zk port in jdbc url with sqlline
sqlline -u jdbc:drill:zk=127.0.0.1:2181

If you use a zk quorum you can specify all the zk nodes in similar fashion.
Also if you use JDBC connectivity with other tools you will need to add the 
drill cluster ID.

For more info and examples see

https://drill.apache.org/docs/using-the-jdbc-driver/ 


--Andries



> On Jun 9, 2016, at 7:25 AM, Sebastian Hamann  wrote:
> 
> Hi,
> First: Thanks for all the help for my last question.
> Now I got another problem:
> I was running drill in embedded mode and my autosetup(download everything 
> from java to drill and set it up and run queries) works perfectly for it. The 
> next step is to bring this to a cluster.
> For testing purposes I want to run zookeeper, drillbit and the drill-client 
> all on my main node for node. If that works the next step would be to start 
> drillbits on all nodes and find a good configuration for the zookeeper, but 
> Iam not so far yet. (just mentioning here if someone has a better idea I am 
> always welcoming new ideas).
> So now to the actual problem:
> So the zookeeper is running on the localhost/127.0.0.1:2181 and I can connect 
> to it/check if its running
> (cfg consists only of tickTime=2000 dataDir=/tmp/zookeeper clientPort=2181).
> Then I start the drillbit with the drillbit.sh on the same node and when I 
> check the status it says that its running.
> When I know try to start the drill client, either via drill-localhost, 
> drill-conf (drill.exec: {cluster-id: "drillbits1",zk.connect: 
> "127.0.0.1:2181"}) or sqlline -u jdbc:drill:zk:127.0.0.1 (which if I 
> understood it right should have all the same effect in my configuration), I 
> get the error message at the bottom (currently drill is always supposed to 
> execute that test query).
> Again, thanks in advance!
> Error:
> No DrillbitEndpoint can be found
> apache drill 1.6.0
> "drill baby drill"
> 0: jdbc:drill:> select count(*) from (values(1));
> 0: jdbc:drill:> No current connection
> java.lang.IllegalStateException: No DrillbitEndpoint can be found
>at 
> com.google.common.base.Preconditions.checkState(Preconditions.java:173)
>at 
> org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:205)
>at 
> org.apache.drill.jdbc.impl.DrillConnectionImpl.(DrillConnectionImpl.java:151)
>at 
> org.apache.drill.jdbc.impl.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:64)
>at 
> org.apache.drill.jdbc.impl.DrillFactory.newConnection(DrillFactory.java:69)
>at 
> net.hydromatic.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:126)
>at org.apache.drill.jdbc.Driver.connect(Driver.java:72)
>at sqlline.DatabaseConnection.connect(DatabaseConnection.java:167)
>at 
> sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:213)
>at sqlline.Commands.close(Commands.java:925)
>at sqlline.Commands.closeall(Commands.java:899)
>at sqlline.SqlLine.begin(SqlLine.java:649)
>at sqlline.SqlLine.start(SqlLine.java:375)
>at sqlline.SqlLine.main(SqlLine.java:268)

Dynamic setting of user id / password for Storage plugin

[Lin, Yaxiong]

Hi,

I am evaluating Drills as the query service for our analytics applications to 
access various data sources and it seems to fill the needs very well. However I 
have one concern/question that I could not find the answer from Drill’s website 
or on google.

My question/concern is that from what I’ve read, the storage plugin 
configuration requires static user id and password set in the configuration 
which storage plugin will use to connect to the backend data source/database. I 
need Drill client to pass the user id and password at query submission to 
storage plugin (e.g. RDMBS) and have storage plugin use that to connect to the 
back end data source/database.  Obviously each client has his/her own set of 
user id and password for each data source/database.  Is there anyway that I can 
achieve this?

Thanks.

Yaxiong Lin
Mayo Clinic

Re: Dynamic setting of user id / password for Storage plugin

[Neeraja Rentachintala]

Have you checked these docs.
https://drill.apache.org/docs/configuring-user-authentication/

On Thu, Jun 9, 2016 at 8:05 AM, Lin, Yaxiong  wrote:

> Hi,
>
> I am evaluating Drills as the query service for our analytics applications
> to access various data sources and it seems to fill the needs very well.
> However I have one concern/question that I could not find the answer from
> Drill’s website or on google.
>
> My question/concern is that from what I’ve read, the storage plugin
> configuration requires static user id and password set in the configuration
> which storage plugin will use to connect to the backend data
> source/database. I need Drill client to pass the user id and password at
> query submission to storage plugin (e.g. RDMBS) and have storage plugin use
> that to connect to the back end data source/database.  Obviously each
> client has his/her own set of user id and password for each data
> source/database.  Is there anyway that I can achieve this?
>
> Thanks.
>
> Yaxiong Lin
> Mayo Clinic
>

Re: Dynamic setting of user id / password for Storage plugin

[Chun Chang]

Yaxiong,

If you still have questions after reading the docs provided by Neeraja,
please let us know. I will be happy to help.

Chun

On Thu, Jun 9, 2016 at 10:40 AM, Neeraja Rentachintala <
nrentachint...@maprtech.com> wrote:

> Have you checked these docs.
> https://drill.apache.org/docs/configuring-user-authentication/
>
> On Thu, Jun 9, 2016 at 8:05 AM, Lin, Yaxiong  wrote:
>
> > Hi,
> >
> > I am evaluating Drills as the query service for our analytics
> applications
> > to access various data sources and it seems to fill the needs very well.
> > However I have one concern/question that I could not find the answer from
> > Drill’s website or on google.
> >
> > My question/concern is that from what I’ve read, the storage plugin
> > configuration requires static user id and password set in the
> configuration
> > which storage plugin will use to connect to the backend data
> > source/database. I need Drill client to pass the user id and password at
> > query submission to storage plugin (e.g. RDMBS) and have storage plugin
> use
> > that to connect to the back end data source/database.  Obviously each
> > client has his/her own set of user id and password for each data
> > source/database.  Is there anyway that I can achieve this?
> >
> > Thanks.
> >
> > Yaxiong Lin
> > Mayo Clinic
> >
>

Re: Dynamic setting of user id / password for Storage plugin

[John Omernik]

I think the original question is more about using the JDBC Storage Plugin
and connections to other sources. I know I've posted a few user posts
related to the security of storage plugins and the idea of passing
credentials to the back end was discussed. The challenge here from a
security perspective is A. Does Drill and the RDBMS share an authenticator?
(i.e. LDAP) if so, how can Drill pass a token to the backend to to ensure
end to end accountability without storing passwords.  This is a non-trivial
challenge in a multi-user system.

My "first step"  approach was to Storage plugins where the ability to query
these could be set within drill (using Filesystem ACLs or other
methodologies) This could allow a drill admin to setup different plugins
with different permissions and assign users there. It defeats the "end
point" (RDBMS)  accountability because unless each user got their own
plugin, there would be shared users via  abit, but that's better than the
everyone can access all the storage plugins.

This is hard in general :)

John

On Thu, Jun 9, 2016 at 1:02 PM, Chun Chang  wrote:

> Yaxiong,
>
> If you still have questions after reading the docs provided by Neeraja,
> please let us know. I will be happy to help.
>
> Chun
>
> On Thu, Jun 9, 2016 at 10:40 AM, Neeraja Rentachintala <
> nrentachint...@maprtech.com> wrote:
>
> > Have you checked these docs.
> > https://drill.apache.org/docs/configuring-user-authentication/
> >
> > On Thu, Jun 9, 2016 at 8:05 AM, Lin, Yaxiong 
> wrote:
> >
> > > Hi,
> > >
> > > I am evaluating Drills as the query service for our analytics
> > applications
> > > to access various data sources and it seems to fill the needs very
> well.
> > > However I have one concern/question that I could not find the answer
> from
> > > Drill’s website or on google.
> > >
> > > My question/concern is that from what I’ve read, the storage plugin
> > > configuration requires static user id and password set in the
> > configuration
> > > which storage plugin will use to connect to the backend data
> > > source/database. I need Drill client to pass the user id and password
> at
> > > query submission to storage plugin (e.g. RDMBS) and have storage plugin
> > use
> > > that to connect to the back end data source/database.  Obviously each
> > > client has his/her own set of user id and password for each data
> > > source/database.  Is there anyway that I can achieve this?
> > >
> > > Thanks.
> > >
> > > Yaxiong Lin
> > > Mayo Clinic
> > >
> >
>

Re: Dynamic setting of user id / password for Storage plugin

[Chun Chang]

That's in deed hard in general. Currently, Drill only supports
impersonation through dfs and hive storage plugin.

On Thu, Jun 9, 2016 at 11:14 AM, John Omernik  wrote:

> I think the original question is more about using the JDBC Storage Plugin
> and connections to other sources. I know I've posted a few user posts
> related to the security of storage plugins and the idea of passing
> credentials to the back end was discussed. The challenge here from a
> security perspective is A. Does Drill and the RDBMS share an authenticator?
> (i.e. LDAP) if so, how can Drill pass a token to the backend to to ensure
> end to end accountability without storing passwords.  This is a non-trivial
> challenge in a multi-user system.
>
> My "first step"  approach was to Storage plugins where the ability to query
> these could be set within drill (using Filesystem ACLs or other
> methodologies) This could allow a drill admin to setup different plugins
> with different permissions and assign users there. It defeats the "end
> point" (RDBMS)  accountability because unless each user got their own
> plugin, there would be shared users via  abit, but that's better than the
> everyone can access all the storage plugins.
>
> This is hard in general :)
>
> John
>
> On Thu, Jun 9, 2016 at 1:02 PM, Chun Chang  wrote:
>
> > Yaxiong,
> >
> > If you still have questions after reading the docs provided by Neeraja,
> > please let us know. I will be happy to help.
> >
> > Chun
> >
> > On Thu, Jun 9, 2016 at 10:40 AM, Neeraja Rentachintala <
> > nrentachint...@maprtech.com> wrote:
> >
> > > Have you checked these docs.
> > > https://drill.apache.org/docs/configuring-user-authentication/
> > >
> > > On Thu, Jun 9, 2016 at 8:05 AM, Lin, Yaxiong 
> > wrote:
> > >
> > > > Hi,
> > > >
> > > > I am evaluating Drills as the query service for our analytics
> > > applications
> > > > to access various data sources and it seems to fill the needs very
> > well.
> > > > However I have one concern/question that I could not find the answer
> > from
> > > > Drill’s website or on google.
> > > >
> > > > My question/concern is that from what I’ve read, the storage plugin
> > > > configuration requires static user id and password set in the
> > > configuration
> > > > which storage plugin will use to connect to the backend data
> > > > source/database. I need Drill client to pass the user id and password
> > at
> > > > query submission to storage plugin (e.g. RDMBS) and have storage
> plugin
> > > use
> > > > that to connect to the back end data source/database.  Obviously each
> > > > client has his/her own set of user id and password for each data
> > > > source/database.  Is there anyway that I can achieve this?
> > > >
> > > > Thanks.
> > > >
> > > > Yaxiong Lin
> > > > Mayo Clinic
> > > >
> > >
> >
>