Is this Location correct? This would only allow connections within the same
host.
If the intent is to re-use the same host as the original connection the
Location can be set to null.
From: Istvan Fodor <i...@istvanfodor.com>
Date: Thursday, April 18, 2024 at 12:02 PM
To: Diego Fernandez <aiguo.fernan...@gmail.com>
Cc: user@arrow.apache.org <user@arrow.apache.org>, Norman Jordan
<norman.jor...@improving.com>
Subject: Re: Arrow Flight JDBC + Arrow RS example
Yes, the endpoint was there, but I noticed that location was missing the port:
let loc = Location {uri: "grpc+tcp://127.0.0.1<http://127.0.0.1>".to_string(),};
I will work with the arrow-rs team to add this as well! Thanks for the help,
very much appreciated!
On Thu, Apr 18, 2024 at 12:52 PM Diego Fernandez
<aiguo.fernan...@gmail.com<mailto:aiguo.fernan...@gmail.com>> wrote:
Glad that solved the header issue!
Honestly, I didn't even know that the driver had retries built in. We've never
seen a retry hit the server.
My guess is that something is failing server side, causing the driver to retry.
Is the server returning the correct results for each of the requests in the
PreparedStatement flow?
Are you correctly specifying the location on the FlightEndpoints for
FlightIinfo?
Something to note is that both ADBC and JDBC `.execute` calls actually use
PreparedStatements under the covers.
On Thu, Apr 18, 2024 at 10:15 AM Istvan Fodor
<i...@istvanfodor.com<mailto:i...@istvanfodor.com>> wrote:
Thank you Diego, this is super helpful!
I just added the headers to the do_handshake response, and now I get past the
authentication issue. I'll follow up with the arrow-rs guys to see if we can
add this change in the repo, it certainly seems to me it would make sense.
I see another issue, not sure if it is driver related or more on the server
side, but through you might have some ideas. When I try to run a query, the
driver seems to call the server in a loop, roughly these are the service calls:
arrow.flight.protocol.FlightService/Handshake
arrow.flight.protocol.sql.ActionCreatePreparedStatementRequest
(
arrow.flight.protocol.sql.CreatePreparedStatement
arrow.flight.protocol.sql.CommandPreparedStatementQuery
arrow.flight.protocol.sql.FetchResults
) x5
The driver eventually bombs out with the following error:
java.sql.SQLException: Error while executing SQL "SELECT 1": Failed to
successfully execute query after 5 attempts.
at cfjd.org.apache.calcite.avatica.Helper.createException(Helper.java:56)
at cfjd.org.apache.calcite.avatica.Helper.createException(Helper.java:41)
at
cfjd.org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:235)
at com.fusiongrid.App.main(App.java:29)
at org.codehaus.mojo.exec.ExecJavaMojo.doMain(ExecJavaMojo.java:385)
at org.codehaus.mojo.exec.ExecJavaMojo.doExec(ExecJavaMojo.java:374)
at org.codehaus.mojo.exec.ExecJavaMojo.lambda$execute$0(ExecJavaMojo.java:296)
at java.base/java.lang.Thread.run(Thread.java:1570)
Caused by: java.lang.RuntimeException: Failed to successfully execute query
after 5 attempts.
at
cfjd.org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:168)
at
cfjd.org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:228)
... 5 more
Have you seen anything like this on other server implementations?
On Thu, Apr 18, 2024 at 11:31 AM Diego Fernandez
<aiguo.fernan...@gmail.com<mailto:aiguo.fernan...@gmail.com>> wrote:
Interesting... so the Rust AFS server is returning the token in the payload? I
believe it should be setting it on the returning headers instead.
In the Java server, you decide on the authentication mechanism. The equivalent
of what you're trying to do is `BearerTokenAuthenticator`, which allows you to
validate the incoming Basic auth header and set the Bearer token header in the
response headers.
After a quick look at the JDBC code, it seems like it's looking for the header
`authorization=Bearer <token>` in the response to set it for following requests.
Maybe the Rust server has a similar auth mechanism to
`BearerTokenAuthenticator`?
On Thu, Apr 18, 2024 at 9:11 AM Istvan Fodor
<i...@istvanfodor.com<mailto:i...@istvanfodor.com>> wrote:
Hi Jordan,
I was testing with the example code in the Apache Arrow Rust repo:
https://github.com/apache/arrow-rs/blob/master/arrow-flight/examples/flight_sql_server.rs
To the best of my knowledge, it returns a token in the payload. I can see in
ngrep that do_handshake is called and this is what it returns:
let result = HandshakeResponse {
protocol_version: 0,
payload: FAKE_TOKEN.into(),
};
let result = Ok(result);
let output = futures::stream::iter(vec![result]);
return Ok(Response::new(Box::pin(output)));
In the Java code it looks like the authorization header is not part of the
metadata that is passed:
No authorization header! metadata = MetadataMap { headers: {"content-type":
"application/grpc", "te": "trailers", "user-agent": "grpc-java-netty/1.60.0",
"username": "admin", "grpc-accept-encoding": "gzip"} }
I logged the metadata on the Rust side and the authorization header was missing
indeed.
IN the Netty logs it looks like the right Basic auth is passed in on
getConnection, and the token comes back fine:
10:50:53.890 [grpc-nio-worker-ELG-1-2] DEBUG
cfjd.io.grpc.netty.NettyClientHandler -- [id: 0x53dab587,
L:/127.0.0.1:59757<http://127.0.0.1:59757> -
R:/127.0.0.1:50050<http://127.0.0.1:50050>] OUTBOUND HEADERS: streamId=3
headers=GrpcHttp2OutboundHeaders[:authority:
0.0.0.0:50050<http://0.0.0.0:50050>, :path:
/arrow.flight.protocol.FlightService/Handshake, :method: POST, :scheme: http,
content-type: application/grpc, te: trailers, user-agent:
grpc-java-netty/1.60.0, username: admin, grpc-accept-encoding: gzip,
authorization: Basic YWRtaW46cGFzc3dvcmQ=] streamDependency=0 weight=16
exclusive=false padding=0 endStream=false
10:50:53.895 [grpc-nio-worker-ELG-1-2] DEBUG
cfjd.io.grpc.netty.NettyClientHandler -- [id: 0x53dab587,
L:/127.0.0.1:59757<http://127.0.0.1:59757> -
R:/127.0.0.1:50050<http://127.0.0.1:50050>] OUTBOUND DATA: streamId=3 padding=0
endStream=true length=5 bytes=0000000000
10:50:53.898 [grpc-nio-worker-ELG-1-2] DEBUG
cfjd.io.grpc.netty.NettyClientHandler -- [id: 0x53dab587,
L:/127.0.0.1:59757<http://127.0.0.1:59757> -
R:/127.0.0.1:50050<http://127.0.0.1:50050>] INBOUND HEADERS: streamId=3
headers=GrpcHttp2ResponseHeaders[:status: 200, content-type: application/grpc,
date: Thu, 18 Apr 2024 15:50:53 GMT] padding=0 endStream=false
10:50:53.899 [grpc-nio-worker-ELG-1-2] DEBUG
cfjd.io.grpc.netty.NettyClientHandler -- [id: 0x53dab587,
L:/127.0.0.1:59757<http://127.0.0.1:59757> -
R:/127.0.0.1:50050<http://127.0.0.1:50050>] INBOUND DATA: streamId=3 padding=0
endStream=false length=17 bytes=000000000c120a757569645f746f6b656e
Subsequent calls in the JDBC/Netty log don't show the authorization header.
On Wed, Apr 17, 2024 at 2:16 PM Istvan Fodor
<i...@istvanfodor.com<mailto:i...@istvanfodor.com>> wrote:
Hi All,
I am trying to write a basic example of a Java/JDBC code querying from the
Arrow Rust example implementation of the Arrow Flight SQL server
(https://github.com/apache/arrow-rs/blob/master/arrow-flight/examples/flight_sql_server.rs).
My impression was that based on the interoperability goals of the Arrow
project, these should work together just fine.
It turns out the in my Java code, I can authenticate (getConnection()) fine
against the Rust server. User/password goes in, and a token comes back, but
subsequent calls (executeQuery() for example) don't include the authorization
token at all (it should look like authorization=Bearer <token>), whereas I
expected the token to just propagate to subsequent calls as it should.
I am using the 15.0.2 flight-sql-jdbc-driver. Any ideas what my issue could be?
Is there any extra setup that we need to do to get JDBC + Basic auth to work
besides supplying user and password parameters?
Thanks,
Istvan Fodor
