Thanks for your help Jacques! Sorry have been out of action for a couple of days.
On 7 November 2015 at 18:27, Jacques Nadeau <[email protected]> wrote: > Glad you've made some progress. > > Not sure why your <storage name>.public is how you have to go about things. > Will have to give that some more thought. At least you're over that hurdle > for the minute. > Yep - bit weird, but not a problem for the time being. > > > > 1. Uuid columns in postgres arent supported - I'll look to contribute > code > > to add support for this if its within my ability. > > > > My gut is that you can do this only in Drill code. Is there an exception > when you try to query or do they just not show up. If the former, than a > Calcite change is also required. If you just get a warning in the log and > the column is missing, then we only need to make changes in Drill's > JdbcRecordReader to provide a JDBC Type => Drill Type mapping. Current > mappings to be added to are here: > > https://github.com/apache/drill/blob/master/contrib/storage-jdbc/src/main/java/org/apache/drill/exec/store/jdbc/JdbcRecordReader.java#L90 Yes I get the column but all of the rows are empty. So I can maybe fix that in the mappings only? I'll see if I can have a look at that when I get chance. I'm having the same issue with CITEXT, case insensitive columns ( http://www.postgresql.org/docs/9.1/static/citext.html) columns as well so might be worth me fixing that up too. Do you have a 'developers start' guide or something that might help me getting master building? > > > > > > > 2. I've added views to convert the uuid columns to postgres 'text' data > > types - but those aren't usable within JOINs in Drill. Is that something > > you plan on fixing? I can cast to VARCHAR(?) but its a bit annoying and I > > bet thats not very good for performance? > > > > What kind of error do you get? Our current mapping is TEXT => VARCHAR so > I'm not sure why the join is failing. Is this a join between two columns in > Postgres or Posgres + Something else? (We push down joins that are all > within Postgres.) > I think this may have been me - I've just tried it again with the views casting the columns to TEXT and it now works - ignore me! If it appears again I'll shout. > > > > > 3. I'm having trouble following the instructions on > > https://drill.apache.org/docs/using-the-jdbc-driver/ to get the JDBC > > client up and running. I think the instructions might be out of date? It > > references having your client use the driver from inside /jars > > named drill-jdbc-all-<version>.jar but that no longer exists? Theres > > just drill-jdbc-<version>.jar and drill-jdbc-storage-<version>.jar. > Trying > > either of those fails in my client (0xDBE) with: > > > > The image was dropped (you can send to me directly if you like). The > correct path is drill/jars/jdbc-driver/. It is possible you missed the > "jdbc-driver" part. You'll need the drill-jdbc-all-* version as that > includes all the dependencies required to make Drill work. We haven't tried > using 0xDBE yet but things should work. > Got it - you're right I missed the /jdbc-driver bit. I've got it 'working' in 0xDBE but not without some funky errors. Need to turn 'synching' off which enumerates the db structure - it hangs otherwise, and I get an UnsupportedOperationException after every query run: java.lang.UnsupportedOperationException at oadd.net.hydromatic.avatica.AvaticaStatement.getMoreResults(AvaticaStatement.java:208) at com.intellij.database.remote.jdbc.impl.RemoteStatementImpl.getMoreResults(RemoteStatementImpl.java:246) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:323) at sun.rmi.transport.Transport$1.run(Transport.java:200) at sun.rmi.transport.Transport$1.run(Transport.java:197) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.Transport.serviceCall(Transport.java:196) at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$255(TCPTransport.java:683) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$$Lambda$2/11679769.run(Unknown Source) <snip> But it works in a pinch! :) > > Sorry Jacques, one last question for now - whats the best way for me to get > > setup for (1)? Is calcite forked within the drill repo or do I need to > pull > > both projects locally and statically deploy jars from my fork of calcite > to > > test in drill? Or can I test calcite in isolation? > > > > We're still forked. The current version is here: > > https://github.com/dremio/calcite/tree/DrillCalcite1.4.0 > > You'll need to do the local deploy for each to make them work together. > There are tests in Calcite for its JdbcAdapter (which Drill leverages) so > fixes for Schema related issues should be possible to test solely within > Calcite. Let me know about what is happening with the UUID column so I can > give you a better direction on steps to take. > Thanks - sounds like I can ignore calcite for now, but pointers on building drill locally would be great and hopefully I can contrib! Cheers. > > thanks, > Jacques >
