[jira] [Commented] (DRILL-5252) A condition returns always true
[
https://issues.apache.org/jira/browse/DRILL-5252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15891725#comment-15891725
]
ASF GitHub Bot commented on DRILL-5252:
---
Github user jinfengni commented on the issue:
https://github.com/apache/drill/pull/745
+1
> A condition returns always true
> ---
>
> Key: DRILL-5252
> URL: https://issues.apache.org/jira/browse/DRILL-5252
> Project: Apache Drill
> Issue Type: Bug
>Reporter: JC
>Priority: Minor
> Labels: ready-to-commit
>
> I've found the following code smell in recent github snapshot.
> Path:
> exec/java-exec/src/main/java/org/apache/drill/exec/expr/EqualityVisitor.java
> {code:java}
> 287
> 288 @Override
> 289 public Boolean visitNullConstant(TypedNullConstant e, LogicalExpression
> value) throws RuntimeException {
> 290 if (!(value instanceof TypedNullConstant)) {
> 291 return false;
> 292 }
> 293 return e.getMajorType().equals(e.getMajorType());
> 294 }
> 295
> {code}
> Should it be like this?
> {code:java}
> 292 }
> 293 return value.getMajorType().equals(e.getMajorType());
> 294 }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (DRILL-4963) Issues when overloading Drill native functions with dynamic UDFs
[
https://issues.apache.org/jira/browse/DRILL-4963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15891718#comment-15891718
]
ASF GitHub Bot commented on DRILL-4963:
---
Github user jinfengni commented on the issue:
https://github.com/apache/drill/pull/701
+1
I have one naive question. Before we dive into MVCC, or try to improve the
current ZK-based approach, have we consider adding a command "refresh function
registry", so that users enforce that function registry across the entire
cluster is consistent? I remember someone (probably @amansinha100 ) once
brought up this idea.
1. How often would user run into the issue caused by inconsistency of
functions in different drillbit? Sounds like we are adding non-negligible
overhead for other queries which do not use dynamic UDF or overloaded
functions, just for the sake of resolving issues of function inconsistency.
2. Drill's function resolution logic not only happens in planning but also
execution time (due to the fact schema may be known in execution time). This
means the function registry could be checked multiple times during the query
lifetime. Adding overhead to each check would slow down the entire query.
3. With the "refresh function registry" command, use has a way to ensure
function registry is consistent after issuing such command. We could tell user
that drillbit could run into various of problems, before run "refresh function
registry", therefore it's always recommend to run such command after user
creates a new UDF.
> Issues when overloading Drill native functions with dynamic UDFs
>
>
> Key: DRILL-4963
> URL: https://issues.apache.org/jira/browse/DRILL-4963
> Project: Apache Drill
> Issue Type: Bug
> Components: Functions - Drill
>Affects Versions: 1.9.0
>Reporter: Roman
>Assignee: Arina Ielchiieva
> Labels: ready-to-commit
> Fix For: Future
>
> Attachments: subquery_udf-1.0.jar, subquery_udf-1.0-sources.jar,
> test_overloading-1.0.jar, test_overloading-1.0-sources.jar
>
>
> I created jar file which overloads 3 DRILL native functions
> (LOG(VARCHAR-REQUIRED), CURRENT_DATE(VARCHAR-REQUIRED) and
> ABS(VARCHAR-REQUIRED,VARCHAR-REQUIRED)) and registered it as dynamic UDF.
> If I try to use my functions I will get errors:
> {code:xml}
> SELECT CURRENT_DATE('test') FROM (VALUES(1));
> {code}
> Error: FUNCTION ERROR: CURRENT_DATE does not support operand types (CHAR)
> SQL Query null
> {code:xml}
> SELECT ABS('test','test') FROM (VALUES(1));
> {code}
> Error: FUNCTION ERROR: ABS does not support operand types (CHAR,CHAR)
> SQL Query null
> {code:xml}
> SELECT LOG('test') FROM (VALUES(1));
> {code}
> Error: SYSTEM ERROR: DrillRuntimeException: Failure while materializing
> expression in constant expression evaluator LOG('test'). Errors:
> Error in expression at index -1. Error: Missing function implementation:
> castTINYINT(VARCHAR-REQUIRED). Full expression: UNKNOWN EXPRESSION.
> But if I rerun all this queries after "DrillRuntimeException", they will run
> correctly. It seems that Drill have not updated the function signature before
> that error. Also if I add jar as usual UDF (copy jar to
> /drill_home/jars/3rdparty and restart drillbits), all queries will run
> correctly without errors.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (DRILL-5301) Add server metadata API
[ https://issues.apache.org/jira/browse/DRILL-5301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15891489#comment-15891489 ] ASF GitHub Bot commented on DRILL-5301: --- Github user laurentgo commented on the issue: https://github.com/apache/drill/pull/764 The cursor change belongs to DRILL-4994 > Add server metadata API > --- > > Key: DRILL-5301 > URL: https://issues.apache.org/jira/browse/DRILL-5301 > Project: Apache Drill > Issue Type: Improvement > Components: Server, Client - C++, Client - Java, Client - JDBC, > Client - ODBC >Reporter: Laurent Goujon >Assignee: Laurent Goujon > Labels: ready-to-commit > > JDBC and ODBC clients exposes lots of metadata regarding server version and > support of various parts of the SQL standard. > Currently the returned information is hardcoded in both clients/drivers which > means that the infomation returned is support as of the client version, not > the server version. > Instead, a new method should be provided to the clients to query the actual > server support. Support on the client or the server should be optional (for > example a client should not use this API if the server doesn't support it and > fallback to default values). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (DRILL-5301) Add server metadata API
[ https://issues.apache.org/jira/browse/DRILL-5301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15891372#comment-15891372 ] ASF GitHub Bot commented on DRILL-5301: --- Github user jinfengni commented on the issue: https://github.com/apache/drill/pull/764 Which JIRA should the last commit (Refactor DrillCursor) belong to, DRILL-4994, 4730, OR 5301? > Add server metadata API > --- > > Key: DRILL-5301 > URL: https://issues.apache.org/jira/browse/DRILL-5301 > Project: Apache Drill > Issue Type: Improvement > Components: Server, Client - C++, Client - Java, Client - JDBC, > Client - ODBC >Reporter: Laurent Goujon >Assignee: Laurent Goujon > Labels: ready-to-commit > > JDBC and ODBC clients exposes lots of metadata regarding server version and > support of various parts of the SQL standard. > Currently the returned information is hardcoded in both clients/drivers which > means that the infomation returned is support as of the client version, not > the server version. > Instead, a new method should be provided to the clients to query the actual > server support. Support on the client or the server should be optional (for > example a client should not use this API if the server doesn't support it and > fallback to default values). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (DRILL-5301) Add server metadata API
[ https://issues.apache.org/jira/browse/DRILL-5301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15891367#comment-15891367 ] Jinfeng Ni commented on DRILL-5301: --- [~laurentgo] PR #764 contains 5 commits, including the patches for DRILL-4994 and DRILL-4730, which also show up in PR#613. What commits should I use for DRILL-4994/DRILL-4730 ( the commits in PR#764, or PR#613), or they are same? Thanks, > Add server metadata API > --- > > Key: DRILL-5301 > URL: https://issues.apache.org/jira/browse/DRILL-5301 > Project: Apache Drill > Issue Type: Improvement > Components: Server, Client - C++, Client - Java, Client - JDBC, > Client - ODBC >Reporter: Laurent Goujon >Assignee: Laurent Goujon > Labels: ready-to-commit > > JDBC and ODBC clients exposes lots of metadata regarding server version and > support of various parts of the SQL standard. > Currently the returned information is hardcoded in both clients/drivers which > means that the infomation returned is support as of the client version, not > the server version. > Instead, a new method should be provided to the clients to query the actual > server support. Support on the client or the server should be optional (for > example a client should not use this API if the server doesn't support it and > fallback to default values). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (DRILL-5290) Provide an option to build operator table once for built-in static functions and reuse it across queries.
[ https://issues.apache.org/jira/browse/DRILL-5290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Padma Penumarthy updated DRILL-5290: Labels: doc-impacting ready-to-commit (was: doc-impacting) > Provide an option to build operator table once for built-in static functions > and reuse it across queries. > - > > Key: DRILL-5290 > URL: https://issues.apache.org/jira/browse/DRILL-5290 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.9.0 >Reporter: Padma Penumarthy >Assignee: Padma Penumarthy > Labels: doc-impacting, ready-to-commit > Fix For: 1.10.0 > > > Currently, DrillOperatorTable which contains standard SQL operators and > functions and Drill User Defined Functions (UDFs) (built-in and dynamic) gets > built for each query as part of creating QueryContext. This is an expensive > operation ( ~30 msec to build) and allocates ~2M on heap for each query. For > high throughput, low latency operational queries, we quickly run out of heap > memory, causing JVM hangs. Build operator table once during startup for > static built-in functions and save in DrillbitContext, so we can reuse it > across queries. > Provide a system/session option to not use dynamic UDFs so we can use the > operator table saved in DrillbitContext and avoid building each time. > *Please note, changes are adding new option exec.udf.use_dynamic which needs > to be documented.* -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (DRILL-5287) Provide option to skip updates of ephemeral state changes in Zookeeper
[ https://issues.apache.org/jira/browse/DRILL-5287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Padma Penumarthy updated DRILL-5287: Reviewer: Sudheesh Katkam > Provide option to skip updates of ephemeral state changes in Zookeeper > -- > > Key: DRILL-5287 > URL: https://issues.apache.org/jira/browse/DRILL-5287 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.9.0 >Reporter: Padma Penumarthy >Assignee: Padma Penumarthy > Labels: doc-impacting, ready-to-commit > Fix For: 1.10.0 > > > We put transient profiles in zookeeper and update state as query progresses > and changes states. It is observed that this adds latency of ~45msec for each > update in the query execution path. This gets even worse when high number of > concurrent queries are in progress. For concurrency=100, the average query > response time even for short queries is 8 sec vs 0.2 sec with these updates > disabled. For short lived queries in a high-throughput scenario, it is of no > value to update state changes in zookeeper. We need an option to disable > these updates for short running operational queries. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (DRILL-5287) Provide option to skip updates of ephemeral state changes in Zookeeper
[ https://issues.apache.org/jira/browse/DRILL-5287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Padma Penumarthy updated DRILL-5287: Labels: doc-impacting ready-to-commit (was: ready-to-commit) > Provide option to skip updates of ephemeral state changes in Zookeeper > -- > > Key: DRILL-5287 > URL: https://issues.apache.org/jira/browse/DRILL-5287 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.9.0 >Reporter: Padma Penumarthy >Assignee: Padma Penumarthy > Labels: doc-impacting, ready-to-commit > Fix For: 1.10.0 > > > We put transient profiles in zookeeper and update state as query progresses > and changes states. It is observed that this adds latency of ~45msec for each > update in the query execution path. This gets even worse when high number of > concurrent queries are in progress. For concurrency=100, the average query > response time even for short queries is 8 sec vs 0.2 sec with these updates > disabled. For short lived queries in a high-throughput scenario, it is of no > value to update state changes in zookeeper. We need an option to disable > these updates for short running operational queries. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (DRILL-5287) Provide option to skip updates of ephemeral state changes in Zookeeper
[ https://issues.apache.org/jira/browse/DRILL-5287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Padma Penumarthy updated DRILL-5287: Labels: ready-to-commit (was: ) > Provide option to skip updates of ephemeral state changes in Zookeeper > -- > > Key: DRILL-5287 > URL: https://issues.apache.org/jira/browse/DRILL-5287 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.9.0 >Reporter: Padma Penumarthy >Assignee: Padma Penumarthy > Labels: doc-impacting, ready-to-commit > Fix For: 1.10.0 > > > We put transient profiles in zookeeper and update state as query progresses > and changes states. It is observed that this adds latency of ~45msec for each > update in the query execution path. This gets even worse when high number of > concurrent queries are in progress. For concurrency=100, the average query > response time even for short queries is 8 sec vs 0.2 sec with these updates > disabled. For short lived queries in a high-throughput scenario, it is of no > value to update state changes in zookeeper. We need an option to disable > these updates for short running operational queries. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (DRILL-5308) Asking an sv2 for its buffer clears the sv2
Paul Rogers created DRILL-5308:
--
Summary: Asking an sv2 for its buffer clears the sv2
Key: DRILL-5308
URL: https://issues.apache.org/jira/browse/DRILL-5308
Project: Apache Drill
Issue Type: Bug
Affects Versions: 1.9.0
Reporter: Paul Rogers
Consider the following code:
{code}
SelectionVector2 sv2 = va.getSelectionVector2();
sv2Size = sv2.getBuffer().capacity();
{code}
When similar code is called for a value vector, we simply get the buffer
capacity. When called for an sv2, we get the capacity *after* clearing the sv2!
The code in question:
{code}
public DrillBuf getBuffer() {
return getBuffer(true);
}
public DrillBuf getBuffer(boolean clear) {
DrillBuf bufferHandle = this.buffer;
if (clear) {
/* Increment the ref count for this buffer */
bufferHandle.retain(1);
/* We are passing ownership of the buffer to the
* caller. clear the buffer from within our selection vector
*/
clear();
}
return bufferHandle;
}
{code}
That is, for a value vector, {{getBuffer()}} simply gets the buffer. For an
sv2, {{getBuffer()}} clears the buffer, then returns it.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (DRILL-5303) Drillbits fail to start when Drill server built with JDK 8 is deployed on a JDK 7 environment
[
https://issues.apache.org/jira/browse/DRILL-5303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15890875#comment-15890875
]
Laurent Goujon commented on DRILL-5303:
---
Java does not provide such guarantee as compiling against version N+1 jdk (with
source and target class using version N) and being able to run on version N:
you need the three of them (bootclasspath, source and target) to match
versions. And this kind of issues happened with all Java releases.
Coming with Java9 I believe, symbols for previous jdks will also be available
meaning you would not need java8 (just for the boot classpath) in order to
target jdk8.
> Drillbits fail to start when Drill server built with JDK 8 is deployed on a
> JDK 7 environment
> -
>
> Key: DRILL-5303
> URL: https://issues.apache.org/jira/browse/DRILL-5303
> Project: Apache Drill
> Issue Type: Bug
> Components: Server, Tools, Build & Test
>Affects Versions: 1.9.0
>Reporter: Abhishek Girish
>
> When Drill is built on a node configured with JDK 8 and is then deployed in a
> JDK 7 environment, Drillbits fail to start and the following errors are seen
> in Drillbit.out:
> {code}
> Exception in thread "main" java.lang.NoSuchMethodError:
> java.util.concurrent.ConcurrentHashMap.keySet()Ljava/util/concurrent/ConcurrentHashMap$KeySetView;
> at
> org.apache.drill.exec.coord.ClusterCoordinator.drillbitRegistered(ClusterCoordinator.java:85)
> at
> org.apache.drill.exec.coord.zk.ZKClusterCoordinator.updateEndpoints(ZKClusterCoordinator.java:266)
> at
> org.apache.drill.exec.coord.zk.ZKClusterCoordinator.start(ZKClusterCoordinator.java:135)
> at org.apache.drill.exec.server.Drillbit.run(Drillbit.java:117)
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:292)
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:272)
> at org.apache.drill.exec.server.Drillbit.main(Drillbit.java:268)
> {code}
> Workaround is to match the Java versions of build and deployment environments.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (DRILL-4708) connection closed unexpectedly
[
https://issues.apache.org/jira/browse/DRILL-4708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Abhishek Girish updated DRILL-4708:
---
Priority: Critical (was: Major)
> connection closed unexpectedly
> --
>
> Key: DRILL-4708
> URL: https://issues.apache.org/jira/browse/DRILL-4708
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - RPC
>Affects Versions: 1.7.0
>Reporter: Chun Chang
>Priority: Critical
>
> Running DRILL functional automation, we often see query failed randomly due
> to the following unexpected connection close error.
> {noformat}
> Execution Failures:
> /root/drillAutomation/framework/framework/resources/Functional/ctas/ctas_flatten/10rows/filter5.q
> Query:
> select * from dfs.ctas_flatten.`filter5_10rows_ctas`
> Failed with exception
> java.sql.SQLException: CONNECTION ERROR: Connection /10.10.100.171:36185 <-->
> drillats4.qa.lab/10.10.100.174:31010 (user client) closed unexpectedly.
> Drillbit down?
> [Error Id: 3d5dad8e-80d0-4c7f-9012-013bf01ce2b7 ]
> at
> org.apache.drill.jdbc.impl.DrillCursor.nextRowInternally(DrillCursor.java:247)
> at org.apache.drill.jdbc.impl.DrillCursor.next(DrillCursor.java:321)
> at
> oadd.net.hydromatic.avatica.AvaticaResultSet.next(AvaticaResultSet.java:187)
> at
> org.apache.drill.jdbc.impl.DrillResultSetImpl.next(DrillResultSetImpl.java:172)
> at
> org.apache.drill.test.framework.DrillTestJdbc.executeQuery(DrillTestJdbc.java:210)
> at
> org.apache.drill.test.framework.DrillTestJdbc.run(DrillTestJdbc.java:99)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> Caused by: oadd.org.apache.drill.common.exceptions.UserException: CONNECTION
> ERROR: Connection /10.10.100.171:36185 <-->
> drillats4.qa.lab/10.10.100.174:31010 (user client) closed unexpectedly.
> Drillbit down?
> [Error Id: 3d5dad8e-80d0-4c7f-9012-013bf01ce2b7 ]
> at
> oadd.org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:543)
> at
> oadd.org.apache.drill.exec.rpc.user.QueryResultHandler$ChannelClosedHandler$1.operationComplete(QueryResultHandler.java:373)
> at
> oadd.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:680)
> at
> oadd.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:603)
> at
> oadd.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:563)
> at
> oadd.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:406)
> at
> oadd.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:82)
> at
> oadd.io.netty.channel.AbstractChannel$CloseFuture.setClosed(AbstractChannel.java:943)
> at
> oadd.io.netty.channel.AbstractChannel$AbstractUnsafe.doClose0(AbstractChannel.java:592)
> at
> oadd.io.netty.channel.AbstractChannel$AbstractUnsafe.close(AbstractChannel.java:584)
> at
> oadd.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.closeOnRead(AbstractNioByteChannel.java:71)
> at
> oadd.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.handleReadException(AbstractNioByteChannel.java:89)
> at
> oadd.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:162)
> at
> oadd.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)
> at
> oadd.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
> at
> oadd.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
> at oadd.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
> at
> oadd.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
> ... 1 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (DRILL-4994) Prepared statement stopped working between 1.8.0 client and < 1.8.0 server
[ https://issues.apache.org/jira/browse/DRILL-4994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15890744#comment-15890744 ] Laurent Goujon commented on DRILL-4994: --- Adding ready-to-commit label as it was also added to DRILL-4730 which is part of the same pull request > Prepared statement stopped working between 1.8.0 client and < 1.8.0 server > -- > > Key: DRILL-4994 > URL: https://issues.apache.org/jira/browse/DRILL-4994 > Project: Apache Drill > Issue Type: Bug >Reporter: Laurent Goujon >Assignee: Laurent Goujon > Labels: ready-to-commit > > Older servers (pre-1.8.0) don't support the prepared statement rpc method, > but the JDBC client doesn't check if it is available or not. The end result > is that the statement is stuck as the server is not responding back. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (DRILL-4994) Prepared statement stopped working between 1.8.0 client and < 1.8.0 server
[ https://issues.apache.org/jira/browse/DRILL-4994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laurent Goujon updated DRILL-4994: -- Labels: ready-to-commit (was: ) > Prepared statement stopped working between 1.8.0 client and < 1.8.0 server > -- > > Key: DRILL-4994 > URL: https://issues.apache.org/jira/browse/DRILL-4994 > Project: Apache Drill > Issue Type: Bug >Reporter: Laurent Goujon >Assignee: Laurent Goujon > Labels: ready-to-commit > > Older servers (pre-1.8.0) don't support the prepared statement rpc method, > but the JDBC client doesn't check if it is available or not. The end result > is that the statement is stuck as the server is not responding back. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (DRILL-5301) Add server metadata API
[ https://issues.apache.org/jira/browse/DRILL-5301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15890210#comment-15890210 ] ASF GitHub Bot commented on DRILL-5301: --- Github user laurentgo commented on the issue: https://github.com/apache/drill/pull/764 yes, pull requests have been kept in sync > Add server metadata API > --- > > Key: DRILL-5301 > URL: https://issues.apache.org/jira/browse/DRILL-5301 > Project: Apache Drill > Issue Type: Improvement > Components: Server, Client - C++, Client - Java, Client - JDBC, > Client - ODBC >Reporter: Laurent Goujon >Assignee: Laurent Goujon > Labels: ready-to-commit > > JDBC and ODBC clients exposes lots of metadata regarding server version and > support of various parts of the SQL standard. > Currently the returned information is hardcoded in both clients/drivers which > means that the infomation returned is support as of the client version, not > the server version. > Instead, a new method should be provided to the clients to query the actual > server support. Support on the client or the server should be optional (for > example a client should not use this API if the server doesn't support it and > fallback to default values). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (DRILL-4730) Update JDBC DatabaseMetaData implementation to use new Metadata APIs
[ https://issues.apache.org/jira/browse/DRILL-4730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15890208#comment-15890208 ] ASF GitHub Bot commented on DRILL-4730: --- Github user laurentgo commented on the issue: https://github.com/apache/drill/pull/613 yes, pull requests have been kept in sync. > Update JDBC DatabaseMetaData implementation to use new Metadata APIs > > > Key: DRILL-4730 > URL: https://issues.apache.org/jira/browse/DRILL-4730 > Project: Apache Drill > Issue Type: Sub-task > Components: Client - JDBC >Reporter: Venki Korukanti >Assignee: Laurent Goujon > Labels: ready-to-commit > > DRILL-4728 is going to add support for new metadata APIs. Replace the > INFORMATION_SCHEMA queries used to get the metadata with the new APIs > provided in Java client. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (DRILL-3510) Add ANSI_QUOTES option so that Drill's SQL Parser will recognize ANSI_SQL identifiers
[
https://issues.apache.org/jira/browse/DRILL-3510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15890078#comment-15890078
]
ASF GitHub Bot commented on DRILL-3510:
---
Github user vdiravka commented on a diff in the pull request:
https://github.com/apache/drill/pull/520#discussion_r9981
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/server/options/TypeValidators.java
---
@@ -204,10 +204,11 @@ public void validate(final OptionValue v, final
OptionManager manager) {
* Validator that checks if the given value is included in a list of
acceptable values. Case insensitive.
*/
public static class EnumeratedStringValidator extends StringValidator {
-private final Set valuesSet = new HashSet<>();
+private final Set valuesSet = new LinkedHashSet<>();
public EnumeratedStringValidator(String name, String def, String...
values) {
super(name, def);
+ valuesSet.add(def.toLowerCase());
--- End diff --
For example
[PARQUET_WRITER_COMPRESSION_TYPE_VALIDATOR](https://github.com/apache/drill/blob/ddcf89548bd33c0cd3e062f1f6d5027eed822372/exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java#L145)
uses mentioned approach - `snappy` value is present twice.
Adding of the default value to the set will allow do not duplicate this
value in the set of acceptable values.
Using of LinkedHashSet will allow to know that the default value is a first
value in the set.
> Add ANSI_QUOTES option so that Drill's SQL Parser will recognize ANSI_SQL
> identifiers
> --
>
> Key: DRILL-3510
> URL: https://issues.apache.org/jira/browse/DRILL-3510
> Project: Apache Drill
> Issue Type: Improvement
> Components: SQL Parser
>Reporter: Jinfeng Ni
>Assignee: Vitalii Diravka
> Labels: doc-impacting
> Fix For: 1.10.0
>
> Attachments: DRILL-3510.patch, DRILL-3510.patch
>
>
> Currently Drill's SQL parser uses backtick as identifier quotes, the same as
> what MySQL does. However, this is different from ANSI SQL specification,
> where double quote is used as identifier quotes.
> MySQL has an option "ANSI_QUOTES", which could be switched on/off by user.
> Drill should follow the same way, so that Drill users do not have to rewrite
> their existing queries, if their queries use double quotes.
> {code}
> SET sql_mode='ANSI_QUOTES';
> {code}
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (DRILL-3510) Add ANSI_QUOTES option so that Drill's SQL Parser will recognize ANSI_SQL identifiers
[
https://issues.apache.org/jira/browse/DRILL-3510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15890082#comment-15890082
]
ASF GitHub Bot commented on DRILL-3510:
---
Github user vdiravka commented on a diff in the pull request:
https://github.com/apache/drill/pull/520#discussion_r99823941
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/SqlConverter.java
---
@@ -161,6 +161,19 @@ public SqlNode parse(String sql) {
SqlParser parser = SqlParser.create(sql, parserConfig);
return parser.parseStmt();
} catch (SqlParseException e) {
+
+ // Attempt to use default back_tick quote character for identifiers,
when QUOTING_IDENTIFIERS_CHARACTER
--- End diff --
Yes, it was desirable behaviour:
[DRILL-3510#comment_1](https://issues.apache.org/jira/browse/DRILL-3510?focusedCommentId=14645147&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14645147)
[DRILL-3510#comment_2](https://issues.apache.org/jira/browse/DRILL-3510?focusedCommentId=14943769&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14943769)
Since calcite doesn't support parsing statement with two different
characters the only way is this approach. But I agree that it isn't an elegant
decision. Therefore I plan to make as you suggested - to return the appropriate
error.
> Add ANSI_QUOTES option so that Drill's SQL Parser will recognize ANSI_SQL
> identifiers
> --
>
> Key: DRILL-3510
> URL: https://issues.apache.org/jira/browse/DRILL-3510
> Project: Apache Drill
> Issue Type: Improvement
> Components: SQL Parser
>Reporter: Jinfeng Ni
>Assignee: Vitalii Diravka
> Labels: doc-impacting
> Fix For: 1.10.0
>
> Attachments: DRILL-3510.patch, DRILL-3510.patch
>
>
> Currently Drill's SQL parser uses backtick as identifier quotes, the same as
> what MySQL does. However, this is different from ANSI SQL specification,
> where double quote is used as identifier quotes.
> MySQL has an option "ANSI_QUOTES", which could be switched on/off by user.
> Drill should follow the same way, so that Drill users do not have to rewrite
> their existing queries, if their queries use double quotes.
> {code}
> SET sql_mode='ANSI_QUOTES';
> {code}
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (DRILL-3510) Add ANSI_QUOTES option so that Drill's SQL Parser will recognize ANSI_SQL identifiers
[
https://issues.apache.org/jira/browse/DRILL-3510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15890079#comment-15890079
]
ASF GitHub Bot commented on DRILL-3510:
---
Github user vdiravka commented on a diff in the pull request:
https://github.com/apache/drill/pull/520#discussion_r99824747
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/rpc/user/UserSession.java ---
@@ -59,9 +62,11 @@
public static final String USER = "user";
public static final String PASSWORD = "password";
public static final String IMPERSONATION_TARGET = "impersonation_target";
+ public static final String QUOTING_IDENTIFIERS_CHARACTER =
"quoting_identifiers_character";
--- End diff --
Agree.
I suppose `QUOTING` is not enough. I think `QUOTING_IDENTIFIERS` should be
good.
> Add ANSI_QUOTES option so that Drill's SQL Parser will recognize ANSI_SQL
> identifiers
> --
>
> Key: DRILL-3510
> URL: https://issues.apache.org/jira/browse/DRILL-3510
> Project: Apache Drill
> Issue Type: Improvement
> Components: SQL Parser
>Reporter: Jinfeng Ni
>Assignee: Vitalii Diravka
> Labels: doc-impacting
> Fix For: 1.10.0
>
> Attachments: DRILL-3510.patch, DRILL-3510.patch
>
>
> Currently Drill's SQL parser uses backtick as identifier quotes, the same as
> what MySQL does. However, this is different from ANSI SQL specification,
> where double quote is used as identifier quotes.
> MySQL has an option "ANSI_QUOTES", which could be switched on/off by user.
> Drill should follow the same way, so that Drill users do not have to rewrite
> their existing queries, if their queries use double quotes.
> {code}
> SET sql_mode='ANSI_QUOTES';
> {code}
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (DRILL-3510) Add ANSI_QUOTES option so that Drill's SQL Parser will recognize ANSI_SQL identifiers
[
https://issues.apache.org/jira/browse/DRILL-3510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15890081#comment-15890081
]
ASF GitHub Bot commented on DRILL-3510:
---
Github user vdiravka commented on a diff in the pull request:
https://github.com/apache/drill/pull/520#discussion_r99830247
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/server/options/TypeValidators.java
---
@@ -204,10 +204,11 @@ public void validate(final OptionValue v, final
OptionManager manager) {
* Validator that checks if the given value is included in a list of
acceptable values. Case insensitive.
*/
public static class EnumeratedStringValidator extends StringValidator {
-private final Set valuesSet = new HashSet<>();
+private final Set valuesSet = new LinkedHashSet<>();
--- End diff --
To show default value first. For example in the output of exception
(TypeValidators.java[#L221](https://github.com/apache/drill/blob/db48298920575cb1c2283e03bdfc7b50e83ae217/exec/java-exec/src/main/java/org/apache/drill/exec/server/options/TypeValidators.java#L221)).
Also I'll replace it with guava one.
> Add ANSI_QUOTES option so that Drill's SQL Parser will recognize ANSI_SQL
> identifiers
> --
>
> Key: DRILL-3510
> URL: https://issues.apache.org/jira/browse/DRILL-3510
> Project: Apache Drill
> Issue Type: Improvement
> Components: SQL Parser
>Reporter: Jinfeng Ni
>Assignee: Vitalii Diravka
> Labels: doc-impacting
> Fix For: 1.10.0
>
> Attachments: DRILL-3510.patch, DRILL-3510.patch
>
>
> Currently Drill's SQL parser uses backtick as identifier quotes, the same as
> what MySQL does. However, this is different from ANSI SQL specification,
> where double quote is used as identifier quotes.
> MySQL has an option "ANSI_QUOTES", which could be switched on/off by user.
> Drill should follow the same way, so that Drill users do not have to rewrite
> their existing queries, if their queries use double quotes.
> {code}
> SET sql_mode='ANSI_QUOTES';
> {code}
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (DRILL-3510) Add ANSI_QUOTES option so that Drill's SQL Parser will recognize ANSI_SQL identifiers
[
https://issues.apache.org/jira/browse/DRILL-3510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15890080#comment-15890080
]
ASF GitHub Bot commented on DRILL-3510:
---
Github user vdiravka commented on a diff in the pull request:
https://github.com/apache/drill/pull/520#discussion_r103463608
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/client/DrillClient.java ---
@@ -695,6 +698,33 @@ public void runQuery(QueryType type,
List planFragments, UserResul
}
/**
+ * Get server properties that represent the list of server session
options.
+ *
+ * @return server properties for the server session options.
+ */
+ public ServerProperties getOptions() throws RpcException {
--- End diff --
Agree. So I'll use the logic from your PR after it'll be merged into the
master.
> Add ANSI_QUOTES option so that Drill's SQL Parser will recognize ANSI_SQL
> identifiers
> --
>
> Key: DRILL-3510
> URL: https://issues.apache.org/jira/browse/DRILL-3510
> Project: Apache Drill
> Issue Type: Improvement
> Components: SQL Parser
>Reporter: Jinfeng Ni
>Assignee: Vitalii Diravka
> Labels: doc-impacting
> Fix For: 1.10.0
>
> Attachments: DRILL-3510.patch, DRILL-3510.patch
>
>
> Currently Drill's SQL parser uses backtick as identifier quotes, the same as
> what MySQL does. However, this is different from ANSI SQL specification,
> where double quote is used as identifier quotes.
> MySQL has an option "ANSI_QUOTES", which could be switched on/off by user.
> Drill should follow the same way, so that Drill users do not have to rewrite
> their existing queries, if their queries use double quotes.
> {code}
> SET sql_mode='ANSI_QUOTES';
> {code}
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (DRILL-3510) Add ANSI_QUOTES option so that Drill's SQL Parser will recognize ANSI_SQL identifiers
[
https://issues.apache.org/jira/browse/DRILL-3510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15890083#comment-15890083
]
ASF GitHub Bot commented on DRILL-3510:
---
Github user vdiravka commented on a diff in the pull request:
https://github.com/apache/drill/pull/520#discussion_r103672254
--- Diff:
exec/jdbc/src/main/java/org/apache/drill/jdbc/impl/DrillDatabaseMetaDataImpl.java
---
@@ -231,11 +274,16 @@ public boolean storesMixedCaseQuotedIdentifiers()
throws SQLException {
return super.storesMixedCaseQuotedIdentifiers();
}
- // TODO(DRILL-3510): Update when Drill accepts standard SQL's double
quote.
@Override
public String getIdentifierQuoteString() throws SQLException {
throwIfClosed();
-return "`";
+Property property =
getServerProperty(PlannerSettings.QUOTING_IDENTIFIERS_CHARACTER_KEY);
+for (Quoting value : Quoting.values()) {
+ if (value.string.equals(property.getValue())) {
+return value.string;
+ }
+}
+throw new SQLException("Unknown quoting identifier character " +
property.getValue());
--- End diff --
Agree. I will add it after merging your's PRs (#613 and #764) to use logic
from there.
> Add ANSI_QUOTES option so that Drill's SQL Parser will recognize ANSI_SQL
> identifiers
> --
>
> Key: DRILL-3510
> URL: https://issues.apache.org/jira/browse/DRILL-3510
> Project: Apache Drill
> Issue Type: Improvement
> Components: SQL Parser
>Reporter: Jinfeng Ni
>Assignee: Vitalii Diravka
> Labels: doc-impacting
> Fix For: 1.10.0
>
> Attachments: DRILL-3510.patch, DRILL-3510.patch
>
>
> Currently Drill's SQL parser uses backtick as identifier quotes, the same as
> what MySQL does. However, this is different from ANSI SQL specification,
> where double quote is used as identifier quotes.
> MySQL has an option "ANSI_QUOTES", which could be switched on/off by user.
> Drill should follow the same way, so that Drill users do not have to rewrite
> their existing queries, if their queries use double quotes.
> {code}
> SET sql_mode='ANSI_QUOTES';
> {code}
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (DRILL-5307) Drill sometimes stops to recognize dfs directories
mehran created DRILL-5307: - Summary: Drill sometimes stops to recognize dfs directories Key: DRILL-5307 URL: https://issues.apache.org/jira/browse/DRILL-5307 Project: Apache Drill Issue Type: Bug Components: SQL Parser Affects Versions: 1.9.0 Reporter: mehran Priority: Critical Drill stops to recognize dfs directories. It happens in random manner. this is the error. >From line 1, column 21 to line 1, column 23: Table 'dfs.root./rep/01/2017' not >found SQL Query null [Error Id: 62c4f977-49d4-4bc5-8d61-9b3a1fdf2b47 on >cent:31010] The query is as follwows. SELECT MAX(ID) FROM dfs.root.`/rep/01/2017` I want to know root cause of the error. Best regards -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (DRILL-5221) cancel message is delayed until queryid or data is received
[ https://issues.apache.org/jira/browse/DRILL-5221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sudheesh Katkam updated DRILL-5221: --- Labels: ready-to-commit (was: ) > cancel message is delayed until queryid or data is received > --- > > Key: DRILL-5221 > URL: https://issues.apache.org/jira/browse/DRILL-5221 > Project: Apache Drill > Issue Type: Improvement > Components: Client - C++ >Affects Versions: 1.9.0 >Reporter: Laurent Goujon >Assignee: Laurent Goujon > Labels: ready-to-commit > > When user is calling the cancel method of the C++ client, the client wait for > a message from the server to reply back with a cancellation message. > In case of queries taking a long time to return batch results, it means > cancellation won't be effective until the next batch is received, instead of > cancelling right away the query (assuming the query id has already been > received, which is generally the case). > It seems this was foreseen by [~vkorukanti] in his initial patch > (https://github.com/vkorukanti/drill/commit/e0ef6349aac48de5828b6d725c2cf013905d18eb) > but was omitted when I backported it post metadata changes. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (DRILL-5167) C++ connector does not set escape string for metadata search pattern
[ https://issues.apache.org/jira/browse/DRILL-5167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sudheesh Katkam updated DRILL-5167: --- Labels: ready-to-commit (was: ) > C++ connector does not set escape string for metadata search pattern > > > Key: DRILL-5167 > URL: https://issues.apache.org/jira/browse/DRILL-5167 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.9.0 >Reporter: Laurent Goujon >Assignee: Laurent Goujon >Priority: Minor > Labels: ready-to-commit > > C++ connector does not set the escape string for search pattern when doing > metadata operation (getCatalogs/getSchema/getTables/getColumns). It is > assumed to be '\\' as returned by DrillMetadata::getSearchEscapeString(), but > because this is not sent over the wire, the server will actually consider > that there's no escape character, and might return different or no result > compared to what has been requested. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (DRILL-5301) Add server metadata API
[ https://issues.apache.org/jira/browse/DRILL-5301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sudheesh Katkam updated DRILL-5301: --- Labels: ready-to-commit (was: ) > Add server metadata API > --- > > Key: DRILL-5301 > URL: https://issues.apache.org/jira/browse/DRILL-5301 > Project: Apache Drill > Issue Type: Improvement > Components: Server, Client - C++, Client - Java, Client - JDBC, > Client - ODBC >Reporter: Laurent Goujon >Assignee: Laurent Goujon > Labels: ready-to-commit > > JDBC and ODBC clients exposes lots of metadata regarding server version and > support of various parts of the SQL standard. > Currently the returned information is hardcoded in both clients/drivers which > means that the infomation returned is support as of the client version, not > the server version. > Instead, a new method should be provided to the clients to query the actual > server support. Support on the client or the server should be optional (for > example a client should not use this API if the server doesn't support it and > fallback to default values). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (DRILL-4730) Update JDBC DatabaseMetaData implementation to use new Metadata APIs
[ https://issues.apache.org/jira/browse/DRILL-4730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sudheesh Katkam updated DRILL-4730: --- Labels: ready-to-commit (was: ) > Update JDBC DatabaseMetaData implementation to use new Metadata APIs > > > Key: DRILL-4730 > URL: https://issues.apache.org/jira/browse/DRILL-4730 > Project: Apache Drill > Issue Type: Sub-task > Components: Client - JDBC >Reporter: Venki Korukanti >Assignee: Laurent Goujon > Labels: ready-to-commit > > DRILL-4728 is going to add support for new metadata APIs. Replace the > INFORMATION_SCHEMA queries used to get the metadata with the new APIs > provided in Java client. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (DRILL-5301) Add server metadata API
[ https://issues.apache.org/jira/browse/DRILL-5301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15889756#comment-15889756 ] ASF GitHub Bot commented on DRILL-5301: --- Github user sudheeshkatkam commented on the issue: https://github.com/apache/drill/pull/764 +1 Is this PR inclusive of comments in #613 (for DRILL-4994, DRILL-4730)? > Add server metadata API > --- > > Key: DRILL-5301 > URL: https://issues.apache.org/jira/browse/DRILL-5301 > Project: Apache Drill > Issue Type: Improvement > Components: Server, Client - C++, Client - Java, Client - JDBC, > Client - ODBC >Reporter: Laurent Goujon >Assignee: Laurent Goujon > > JDBC and ODBC clients exposes lots of metadata regarding server version and > support of various parts of the SQL standard. > Currently the returned information is hardcoded in both clients/drivers which > means that the infomation returned is support as of the client version, not > the server version. > Instead, a new method should be provided to the clients to query the actual > server support. Support on the client or the server should be optional (for > example a client should not use this API if the server doesn't support it and > fallback to default values). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
