[jira] [Commented] (DRILL-4946) org.objectweb.asm.tree.analysis.AnalyzerException printed to console in embedded mode
[
https://issues.apache.org/jira/browse/DRILL-4946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1558#comment-1558
]
Chunhui Shi commented on DRILL-4946:
[~jacq...@dremio.com] Thanks. We may want to investigate more on this scalar
replacement later. What worries me is, if in some cases it generates 'invalid'
byte code sequence and we can catch with an exception, in some other cases, it
may generate 'incorrect' byte code but we don't have any exception raised.
> org.objectweb.asm.tree.analysis.AnalyzerException printed to console in
> embedded mode
> -
>
> Key: DRILL-4946
> URL: https://issues.apache.org/jira/browse/DRILL-4946
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Chunhui Shi
>Assignee: Chunhui Shi
>Priority: Critical
>
> Testing by querying a json file got AnalyzerException printed.
> The problem was due to scalar_replacement mode is default to be 'try', and
> org.objectweb.asm.util.CheckMethodAdapter is printing stack trace to stderr.
> [shi@cshi-centos1 private-drill]$ cat /tmp/conv.json
> {"row": "0", "key": "\\x4a\\x31\\x39\\x38", "key2": "4a313938", "kp1":
> "4a31", "kp2": "38"}
> {"row": "1", "key": null, "key2": null, "kp1": null, "kp2": null}
> {"row": "2", "key": "\\x4e\\x4f\\x39\\x51", "key2": "4e4f3951", "kp1":
> "4e4f", "kp2": "51"}
> {"row": "3", "key": "\\x6e\\x6f\\x39\\x31", "key2": "6e6f3931", "kp1":
> "6e6f", "kp2": "31"}
> 0: jdbc:drill:zk=local> SELECT convert_from(binary_string(key), 'INT_BE') as
> intkey from dfs.`/tmp/conv.json`;
> org.objectweb.asm.tree.analysis.AnalyzerException: Error at instruction 158:
> Expected an object reference, but found .
> at org.objectweb.asm.tree.analysis.Analyzer.analyze(Analyzer.java:294)
> at
> org.objectweb.asm.util.CheckMethodAdapter$1.visitEnd(CheckMethodAdapter.java:450)
> at org.objectweb.asm.MethodVisitor.visitEnd(MethodVisitor.java:877)
> at
> org.objectweb.asm.util.CheckMethodAdapter.visitEnd(CheckMethodAdapter.java:1028)
> at org.objectweb.asm.MethodVisitor.visitEnd(MethodVisitor.java:877)
> at
> org.apache.drill.exec.compile.CheckMethodVisitorFsm.visitEnd(CheckMethodVisitorFsm.java:114)
> at org.objectweb.asm.MethodVisitor.visitEnd(MethodVisitor.java:877)
> at
> org.apache.drill.exec.compile.CheckMethodVisitorFsm.visitEnd(CheckMethodVisitorFsm.java:114)
> at org.objectweb.asm.MethodVisitor.visitEnd(MethodVisitor.java:877)
> at org.objectweb.asm.MethodVisitor.visitEnd(MethodVisitor.java:877)
> at
> org.apache.drill.exec.compile.bytecode.InstructionModifier.visitEnd(InstructionModifier.java:508)
> at org.objectweb.asm.tree.MethodNode.accept(MethodNode.java:837)
> at
> org.apache.drill.exec.compile.bytecode.ScalarReplacementNode.visitEnd(ScalarReplacementNode.java:87)
> at org.objectweb.asm.MethodVisitor.visitEnd(MethodVisitor.java:877)
> at
> org.apache.drill.exec.compile.bytecode.AloadPopRemover.visitEnd(AloadPopRemover.java:136)
> at org.objectweb.asm.tree.MethodNode.accept(MethodNode.java:837)
> at org.objectweb.asm.tree.MethodNode.accept(MethodNode.java:726)
> at org.objectweb.asm.tree.ClassNode.accept(ClassNode.java:412)
> at
> org.apache.drill.exec.compile.MergeAdapter.getMergedClass(MergeAdapter.java:223)
> at
> org.apache.drill.exec.compile.ClassTransformer.getImplementationClass(ClassTransformer.java:263)
> at
> org.apache.drill.exec.compile.CodeCompiler$Loader.load(CodeCompiler.java:78)
> at
> org.apache.drill.exec.compile.CodeCompiler$Loader.load(CodeCompiler.java:74)
> at
> com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3527)
> at
> com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2319)
> at
> com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2282)
> at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2197)
> at com.google.common.cache.LocalCache.get(LocalCache.java:3937)
> at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3941)
> at
> com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4824)
> at
> org.apache.drill.exec.compile.CodeCompiler.getImplementationClass(CodeCompiler.java:63)
> at
> org.apache.drill.exec.compile.CodeCompiler.getImplementationClass(CodeCompiler.java:56)
> at
> org.apache.drill.exec.ops.FragmentContext.getImplementationClass(FragmentContext.java:310)
> at
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.setupNewSchema(ProjectRecordBatch.java:484)
> at
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:7
[jira] [Commented] (DRILL-4946) org.objectweb.asm.tree.analysis.AnalyzerException printed to console in embedded mode
[
https://issues.apache.org/jira/browse/DRILL-4946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15581070#comment-15581070
]
Jacques Nadeau commented on DRILL-4946:
---
For reference: the scalar replacement code is known to not work in some
situations. In those situations, in production, we fallback to skipping it
since it is simply a performance optimization. In development tests we fail
queries so we don't accidentally introduce scalar replacement regressions.
> org.objectweb.asm.tree.analysis.AnalyzerException printed to console in
> embedded mode
> -
>
> Key: DRILL-4946
> URL: https://issues.apache.org/jira/browse/DRILL-4946
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Chunhui Shi
>Assignee: Chunhui Shi
>Priority: Critical
>
> Testing by querying a json file got AnalyzerException printed.
> The problem was due to scalar_replacement mode is default to be 'try', and
> org.objectweb.asm.util.CheckMethodAdapter is printing stack trace to stderr.
> [shi@cshi-centos1 private-drill]$ cat /tmp/conv.json
> {"row": "0", "key": "\\x4a\\x31\\x39\\x38", "key2": "4a313938", "kp1":
> "4a31", "kp2": "38"}
> {"row": "1", "key": null, "key2": null, "kp1": null, "kp2": null}
> {"row": "2", "key": "\\x4e\\x4f\\x39\\x51", "key2": "4e4f3951", "kp1":
> "4e4f", "kp2": "51"}
> {"row": "3", "key": "\\x6e\\x6f\\x39\\x31", "key2": "6e6f3931", "kp1":
> "6e6f", "kp2": "31"}
> 0: jdbc:drill:zk=local> SELECT convert_from(binary_string(key), 'INT_BE') as
> intkey from dfs.`/tmp/conv.json`;
> org.objectweb.asm.tree.analysis.AnalyzerException: Error at instruction 158:
> Expected an object reference, but found .
> at org.objectweb.asm.tree.analysis.Analyzer.analyze(Analyzer.java:294)
> at
> org.objectweb.asm.util.CheckMethodAdapter$1.visitEnd(CheckMethodAdapter.java:450)
> at org.objectweb.asm.MethodVisitor.visitEnd(MethodVisitor.java:877)
> at
> org.objectweb.asm.util.CheckMethodAdapter.visitEnd(CheckMethodAdapter.java:1028)
> at org.objectweb.asm.MethodVisitor.visitEnd(MethodVisitor.java:877)
> at
> org.apache.drill.exec.compile.CheckMethodVisitorFsm.visitEnd(CheckMethodVisitorFsm.java:114)
> at org.objectweb.asm.MethodVisitor.visitEnd(MethodVisitor.java:877)
> at
> org.apache.drill.exec.compile.CheckMethodVisitorFsm.visitEnd(CheckMethodVisitorFsm.java:114)
> at org.objectweb.asm.MethodVisitor.visitEnd(MethodVisitor.java:877)
> at org.objectweb.asm.MethodVisitor.visitEnd(MethodVisitor.java:877)
> at
> org.apache.drill.exec.compile.bytecode.InstructionModifier.visitEnd(InstructionModifier.java:508)
> at org.objectweb.asm.tree.MethodNode.accept(MethodNode.java:837)
> at
> org.apache.drill.exec.compile.bytecode.ScalarReplacementNode.visitEnd(ScalarReplacementNode.java:87)
> at org.objectweb.asm.MethodVisitor.visitEnd(MethodVisitor.java:877)
> at
> org.apache.drill.exec.compile.bytecode.AloadPopRemover.visitEnd(AloadPopRemover.java:136)
> at org.objectweb.asm.tree.MethodNode.accept(MethodNode.java:837)
> at org.objectweb.asm.tree.MethodNode.accept(MethodNode.java:726)
> at org.objectweb.asm.tree.ClassNode.accept(ClassNode.java:412)
> at
> org.apache.drill.exec.compile.MergeAdapter.getMergedClass(MergeAdapter.java:223)
> at
> org.apache.drill.exec.compile.ClassTransformer.getImplementationClass(ClassTransformer.java:263)
> at
> org.apache.drill.exec.compile.CodeCompiler$Loader.load(CodeCompiler.java:78)
> at
> org.apache.drill.exec.compile.CodeCompiler$Loader.load(CodeCompiler.java:74)
> at
> com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3527)
> at
> com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2319)
> at
> com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2282)
> at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2197)
> at com.google.common.cache.LocalCache.get(LocalCache.java:3937)
> at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3941)
> at
> com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4824)
> at
> org.apache.drill.exec.compile.CodeCompiler.getImplementationClass(CodeCompiler.java:63)
> at
> org.apache.drill.exec.compile.CodeCompiler.getImplementationClass(CodeCompiler.java:56)
> at
> org.apache.drill.exec.ops.FragmentContext.getImplementationClass(FragmentContext.java:310)
> at
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.setupNewSchema(ProjectRecordBatch.java:484)
> at
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:78)
>
[jira] [Commented] (DRILL-4946) org.objectweb.asm.tree.analysis.AnalyzerException printed to console in embedded mode
[
https://issues.apache.org/jira/browse/DRILL-4946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15581061#comment-15581061
]
ASF GitHub Bot commented on DRILL-4946:
---
GitHub user chunhui-shi opened a pull request:
https://github.com/apache/drill/pull/619
DRILL-4946: redirect System.err so users under embedded mode won't se…
…e error stacks printed due to errors occurred in scalar replacement.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/chunhui-shi/drill DRILL-4946
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/drill/pull/619.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #619
commit ea41861220f1e6f594d644f6b4eb8129a216f8b8
Author: chunhui-shi
Date: 2016-10-14T20:34:11Z
DRILL-4946: redirect System.err so users under embedded mode won't see
error stacks printed due to errors occurred in scalar replacement.
> org.objectweb.asm.tree.analysis.AnalyzerException printed to console in
> embedded mode
> -
>
> Key: DRILL-4946
> URL: https://issues.apache.org/jira/browse/DRILL-4946
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Chunhui Shi
>Assignee: Chunhui Shi
>Priority: Critical
>
> Testing by querying a json file got AnalyzerException printed.
> The problem was due to scalar_replacement mode is default to be 'try', and
> org.objectweb.asm.util.CheckMethodAdapter is printing stack trace to stderr.
> [shi@cshi-centos1 private-drill]$ cat /tmp/conv.json
> {"row": "0", "key": "\\x4a\\x31\\x39\\x38", "key2": "4a313938", "kp1":
> "4a31", "kp2": "38"}
> {"row": "1", "key": null, "key2": null, "kp1": null, "kp2": null}
> {"row": "2", "key": "\\x4e\\x4f\\x39\\x51", "key2": "4e4f3951", "kp1":
> "4e4f", "kp2": "51"}
> {"row": "3", "key": "\\x6e\\x6f\\x39\\x31", "key2": "6e6f3931", "kp1":
> "6e6f", "kp2": "31"}
> 0: jdbc:drill:zk=local> SELECT convert_from(binary_string(key), 'INT_BE') as
> intkey from dfs.`/tmp/conv.json`;
> org.objectweb.asm.tree.analysis.AnalyzerException: Error at instruction 158:
> Expected an object reference, but found .
> at org.objectweb.asm.tree.analysis.Analyzer.analyze(Analyzer.java:294)
> at
> org.objectweb.asm.util.CheckMethodAdapter$1.visitEnd(CheckMethodAdapter.java:450)
> at org.objectweb.asm.MethodVisitor.visitEnd(MethodVisitor.java:877)
> at
> org.objectweb.asm.util.CheckMethodAdapter.visitEnd(CheckMethodAdapter.java:1028)
> at org.objectweb.asm.MethodVisitor.visitEnd(MethodVisitor.java:877)
> at
> org.apache.drill.exec.compile.CheckMethodVisitorFsm.visitEnd(CheckMethodVisitorFsm.java:114)
> at org.objectweb.asm.MethodVisitor.visitEnd(MethodVisitor.java:877)
> at
> org.apache.drill.exec.compile.CheckMethodVisitorFsm.visitEnd(CheckMethodVisitorFsm.java:114)
> at org.objectweb.asm.MethodVisitor.visitEnd(MethodVisitor.java:877)
> at org.objectweb.asm.MethodVisitor.visitEnd(MethodVisitor.java:877)
> at
> org.apache.drill.exec.compile.bytecode.InstructionModifier.visitEnd(InstructionModifier.java:508)
> at org.objectweb.asm.tree.MethodNode.accept(MethodNode.java:837)
> at
> org.apache.drill.exec.compile.bytecode.ScalarReplacementNode.visitEnd(ScalarReplacementNode.java:87)
> at org.objectweb.asm.MethodVisitor.visitEnd(MethodVisitor.java:877)
> at
> org.apache.drill.exec.compile.bytecode.AloadPopRemover.visitEnd(AloadPopRemover.java:136)
> at org.objectweb.asm.tree.MethodNode.accept(MethodNode.java:837)
> at org.objectweb.asm.tree.MethodNode.accept(MethodNode.java:726)
> at org.objectweb.asm.tree.ClassNode.accept(ClassNode.java:412)
> at
> org.apache.drill.exec.compile.MergeAdapter.getMergedClass(MergeAdapter.java:223)
> at
> org.apache.drill.exec.compile.ClassTransformer.getImplementationClass(ClassTransformer.java:263)
> at
> org.apache.drill.exec.compile.CodeCompiler$Loader.load(CodeCompiler.java:78)
> at
> org.apache.drill.exec.compile.CodeCompiler$Loader.load(CodeCompiler.java:74)
> at
> com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3527)
> at
> com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2319)
> at
> com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2282)
> at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2197)
> at com.google.common.cache.LocalCache.get(LocalCache.java:3937)
> at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3941)
> at
> com.google.common.cache.Lo
[jira] [Closed] (DRILL-4876) Remain disconnected connection
[ https://issues.apache.org/jira/browse/DRILL-4876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Kojima closed DRILL-4876. Resolution: Information Provided > Remain disconnected connection > -- > > Key: DRILL-4876 > URL: https://issues.apache.org/jira/browse/DRILL-4876 > Project: Apache Drill > Issue Type: Bug > Components: Client - JDBC >Affects Versions: 1.6.0, 1.7.0, 1.8.0 > Environment: CentOS 7 >Reporter: Takuya Kojima >Assignee: Sorabh Hamirwasia >Priority: Minor > Attachments: 1_normal.png, 2_after_restart.png, > 3_try_to_connect_after_restart.png, > 4_disconnected_after_minEvictableIdleTimeMillis.png, > 5_after_disconnected.png, DrillClientConnectionResetTest.java, > drill-connection-pool.txt > > > I'm using drill via Java Application on Tomcat with JDBC driver. > I found that disconnected connection is not released when restart a drillbit. > Drillbit is restarted, but JDBC's connection keeps to try to connect the > connection which started before restart. > Expected behavior is its connection release and reconnect when drillbit is > restarted, but as a matter of fact, the connection will be released after > elapsed time of "minEvictableIdleTimeMillis" setting. > As a result, the application can't connect in the meantime despite drillbit > is active. > I thought this is not major issue, but Postgres and Vertica's JDBC driver > works well in the same situation. I spend the much time to identify the > cause, so I create a new issue of this. > The attached is log and JMX's monitor graph with 1.6.0's JDBC driver, but I > also get it with 1.7.0 and 1.8.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-4876) Remain disconnected connection
[ https://issues.apache.org/jira/browse/DRILL-4876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15581012#comment-15581012 ] Takuya Kojima commented on DRILL-4876: -- Hi Sorabh, All right, now I opened the new ticket and will close this one. Thank you for your support. It was really great help. Thanks, Takuya > Remain disconnected connection > -- > > Key: DRILL-4876 > URL: https://issues.apache.org/jira/browse/DRILL-4876 > Project: Apache Drill > Issue Type: Bug > Components: Client - JDBC >Affects Versions: 1.6.0, 1.7.0, 1.8.0 > Environment: CentOS 7 >Reporter: Takuya Kojima >Assignee: Sorabh Hamirwasia >Priority: Minor > Attachments: 1_normal.png, 2_after_restart.png, > 3_try_to_connect_after_restart.png, > 4_disconnected_after_minEvictableIdleTimeMillis.png, > 5_after_disconnected.png, DrillClientConnectionResetTest.java, > drill-connection-pool.txt > > > I'm using drill via Java Application on Tomcat with JDBC driver. > I found that disconnected connection is not released when restart a drillbit. > Drillbit is restarted, but JDBC's connection keeps to try to connect the > connection which started before restart. > Expected behavior is its connection release and reconnect when drillbit is > restarted, but as a matter of fact, the connection will be released after > elapsed time of "minEvictableIdleTimeMillis" setting. > As a result, the application can't connect in the meantime despite drillbit > is active. > I thought this is not major issue, but Postgres and Vertica's JDBC driver > works well in the same situation. I spend the much time to identify the > cause, so I create a new issue of this. > The attached is log and JMX's monitor graph with 1.6.0's JDBC driver, but I > also get it with 1.7.0 and 1.8.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-4948) Re-connection without explicit new connection
Takuya Kojima created DRILL-4948: Summary: Re-connection without explicit new connection Key: DRILL-4948 URL: https://issues.apache.org/jira/browse/DRILL-4948 Project: Apache Drill Issue Type: Improvement Components: Client - JDBC Affects Versions: Future Reporter: Takuya Kojima Priority: Minor Current JDBC driver's behavior is basically need to abort old connection and create new one when lost the connection. Re-connection by the application is not bad, but traditional DB products(e.g. Postgres, Vertica) JDBC driver can handle it without explicit re-connection. So, it would be great enhancement for new comer from another DB products world. See also this closed bug ticket: #4876 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-4876) Remain disconnected connection
[ https://issues.apache.org/jira/browse/DRILL-4876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15580360#comment-15580360 ] Sorabh Hamirwasia commented on DRILL-4876: -- Hi Takuya, You can close this JIRA ticket and please open a new one for the enhancement. For now let's mark it for Furture. It would be great to put a link for this issue in the new JIRA as well just to provide some more background information in future. Thanks, Sorabh > Remain disconnected connection > -- > > Key: DRILL-4876 > URL: https://issues.apache.org/jira/browse/DRILL-4876 > Project: Apache Drill > Issue Type: Bug > Components: Client - JDBC >Affects Versions: 1.6.0, 1.7.0, 1.8.0 > Environment: CentOS 7 >Reporter: Takuya Kojima >Assignee: Sorabh Hamirwasia >Priority: Minor > Attachments: 1_normal.png, 2_after_restart.png, > 3_try_to_connect_after_restart.png, > 4_disconnected_after_minEvictableIdleTimeMillis.png, > 5_after_disconnected.png, DrillClientConnectionResetTest.java, > drill-connection-pool.txt > > > I'm using drill via Java Application on Tomcat with JDBC driver. > I found that disconnected connection is not released when restart a drillbit. > Drillbit is restarted, but JDBC's connection keeps to try to connect the > connection which started before restart. > Expected behavior is its connection release and reconnect when drillbit is > restarted, but as a matter of fact, the connection will be released after > elapsed time of "minEvictableIdleTimeMillis" setting. > As a result, the application can't connect in the meantime despite drillbit > is active. > I thought this is not major issue, but Postgres and Vertica's JDBC driver > works well in the same situation. I spend the much time to identify the > cause, so I create a new issue of this. > The attached is log and JMX's monitor graph with 1.6.0's JDBC driver, but I > also get it with 1.7.0 and 1.8.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
