[jira] [Commented] (DRILL-6722) Query from parquet with case-then and arithmetic operation returns a wrong result
[
https://issues.apache.org/jira/browse/DRILL-6722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16673478#comment-16673478
]
Oleg Zinoviev commented on DRILL-6722:
--
At this stage, it seems to me that the whole logic of the MergeAdapter is
broken. Neither the variable index in ValueHolderSub, nor mapping local
variable to frame slot in MethodAnalyzer does not track source code control
flow.
Bug reason:
Then
ALOAD[P]
ASTORE[N]
executes, ValueHolderSub linked with variable N changes it's `first` field to P
holder `first` field. Since ValueHolderSub shared between all variables, linked
to him, it leads to all this varaibles become assigned to P.
If I make first field final, CASE fails:
CASE compiles into
var res;
{code:java}
if (expression) {
res = v1;
else {
res = v2;
}
{code}
Then Analyzer calculates Frame variables, it ignores if - else logic and
assigne v2 to res slot.
P.S.
https://stackoverflow.com/questions/25109942/is-there-a-better-explanation-of-stack-map-frames
Can be Analyzed is intended only for variable type inferrence?
> Query from parquet with case-then and arithmetic operation returns a wrong
> result
> -
>
> Key: DRILL-6722
> URL: https://issues.apache.org/jira/browse/DRILL-6722
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Codegen
>Affects Versions: 1.14.0
>Reporter: Oleg Zinoviev
>Priority: Major
> Attachments: JaininoJava.class, JaininoJava2_merged.class,
> correct.csv, result.csv
>
>
> Steps to reproduce:
> 1) Create sample table:
> {code:sql}
> create table dfs.tmp.test as
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> {code}
> 2) Execute query:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from dfs.tmp.test s
> {code}
> 3) Drill returns: [^result.csv]
> 4) Result of query without parquet:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from (
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> ) s
> {code}
> [^correct.csv]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6722) Query from parquet with case-then and arithmetic operation returns a wrong result
[
https://issues.apache.org/jira/browse/DRILL-6722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16672543#comment-16672543
]
shuifeng lu commented on DRILL-6722:
Got it! Appreciate your help.
> Query from parquet with case-then and arithmetic operation returns a wrong
> result
> -
>
> Key: DRILL-6722
> URL: https://issues.apache.org/jira/browse/DRILL-6722
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Codegen
>Affects Versions: 1.14.0
>Reporter: Oleg Zinoviev
>Priority: Major
> Attachments: JaininoJava.class, JaininoJava2_merged.class,
> correct.csv, result.csv
>
>
> Steps to reproduce:
> 1) Create sample table:
> {code:sql}
> create table dfs.tmp.test as
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> {code}
> 2) Execute query:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from dfs.tmp.test s
> {code}
> 3) Drill returns: [^result.csv]
> 4) Result of query without parquet:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from (
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> ) s
> {code}
> [^correct.csv]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6722) Query from parquet with case-then and arithmetic operation returns a wrong result
[
https://issues.apache.org/jira/browse/DRILL-6722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16671736#comment-16671736
]
Oleg Zinoviev commented on DRILL-6722:
--
[~shuifeng]
For ClassNode:
{code:java}
ClassWriter cw = new ClassWriter(ClassWriter.COMPUTE_FRAMES);
generatedMerged.accept(cw);
Files.write(Paths.get("/home/ozinoviev/Class.class"), cw.toByteArray());
{code}
> Query from parquet with case-then and arithmetic operation returns a wrong
> result
> -
>
> Key: DRILL-6722
> URL: https://issues.apache.org/jira/browse/DRILL-6722
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Codegen
>Affects Versions: 1.14.0
>Reporter: Oleg Zinoviev
>Priority: Major
> Attachments: JaininoJava.class, JaininoJava2_merged.class,
> correct.csv, result.csv
>
>
> Steps to reproduce:
> 1) Create sample table:
> {code:sql}
> create table dfs.tmp.test as
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> {code}
> 2) Execute query:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from dfs.tmp.test s
> {code}
> 3) Drill returns: [^result.csv]
> 4) Result of query without parquet:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from (
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> ) s
> {code}
> [^correct.csv]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6722) Query from parquet with case-then and arithmetic operation returns a wrong result
[
https://issues.apache.org/jira/browse/DRILL-6722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16671093#comment-16671093
]
shuifeng lu commented on DRILL-6722:
May I ask again about saving class to file?
I can't find the right way to write bytes into file in Evaluate expression
pop-up, did I miss something?
> Query from parquet with case-then and arithmetic operation returns a wrong
> result
> -
>
> Key: DRILL-6722
> URL: https://issues.apache.org/jira/browse/DRILL-6722
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Codegen
>Affects Versions: 1.14.0
>Reporter: Oleg Zinoviev
>Priority: Major
> Attachments: JaininoJava.class, JaininoJava2_merged.class,
> correct.csv, result.csv
>
>
> Steps to reproduce:
> 1) Create sample table:
> {code:sql}
> create table dfs.tmp.test as
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> {code}
> 2) Execute query:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from dfs.tmp.test s
> {code}
> 3) Drill returns: [^result.csv]
> 4) Result of query without parquet:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from (
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> ) s
> {code}
> [^correct.csv]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6722) Query from parquet with case-then and arithmetic operation returns a wrong result
[
https://issues.apache.org/jira/browse/DRILL-6722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669986#comment-16669986
]
shuifeng lu commented on DRILL-6722:
Oh, sorry, saveCodeForDebugging is setting to true for debugging.:D
> Query from parquet with case-then and arithmetic operation returns a wrong
> result
> -
>
> Key: DRILL-6722
> URL: https://issues.apache.org/jira/browse/DRILL-6722
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Codegen
>Affects Versions: 1.14.0
>Reporter: Oleg Zinoviev
>Priority: Major
> Attachments: JaininoJava.class, JaininoJava2_merged.class,
> correct.csv, result.csv
>
>
> Steps to reproduce:
> 1) Create sample table:
> {code:sql}
> create table dfs.tmp.test as
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> {code}
> 2) Execute query:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from dfs.tmp.test s
> {code}
> 3) Drill returns: [^result.csv]
> 4) Result of query without parquet:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from (
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> ) s
> {code}
> [^correct.csv]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6722) Query from parquet with case-then and arithmetic operation returns a wrong result
[
https://issues.apache.org/jira/browse/DRILL-6722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669989#comment-16669989
]
Oleg Zinoviev commented on DRILL-6722:
--
I'll try to provide PR tomorrow :)
> Query from parquet with case-then and arithmetic operation returns a wrong
> result
> -
>
> Key: DRILL-6722
> URL: https://issues.apache.org/jira/browse/DRILL-6722
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Codegen
>Affects Versions: 1.14.0
>Reporter: Oleg Zinoviev
>Priority: Major
> Attachments: JaininoJava.class, JaininoJava2_merged.class,
> correct.csv, result.csv
>
>
> Steps to reproduce:
> 1) Create sample table:
> {code:sql}
> create table dfs.tmp.test as
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> {code}
> 2) Execute query:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from dfs.tmp.test s
> {code}
> 3) Drill returns: [^result.csv]
> 4) Result of query without parquet:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from (
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> ) s
> {code}
> [^correct.csv]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6722) Query from parquet with case-then and arithmetic operation returns a wrong result
[
https://issues.apache.org/jira/browse/DRILL-6722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669976#comment-16669976
]
Oleg Zinoviev commented on DRILL-6722:
--
I have a latest master revision, and bug is still here... Could it be that your
drill.exec.compile.prefer_plain_java value is true?
{noformat}
By the way, how to generate this class without setting saveCodeForDebugging to
true?{noformat}
Breakpoint -> Evaluate expression -> Write a byte array with class into file ->
Use any decompiler
> Query from parquet with case-then and arithmetic operation returns a wrong
> result
> -
>
> Key: DRILL-6722
> URL: https://issues.apache.org/jira/browse/DRILL-6722
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Codegen
>Affects Versions: 1.14.0
>Reporter: Oleg Zinoviev
>Priority: Major
> Attachments: JaininoJava.class, JaininoJava2_merged.class,
> correct.csv, result.csv
>
>
> Steps to reproduce:
> 1) Create sample table:
> {code:sql}
> create table dfs.tmp.test as
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> {code}
> 2) Execute query:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from dfs.tmp.test s
> {code}
> 3) Drill returns: [^result.csv]
> 4) Result of query without parquet:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from (
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> ) s
> {code}
> [^correct.csv]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6722) Query from parquet with case-then and arithmetic operation returns a wrong result
[
https://issues.apache.org/jira/browse/DRILL-6722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669968#comment-16669968
]
shuifeng lu commented on DRILL-6722:
[~le.louch] Thanks for your reply.
This test works fine with DRILL-6763 on my laptop.
By the way, how to generate this class without setting saveCodeForDebugging to
true?
I've been struggling with it for a time.
> Query from parquet with case-then and arithmetic operation returns a wrong
> result
> -
>
> Key: DRILL-6722
> URL: https://issues.apache.org/jira/browse/DRILL-6722
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Codegen
>Affects Versions: 1.14.0
>Reporter: Oleg Zinoviev
>Priority: Major
> Attachments: JaininoJava.class, JaininoJava2_merged.class,
> correct.csv, result.csv
>
>
> Steps to reproduce:
> 1) Create sample table:
> {code:sql}
> create table dfs.tmp.test as
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> {code}
> 2) Execute query:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from dfs.tmp.test s
> {code}
> 3) Drill returns: [^result.csv]
> 4) Result of query without parquet:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from (
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> ) s
> {code}
> [^correct.csv]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6722) Query from parquet with case-then and arithmetic operation returns a wrong result
[
https://issues.apache.org/jira/browse/DRILL-6722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669937#comment-16669937
]
Oleg Zinoviev commented on DRILL-6722:
--
[~shuifeng], thanks for your comment.
This is generated class file before ValueHolderReplacementVisitor call:
[^JaininoJava.class]
This is generated class file after ValueHolderReplacementVisitor call:
[^JaininoJava2_merged.class]
Decompiled code of doEval method:
{code:java}
boolean var3 = false;
boolean var4 = false;
int var27 = this.vv1.getAccessor().get(inIndex);
boolean var5 = false;
int var28 = this.vv5.getAccessor().get(inIndex);
boolean var6 = false;
boolean var7 = false;
if (!Double.isNaN((double)var28) && (!Double.isNaN((double)var27) ||
!Double.isNaN((double)var28))) {
if (Double.isNaN((double)var27) && !Double.isNaN((double)var28)) {
var7 = true;
} else {
var7 = var27 > var28;
}
} else {
var7 = false;
}
int var26;
if (var7) {
var26 = var27;
} else {
var26 = var28;
}
this.vv10.getMutator().set(outIndex, var26);
boolean var8 = false;
boolean var9 = false;
int var29 = var26 - var26; // << Error reason. Bug somewhere in
ValueHolderReplacementVisitor.
boolean var10 = false;
boolean var11 = false;
int var30 = Math.abs(var29);
this.vv15.getMutator().set(outIndex, var30);
{code}
> Query from parquet with case-then and arithmetic operation returns a wrong
> result
> -
>
> Key: DRILL-6722
> URL: https://issues.apache.org/jira/browse/DRILL-6722
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Codegen
>Affects Versions: 1.14.0
>Reporter: Oleg Zinoviev
>Priority: Major
> Attachments: JaininoJava.class, JaininoJava2_merged.class,
> correct.csv, result.csv
>
>
> Steps to reproduce:
> 1) Create sample table:
> {code:sql}
> create table dfs.tmp.test as
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> {code}
> 2) Execute query:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from dfs.tmp.test s
> {code}
> 3) Drill returns: [^result.csv]
> 4) Result of query without parquet:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from (
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> ) s
> {code}
> [^correct.csv]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6722) Query from parquet with case-then and arithmetic operation returns a wrong result
[
https://issues.apache.org/jira/browse/DRILL-6722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621881#comment-16621881
]
shuifeng lu commented on DRILL-6722:
It works fine when override the value of drill.exec.compile.prefer_plain_java
to true.
after some tests and debugging, projector.projectRecords in
ProjectRecordBatch.doWork() works incorrectly.
It is not easy to debug due to lack of debug info since if set
saveCodeForDebugging to true, this works fine.
any suggestion is appreciated.
> Query from parquet with case-then and arithmetic operation returns a wrong
> result
> -
>
> Key: DRILL-6722
> URL: https://issues.apache.org/jira/browse/DRILL-6722
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Codegen
>Affects Versions: 1.14.0
>Reporter: Oleg Zinoviev
>Priority: Major
> Attachments: correct.csv, result.csv
>
>
> Steps to reproduce:
> 1) Create sample table:
> {code:sql}
> create table dfs.tmp.test as
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> {code}
> 2) Execute query:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from dfs.tmp.test s
> {code}
> 3) Drill returns: [^result.csv]
> 4) Result of query without parquet:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from (
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> ) s
> {code}
> [^correct.csv]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6722) Query from parquet with case-then and arithmetic operation returns a wrong result
[
https://issues.apache.org/jira/browse/DRILL-6722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596670#comment-16596670
]
Oleg Zinoviev commented on DRILL-6722:
--
1) Bug can be reproduced without abs call
2) alias of case-then column has no affect on bug
> Query from parquet with case-then and arithmetic operation returns a wrong
> result
> -
>
> Key: DRILL-6722
> URL: https://issues.apache.org/jira/browse/DRILL-6722
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.14.0
>Reporter: Oleg Zinoviev
>Priority: Major
> Attachments: correct.csv, result.csv
>
>
> Steps to reproduce:
> 1) Create sample table:
> {code:sql}
> create table dfs.tmp.test as
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> {code}
> 2) Execute query:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from dfs.tmp.test s
> {code}
> 3) Drill returns: [^result.csv]
> 4) Result of query without parquet:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from (
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> ) s
> {code}
> [^correct.csv]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
