[jira] [Created] (DRILL-7781) NearestDate function returns an incorrect result when calculating the quarter
Oleg Zinoviev created DRILL-7781:
Summary: NearestDate function returns an incorrect result when
calculating the quarter
Key: DRILL-7781
URL: https://issues.apache.org/jira/browse/DRILL-7781
Project: Apache Drill
Issue Type: Bug
Affects Versions: 1.17.0, 1.16.0
Reporter: Oleg Zinoviev
||SQL||Expected||Actual||
|nearestDate('2019-03-01 00:00:00', 'quarter')|2019-01-01 00:00:00|2019-04-01
00:00:00|
|nearestDate('2019-06-01 00:00:00', 'quarter')|2019-04-01 00:00:00|2019-07-01
00:00:00|
|nearestDate('2019-09-01 00:00:00', 'quarter')|2019-07-01 00:00:00|2019-10-01
00:00:00|
|nearestDate('2019-12-01 00:00:00', 'quarter')|2019-10-01 00:00:00|SYSTEM
ERROR: DateTimeException: Invalid value for MonthOfYear (valid values 1 - 12):
13|
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (DRILL-7773) Incorrect conversion result to TIME_EPOCH_BE
Oleg Zinoviev created DRILL-7773:
Summary: Incorrect conversion result to TIME_EPOCH_BE
Key: DRILL-7773
URL: https://issues.apache.org/jira/browse/DRILL-7773
Project: Apache Drill
Issue Type: Bug
Affects Versions: 1.17.0
Reporter: Oleg Zinoviev
Run query:
{code:sql}
select cast(convert_from(convert_to('23:30:21', 'TIME_EPOCH_BE'),
'TIME_EPOCH_BE') as time) as t
{code}
Expected result:
23:30:21
Actual result:
00:56:23.295
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (DRILL-7721) Inconsistent behavior of external_sort and topN
[
https://issues.apache.org/jira/browse/DRILL-7721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleg Zinoviev updated DRILL-7721:
-
Description:
The TopN operator with the same sort field value can change the order of the
rows relative to the original (unstable sort). The External_sort operator
preserves the original row order (at least in the example). This inconsistency
breaks the implementation of data pagination.
Example:
1) Create table
{code:sql}
create table dfs.test_sort as (
select 1 as index, 0 as value
union all
select 1 as index, 1 as value
union all
select 1 as index, -1 as value
union all
select 2 as index, 0 as value
union all
select 2 as index, 1 as value
union all
select 2 as index, -1 as value
union all
select -1 as index, 0 as value
union all
select -1 as index, 1 as value
union all
select -1 as index, -1 as value
)
{code}
2) Execute query:
{code:sql}
select * from dfs.test_sort s order by s.index limit 100 // Should be TopN sort
{code}
3) Execute query:
{code:sql}
select * from dfs.test_sort s order by s.index limit 1000 // Should be
External_sort
{code}
4) Compare results:)
!topn.png! !sort.png!
was:
The TopN operator with the same sort field value can change the order of the
lines relative to the original (unstable sort). The External_sort operator
preserves the original row order (at least in the example). This inconsistency
breaks the implementation of data pagination.
Example:
1) Create table
{code:sql}
create table dfs.test_sort as (
select 1 as index, 0 as value
union all
select 1 as index, 1 as value
union all
select 1 as index, -1 as value
union all
select 2 as index, 0 as value
union all
select 2 as index, 1 as value
union all
select 2 as index, -1 as value
union all
select -1 as index, 0 as value
union all
select -1 as index, 1 as value
union all
select -1 as index, -1 as value
)
{code}
2) Execute query:
{code:sql}
select * from dfs.test_sort s order by s.index limit 100 // Should be TopN sort
{code}
3) Execute query:
{code:sql}
select * from dfs.test_sort s order by s.index limit 1000 // Should be
External_sort
{code}
4) Compare results:)
!topn.png! !sort.png!
> Inconsistent behavior of external_sort and topN
> ---
>
> Key: DRILL-7721
> URL: https://issues.apache.org/jira/browse/DRILL-7721
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.17.0
>Reporter: Oleg Zinoviev
>Priority: Major
> Attachments: sort.png, topn.png
>
>
> The TopN operator with the same sort field value can change the order of the
> rows relative to the original (unstable sort). The External_sort operator
> preserves the original row order (at least in the example). This
> inconsistency breaks the implementation of data pagination.
> Example:
> 1) Create table
> {code:sql}
> create table dfs.test_sort as (
> select 1 as index, 0 as value
> union all
> select 1 as index, 1 as value
> union all
> select 1 as index, -1 as value
> union all
> select 2 as index, 0 as value
> union all
> select 2 as index, 1 as value
> union all
> select 2 as index, -1 as value
> union all
> select -1 as index, 0 as value
> union all
> select -1 as index, 1 as value
> union all
> select -1 as index, -1 as value
> )
> {code}
> 2) Execute query:
> {code:sql}
> select * from dfs.test_sort s order by s.index limit 100 // Should be TopN
> sort
> {code}
> 3) Execute query:
> {code:sql}
> select * from dfs.test_sort s order by s.index limit 1000 // Should be
> External_sort
> {code}
> 4) Compare results:)
> !topn.png! !sort.png!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (DRILL-7721) Inconsistent behavior of external_sort and topN
Oleg Zinoviev created DRILL-7721:
Summary: Inconsistent behavior of external_sort and topN
Key: DRILL-7721
URL: https://issues.apache.org/jira/browse/DRILL-7721
Project: Apache Drill
Issue Type: Bug
Affects Versions: 1.17.0
Reporter: Oleg Zinoviev
Attachments: sort.png, topn.png
The TopN operator with the same sort field value can change the order of the
lines relative to the original (unstable sort). The External_sort operator
preserves the original row order (at least in the example). This inconsistency
breaks the implementation of data pagination.
Example:
1) Create table
{code:sql}
create table dfs.test_sort as (
select 1 as index, 0 as value
union all
select 1 as index, 1 as value
union all
select 1 as index, -1 as value
union all
select 2 as index, 0 as value
union all
select 2 as index, 1 as value
union all
select 2 as index, -1 as value
union all
select -1 as index, 0 as value
union all
select -1 as index, 1 as value
union all
select -1 as index, -1 as value
)
{code}
2) Execute query:
{code:sql}
select * from dfs.test_sort s order by s.index limit 100 // Should be TopN sort
{code}
3) Execute query:
{code:sql}
select * from dfs.test_sort s order by s.index limit 1000 // Should be
External_sort
{code}
4) Compare results:)
!topn.png! !sort.png!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (DRILL-7622) Compilation error when using HLL / TDigest with `group by` clause
[ https://issues.apache.org/jira/browse/DRILL-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Zinoviev updated DRILL-7622: - Summary: Compilation error when using HLL / TDigest with `group by` clause (was: CompileException in HashAggBatch.createAggregator for HLL/tdigest function) > Compilation error when using HLL / TDigest with `group by` clause > - > > Key: DRILL-7622 > URL: https://issues.apache.org/jira/browse/DRILL-7622 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.17.0 >Reporter: Oleg Zinoviev >Assignee: Oleg Zinoviev >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (DRILL-7622) CompileException in HashAggBatch.createAggregator for HLL/tdigest function
Oleg Zinoviev created DRILL-7622: Summary: CompileException in HashAggBatch.createAggregator for HLL/tdigest function Key: DRILL-7622 URL: https://issues.apache.org/jira/browse/DRILL-7622 Project: Apache Drill Issue Type: Bug Affects Versions: 1.17.0 Reporter: Oleg Zinoviev Assignee: Oleg Zinoviev -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (DRILL-5530) IntVectors are sometimes not zero-filled on allocation
[
https://issues.apache.org/jira/browse/DRILL-5530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17041670#comment-17041670
]
Oleg Zinoviev commented on DRILL-5530:
--
[~Paul.Rogers]
Thank you very much for the detailed explanations.
> IntVectors are sometimes not zero-filled on allocation
> --
>
> Key: DRILL-5530
> URL: https://issues.apache.org/jira/browse/DRILL-5530
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.8.0
>Reporter: Paul Rogers
>Assignee: Paul Rogers
>Priority: Major
>
> Much of Drill's vector code is based on the assumption that vectors are
> initialized to 0s.
> For example, consider the "bit" (is-set) vector used for nullable types. The
> vector is presumed initialized to 0 so that all fields are set to not-set
> (e.g. null) by default. For example, to allocate a new vector the code
> (slightly edited) is:
> {code}
> private void allocateBytes(final long size) {
> final int curSize = (int) size;
> clear();
> data = allocator.buffer(curSize);
> data.setZero(0, data.capacity()); // Zero whole vector
> allocationSizeInBytes = curSize;
> }
> {code}
> The code to reallocate (grow) the vector zeros the new slots:
> {code}
> public void reAlloc() {
> final long newAllocationSize = allocationSizeInBytes * 2L;
> final int curSize = (int)newAllocationSize;
> final DrillBuf newBuf = allocator.buffer(curSize);
> newBuf.setZero(0, newBuf.capacity());
> newBuf.setBytes(0, data, 0, data.capacity()); // Zero new part
> data.release();
> data = newBuf;
> allocationSizeInBytes = curSize;
> }
> {code}
> However, it turns out that other vectors omit the initial zero step. Consider
> the {{IntVector}}:
> {code}
> private void allocateBytes(final long size) {
> final int curSize = (int)size;
> clear();
> data = allocator.buffer(curSize);
> // Notice no call here to zero the buffer
> data.readerIndex(0);
> allocationSizeInBytes = curSize;
> }
> {code}
> Now things start to get strange. If the buffer is newly allocated by Netty
> from the OS, it will contain zeros. Thus, most test cases will get a zero
> buffer. But, [Netty does *not* zero the
> buffer|http://netty.io/wiki/using-as-a-generic-library.html#performance] if
> the buffer is recycled from Netty's free list.
> Further, the reallocation for ints *does* zero the new half of the buffer:
> {code}
> public void reAlloc() {
> final long newAllocationSize = allocationSizeInBytes * 2L;
> final DrillBuf newBuf = allocator.buffer((int)newAllocationSize);
> newBuf.setBytes(0, data, 0, data.capacity());
> final int halfNewCapacity = newBuf.capacity() / 2;
> newBuf.setZero(halfNewCapacity, halfNewCapacity); // Zero new bytes
> newBuf.writerIndex(data.writerIndex());
> data.release(1);
> data = newBuf;
> allocationSizeInBytes = (int)newAllocationSize;
> }
> {code}
> The result is that, most times, the bytes of the vector will be zero. But,
> the first allocation may be non-zero if Netty reuses an existing block from
> Netty's free list.
> If any code assumes that values default to zero, that code will fail -- but
> only for the first few bytes and only if run long enough for the buffer to be
> a "recycled" "dirty" one.
> This was found by a test that filled vectors with runs of 'X' values for one
> test, followed by another test that allocated an int vector and inspected its
> contents.
> This asymmetry is just asking for bugs to appear. To make clear that only bit
> vectors are zero filled:
> * Remove the {{setZero()}} call when reallocating vectors.
> * When assertions are on, fill the buffer with dummy data (such as
> 0b1010_1010) to flush out any code that happens to depend on zero values.
> * Code that needs to "back-fill" values (such as when columns are missing in
> some rows in a reader) should do the back-filling explicitly rather than
> assuming zeros.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (DRILL-5530) IntVectors are sometimes not zero-filled on allocation
[
https://issues.apache.org/jira/browse/DRILL-5530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17041050#comment-17041050
]
Oleg Zinoviev commented on DRILL-5530:
--
[~paul-rogers]
I accidentally discovered a similar behavior when sequentially allocating and
freeing a buffer. Here is a test that demonstrates this behavior.
{code:java}
try (OperatorFixture operatorFixture = new
OperatorFixture.Builder(baseDirTestWatcher).build()) {
DrillBuf buffer = operatorFixture.allocator().buffer(8);
buffer.setByte(0, 1);
long addr = buffer.memoryAddress();
buffer.release();
buffer = operatorFixture.allocator().buffer(8);
Assert.assertEquals(addr, buffer.memoryAddress());// same address
Assert.assertEquals(0, buffer.getByte(0)); // <== fail, since
buffer.getByte(0) return 1
}
{code}
In my case, a non-zero buffer is returned from PooledByteBufAllocatorL ::
threadCache () :: directArena :: allocate (...)
> IntVectors are sometimes not zero-filled on allocation
> --
>
> Key: DRILL-5530
> URL: https://issues.apache.org/jira/browse/DRILL-5530
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.8.0
>Reporter: Paul Rogers
>Priority: Major
>
> Much of Drill's vector code is based on the assumption that vectors are
> initialized to 0s.
> For example, consider the "bit" (is-set) vector used for nullable types. The
> vector is presumed initialized to 0 so that all fields are set to not-set
> (e.g. null) by default. For example, to allocate a new vector the code
> (slightly edited) is:
> {code}
> private void allocateBytes(final long size) {
> final int curSize = (int) size;
> clear();
> data = allocator.buffer(curSize);
> data.setZero(0, data.capacity()); // Zero whole vector
> allocationSizeInBytes = curSize;
> }
> {code}
> The code to reallocate (grow) the vector zeros the new slots:
> {code}
> public void reAlloc() {
> final long newAllocationSize = allocationSizeInBytes * 2L;
> final int curSize = (int)newAllocationSize;
> final DrillBuf newBuf = allocator.buffer(curSize);
> newBuf.setZero(0, newBuf.capacity());
> newBuf.setBytes(0, data, 0, data.capacity()); // Zero new part
> data.release();
> data = newBuf;
> allocationSizeInBytes = curSize;
> }
> {code}
> However, it turns out that other vectors omit the initial zero step. Consider
> the {{IntVector}}:
> {code}
> private void allocateBytes(final long size) {
> final int curSize = (int)size;
> clear();
> data = allocator.buffer(curSize);
> // Notice no call here to zero the buffer
> data.readerIndex(0);
> allocationSizeInBytes = curSize;
> }
> {code}
> Now things start to get strange. If the buffer is newly allocated by Netty
> from the OS, it will contain zeros. Thus, most test cases will get a zero
> buffer. But, [Netty does *not* zero the
> buffer|http://netty.io/wiki/using-as-a-generic-library.html#performance] if
> the buffer is recycled from Netty's free list.
> Further, the reallocation for ints *does* zero the new half of the buffer:
> {code}
> public void reAlloc() {
> final long newAllocationSize = allocationSizeInBytes * 2L;
> final DrillBuf newBuf = allocator.buffer((int)newAllocationSize);
> newBuf.setBytes(0, data, 0, data.capacity());
> final int halfNewCapacity = newBuf.capacity() / 2;
> newBuf.setZero(halfNewCapacity, halfNewCapacity); // Zero new bytes
> newBuf.writerIndex(data.writerIndex());
> data.release(1);
> data = newBuf;
> allocationSizeInBytes = (int)newAllocationSize;
> }
> {code}
> The result is that, most times, the bytes of the vector will be zero. But,
> the first allocation may be non-zero if Netty reuses an existing block from
> Netty's free list.
> If any code assumes that values default to zero, that code will fail -- but
> only for the first few bytes and only if run long enough for the buffer to be
> a "recycled" "dirty" one.
> This was found by a test that filled vectors with runs of 'X' values for one
> test, followed by another test that allocated an int vector and inspected its
> contents.
> This asymmetry is just asking for bugs to appear. To make clear that only bit
> vectors are zero filled:
> * Remove the {{setZero()}} call when reallocating vectors.
> * When assertions are on, fill the buffer with dummy data (such as
> 0b1010_1010) to flush out any code that happens to depend on zero values.
> * Code that needs to "back-fill" values (such as when columns are missing in
> some rows in a reader) should do the back-filling explicitly rather than
> assuming zeros.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (DRILL-7586) drill-hive-exec-shaded contains commons-lang3 version 3.1
Oleg Zinoviev created DRILL-7586: Summary: drill-hive-exec-shaded contains commons-lang3 version 3.1 Key: DRILL-7586 URL: https://issues.apache.org/jira/browse/DRILL-7586 Project: Apache Drill Issue Type: Bug Affects Versions: 1.17.0, 1.16.0 Reporter: Oleg Zinoviev Assignee: Oleg Zinoviev org.apache.hive:hive-exec contains shaded commons-lang3 library version 3.1. During the Drill build, these classes are added to the Vasya library. This causes an incorrect version of StringUtils to be loaded in runtime (since the jars directory is found in classpath earlier than jars/3rdparty) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (DRILL-7546) DrillbitContext.getEndpoint state is allways "startup"
[ https://issues.apache.org/jira/browse/DRILL-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021290#comment-17021290 ] Oleg Zinoviev commented on DRILL-7546: -- Same for httpPort in DrillbitContext:: getEndpoint (allways not set) > DrillbitContext.getEndpoint state is allways "startup" > -- > > Key: DRILL-7546 > URL: https://issues.apache.org/jira/browse/DRILL-7546 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.16.0 >Reporter: Oleg Zinoviev >Priority: Major > > For now DrillbitContext:: getEndpoint::getState always returns STARTUP. Since > CoordinationProtos.DrillbitEndpoint is actively used in comparisons on equals > or as a key in hash map (e.g. in AssignmentCreator). this can lead to > situations where the comparison for the current node returns false if the > node information obtained from the coordinator and from > DrillbitContext::getEndpoint is compared. > I think I see this situation in AssignmentCreator. Could you check this > behavior? Unfortunately, I don't understand how to write a test that > demonstrates this. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (DRILL-7546) DrillbitContext.getEndpoint state is allways "startup"
[ https://issues.apache.org/jira/browse/DRILL-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Zinoviev updated DRILL-7546: - Description: For now DrillbitContext:: getEndpoint::getState always returns STARTUP. Since CoordinationProtos.DrillbitEndpoint is actively used in comparisons on equals or as a key in hash map (e.g. in AssignmentCreator). this can lead to situations where the comparison for the current node returns false if the node information obtained from the coordinator and from DrillbitContext::getEndpoint is compared. I think I see this situation in AssignmentCreator. Could you check this behavior? Unfortunately, I don't understand how to write a test that demonstrates this. was: For now DrillbitContext:: getEndpoint::getState always returns STARTUP. Since CoordinationProtos.DrillbitEndpoint is actively used in comparisons on equals or as a key in hash map (e.g. in AssignmentCreator). this can lead to situations where the comparison for the current node returns false if the node information obtained from the coordinator and from DrillbitContext::getEndpoint is compared. I think I see this situation in Assignment Creator. Could you check this behavior? Unfortunately, I don't understand how to write a test that demonstrates this. > DrillbitContext.getEndpoint state is allways "startup" > -- > > Key: DRILL-7546 > URL: https://issues.apache.org/jira/browse/DRILL-7546 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.16.0 >Reporter: Oleg Zinoviev >Priority: Major > > For now DrillbitContext:: getEndpoint::getState always returns STARTUP. Since > CoordinationProtos.DrillbitEndpoint is actively used in comparisons on equals > or as a key in hash map (e.g. in AssignmentCreator). this can lead to > situations where the comparison for the current node returns false if the > node information obtained from the coordinator and from > DrillbitContext::getEndpoint is compared. > I think I see this situation in AssignmentCreator. Could you check this > behavior? Unfortunately, I don't understand how to write a test that > demonstrates this. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (DRILL-7546) DrillbitContext.getCurrentEndpoint state is allways "startup"
[ https://issues.apache.org/jira/browse/DRILL-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Zinoviev updated DRILL-7546: - Description: For now DrillbitContext:: getEndpoint::getState always returns STARTUP. Since CoordinationProtos.DrillbitEndpoint is actively used in comparisons on equals or as a key in hash map (e.g. in AssignmentCreator). this can lead to situations where the comparison for the current node returns false if the node information obtained from the coordinator and from DrillbitContext::getEndpoint is compared. I think I see this situation in Assignment Creator. Could you check this behavior? Unfortunately, I don't understand how to write a test that demonstrates this. was: For now DrillbitContext:: getEndpoint::getState always returns STARTUP. Since CoordinationProtos.DrillbitEndpoint is actively used in comparisons on equals or as a key in hash map (e.g. in AssignmentCreator). this can lead to situations where the comparison for the current node returns false if the node information obtained from the coordinator and from DrillbitContext::getEndpoint is compared. Мне кажется, что я наблюдаю такую ситуацию в AssignmentCreator. Не могли бы вы проверить это поведение? К сожалению я не понимаю, как написать тест, демонстрирующий это. > DrillbitContext.getCurrentEndpoint state is allways "startup" > - > > Key: DRILL-7546 > URL: https://issues.apache.org/jira/browse/DRILL-7546 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.16.0 >Reporter: Oleg Zinoviev >Priority: Major > > For now DrillbitContext:: getEndpoint::getState always returns STARTUP. Since > CoordinationProtos.DrillbitEndpoint is actively used in comparisons on equals > or as a key in hash map (e.g. in AssignmentCreator). this can lead to > situations where the comparison for the current node returns false if the > node information obtained from the coordinator and from > DrillbitContext::getEndpoint is compared. > I think I see this situation in Assignment Creator. Could you check this > behavior? Unfortunately, I don't understand how to write a test that > demonstrates this. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (DRILL-7546) DrillbitContext.getEndpoint state is allways "startup"
[ https://issues.apache.org/jira/browse/DRILL-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Zinoviev updated DRILL-7546: - Summary: DrillbitContext.getEndpoint state is allways "startup" (was: DrillbitContext.getCurrentEndpoint state is allways "startup") > DrillbitContext.getEndpoint state is allways "startup" > -- > > Key: DRILL-7546 > URL: https://issues.apache.org/jira/browse/DRILL-7546 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.16.0 >Reporter: Oleg Zinoviev >Priority: Major > > For now DrillbitContext:: getEndpoint::getState always returns STARTUP. Since > CoordinationProtos.DrillbitEndpoint is actively used in comparisons on equals > or as a key in hash map (e.g. in AssignmentCreator). this can lead to > situations where the comparison for the current node returns false if the > node information obtained from the coordinator and from > DrillbitContext::getEndpoint is compared. > I think I see this situation in Assignment Creator. Could you check this > behavior? Unfortunately, I don't understand how to write a test that > demonstrates this. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (DRILL-7546) DrillbitContext.getCurrentEndpoint state is allways "startup"
Oleg Zinoviev created DRILL-7546: Summary: DrillbitContext.getCurrentEndpoint state is allways "startup" Key: DRILL-7546 URL: https://issues.apache.org/jira/browse/DRILL-7546 Project: Apache Drill Issue Type: Bug Affects Versions: 1.16.0 Reporter: Oleg Zinoviev For now DrillbitContext:: getEndpoint::getState always returns STARTUP. Since CoordinationProtos.DrillbitEndpoint is actively used in comparisons on equals or as a key in hash map (e.g. in AssignmentCreator). this can lead to situations where the comparison for the current node returns false if the node information obtained from the coordinator and from DrillbitContext::getEndpoint is compared. Мне кажется, что я наблюдаю такую ситуацию в AssignmentCreator. Не могли бы вы проверить это поведение? К сожалению я не понимаю, как написать тест, демонстрирующий это. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (DRILL-7362) COUNT(*) on JSON with outer list results in JsonParse error
Oleg Zinoviev created DRILL-7362: Summary: COUNT(*) on JSON with outer list results in JsonParse error Key: DRILL-7362 URL: https://issues.apache.org/jira/browse/DRILL-7362 Project: Apache Drill Issue Type: Bug Reporter: Oleg Zinoviev Count from a JSON file with a outer array results in JsonParseException: Cannot read from the middle of a record. Current token was START_ARRAY. P.S. A simple select from such file works normally. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Closed] (DRILL-7355) Reading a offset based vectors may produce IndexOutOfBoundsException
[ https://issues.apache.org/jira/browse/DRILL-7355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Zinoviev closed DRILL-7355. Resolution: Cannot Reproduce > Reading a offset based vectors may produce IndexOutOfBoundsException > > > Key: DRILL-7355 > URL: https://issues.apache.org/jira/browse/DRILL-7355 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.16.0 >Reporter: Oleg Zinoviev >Priority: Major > > Var16Char, VarBinary, VarChar and VarDecimal vectors contain a UInt4 offset > vector. During the writing of these vectors, the offset value overflow is not > processed. Therefore, upon subsequent reading of such a vector to the > corresponding holder, the value in the "end" field may be less than the value > of the "start" field. In some cases, the "start" and "end" values are used to > get the size of the buffer (for example,ParquetOutputRecordWriter). This > causes the target buffer size to be negative. > Stack trace: > Caused by: java.lang.IndexOutOfBoundsException: index: 17, length: -799 > (expected: range(0, 524288)) > at > io.netty.buffer.AbstractByteBuf.checkIndex0(AbstractByteBuf.java:1125) > at io.netty.buffer.AbstractByteBuf.checkIndex(AbstractByteBuf.java:1120) > at > io.netty.buffer.PooledUnsafeDirectByteBuf.nioBuffer(PooledUnsafeDirectByteBuf.java:244) > at io.netty.buffer.WrappedByteBuf.nioBuffer(WrappedByteBuf.java:734) > at > io.netty.buffer.UnsafeDirectLittleEndian.nioBuffer(UnsafeDirectLittleEndian.java:34) > at io.netty.buffer.DrillBuf.nioBuffer(DrillBuf.java:364) > at > org.apache.drill.exec.store.ParquetOutputRecordWriter$VarCharParquetConverter.writeField(ParquetOutputRecordWriter.java:1824) > at > org.apache.drill.exec.store.EventBasedRecordWriter.write(EventBasedRecordWriter.java:48) > at > org.apache.drill.exec.physical.impl.WriterRecordBatch.innerNext(WriterRecordBatch.java:111) > at > org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:186) > at > org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:104) > at > org.apache.drill.exec.physical.impl.SingleSenderCreator$SingleSenderRootExec.innerNext(SingleSenderCreator.java:93) > at > org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:94) > at > org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:296) > at > org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:283) -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (DRILL-7355) Reading a offset based vectors may produce IndexOutOfBoundsException
[ https://issues.apache.org/jira/browse/DRILL-7355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913379#comment-16913379 ] Oleg Zinoviev commented on DRILL-7355: -- [~arina], looks like it's already fixed, since I cannot reproduce it on drill 1.17.0. > Reading a offset based vectors may produce IndexOutOfBoundsException > > > Key: DRILL-7355 > URL: https://issues.apache.org/jira/browse/DRILL-7355 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.16.0 >Reporter: Oleg Zinoviev >Priority: Major > > Var16Char, VarBinary, VarChar and VarDecimal vectors contain a UInt4 offset > vector. During the writing of these vectors, the offset value overflow is not > processed. Therefore, upon subsequent reading of such a vector to the > corresponding holder, the value in the "end" field may be less than the value > of the "start" field. In some cases, the "start" and "end" values are used to > get the size of the buffer (for example,ParquetOutputRecordWriter). This > causes the target buffer size to be negative. > Stack trace: > Caused by: java.lang.IndexOutOfBoundsException: index: 17, length: -799 > (expected: range(0, 524288)) > at > io.netty.buffer.AbstractByteBuf.checkIndex0(AbstractByteBuf.java:1125) > at io.netty.buffer.AbstractByteBuf.checkIndex(AbstractByteBuf.java:1120) > at > io.netty.buffer.PooledUnsafeDirectByteBuf.nioBuffer(PooledUnsafeDirectByteBuf.java:244) > at io.netty.buffer.WrappedByteBuf.nioBuffer(WrappedByteBuf.java:734) > at > io.netty.buffer.UnsafeDirectLittleEndian.nioBuffer(UnsafeDirectLittleEndian.java:34) > at io.netty.buffer.DrillBuf.nioBuffer(DrillBuf.java:364) > at > org.apache.drill.exec.store.ParquetOutputRecordWriter$VarCharParquetConverter.writeField(ParquetOutputRecordWriter.java:1824) > at > org.apache.drill.exec.store.EventBasedRecordWriter.write(EventBasedRecordWriter.java:48) > at > org.apache.drill.exec.physical.impl.WriterRecordBatch.innerNext(WriterRecordBatch.java:111) > at > org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:186) > at > org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:104) > at > org.apache.drill.exec.physical.impl.SingleSenderCreator$SingleSenderRootExec.innerNext(SingleSenderCreator.java:93) > at > org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:94) > at > org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:296) > at > org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:283) -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (DRILL-7355) Reading a offset based vectors may produce IndexOutOfBoundsException
[ https://issues.apache.org/jira/browse/DRILL-7355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Zinoviev updated DRILL-7355: - Description: Var16Char, VarBinary, VarChar and VarDecimal vectors contain a UInt4 offset vector. During the writing of these vectors, the offset value overflow is not processed. Therefore, upon subsequent reading of such a vector to the corresponding holder, the value in the "end" field may be less than the value of the "start" field. In some cases, the "start" and "end" values are used to get the size of the buffer (for example,ParquetOutputRecordWriter). This causes the target buffer size to be negative. Stack trace: Caused by: java.lang.IndexOutOfBoundsException: index: 17, length: -799 (expected: range(0, 524288)) at io.netty.buffer.AbstractByteBuf.checkIndex0(AbstractByteBuf.java:1125) at io.netty.buffer.AbstractByteBuf.checkIndex(AbstractByteBuf.java:1120) at io.netty.buffer.PooledUnsafeDirectByteBuf.nioBuffer(PooledUnsafeDirectByteBuf.java:244) at io.netty.buffer.WrappedByteBuf.nioBuffer(WrappedByteBuf.java:734) at io.netty.buffer.UnsafeDirectLittleEndian.nioBuffer(UnsafeDirectLittleEndian.java:34) at io.netty.buffer.DrillBuf.nioBuffer(DrillBuf.java:364) at org.apache.drill.exec.store.ParquetOutputRecordWriter$VarCharParquetConverter.writeField(ParquetOutputRecordWriter.java:1824) at org.apache.drill.exec.store.EventBasedRecordWriter.write(EventBasedRecordWriter.java:48) at org.apache.drill.exec.physical.impl.WriterRecordBatch.innerNext(WriterRecordBatch.java:111) at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:186) at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:104) at org.apache.drill.exec.physical.impl.SingleSenderCreator$SingleSenderRootExec.innerNext(SingleSenderCreator.java:93) at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:94) at org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:296) at org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:283) was: Var16Char, VarBinary, VarChar and VarDecimal vectors contain a UInt4 offset vector. When writing and reading these vectors, the overflow of the offset value is not processed. While reading values into the corresponding holder, the values of their offset vectors are written into the "start" and "end" fields as ordinary integers. w In some cases, the "start" and "end" values are used to get the size of the buffer (for example,ParquetOutputRecordWriter). This causes the target buffer size to be negative. Stack trace: Caused by: java.lang.IndexOutOfBoundsException: index: 17, length: -799 (expected: range(0, 524288)) at io.netty.buffer.AbstractByteBuf.checkIndex0(AbstractByteBuf.java:1125) at io.netty.buffer.AbstractByteBuf.checkIndex(AbstractByteBuf.java:1120) at io.netty.buffer.PooledUnsafeDirectByteBuf.nioBuffer(PooledUnsafeDirectByteBuf.java:244) at io.netty.buffer.WrappedByteBuf.nioBuffer(WrappedByteBuf.java:734) at io.netty.buffer.UnsafeDirectLittleEndian.nioBuffer(UnsafeDirectLittleEndian.java:34) at io.netty.buffer.DrillBuf.nioBuffer(DrillBuf.java:364) at org.apache.drill.exec.store.ParquetOutputRecordWriter$VarCharParquetConverter.writeField(ParquetOutputRecordWriter.java:1824) at org.apache.drill.exec.store.EventBasedRecordWriter.write(EventBasedRecordWriter.java:48) at org.apache.drill.exec.physical.impl.WriterRecordBatch.innerNext(WriterRecordBatch.java:111) at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:186) at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:104) at org.apache.drill.exec.physical.impl.SingleSenderCreator$SingleSenderRootExec.innerNext(SingleSenderCreator.java:93) at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:94) at org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:296) at org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:283) > Reading a offset based vectors may produce IndexOutOfBoundsException > > > Key: DRILL-7355 > URL: https://issues.apache.org/jira/browse/DRILL-7355 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.16.0 >Reporter: Oleg Zinoviev >Priority: Major > > Var16Char, VarBinary, VarChar and VarDecimal vectors contain a UInt4 offset > vector. During the writing of these vectors, the offset value overflow is not > processed. Therefore, upon subsequent reading of such a vector
[jira] [Updated] (DRILL-7355) Reading a offset based vectors may produce IndexOutOfBoundsException
[ https://issues.apache.org/jira/browse/DRILL-7355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Zinoviev updated DRILL-7355: - Description: Var16Char, VarBinary, VarChar and VarDecimal vectors contain a UInt4 offset vector. When writing and reading these vectors, the overflow of the offset value is not processed. While reading values into the corresponding holder, the values of their offset vectors are written into the "start" and "end" fields as ordinary integers. w In some cases, the "start" and "end" values are used to get the size of the buffer (for example,ParquetOutputRecordWriter). This causes the target buffer size to be negative. Stack trace: Caused by: java.lang.IndexOutOfBoundsException: index: 17, length: -799 (expected: range(0, 524288)) at io.netty.buffer.AbstractByteBuf.checkIndex0(AbstractByteBuf.java:1125) at io.netty.buffer.AbstractByteBuf.checkIndex(AbstractByteBuf.java:1120) at io.netty.buffer.PooledUnsafeDirectByteBuf.nioBuffer(PooledUnsafeDirectByteBuf.java:244) at io.netty.buffer.WrappedByteBuf.nioBuffer(WrappedByteBuf.java:734) at io.netty.buffer.UnsafeDirectLittleEndian.nioBuffer(UnsafeDirectLittleEndian.java:34) at io.netty.buffer.DrillBuf.nioBuffer(DrillBuf.java:364) at org.apache.drill.exec.store.ParquetOutputRecordWriter$VarCharParquetConverter.writeField(ParquetOutputRecordWriter.java:1824) at org.apache.drill.exec.store.EventBasedRecordWriter.write(EventBasedRecordWriter.java:48) at org.apache.drill.exec.physical.impl.WriterRecordBatch.innerNext(WriterRecordBatch.java:111) at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:186) at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:104) at org.apache.drill.exec.physical.impl.SingleSenderCreator$SingleSenderRootExec.innerNext(SingleSenderCreator.java:93) at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:94) at org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:296) at org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:283) was: * to be fixed * Stack trace: Caused by: java.lang.IndexOutOfBoundsException: index: 17, length: -799 (expected: range(0, 524288)) at io.netty.buffer.AbstractByteBuf.checkIndex0(AbstractByteBuf.java:1125) at io.netty.buffer.AbstractByteBuf.checkIndex(AbstractByteBuf.java:1120) at io.netty.buffer.PooledUnsafeDirectByteBuf.nioBuffer(PooledUnsafeDirectByteBuf.java:244) at io.netty.buffer.WrappedByteBuf.nioBuffer(WrappedByteBuf.java:734) at io.netty.buffer.UnsafeDirectLittleEndian.nioBuffer(UnsafeDirectLittleEndian.java:34) at io.netty.buffer.DrillBuf.nioBuffer(DrillBuf.java:364) at org.apache.drill.exec.store.ParquetOutputRecordWriter$VarCharParquetConverter.writeField(ParquetOutputRecordWriter.java:1824) at org.apache.drill.exec.store.EventBasedRecordWriter.write(EventBasedRecordWriter.java:48) at org.apache.drill.exec.physical.impl.WriterRecordBatch.innerNext(WriterRecordBatch.java:111) at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:186) at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:104) at org.apache.drill.exec.physical.impl.SingleSenderCreator$SingleSenderRootExec.innerNext(SingleSenderCreator.java:93) at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:94) at org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:296) at org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:283) > Reading a offset based vectors may produce IndexOutOfBoundsException > > > Key: DRILL-7355 > URL: https://issues.apache.org/jira/browse/DRILL-7355 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.16.0 >Reporter: Oleg Zinoviev >Priority: Major > > Var16Char, VarBinary, VarChar and VarDecimal vectors contain a UInt4 offset > vector. When writing and reading these vectors, the overflow of the offset > value is not processed. While reading values into the corresponding holder, > the values of their offset vectors are written into the "start" and "end" > fields as ordinary integers. w In some cases, the "start" and "end" values > are used to get the size of the buffer (for > example,ParquetOutputRecordWriter). This causes the target buffer size to be > negative. > Stack trace: > Caused by: java.lang.IndexOutOfBoundsException: index: 17, length: -799 > (expected: range(0, 524288)) > at >
[jira] [Commented] (DRILL-7355) Reading a offset based vectors may produce IndexOutOfBoundsException
[ https://issues.apache.org/jira/browse/DRILL-7355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913284#comment-16913284 ] Oleg Zinoviev commented on DRILL-7355: -- Sorry, I missed that UInt4 is a Java 'int' vector. Description need to be fixed > Reading a offset based vectors may produce IndexOutOfBoundsException > > > Key: DRILL-7355 > URL: https://issues.apache.org/jira/browse/DRILL-7355 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.16.0 > Environment: Stack trace: > Caused by: java.lang.IndexOutOfBoundsException: index: 17, length: -799 > (expected: range(0, 524288)) > at > io.netty.buffer.AbstractByteBuf.checkIndex0(AbstractByteBuf.java:1125) > at io.netty.buffer.AbstractByteBuf.checkIndex(AbstractByteBuf.java:1120) > at > io.netty.buffer.PooledUnsafeDirectByteBuf.nioBuffer(PooledUnsafeDirectByteBuf.java:244) > at io.netty.buffer.WrappedByteBuf.nioBuffer(WrappedByteBuf.java:734) > at > io.netty.buffer.UnsafeDirectLittleEndian.nioBuffer(UnsafeDirectLittleEndian.java:34) > at io.netty.buffer.DrillBuf.nioBuffer(DrillBuf.java:364) > at > org.apache.drill.exec.store.ParquetOutputRecordWriter$VarCharParquetConverter.writeField(ParquetOutputRecordWriter.java:1824) > at > org.apache.drill.exec.store.EventBasedRecordWriter.write(EventBasedRecordWriter.java:48) > at > org.apache.drill.exec.physical.impl.WriterRecordBatch.innerNext(WriterRecordBatch.java:111) > at > org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:186) > at > org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:104) > at > org.apache.drill.exec.physical.impl.SingleSenderCreator$SingleSenderRootExec.innerNext(SingleSenderCreator.java:93) > at > org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:94) > at > org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:296) > at > org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:283) >Reporter: Oleg Zinoviev >Priority: Major > > * to be fixed * -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (DRILL-7355) Reading a offset based vectors may produce IndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/DRILL-7355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleg Zinoviev updated DRILL-7355:
-
Description: * to be fixed * (was: Var16Char, VarBinary, VarChar and
VarDecimal vectors contain a UInt4 offset vector. When reading values into the
corresponding holder, the values of their offset vectors are written into the
"start" and "end" fields as ordinary integers. In the subsequent operations is
not considered that originally this values were unsigned integers and (for
example in the calculation of the buffer size at ParquetOutputRecordWriter)
usual subtraction "end" - "start" used. This causes the target buffer size to
be negative.
Stack trace:
{noformat}
Caused by: java.lang.IndexOutOfBoundsException: index: 17, length: -799
(expected: range(0, 524288))
at
io.netty.buffer.AbstractByteBuf.checkIndex0(AbstractByteBuf.java:1125)
at io.netty.buffer.AbstractByteBuf.checkIndex(AbstractByteBuf.java:1120)
at
io.netty.buffer.PooledUnsafeDirectByteBuf.nioBuffer(PooledUnsafeDirectByteBuf.java:244)
at io.netty.buffer.WrappedByteBuf.nioBuffer(WrappedByteBuf.java:734)
at
io.netty.buffer.UnsafeDirectLittleEndian.nioBuffer(UnsafeDirectLittleEndian.java:34)
at io.netty.buffer.DrillBuf.nioBuffer(DrillBuf.java:364)
at
org.apache.drill.exec.store.ParquetOutputRecordWriter$VarCharParquetConverter.writeField(ParquetOutputRecordWriter.java:1824)
at
org.apache.drill.exec.store.EventBasedRecordWriter.write(EventBasedRecordWriter.java:48)
at
org.apache.drill.exec.physical.impl.WriterRecordBatch.innerNext(WriterRecordBatch.java:111)
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:186)
at
org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:104)
at
org.apache.drill.exec.physical.impl.SingleSenderCreator$SingleSenderRootExec.innerNext(SingleSenderCreator.java:93)
at
org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:94)
at
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:296)
at
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:283)
{noformat})
Environment:
Stack trace:
Caused by: java.lang.IndexOutOfBoundsException: index: 17, length: -799
(expected: range(0, 524288))
at
io.netty.buffer.AbstractByteBuf.checkIndex0(AbstractByteBuf.java:1125)
at io.netty.buffer.AbstractByteBuf.checkIndex(AbstractByteBuf.java:1120)
at
io.netty.buffer.PooledUnsafeDirectByteBuf.nioBuffer(PooledUnsafeDirectByteBuf.java:244)
at io.netty.buffer.WrappedByteBuf.nioBuffer(WrappedByteBuf.java:734)
at
io.netty.buffer.UnsafeDirectLittleEndian.nioBuffer(UnsafeDirectLittleEndian.java:34)
at io.netty.buffer.DrillBuf.nioBuffer(DrillBuf.java:364)
at
org.apache.drill.exec.store.ParquetOutputRecordWriter$VarCharParquetConverter.writeField(ParquetOutputRecordWriter.java:1824)
at
org.apache.drill.exec.store.EventBasedRecordWriter.write(EventBasedRecordWriter.java:48)
at
org.apache.drill.exec.physical.impl.WriterRecordBatch.innerNext(WriterRecordBatch.java:111)
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:186)
at
org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:104)
at
org.apache.drill.exec.physical.impl.SingleSenderCreator$SingleSenderRootExec.innerNext(SingleSenderCreator.java:93)
at
org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:94)
at
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:296)
at
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:283)
> Reading a offset based vectors may produce IndexOutOfBoundsException
>
>
> Key: DRILL-7355
> URL: https://issues.apache.org/jira/browse/DRILL-7355
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.16.0
> Environment: Stack trace:
> Caused by: java.lang.IndexOutOfBoundsException: index: 17, length: -799
> (expected: range(0, 524288))
> at
> io.netty.buffer.AbstractByteBuf.checkIndex0(AbstractByteBuf.java:1125)
> at io.netty.buffer.AbstractByteBuf.checkIndex(AbstractByteBuf.java:1120)
> at
> io.netty.buffer.PooledUnsafeDirectByteBuf.nioBuffer(PooledUnsafeDirectByteBuf.java:244)
> at io.netty.buffer.WrappedByteBuf.nioBuffer(WrappedByteBuf.java:734)
> at
> io.netty.buffer.UnsafeDirectLittleEndian.nioBuffer(UnsafeDirectLittleEndian.java:34)
> at io.netty.buffer.DrillBuf.nioBuffer(DrillBuf.java:364)
> at
>
[jira] [Commented] (DRILL-7355) Reading a offset based vectors may produce IndexOutOfBoundsException
[ https://issues.apache.org/jira/browse/DRILL-7355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913276#comment-16913276 ] Oleg Zinoviev commented on DRILL-7355: -- Any chance this can be fixed in 1.17.0? > Reading a offset based vectors may produce IndexOutOfBoundsException > > > Key: DRILL-7355 > URL: https://issues.apache.org/jira/browse/DRILL-7355 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.16.0 > Environment: Stack trace: > Caused by: java.lang.IndexOutOfBoundsException: index: 17, length: -799 > (expected: range(0, 524288)) > at > io.netty.buffer.AbstractByteBuf.checkIndex0(AbstractByteBuf.java:1125) > at io.netty.buffer.AbstractByteBuf.checkIndex(AbstractByteBuf.java:1120) > at > io.netty.buffer.PooledUnsafeDirectByteBuf.nioBuffer(PooledUnsafeDirectByteBuf.java:244) > at io.netty.buffer.WrappedByteBuf.nioBuffer(WrappedByteBuf.java:734) > at > io.netty.buffer.UnsafeDirectLittleEndian.nioBuffer(UnsafeDirectLittleEndian.java:34) > at io.netty.buffer.DrillBuf.nioBuffer(DrillBuf.java:364) > at > org.apache.drill.exec.store.ParquetOutputRecordWriter$VarCharParquetConverter.writeField(ParquetOutputRecordWriter.java:1824) > at > org.apache.drill.exec.store.EventBasedRecordWriter.write(EventBasedRecordWriter.java:48) > at > org.apache.drill.exec.physical.impl.WriterRecordBatch.innerNext(WriterRecordBatch.java:111) > at > org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:186) > at > org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:104) > at > org.apache.drill.exec.physical.impl.SingleSenderCreator$SingleSenderRootExec.innerNext(SingleSenderCreator.java:93) > at > org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:94) > at > org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:296) > at > org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:283) >Reporter: Oleg Zinoviev >Priority: Major > > Var16Char, VarBinary, VarChar and VarDecimal vectors contain a UInt4 offset > vector. When reading values into the corresponding holder, the values of > their offset vectors are written into the "start" and "end" fields as > ordinary integers. In the subsequent operations is not considered that > originally this values were unsigned integers and (for example in the > calculation of the buffer size at ParquetOutputRecordWriter) usual > subtraction "end" - "start" used. This causes the target buffer size to be > negative. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (DRILL-7355) Reading a offset based vectors may produce IndexOutOfBoundsException
[ https://issues.apache.org/jira/browse/DRILL-7355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Zinoviev updated DRILL-7355: - Summary: Reading a offset based vectors may produce IndexOutOfBoundsException (was: Reading a offset based vectors into holder may produce IndexOutOfBoundsException) > Reading a offset based vectors may produce IndexOutOfBoundsException > > > Key: DRILL-7355 > URL: https://issues.apache.org/jira/browse/DRILL-7355 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.16.0 > Environment: Stack trace: > Caused by: java.lang.IndexOutOfBoundsException: index: 17, length: -799 > (expected: range(0, 524288)) > at > io.netty.buffer.AbstractByteBuf.checkIndex0(AbstractByteBuf.java:1125) > at io.netty.buffer.AbstractByteBuf.checkIndex(AbstractByteBuf.java:1120) > at > io.netty.buffer.PooledUnsafeDirectByteBuf.nioBuffer(PooledUnsafeDirectByteBuf.java:244) > at io.netty.buffer.WrappedByteBuf.nioBuffer(WrappedByteBuf.java:734) > at > io.netty.buffer.UnsafeDirectLittleEndian.nioBuffer(UnsafeDirectLittleEndian.java:34) > at io.netty.buffer.DrillBuf.nioBuffer(DrillBuf.java:364) > at > org.apache.drill.exec.store.ParquetOutputRecordWriter$VarCharParquetConverter.writeField(ParquetOutputRecordWriter.java:1824) > at > org.apache.drill.exec.store.EventBasedRecordWriter.write(EventBasedRecordWriter.java:48) > at > org.apache.drill.exec.physical.impl.WriterRecordBatch.innerNext(WriterRecordBatch.java:111) > at > org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:186) > at > org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:104) > at > org.apache.drill.exec.physical.impl.SingleSenderCreator$SingleSenderRootExec.innerNext(SingleSenderCreator.java:93) > at > org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:94) > at > org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:296) > at > org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:283) >Reporter: Oleg Zinoviev >Priority: Major > > Var16Char, VarBinary, VarChar and VarDecimal vectors contain a UInt4 offset > vector. When reading values into the corresponding holder, the values of > their offset vectors are written into the "start" and "end" fields as > ordinary integers. In the subsequent operations is not considered that > originally this values were unsigned integers and (for example in the > calculation of the buffer size at ParquetOutputRecordWriter) usual > subtraction "end" - "start" used. This causes the target buffer size to be > negative. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (DRILL-7355) Reading a offset based vectors into holder may produce IndexOutOfBoundsException
Oleg Zinoviev created DRILL-7355: Summary: Reading a offset based vectors into holder may produce IndexOutOfBoundsException Key: DRILL-7355 URL: https://issues.apache.org/jira/browse/DRILL-7355 Project: Apache Drill Issue Type: Bug Affects Versions: 1.16.0 Environment: Stack trace: Caused by: java.lang.IndexOutOfBoundsException: index: 17, length: -799 (expected: range(0, 524288)) at io.netty.buffer.AbstractByteBuf.checkIndex0(AbstractByteBuf.java:1125) at io.netty.buffer.AbstractByteBuf.checkIndex(AbstractByteBuf.java:1120) at io.netty.buffer.PooledUnsafeDirectByteBuf.nioBuffer(PooledUnsafeDirectByteBuf.java:244) at io.netty.buffer.WrappedByteBuf.nioBuffer(WrappedByteBuf.java:734) at io.netty.buffer.UnsafeDirectLittleEndian.nioBuffer(UnsafeDirectLittleEndian.java:34) at io.netty.buffer.DrillBuf.nioBuffer(DrillBuf.java:364) at org.apache.drill.exec.store.ParquetOutputRecordWriter$VarCharParquetConverter.writeField(ParquetOutputRecordWriter.java:1824) at org.apache.drill.exec.store.EventBasedRecordWriter.write(EventBasedRecordWriter.java:48) at org.apache.drill.exec.physical.impl.WriterRecordBatch.innerNext(WriterRecordBatch.java:111) at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:186) at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:104) at org.apache.drill.exec.physical.impl.SingleSenderCreator$SingleSenderRootExec.innerNext(SingleSenderCreator.java:93) at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:94) at org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:296) at org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:283) Reporter: Oleg Zinoviev Var16Char, VarBinary, VarChar and VarDecimal vectors contain a UInt4 offset vector. When reading values into the corresponding holder, the values of their offset vectors are written into the "start" and "end" fields as ordinary integers. In the subsequent operations is not considered that originally this values were unsigned integers and (for example in the calculation of the buffer size at ParquetOutputRecordWriter) usual subtraction "end" - "start" used. This causes the target buffer size to be negative. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (DRILL-7349) ClusterTest dfs.tmp storage inconsistence with cluster size > 1
[
https://issues.apache.org/jira/browse/DRILL-7349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleg Zinoviev updated DRILL-7349:
-
Description:
ClusterTest#dirTestWatcher works inconsistent if cluster size is greater when 1.
For example, test
{code:java}
public class TestClusterTest extends ClusterTest {
@BeforeClass
public static void setUp() throws Exception {
startCluster(ClusterFixture.builder(dirTestWatcher).clusterSize(2));
}
@Test
public void testWriteFile() throws Exception {
final String outputFileName = "some_file_name";
final File outputFile =
FileUtils.getFile(dirTestWatcher.getDfsTestTmpDir(), outputFileName);
queryBuilder().sql("CREATE TABLE dfs.tmp.%s AS SELECT * FROM
cp.`employee.json` limit 1", outputFileName).run();
assertTrue(outputFile.exists());
}
}
{code}
may fail on assertion.
Reason: several directories for the storage `dfs.tmp` (possibly 1 for each
node) was created, and dirTestWatcher.getDfsTestTmpDir () may return the wrong
one.
was:
ClusterTest#dirTestWatcher works inconsistent if cluster size is greater when 1.
For example, test
{code:java}
public class TestClusterTest extends ClusterTest {
@BeforeClass
public static void setUp() throws Exception {
startCluster(ClusterFixture.builder(dirTestWatcher).clusterSize(2));
}
@Test
public void testWriteFile() throws Exception {
final String outputFileName =
"testparquetwriteremptyfiles_testwriteemptyfile";
final File outputFile =
FileUtils.getFile(dirTestWatcher.getDfsTestTmpDir(), outputFileName);
queryBuilder().sql("CREATE TABLE dfs.tmp.%s AS SELECT * FROM
cp.`employee.json` limit 1", outputFileName).run();
assertTrue(outputFile.exists());
}
}
{code}
may fail on assertion.
Reason: several directories for the storage `dfs.tmp` (possibly 1 for each
node) was created, and dirTestWatcher.getDfsTestTmpDir () may return the wrong
one.
> ClusterTest dfs.tmp storage inconsistence with cluster size > 1
> ---
>
> Key: DRILL-7349
> URL: https://issues.apache.org/jira/browse/DRILL-7349
> Project: Apache Drill
> Issue Type: Bug
> Components: Tools, Build Test
>Affects Versions: 1.17.0
>Reporter: Oleg Zinoviev
>Priority: Minor
>
> ClusterTest#dirTestWatcher works inconsistent if cluster size is greater when
> 1.
> For example, test
> {code:java}
> public class TestClusterTest extends ClusterTest {
> @BeforeClass
> public static void setUp() throws Exception {
> startCluster(ClusterFixture.builder(dirTestWatcher).clusterSize(2));
> }
> @Test
> public void testWriteFile() throws Exception {
> final String outputFileName = "some_file_name";
> final File outputFile =
> FileUtils.getFile(dirTestWatcher.getDfsTestTmpDir(), outputFileName);
> queryBuilder().sql("CREATE TABLE dfs.tmp.%s AS SELECT * FROM
> cp.`employee.json` limit 1", outputFileName).run();
> assertTrue(outputFile.exists());
> }
> }
> {code}
> may fail on assertion.
> Reason: several directories for the storage `dfs.tmp` (possibly 1 for each
> node) was created, and dirTestWatcher.getDfsTestTmpDir () may return the
> wrong one.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Created] (DRILL-7349) ClusterTest dfs.tmp storage inconsistence with cluster size > 1
Oleg Zinoviev created DRILL-7349:
Summary: ClusterTest dfs.tmp storage inconsistence with cluster
size > 1
Key: DRILL-7349
URL: https://issues.apache.org/jira/browse/DRILL-7349
Project: Apache Drill
Issue Type: Bug
Components: Tools, Build Test
Affects Versions: 1.17.0
Reporter: Oleg Zinoviev
ClusterTest#dirTestWatcher works inconsistent if cluster size is greater when 1.
For example, test
{code:java}
public class TestClusterTest extends ClusterTest {
@BeforeClass
public static void setUp() throws Exception {
startCluster(ClusterFixture.builder(dirTestWatcher).clusterSize(2));
}
@Test
public void testWriteFile() throws Exception {
final String outputFileName =
"testparquetwriteremptyfiles_testwriteemptyfile";
final File outputFile =
FileUtils.getFile(dirTestWatcher.getDfsTestTmpDir(), outputFileName);
queryBuilder().sql("CREATE TABLE dfs.tmp.%s AS SELECT * FROM
cp.`employee.json` limit 1", outputFileName).run();
assertTrue(outputFile.exists());
}
}
{code}
may fail on assertion.
Reason: several directories for the storage `dfs.tmp` (possibly 1 for each
node) was created, and dirTestWatcher.getDfsTestTmpDir () may return the wrong
one.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (DRILL-7341) Vector reAlloc may fails after exchange.
[ https://issues.apache.org/jira/browse/DRILL-7341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Zinoviev updated DRILL-7341: - Summary: Vector reAlloc may fails after exchange. (was: Vector reAllocation may fails after exchange.) > Vector reAlloc may fails after exchange. > > > Key: DRILL-7341 > URL: https://issues.apache.org/jira/browse/DRILL-7341 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.16.0 >Reporter: Oleg Zinoviev >Priority: Major > Attachments: stacktrace.log > > > There are several methods that modify the BaseDataValueVector#data field. > Some of them, such as BaseDataValueVector#exchange, do not change > allocationSizeInBytes. > Therefore, if BaseDataValueVector#exchange was executed for vectors with > different size, *ValueVector#reAlloc may create a buffer of insufficient size. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (DRILL-7341) Vector reAllocation may fails after exchange.
Oleg Zinoviev created DRILL-7341: Summary: Vector reAllocation may fails after exchange. Key: DRILL-7341 URL: https://issues.apache.org/jira/browse/DRILL-7341 Project: Apache Drill Issue Type: Bug Affects Versions: 1.16.0 Reporter: Oleg Zinoviev Attachments: stacktrace.log There are several methods that modify the BaseDataValueVector#data field. Some of them, such as BaseDataValueVector#exchange, do not change allocationSizeInBytes. Therefore, if BaseDataValueVector#exchange was executed for vectors with different size, *ValueVector#reAlloc may create a buffer of insufficient size. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (DRILL-7263) Quoted column names clashes with functions
Oleg Zinoviev created DRILL-7263:
Summary: Quoted column names clashes with functions
Key: DRILL-7263
URL: https://issues.apache.org/jira/browse/DRILL-7263
Project: Apache Drill
Issue Type: Bug
Affects Versions: 1.16.0, 1.15.0
Reporter: Oleg Zinoviev
If the quoted column name without specifying the source alias, matches the
function name without parameters, Apache Drill executes function.
Query:
{code:sql}
select `CURRENT_TIMESTAMP` from (select 1 as `CURRENT_TIMESTAMP`) s
{code}
Result:
||CURRENT_TIMESTAMP||
|2019-05-17T14:23:17.708|
Expected:
||CURRENT_TIMESTAMP||
|1|
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-7050) RexNode convert exception in subquery
[
https://issues.apache.org/jira/browse/DRILL-7050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16840298#comment-16840298
]
Oleg Zinoviev commented on DRILL-7050:
--
Could you check this query:
{code:sql}
select t1.id,
(select count(t2.id)
from (
select 1 as id
union all
select 2 as id
) t2 where t2.id = t1.id)
from (
select 1 as id
union all
select 2 as id
) t1
{code}
After this fix it fails with
{noformat}
SYSTEM ERROR: UnsupportedOperationException: Adding Implicit RowID column is
not supported for ValuesPrel operator
{noformat}
> RexNode convert exception in subquery
> -
>
> Key: DRILL-7050
> URL: https://issues.apache.org/jira/browse/DRILL-7050
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.14.0, 1.15.0
>Reporter: Oleg Zinoviev
>Assignee: Volodymyr Vysotskyi
>Priority: Major
> Labels: ready-to-commit
> Fix For: 1.17.0
>
>
> If the query contains a subquery whose filters are associated with the main
> query, an error occurs: *PLAN ERROR: Cannot convert RexNode to equivalent
> Drill expression. RexNode Class: org.apache.calcite.rex.RexCorrelVariable*
> Steps to reproduce:
> 1) Create source table (or view, doesn't matter)
> {code:sql}
> create table dfs.root.source as (
> select 1 as id union all select 2 as id
> )
> {code}
> 2) Execute query
> {code:sql}
> select t1.id,
> (select count(t2.id)
> from dfs.root.source t2 where t2.id = t1.id)
> from dfs.root.source t1
> {code}
> Reason:
> Method
> {code:java}org.apache.calcite.sql2rel.SqlToRelConverter.Blackboard.lookupExp{code}
> call {code:java}RexBuilder.makeCorrel{code} in some cases
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-7156) Empty Parquet is not getting created if 0 records in result
[ https://issues.apache.org/jira/browse/DRILL-7156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16840295#comment-16840295 ] Oleg Zinoviev commented on DRILL-7156: -- I had to deal with this problem. As a result, I made my own Apache Drill build, which supports the creation and reading of empty parquet files. I can try to prepare a patch in the main repository, if this is really a Drill problem. > Empty Parquet is not getting created if 0 records in result > --- > > Key: DRILL-7156 > URL: https://issues.apache.org/jira/browse/DRILL-7156 > Project: Apache Drill > Issue Type: Bug > Components: Storage - Parquet >Reporter: Sayalee Bhanavase >Priority: Blocker > > I am creating parquet tables out of joins. If there is no record in join, it > does not create empty. table and when I reused the table my further script > fails. > Has anyone faced this issue? Any suggestion or workaround? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-7103) BSON Recored reader: wrong timestamp values
[ https://issues.apache.org/jira/browse/DRILL-7103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16792684#comment-16792684 ] Oleg Zinoviev commented on DRILL-7103: -- Way to reproduce: Try to read timestamp after 1970-01-25 (in my case it was 2015-01-01) from mongoDB > BSON Recored reader: wrong timestamp values > --- > > Key: DRILL-7103 > URL: https://issues.apache.org/jira/browse/DRILL-7103 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.14.0, 1.15.0 >Reporter: Oleg Zinoviev >Priority: Major > > The BsonRecordReader # writeTimeStamp method converts the DateTime#getMillis > () result to an integer. This leads to arithmetic overflow if the date is > after 1970-01-25. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (DRILL-7103) BSON Recored reader: wrong timestamp values
Oleg Zinoviev created DRILL-7103: Summary: BSON Recored reader: wrong timestamp values Key: DRILL-7103 URL: https://issues.apache.org/jira/browse/DRILL-7103 Project: Apache Drill Issue Type: Bug Affects Versions: 1.15.0, 1.14.0 Reporter: Oleg Zinoviev The BsonRecordReader # writeTimeStamp method converts the DateTime#getMillis () result to an integer. This leads to arithmetic overflow if the date is after 1970-01-25. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-7050) RexNode convert exception in subquery
[
https://issues.apache.org/jira/browse/DRILL-7050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleg Zinoviev updated DRILL-7050:
-
Description:
If the query contains a subquery whose filters are associated with the main
query, an error occurs: *PLAN ERROR: Cannot convert RexNode to equivalent Drill
expression. RexNode Class: org.apache.calcite.rex.RexCorrelVariable*
Steps to reproduce:
1) Create source table (or view, doesn't matter)
{code:sql}
create table dfs.root.source as (
select 1 as id union all select 2 as id
)
{code}
2) Execute query
{code:sql}
select t1.id,
(select count(t2.id)
from dfs.root.source t2 where t2.id = t1.id)
from dfs.root.source t1
{code}
Reason:
Method
{code:java}org.apache.calcite.sql2rel.SqlToRelConverter.Blackboard.lookupExp{code}
call {code:java}RexBuilder.makeCorrel{code} in some cases
was:
If the query contains a subquery whose filters are associated with the main
query, an error occurs: *PLAN ERROR: Cannot convert RexNode to equivalent Drill
expression. RexNode Class: org.apache.calcite.rex.RexCorrelVariable*
Steps to reproduce:
1) Create source table (or view, doesn't matter)
{code:sql}
create table dfs.root.source as (
select 1 as id union all select 2 as id
)
{code}
2) Execute query
{code:sql}
select t1.id,
(select count(t2.id)
from storage.datasets.source t2 where t2.id = t1.id)
from storage.datasets.source t1
{code}
Reason:
Method
{code:java}org.apache.calcite.sql2rel.SqlToRelConverter.Blackboard.lookupExp{code}
call {code:java}RexBuilder.makeCorrel{code} in some cases
> RexNode convert exception in subquery
> -
>
> Key: DRILL-7050
> URL: https://issues.apache.org/jira/browse/DRILL-7050
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.14.0, 1.15.0
>Reporter: Oleg Zinoviev
>Priority: Major
>
> If the query contains a subquery whose filters are associated with the main
> query, an error occurs: *PLAN ERROR: Cannot convert RexNode to equivalent
> Drill expression. RexNode Class: org.apache.calcite.rex.RexCorrelVariable*
> Steps to reproduce:
> 1) Create source table (or view, doesn't matter)
> {code:sql}
> create table dfs.root.source as (
> select 1 as id union all select 2 as id
> )
> {code}
> 2) Execute query
> {code:sql}
> select t1.id,
> (select count(t2.id)
> from dfs.root.source t2 where t2.id = t1.id)
> from dfs.root.source t1
> {code}
> Reason:
> Method
> {code:java}org.apache.calcite.sql2rel.SqlToRelConverter.Blackboard.lookupExp{code}
> call {code:java}RexBuilder.makeCorrel{code} in some cases
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-7050) RexNode convert exception in subquery
[
https://issues.apache.org/jira/browse/DRILL-7050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleg Zinoviev updated DRILL-7050:
-
Description:
If the query contains a subquery whose filters are associated with the main
query, an error occurs: *PLAN ERROR: Cannot convert RexNode to equivalent Drill
expression. RexNode Class: org.apache.calcite.rex.RexCorrelVariable*
Steps to reproduce:
1) Create source table (or view, doesn't matter)
{code:sql}
create table dfs.root.source as (
select 1 as id union all select 2 as id
)
{code}
2) Execute query
{code:sql}
select t1.id,
(select count(t2.id)
from storage.datasets.source t2 where t2.id = t1.id)
from storage.datasets.source t1
{code}
Reason:
Method
{code:java}org.apache.calcite.sql2rel.SqlToRelConverter.Blackboard.lookupExp{code}
call {code:java}RexBuilder.makeCorrel{code} in some cases
was:
If the query contains a subquery whose filters are associated with the main
query, an error occurs: ```PLAN ERROR: Cannot convert RexNode to equivalent
Drill expression. RexNode Class: org.apache.calcite.rex.RexCorrelVariable```
Steps to reproduce:
1) Create source table (or view, doesn't matter)
{code:sql}
create table dfs.root.source as (
select 1 as id union all select 2 as id
)
{code}
2) Execute query
{code:sql}
select t1.id,
(select count(t2.id)
from storage.datasets.source t2 where t2.id = t1.id)
from storage.datasets.source t1
{code}
Reason:
Method
{code:java}org.apache.calcite.sql2rel.SqlToRelConverter.Blackboard.lookupExp{code}
call {code:java}RexBuilder.makeCorrel{code} in some cases
> RexNode convert exception in subquery
> -
>
> Key: DRILL-7050
> URL: https://issues.apache.org/jira/browse/DRILL-7050
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.14.0, 1.15.0
>Reporter: Oleg Zinoviev
>Priority: Major
>
> If the query contains a subquery whose filters are associated with the main
> query, an error occurs: *PLAN ERROR: Cannot convert RexNode to equivalent
> Drill expression. RexNode Class: org.apache.calcite.rex.RexCorrelVariable*
> Steps to reproduce:
> 1) Create source table (or view, doesn't matter)
> {code:sql}
> create table dfs.root.source as (
> select 1 as id union all select 2 as id
> )
> {code}
> 2) Execute query
> {code:sql}
> select t1.id,
> (select count(t2.id)
> from storage.datasets.source t2 where t2.id = t1.id)
> from storage.datasets.source t1
> {code}
> Reason:
> Method
> {code:java}org.apache.calcite.sql2rel.SqlToRelConverter.Blackboard.lookupExp{code}
> call {code:java}RexBuilder.makeCorrel{code} in some cases
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (DRILL-7050) RexNode convert exception in subquery
Oleg Zinoviev created DRILL-7050: Summary: RexNode convert exception in subquery Key: DRILL-7050 URL: https://issues.apache.org/jira/browse/DRILL-7050 Project: Apache Drill Issue Type: Bug Affects Versions: 1.15.0, 1.14.0 Reporter: Oleg Zinoviev If the query contains a subquery whose filters are associated with the main query, an error occurs: ```PLAN ERROR: Cannot convert RexNode to equivalent Drill expression. RexNode Class: org.apache.calcite.rex.RexCorrelVariable``` Steps to reproduce: 1) Create source table (or view, doesn't matter) ```sql create table dfs.root.source as ( select 1 as id union all select 2 as id ) ``` 2) Execute query ```sql select t1.id, (select count(t2.id) from storage.datasets.source t2 where t2.id = t1.id) from storage.datasets.source t1 ``` Reason: Method ```org.apache.calcite.sql2rel.SqlToRelConverter.Blackboard.lookupExp``` call ```RexBuilder.makeCorrel``` in some cases -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-7050) RexNode convert exception in subquery
[
https://issues.apache.org/jira/browse/DRILL-7050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleg Zinoviev updated DRILL-7050:
-
Description:
If the query contains a subquery whose filters are associated with the main
query, an error occurs: ```PLAN ERROR: Cannot convert RexNode to equivalent
Drill expression. RexNode Class: org.apache.calcite.rex.RexCorrelVariable```
Steps to reproduce:
1) Create source table (or view, doesn't matter)
{code:sql}
create table dfs.root.source as (
select 1 as id union all select 2 as id
)
{code}
2) Execute query
{code:sql}
select t1.id,
(select count(t2.id)
from storage.datasets.source t2 where t2.id = t1.id)
from storage.datasets.source t1
{code}
Reason:
Method
{code:java}org.apache.calcite.sql2rel.SqlToRelConverter.Blackboard.lookupExp{code}
call {code:java}RexBuilder.makeCorrel{code} in some cases
was:
If the query contains a subquery whose filters are associated with the main
query, an error occurs: ```PLAN ERROR: Cannot convert RexNode to equivalent
Drill expression. RexNode Class: org.apache.calcite.rex.RexCorrelVariable```
Steps to reproduce:
1) Create source table (or view, doesn't matter)
```sql
create table dfs.root.source as (
select 1 as id union all select 2 as id
)
```
2) Execute query
```sql
select t1.id,
(select count(t2.id)
from storage.datasets.source t2 where t2.id = t1.id)
from storage.datasets.source t1
```
Reason:
Method ```org.apache.calcite.sql2rel.SqlToRelConverter.Blackboard.lookupExp```
call ```RexBuilder.makeCorrel``` in some cases
> RexNode convert exception in subquery
> -
>
> Key: DRILL-7050
> URL: https://issues.apache.org/jira/browse/DRILL-7050
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.14.0, 1.15.0
>Reporter: Oleg Zinoviev
>Priority: Major
>
> If the query contains a subquery whose filters are associated with the main
> query, an error occurs: ```PLAN ERROR: Cannot convert RexNode to equivalent
> Drill expression. RexNode Class: org.apache.calcite.rex.RexCorrelVariable```
> Steps to reproduce:
> 1) Create source table (or view, doesn't matter)
> {code:sql}
> create table dfs.root.source as (
> select 1 as id union all select 2 as id
> )
> {code}
> 2) Execute query
> {code:sql}
> select t1.id,
> (select count(t2.id)
> from storage.datasets.source t2 where t2.id = t1.id)
> from storage.datasets.source t1
> {code}
> Reason:
> Method
> {code:java}org.apache.calcite.sql2rel.SqlToRelConverter.Blackboard.lookupExp{code}
> call {code:java}RexBuilder.makeCorrel{code} in some cases
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-6867) WebUI Query editor cursor position
[ https://issues.apache.org/jira/browse/DRILL-6867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Zinoviev updated DRILL-6867: - Priority: Minor (was: Major) > WebUI Query editor cursor position > -- > > Key: DRILL-6867 > URL: https://issues.apache.org/jira/browse/DRILL-6867 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.14.0 >Reporter: Oleg Zinoviev >Priority: Minor > Attachments: query.png > > > In Firefox (at least on Ununtu 18.10), the position of the cursor does not > match the text when editing the SQL-query > !query.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (DRILL-6867) WebUI Query editor cursor position
Oleg Zinoviev created DRILL-6867: Summary: WebUI Query editor cursor position Key: DRILL-6867 URL: https://issues.apache.org/jira/browse/DRILL-6867 Project: Apache Drill Issue Type: Bug Reporter: Oleg Zinoviev Attachments: query In Firefox (at least on Ununtu 18.10), the position of the cursor does not match the text when editing the SQL-query [^query] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6867) WebUI Query editor cursor position
[ https://issues.apache.org/jira/browse/DRILL-6867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Zinoviev updated DRILL-6867: - Attachment: query.png > WebUI Query editor cursor position > -- > > Key: DRILL-6867 > URL: https://issues.apache.org/jira/browse/DRILL-6867 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.14.0 >Reporter: Oleg Zinoviev >Priority: Major > Attachments: query.png > > > In Firefox (at least on Ununtu 18.10), the position of the cursor does not > match the text when editing the SQL-query > [^query] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6867) WebUI Query editor cursor position
[ https://issues.apache.org/jira/browse/DRILL-6867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16696568#comment-16696568 ] Oleg Zinoviev commented on DRILL-6867: -- Reason: Courier font is missing in Ubuntu 18.10 Possible fix: Add few additional font families to query-editor-format > WebUI Query editor cursor position > -- > > Key: DRILL-6867 > URL: https://issues.apache.org/jira/browse/DRILL-6867 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.14.0 >Reporter: Oleg Zinoviev >Priority: Major > Attachments: query.png > > > In Firefox (at least on Ununtu 18.10), the position of the cursor does not > match the text when editing the SQL-query > !query.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6867) WebUI Query editor cursor position
[ https://issues.apache.org/jira/browse/DRILL-6867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Zinoviev updated DRILL-6867: - Attachment: (was: query) > WebUI Query editor cursor position > -- > > Key: DRILL-6867 > URL: https://issues.apache.org/jira/browse/DRILL-6867 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.14.0 >Reporter: Oleg Zinoviev >Priority: Major > Attachments: query.png > > > In Firefox (at least on Ununtu 18.10), the position of the cursor does not > match the text when editing the SQL-query > [^query] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6867) WebUI Query editor cursor position
[ https://issues.apache.org/jira/browse/DRILL-6867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Zinoviev updated DRILL-6867: - Description: In Firefox (at least on Ununtu 18.10), the position of the cursor does not match the text when editing the SQL-query !query.png! was: In Firefox (at least on Ununtu 18.10), the position of the cursor does not match the text when editing the SQL-query [^query] > WebUI Query editor cursor position > -- > > Key: DRILL-6867 > URL: https://issues.apache.org/jira/browse/DRILL-6867 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.14.0 >Reporter: Oleg Zinoviev >Priority: Major > Attachments: query.png > > > In Firefox (at least on Ununtu 18.10), the position of the cursor does not > match the text when editing the SQL-query > !query.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6867) WebUI Query editor cursor position
[ https://issues.apache.org/jira/browse/DRILL-6867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Zinoviev updated DRILL-6867: - Affects Version/s: 1.14.0 > WebUI Query editor cursor position > -- > > Key: DRILL-6867 > URL: https://issues.apache.org/jira/browse/DRILL-6867 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.14.0 >Reporter: Oleg Zinoviev >Priority: Major > Attachments: query > > > In Firefox (at least on Ununtu 18.10), the position of the cursor does not > match the text when editing the SQL-query > [^query] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (DRILL-6722) Query from parquet with case-then and arithmetic operation returns a wrong result
[
https://issues.apache.org/jira/browse/DRILL-6722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleg Zinoviev updated DRILL-6722:
-
Comment: was deleted
(was: At this stage, it seems to me that the whole logic of the
ScalarReplacementNode is broken. Neither the variable index in ValueHolderSub,
nor mapping local variable to frame slot in MethodAnalyzer does not track
source code control flow.
Bug reason:
Then
ALOAD[P]
ASTORE[N]
executes, ValueHolderSub linked with variable P changes it's `first` field to N
holder `first` field. Since ValueHolderSub shared between all variables, linked
to him, it leads to all this varaibles become assigned to N.
If I make first field final, CASE fails:
CASE compiles into
{code:java}
var res;
if (expression) {
res = v1;
else {
res = v2;
}
{code}
Then Analyzer calculates Frame variables, it ignores if - else logic and
assigne v2 to res slot.
P.S.
https://stackoverflow.com/questions/25109942/is-there-a-better-explanation-of-stack-map-frames
Can be Analyzed is intended only for variable type inferrence?
)
> Query from parquet with case-then and arithmetic operation returns a wrong
> result
> -
>
> Key: DRILL-6722
> URL: https://issues.apache.org/jira/browse/DRILL-6722
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Codegen
>Affects Versions: 1.14.0
>Reporter: Oleg Zinoviev
>Priority: Major
> Attachments: JaininoJava.class, JaininoJava2_merged.class,
> correct.csv, result.csv
>
>
> Steps to reproduce:
> 1) Create sample table:
> {code:sql}
> create table dfs.tmp.test as
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> {code}
> 2) Execute query:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from dfs.tmp.test s
> {code}
> 3) Drill returns: [^result.csv]
> 4) Result of query without parquet:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from (
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> ) s
> {code}
> [^correct.csv]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (DRILL-6722) Query from parquet with case-then and arithmetic operation returns a wrong result
[
https://issues.apache.org/jira/browse/DRILL-6722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16673478#comment-16673478
]
Oleg Zinoviev edited comment on DRILL-6722 at 11/2/18 5:53 PM:
---
At this stage, it seems to me that the whole logic of the ScalarReplacementNode
is broken. Neither the variable index in ValueHolderSub, nor mapping local
variable to frame slot in MethodAnalyzer does not track source code control
flow.
Bug reason:
Then
ALOAD[P]
ASTORE[N]
executes, ValueHolderSub linked with variable P changes it's `first` field to N
holder `first` field. Since ValueHolderSub shared between all variables, linked
to him, it leads to all this varaibles become assigned to P.
If I make first field final, CASE fails:
CASE compiles into
{code:java}
var res;
if (expression) {
res = v1;
else {
res = v2;
}
{code}
Then Analyzer calculates Frame variables, it ignores if - else logic and
assigne v2 to res slot.
P.S.
https://stackoverflow.com/questions/25109942/is-there-a-better-explanation-of-stack-map-frames
Can be Analyzed is intended only for variable type inferrence?
was (Author: le.louch):
At this stage, it seems to me that the whole logic of the ScalarReplacementNode
is broken. Neither the variable index in ValueHolderSub, nor mapping local
variable to frame slot in MethodAnalyzer does not track source code control
flow.
Bug reason:
Then
ALOAD[P]
ASTORE[N]
executes, ValueHolderSub linked with variable N changes it's `first` field to P
holder `first` field. Since ValueHolderSub shared between all variables, linked
to him, it leads to all this varaibles become assigned to P.
If I make first field final, CASE fails:
CASE compiles into
{code:java}
var res;
if (expression) {
res = v1;
else {
res = v2;
}
{code}
Then Analyzer calculates Frame variables, it ignores if - else logic and
assigne v2 to res slot.
P.S.
https://stackoverflow.com/questions/25109942/is-there-a-better-explanation-of-stack-map-frames
Can be Analyzed is intended only for variable type inferrence?
> Query from parquet with case-then and arithmetic operation returns a wrong
> result
> -
>
> Key: DRILL-6722
> URL: https://issues.apache.org/jira/browse/DRILL-6722
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Codegen
>Affects Versions: 1.14.0
>Reporter: Oleg Zinoviev
>Priority: Major
> Attachments: JaininoJava.class, JaininoJava2_merged.class,
> correct.csv, result.csv
>
>
> Steps to reproduce:
> 1) Create sample table:
> {code:sql}
> create table dfs.tmp.test as
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> {code}
> 2) Execute query:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from dfs.tmp.test s
> {code}
> 3) Drill returns: [^result.csv]
> 4) Result of query without parquet:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from (
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> ) s
> {code}
> [^correct.csv]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (DRILL-6722) Query from parquet with case-then and arithmetic operation returns a wrong result
[
https://issues.apache.org/jira/browse/DRILL-6722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16673478#comment-16673478
]
Oleg Zinoviev edited comment on DRILL-6722 at 11/2/18 5:53 PM:
---
At this stage, it seems to me that the whole logic of the ScalarReplacementNode
is broken. Neither the variable index in ValueHolderSub, nor mapping local
variable to frame slot in MethodAnalyzer does not track source code control
flow.
Bug reason:
Then
ALOAD[P]
ASTORE[N]
executes, ValueHolderSub linked with variable P changes it's `first` field to N
holder `first` field. Since ValueHolderSub shared between all variables, linked
to him, it leads to all this varaibles become assigned to N.
If I make first field final, CASE fails:
CASE compiles into
{code:java}
var res;
if (expression) {
res = v1;
else {
res = v2;
}
{code}
Then Analyzer calculates Frame variables, it ignores if - else logic and
assigne v2 to res slot.
P.S.
https://stackoverflow.com/questions/25109942/is-there-a-better-explanation-of-stack-map-frames
Can be Analyzed is intended only for variable type inferrence?
was (Author: le.louch):
At this stage, it seems to me that the whole logic of the ScalarReplacementNode
is broken. Neither the variable index in ValueHolderSub, nor mapping local
variable to frame slot in MethodAnalyzer does not track source code control
flow.
Bug reason:
Then
ALOAD[P]
ASTORE[N]
executes, ValueHolderSub linked with variable P changes it's `first` field to N
holder `first` field. Since ValueHolderSub shared between all variables, linked
to him, it leads to all this varaibles become assigned to P.
If I make first field final, CASE fails:
CASE compiles into
{code:java}
var res;
if (expression) {
res = v1;
else {
res = v2;
}
{code}
Then Analyzer calculates Frame variables, it ignores if - else logic and
assigne v2 to res slot.
P.S.
https://stackoverflow.com/questions/25109942/is-there-a-better-explanation-of-stack-map-frames
Can be Analyzed is intended only for variable type inferrence?
> Query from parquet with case-then and arithmetic operation returns a wrong
> result
> -
>
> Key: DRILL-6722
> URL: https://issues.apache.org/jira/browse/DRILL-6722
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Codegen
>Affects Versions: 1.14.0
>Reporter: Oleg Zinoviev
>Priority: Major
> Attachments: JaininoJava.class, JaininoJava2_merged.class,
> correct.csv, result.csv
>
>
> Steps to reproduce:
> 1) Create sample table:
> {code:sql}
> create table dfs.tmp.test as
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> {code}
> 2) Execute query:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from dfs.tmp.test s
> {code}
> 3) Drill returns: [^result.csv]
> 4) Result of query without parquet:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from (
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> ) s
> {code}
> [^correct.csv]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (DRILL-6722) Query from parquet with case-then and arithmetic operation returns a wrong result
[
https://issues.apache.org/jira/browse/DRILL-6722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16673478#comment-16673478
]
Oleg Zinoviev edited comment on DRILL-6722 at 11/2/18 5:51 PM:
---
At this stage, it seems to me that the whole logic of the ScalarReplacementNode
is broken. Neither the variable index in ValueHolderSub, nor mapping local
variable to frame slot in MethodAnalyzer does not track source code control
flow.
Bug reason:
Then
ALOAD[P]
ASTORE[N]
executes, ValueHolderSub linked with variable N changes it's `first` field to P
holder `first` field. Since ValueHolderSub shared between all variables, linked
to him, it leads to all this varaibles become assigned to P.
If I make first field final, CASE fails:
CASE compiles into
{code:java}
var res;
if (expression) {
res = v1;
else {
res = v2;
}
{code}
Then Analyzer calculates Frame variables, it ignores if - else logic and
assigne v2 to res slot.
P.S.
https://stackoverflow.com/questions/25109942/is-there-a-better-explanation-of-stack-map-frames
Can be Analyzed is intended only for variable type inferrence?
was (Author: le.louch):
At this stage, it seems to me that the whole logic of the MergeAdapter is
broken. Neither the variable index in ValueHolderSub, nor mapping local
variable to frame slot in MethodAnalyzer does not track source code control
flow.
Bug reason:
Then
ALOAD[P]
ASTORE[N]
executes, ValueHolderSub linked with variable N changes it's `first` field to P
holder `first` field. Since ValueHolderSub shared between all variables, linked
to him, it leads to all this varaibles become assigned to P.
If I make first field final, CASE fails:
CASE compiles into
{code:java}
var res;
if (expression) {
res = v1;
else {
res = v2;
}
{code}
Then Analyzer calculates Frame variables, it ignores if - else logic and
assigne v2 to res slot.
P.S.
https://stackoverflow.com/questions/25109942/is-there-a-better-explanation-of-stack-map-frames
Can be Analyzed is intended only for variable type inferrence?
> Query from parquet with case-then and arithmetic operation returns a wrong
> result
> -
>
> Key: DRILL-6722
> URL: https://issues.apache.org/jira/browse/DRILL-6722
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Codegen
>Affects Versions: 1.14.0
>Reporter: Oleg Zinoviev
>Priority: Major
> Attachments: JaininoJava.class, JaininoJava2_merged.class,
> correct.csv, result.csv
>
>
> Steps to reproduce:
> 1) Create sample table:
> {code:sql}
> create table dfs.tmp.test as
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> {code}
> 2) Execute query:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from dfs.tmp.test s
> {code}
> 3) Drill returns: [^result.csv]
> 4) Result of query without parquet:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from (
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> ) s
> {code}
> [^correct.csv]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (DRILL-6722) Query from parquet with case-then and arithmetic operation returns a wrong result
[
https://issues.apache.org/jira/browse/DRILL-6722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16673478#comment-16673478
]
Oleg Zinoviev edited comment on DRILL-6722 at 11/2/18 5:48 PM:
---
At this stage, it seems to me that the whole logic of the MergeAdapter is
broken. Neither the variable index in ValueHolderSub, nor mapping local
variable to frame slot in MethodAnalyzer does not track source code control
flow.
Bug reason:
Then
ALOAD[P]
ASTORE[N]
executes, ValueHolderSub linked with variable N changes it's `first` field to P
holder `first` field. Since ValueHolderSub shared between all variables, linked
to him, it leads to all this varaibles become assigned to P.
If I make first field final, CASE fails:
CASE compiles into
{code:java}
var res;
if (expression) {
res = v1;
else {
res = v2;
}
{code}
Then Analyzer calculates Frame variables, it ignores if - else logic and
assigne v2 to res slot.
P.S.
https://stackoverflow.com/questions/25109942/is-there-a-better-explanation-of-stack-map-frames
Can be Analyzed is intended only for variable type inferrence?
was (Author: le.louch):
At this stage, it seems to me that the whole logic of the MergeAdapter is
broken. Neither the variable index in ValueHolderSub, nor mapping local
variable to frame slot in MethodAnalyzer does not track source code control
flow.
Bug reason:
Then
ALOAD[P]
ASTORE[N]
executes, ValueHolderSub linked with variable N changes it's `first` field to P
holder `first` field. Since ValueHolderSub shared between all variables, linked
to him, it leads to all this varaibles become assigned to P.
If I make first field final, CASE fails:
CASE compiles into
{code:java}
var res;
if (expression) {
res = v1;
else {
res = v2;
}
{code}
Then Analyzer calculates Frame variables, it ignores if - else logic and
assigne v2 to res slot.
P.S.
https://stackoverflow.com/questions/25109942/is-there-a-better-explanation-of-stack-map-frames
Can be Analyzed is intended only for variable type inferrence?
> Query from parquet with case-then and arithmetic operation returns a wrong
> result
> -
>
> Key: DRILL-6722
> URL: https://issues.apache.org/jira/browse/DRILL-6722
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Codegen
>Affects Versions: 1.14.0
>Reporter: Oleg Zinoviev
>Priority: Major
> Attachments: JaininoJava.class, JaininoJava2_merged.class,
> correct.csv, result.csv
>
>
> Steps to reproduce:
> 1) Create sample table:
> {code:sql}
> create table dfs.tmp.test as
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> {code}
> 2) Execute query:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from dfs.tmp.test s
> {code}
> 3) Drill returns: [^result.csv]
> 4) Result of query without parquet:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from (
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> ) s
> {code}
> [^correct.csv]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (DRILL-6722) Query from parquet with case-then and arithmetic operation returns a wrong result
[
https://issues.apache.org/jira/browse/DRILL-6722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16673478#comment-16673478
]
Oleg Zinoviev edited comment on DRILL-6722 at 11/2/18 5:47 PM:
---
At this stage, it seems to me that the whole logic of the MergeAdapter is
broken. Neither the variable index in ValueHolderSub, nor mapping local
variable to frame slot in MethodAnalyzer does not track source code control
flow.
Bug reason:
Then
ALOAD[P]
ASTORE[N]
executes, ValueHolderSub linked with variable N changes it's `first` field to P
holder `first` field. Since ValueHolderSub shared between all variables, linked
to him, it leads to all this varaibles become assigned to P.
If I make first field final, CASE fails:
CASE compiles into
{code:java}
var res;
if (expression) {
res = v1;
else {
res = v2;
}
{code}
Then Analyzer calculates Frame variables, it ignores if - else logic and
assigne v2 to res slot.
P.S.
https://stackoverflow.com/questions/25109942/is-there-a-better-explanation-of-stack-map-frames
Can be Analyzed is intended only for variable type inferrence?
was (Author: le.louch):
At this stage, it seems to me that the whole logic of the MergeAdapter is
broken. Neither the variable index in ValueHolderSub, nor mapping local
variable to frame slot in MethodAnalyzer does not track source code control
flow.
Bug reason:
Then
ALOAD[P]
ASTORE[N]
executes, ValueHolderSub linked with variable N changes it's `first` field to P
holder `first` field. Since ValueHolderSub shared between all variables, linked
to him, it leads to all this varaibles become assigned to P.
If I make first field final, CASE fails:
CASE compiles into
var res;
{code:java}
if (expression) {
res = v1;
else {
res = v2;
}
{code}
Then Analyzer calculates Frame variables, it ignores if - else logic and
assigne v2 to res slot.
P.S.
https://stackoverflow.com/questions/25109942/is-there-a-better-explanation-of-stack-map-frames
Can be Analyzed is intended only for variable type inferrence?
> Query from parquet with case-then and arithmetic operation returns a wrong
> result
> -
>
> Key: DRILL-6722
> URL: https://issues.apache.org/jira/browse/DRILL-6722
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Codegen
>Affects Versions: 1.14.0
>Reporter: Oleg Zinoviev
>Priority: Major
> Attachments: JaininoJava.class, JaininoJava2_merged.class,
> correct.csv, result.csv
>
>
> Steps to reproduce:
> 1) Create sample table:
> {code:sql}
> create table dfs.tmp.test as
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> {code}
> 2) Execute query:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from dfs.tmp.test s
> {code}
> 3) Drill returns: [^result.csv]
> 4) Result of query without parquet:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from (
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> ) s
> {code}
> [^correct.csv]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6722) Query from parquet with case-then and arithmetic operation returns a wrong result
[
https://issues.apache.org/jira/browse/DRILL-6722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16673478#comment-16673478
]
Oleg Zinoviev commented on DRILL-6722:
--
At this stage, it seems to me that the whole logic of the MergeAdapter is
broken. Neither the variable index in ValueHolderSub, nor mapping local
variable to frame slot in MethodAnalyzer does not track source code control
flow.
Bug reason:
Then
ALOAD[P]
ASTORE[N]
executes, ValueHolderSub linked with variable N changes it's `first` field to P
holder `first` field. Since ValueHolderSub shared between all variables, linked
to him, it leads to all this varaibles become assigned to P.
If I make first field final, CASE fails:
CASE compiles into
var res;
{code:java}
if (expression) {
res = v1;
else {
res = v2;
}
{code}
Then Analyzer calculates Frame variables, it ignores if - else logic and
assigne v2 to res slot.
P.S.
https://stackoverflow.com/questions/25109942/is-there-a-better-explanation-of-stack-map-frames
Can be Analyzed is intended only for variable type inferrence?
> Query from parquet with case-then and arithmetic operation returns a wrong
> result
> -
>
> Key: DRILL-6722
> URL: https://issues.apache.org/jira/browse/DRILL-6722
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Codegen
>Affects Versions: 1.14.0
>Reporter: Oleg Zinoviev
>Priority: Major
> Attachments: JaininoJava.class, JaininoJava2_merged.class,
> correct.csv, result.csv
>
>
> Steps to reproduce:
> 1) Create sample table:
> {code:sql}
> create table dfs.tmp.test as
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> {code}
> 2) Execute query:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from dfs.tmp.test s
> {code}
> 3) Drill returns: [^result.csv]
> 4) Result of query without parquet:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from (
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> ) s
> {code}
> [^correct.csv]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6722) Query from parquet with case-then and arithmetic operation returns a wrong result
[
https://issues.apache.org/jira/browse/DRILL-6722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16671736#comment-16671736
]
Oleg Zinoviev commented on DRILL-6722:
--
[~shuifeng]
For ClassNode:
{code:java}
ClassWriter cw = new ClassWriter(ClassWriter.COMPUTE_FRAMES);
generatedMerged.accept(cw);
Files.write(Paths.get("/home/ozinoviev/Class.class"), cw.toByteArray());
{code}
> Query from parquet with case-then and arithmetic operation returns a wrong
> result
> -
>
> Key: DRILL-6722
> URL: https://issues.apache.org/jira/browse/DRILL-6722
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Codegen
>Affects Versions: 1.14.0
>Reporter: Oleg Zinoviev
>Priority: Major
> Attachments: JaininoJava.class, JaininoJava2_merged.class,
> correct.csv, result.csv
>
>
> Steps to reproduce:
> 1) Create sample table:
> {code:sql}
> create table dfs.tmp.test as
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> {code}
> 2) Execute query:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from dfs.tmp.test s
> {code}
> 3) Drill returns: [^result.csv]
> 4) Result of query without parquet:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from (
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> ) s
> {code}
> [^correct.csv]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6722) Query from parquet with case-then and arithmetic operation returns a wrong result
[
https://issues.apache.org/jira/browse/DRILL-6722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669989#comment-16669989
]
Oleg Zinoviev commented on DRILL-6722:
--
I'll try to provide PR tomorrow :)
> Query from parquet with case-then and arithmetic operation returns a wrong
> result
> -
>
> Key: DRILL-6722
> URL: https://issues.apache.org/jira/browse/DRILL-6722
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Codegen
>Affects Versions: 1.14.0
>Reporter: Oleg Zinoviev
>Priority: Major
> Attachments: JaininoJava.class, JaininoJava2_merged.class,
> correct.csv, result.csv
>
>
> Steps to reproduce:
> 1) Create sample table:
> {code:sql}
> create table dfs.tmp.test as
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> {code}
> 2) Execute query:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from dfs.tmp.test s
> {code}
> 3) Drill returns: [^result.csv]
> 4) Result of query without parquet:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from (
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> ) s
> {code}
> [^correct.csv]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6722) Query from parquet with case-then and arithmetic operation returns a wrong result
[
https://issues.apache.org/jira/browse/DRILL-6722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669976#comment-16669976
]
Oleg Zinoviev commented on DRILL-6722:
--
I have a latest master revision, and bug is still here... Could it be that your
drill.exec.compile.prefer_plain_java value is true?
{noformat}
By the way, how to generate this class without setting saveCodeForDebugging to
true?{noformat}
Breakpoint -> Evaluate expression -> Write a byte array with class into file ->
Use any decompiler
> Query from parquet with case-then and arithmetic operation returns a wrong
> result
> -
>
> Key: DRILL-6722
> URL: https://issues.apache.org/jira/browse/DRILL-6722
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Codegen
>Affects Versions: 1.14.0
>Reporter: Oleg Zinoviev
>Priority: Major
> Attachments: JaininoJava.class, JaininoJava2_merged.class,
> correct.csv, result.csv
>
>
> Steps to reproduce:
> 1) Create sample table:
> {code:sql}
> create table dfs.tmp.test as
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> {code}
> 2) Execute query:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from dfs.tmp.test s
> {code}
> 3) Drill returns: [^result.csv]
> 4) Result of query without parquet:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from (
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> ) s
> {code}
> [^correct.csv]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6722) Query from parquet with case-then and arithmetic operation returns a wrong result
[
https://issues.apache.org/jira/browse/DRILL-6722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669937#comment-16669937
]
Oleg Zinoviev commented on DRILL-6722:
--
[~shuifeng], thanks for your comment.
This is generated class file before ValueHolderReplacementVisitor call:
[^JaininoJava.class]
This is generated class file after ValueHolderReplacementVisitor call:
[^JaininoJava2_merged.class]
Decompiled code of doEval method:
{code:java}
boolean var3 = false;
boolean var4 = false;
int var27 = this.vv1.getAccessor().get(inIndex);
boolean var5 = false;
int var28 = this.vv5.getAccessor().get(inIndex);
boolean var6 = false;
boolean var7 = false;
if (!Double.isNaN((double)var28) && (!Double.isNaN((double)var27) ||
!Double.isNaN((double)var28))) {
if (Double.isNaN((double)var27) && !Double.isNaN((double)var28)) {
var7 = true;
} else {
var7 = var27 > var28;
}
} else {
var7 = false;
}
int var26;
if (var7) {
var26 = var27;
} else {
var26 = var28;
}
this.vv10.getMutator().set(outIndex, var26);
boolean var8 = false;
boolean var9 = false;
int var29 = var26 - var26; // << Error reason. Bug somewhere in
ValueHolderReplacementVisitor.
boolean var10 = false;
boolean var11 = false;
int var30 = Math.abs(var29);
this.vv15.getMutator().set(outIndex, var30);
{code}
> Query from parquet with case-then and arithmetic operation returns a wrong
> result
> -
>
> Key: DRILL-6722
> URL: https://issues.apache.org/jira/browse/DRILL-6722
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Codegen
>Affects Versions: 1.14.0
>Reporter: Oleg Zinoviev
>Priority: Major
> Attachments: JaininoJava.class, JaininoJava2_merged.class,
> correct.csv, result.csv
>
>
> Steps to reproduce:
> 1) Create sample table:
> {code:sql}
> create table dfs.tmp.test as
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> {code}
> 2) Execute query:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from dfs.tmp.test s
> {code}
> 3) Drill returns: [^result.csv]
> 4) Result of query without parquet:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from (
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> ) s
> {code}
> [^correct.csv]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-6722) Query from parquet with case-then and arithmetic operation returns a wrong result
[
https://issues.apache.org/jira/browse/DRILL-6722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleg Zinoviev updated DRILL-6722:
-
Attachment: JaininoJava.class
> Query from parquet with case-then and arithmetic operation returns a wrong
> result
> -
>
> Key: DRILL-6722
> URL: https://issues.apache.org/jira/browse/DRILL-6722
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Codegen
>Affects Versions: 1.14.0
>Reporter: Oleg Zinoviev
>Priority: Major
> Attachments: JaininoJava.class, JaininoJava2_merged.class,
> correct.csv, result.csv
>
>
> Steps to reproduce:
> 1) Create sample table:
> {code:sql}
> create table dfs.tmp.test as
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> {code}
> 2) Execute query:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from dfs.tmp.test s
> {code}
> 3) Drill returns: [^result.csv]
> 4) Result of query without parquet:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from (
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> ) s
> {code}
> [^correct.csv]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-6722) Query from parquet with case-then and arithmetic operation returns a wrong result
[
https://issues.apache.org/jira/browse/DRILL-6722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleg Zinoviev updated DRILL-6722:
-
Attachment: JaininoJava2_merged.class
> Query from parquet with case-then and arithmetic operation returns a wrong
> result
> -
>
> Key: DRILL-6722
> URL: https://issues.apache.org/jira/browse/DRILL-6722
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Codegen
>Affects Versions: 1.14.0
>Reporter: Oleg Zinoviev
>Priority: Major
> Attachments: JaininoJava.class, JaininoJava2_merged.class,
> correct.csv, result.csv
>
>
> Steps to reproduce:
> 1) Create sample table:
> {code:sql}
> create table dfs.tmp.test as
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> {code}
> 2) Execute query:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from dfs.tmp.test s
> {code}
> 3) Drill returns: [^result.csv]
> 4) Result of query without parquet:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from (
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> ) s
> {code}
> [^correct.csv]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6722) Query from parquet with case-then and arithmetic operation returns a wrong result
[
https://issues.apache.org/jira/browse/DRILL-6722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596670#comment-16596670
]
Oleg Zinoviev commented on DRILL-6722:
--
1) Bug can be reproduced without abs call
2) alias of case-then column has no affect on bug
> Query from parquet with case-then and arithmetic operation returns a wrong
> result
> -
>
> Key: DRILL-6722
> URL: https://issues.apache.org/jira/browse/DRILL-6722
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.14.0
>Reporter: Oleg Zinoviev
>Priority: Major
> Attachments: correct.csv, result.csv
>
>
> Steps to reproduce:
> 1) Create sample table:
> {code:sql}
> create table dfs.tmp.test as
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> {code}
> 2) Execute query:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from dfs.tmp.test s
> {code}
> 3) Drill returns: [^result.csv]
> 4) Result of query without parquet:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from (
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> ) s
> {code}
> [^correct.csv]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-6722) Query from parquet with case-then and arithmetic operation returns a wrong result
[
https://issues.apache.org/jira/browse/DRILL-6722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleg Zinoviev updated DRILL-6722:
-
Summary: Query from parquet with case-then and arithmetic operation returns
a wrong result (was: Query from parquet with case-then and scalar operation
returns a wrong result)
> Query from parquet with case-then and arithmetic operation returns a wrong
> result
> -
>
> Key: DRILL-6722
> URL: https://issues.apache.org/jira/browse/DRILL-6722
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.14.0
>Reporter: Oleg Zinoviev
>Priority: Major
> Attachments: correct.csv, result.csv
>
>
> Steps to reproduce:
> 1) Create sample table:
> {code:sql}
> create table dfs.tmp.test as
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> {code}
> 2) Execute query:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from dfs.tmp.test s
> {code}
> 3) Drill returns: [^result.csv]
> 4) Result of query without parquet:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from (
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> ) s
> {code}
> [^correct.csv]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-6722) Query from parquet with case-then and scalar operation returns a wrong result
[
https://issues.apache.org/jira/browse/DRILL-6722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleg Zinoviev updated DRILL-6722:
-
Summary: Query from parquet with case-then and scalar operation returns a
wrong result (was: Query from parket with case-then and scalar operation
returns a wrong result)
> Query from parquet with case-then and scalar operation returns a wrong result
> -
>
> Key: DRILL-6722
> URL: https://issues.apache.org/jira/browse/DRILL-6722
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.14.0
>Reporter: Oleg Zinoviev
>Priority: Major
> Attachments: correct.csv, result.csv
>
>
> Steps to reproduce:
> 1) Create sample table:
> {code:sql}
> create table dfs.tmp.test as
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> {code}
> 2) Execute query:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from dfs.tmp.test s
> {code}
> 3) Drill returns: [^result.csv]
> 4) Result of query without parquet:
> {code:sql}
> select
> case when s.a > s.b then s.a else s.b end as b,
> abs(s.a - s.b) as d
> from (
> select 1 as a, 2 as b
> union all
> select 3 as a, 2 as b
> union all
> select 1 as a, 4 as b
> union all
> select 2 as a, 2 as b
> ) s
> {code}
> [^correct.csv]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (DRILL-6722) Query from parket with case-then and scalar operation returns a wrong result
Oleg Zinoviev created DRILL-6722:
Summary: Query from parket with case-then and scalar operation
returns a wrong result
Key: DRILL-6722
URL: https://issues.apache.org/jira/browse/DRILL-6722
Project: Apache Drill
Issue Type: Bug
Affects Versions: 1.14.0
Reporter: Oleg Zinoviev
Attachments: correct.csv, result.csv
Steps to reproduce:
1) Create sample table:
{code:sql}
create table dfs.tmp.test as
select 1 as a, 2 as b
union all
select 3 as a, 2 as b
union all
select 1 as a, 4 as b
union all
select 2 as a, 2 as b
{code}
2) Execute query:
{code:sql}
select
case when s.a > s.b then s.a else s.b end as b,
abs(s.a - s.b) as d
from dfs.tmp.test s
{code}
3) Drill returns: [^result.csv]
4) Result of query without parquet:
{code:sql}
select
case when s.a > s.b then s.a else s.b end as b,
abs(s.a - s.b) as d
from (
select 1 as a, 2 as b
union all
select 3 as a, 2 as b
union all
select 1 as a, 4 as b
union all
select 2 as a, 2 as b
) s
{code}
[^correct.csv]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6721) SchemalessScan cannot be deserialized by Jackson
[
https://issues.apache.org/jira/browse/DRILL-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596150#comment-16596150
]
Oleg Zinoviev commented on DRILL-6721:
--
Some stacktrace:
2018-08-29 12:20:34,215 [24799aa1-2d41-73f7-dd8d-109d3044d952:frag:4:0] ERROR
o.a.d.e.w.fragment.FragmentExecutor - SYSTEM ERROR: JsonMappingException: No
suitable constructor found for type [simple type, class
org.apache.drill.exec.physical.base.SchemalessScan]: can not instantiate from
JSON object (missing default constructor or creator, or perhaps need to
add/enable type information?)
at [Source: {
"pop" : "broadcast-sender",
"@id" : 0,
"receiver-major-fragment" : 1,
"child" : {
"pop" : "project",
"@id" : 1,
"exprs" : [ {
"ref" : "`T63¦¦**`",
"expr" : "`T63¦¦**`"
}, {
"ref" : "`$f1`",
"expr" : "`T63¦¦action_time`"
}, {
"ref" : "`$f2`",
"expr" : "`T63¦¦tab_number`"
} ],
"child" : {
"pop" : "project",
"@id" : 2,
"exprs" : [ {
"ref" : "`T63¦¦**`",
"expr" : "`**`"
} ],
"child" : {
"pop" : "schemaless-scan",
"@id" : 3,
"userName" : "root",
"cost" : 0.0
},
"outputProj" : false,
"initialAllocation" : 100,
"maxAllocation" : 100,
"cost" : 1.0
},
"outputProj" : false,
"initialAllocation" : 100,
"maxAllocation" : 100,
"cost" : 1.0
},
"destinations" : [ {
"minorFragmentId" : 0,
"endpoint" :
"CiBkZW1vLWlkdnAtZGF0YS1pZHZwLWRhdGEtZHJpbGwtMRCi8gEYo/IBIKTyATIGMS4xMy4wOAE="
}, {
"minorFragmentId" : 1,
"endpoint" :
"CiBkZW1vLWlkdnAtZGF0YS1pZHZwLWRhdGEtZHJpbGwtMRCi8gEYo/IBIKTyATIGMS4xMy4wOAE="
}, {
"minorFragmentId" : 2,
"endpoint" :
"CiBkZW1vLWlkdnAtZGF0YS1pZHZwLWRhdGEtZHJpbGwtMRCi8gEYo/IBIKTyATIGMS4xMy4wOAE="
}, {
"minorFragmentId" : 3,
"endpoint" :
"CiBkZW1vLWlkdnAtZGF0YS1pZHZwLWRhdGEtZHJpbGwtMRCi8gEYo/IBIKTyATIGMS4xMy4wOAE="
}, {
"minorFragmentId" : 4,
"endpoint" :
"CiBkZW1vLWlkdnAtZGF0YS1pZHZwLWRhdGEtZHJpbGwtMRCi8gEYo/IBIKTyATIGMS4xMy4wOAE="
}, {
"minorFragmentId" : 5,
"endpoint" :
"CiBkZW1vLWlkdnAtZGF0YS1pZHZwLWRhdGEtZHJpbGwtMRCi8gEYo/IBIKTyATIGMS4xMy4wOAE="
}, {
"minorFragmentId" : 6,
"endpoint" :
"CiBkZW1vLWlkdnAtZGF0YS1pZHZwLWRhdGEtZHJpbGwtMRCi8gEYo/IBIKTyATIGMS4xMy4wOAE="
} ],
"initialAllocation" : 100,
"maxAllocation" : 100,
"cost" : 1.0
}; line: 27, column: 9] (through reference chain:
org.apache.drill.exec.physical.config.BroadcastSender["child"]->org.apache.drill.exec.physical.config.Project["child"]->org.apache.drill.exec.physical.config.Project["child"])
> SchemalessScan cannot be deserialized by Jackson
>
>
> Key: DRILL-6721
> URL: https://issues.apache.org/jira/browse/DRILL-6721
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.13.0, 1.14.0
>Reporter: Oleg Zinoviev
>Priority: Major
>
> SchemalessScan has a JsonTypeName annotation, but default (or marked with
> @JsonCreator) constructor is missing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (DRILL-6721) SchemalessScan cannot be deserialized by Jackson
Oleg Zinoviev created DRILL-6721: Summary: SchemalessScan cannot be deserialized by Jackson Key: DRILL-6721 URL: https://issues.apache.org/jira/browse/DRILL-6721 Project: Apache Drill Issue Type: Bug Affects Versions: 1.14.0, 1.13.0, 1.15.0 Reporter: Oleg Zinoviev SchemalessScan has a JsonTypeName annotation, but default (or marked with @JsonCreator) constructor is missing. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6717) lower and upper functions not works with national charactes
[
https://issues.apache.org/jira/browse/DRILL-6717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16595365#comment-16595365
]
Oleg Zinoviev commented on DRILL-6717:
--
Sorry for duplicate
> lower and upper functions not works with national charactes
> ---
>
> Key: DRILL-6717
> URL: https://issues.apache.org/jira/browse/DRILL-6717
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.14.0, 1.15.0
>Reporter: Oleg Zinoviev
>Assignee: Oleg Zinoviev
>Priority: Major
> Attachments: result.csv, sample.json
>
>
> upper() and lower() functions works only with a ascii characters.
> sample dataset: [^sample.json]
> query:
> {code:sql}
> select
> upper(ascii_l) as upper_ascii,
> upper(national_l) as upper_national,
> lower(ascii_u) as lower_ascii,
> lower(national_u) as lower_national
> from dfs.`sample.json`
> {code}
> drill response: [^result.csv]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (DRILL-6717) lower and upper functions not works with national charactes
Oleg Zinoviev created DRILL-6717:
Summary: lower and upper functions not works with national
charactes
Key: DRILL-6717
URL: https://issues.apache.org/jira/browse/DRILL-6717
Project: Apache Drill
Issue Type: Bug
Affects Versions: 1.14.0, 1.15.0
Reporter: Oleg Zinoviev
Attachments: result.csv, sample.json
upper() and lower() functions works only with a ascii characters.
sample dataset: [^sample.json]
query:
{code:sql}
select
upper(ascii_l) as upper_ascii,
upper(national_l) as upper_national,
lower(ascii_u) as lower_ascii,
lower(national_u) as lower_national
from dfs.`sample.json`
{code}
drill response: [^result.csv]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (DRILL-6712) Creation of jdbc storage plugin fails with NoSuchMethod
[
https://issues.apache.org/jira/browse/DRILL-6712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16593760#comment-16593760
]
Oleg Zinoviev edited comment on DRILL-6712 at 8/27/18 2:40 PM:
---
I just try to create a new jdbc storage plugin:
{
"type": "jdbc",
"driver": "oracle.jdbc.OracleDriver",
"url": "jdbc:oracle:thin:@***",
"username": "***",
"password": "***",
"enabled": true
}
Drill response was "Please retry: Error while creating/ updating storage :
org.apache.commons.pool2.impl.GenericObjectPool.setTestOnCreate(Z)V"
*All unit tests pass, so I am wondering how it slipped through.*
Maybe test classpath differs from bundled jars? Calcite 1.17.0 has it's own
commons-pool2 dependency (2.5.0 version:
https://mvnrepository.com/artifact/org.apache.calcite/calcite-core/1.17.0), but
drill java-exec has a commons-pool2:2.1 dependency.
was (Author: le.louch):
I just try to create a new jdbc storage plugin:
{
"type": "jdbc",
"driver": "oracle.jdbc.OracleDriver",
"url": "jdbc:oracle:thin:@***",
"username": "***",
"password": "***",
"enabled": true
}
Drill response was "Please retry: Error while creating/ updating storage :
org.apache.commons.pool2.impl.GenericObjectPool.setTestOnCreate(Z)V"
*All unit tests pass, so I am wondering how it slipped through.*
Test classpath differs from bundled jars. Calcite 1.17.0 has it's own
commons-pool2 dependency (2.5.0 version:
https://mvnrepository.com/artifact/org.apache.calcite/calcite-core/1.17.0), but
drill java-exec has a commons-pool2:2.1 dependency.
> Creation of jdbc storage plugin fails with NoSuchMethod
> ---
>
> Key: DRILL-6712
> URL: https://issues.apache.org/jira/browse/DRILL-6712
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - JDBC
>Affects Versions: 1.15.0
>Reporter: Oleg Zinoviev
>Priority: Major
>
> Creation of jdbc-storage-plugin falis with:
> Caused by: java.lang.NoSuchMethodError:
> org.apache.commons.pool2.impl.GenericObjectPool.setTestOnCreate(Z)V
> at
> org.apache.commons.dbcp2.BasicDataSource.createConnectionPool(BasicDataSource.java:2224)
> ~[commons-dbcp2-2.4.0.jar:2.4.0]
> at
> org.apache.commons.dbcp2.BasicDataSource.createDataSource(BasicDataSource.java:2085)
> ~[commons-dbcp2-2.4.0.jar:2.4.0]
> at
> org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:1525)
> ~[commons-dbcp2-2.4.0.jar:2.4.0]
> at
> org.apache.calcite.adapter.jdbc.JdbcUtils$DialectPool.get(JdbcUtils.java:78)
> ~[calcite-core-1.17.0-drill-r0.jar:1.17.0-drill-r0]
> at
> org.apache.calcite.adapter.jdbc.JdbcSchema.createDialect(JdbcSchema.java:184)
> ~[calcite-core-1.17.0-drill-r0.jar:1.17.0-drill-r0]
> at
> org.apache.drill.exec.store.jdbc.JdbcStoragePlugin.(JdbcStoragePlugin.java:101)
> ~[drill-jdbc-storage-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT]
> Reason:
> bundled dependency commons-pool2:2.1
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6712) Creation of jdbc storage plugin fails with NoSuchMethod
[
https://issues.apache.org/jira/browse/DRILL-6712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16593760#comment-16593760
]
Oleg Zinoviev commented on DRILL-6712:
--
I just try to create a new jdbc storage plugin:
{
"type": "jdbc",
"driver": "oracle.jdbc.OracleDriver",
"url": "jdbc:oracle:thin:@***",
"username": "***",
"password": "***",
"enabled": true
}
Drill response was "Please retry: Error while creating/ updating storage :
org.apache.commons.pool2.impl.GenericObjectPool.setTestOnCreate(Z)V"
All unit tests pass, so I am wondering how it slipped through.
Test classpath differs from bundled jars. Calcite 1.17.0 has it's own
commons-pool2 dependency (2.5.0 version:
https://mvnrepository.com/artifact/org.apache.calcite/calcite-core/1.17.0), but
drill java-exec has a commons-pool2:2.1 dependency.
> Creation of jdbc storage plugin fails with NoSuchMethod
> ---
>
> Key: DRILL-6712
> URL: https://issues.apache.org/jira/browse/DRILL-6712
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - JDBC
>Affects Versions: 1.15.0
>Reporter: Oleg Zinoviev
>Priority: Major
>
> Creation of jdbc-storage-plugin falis with:
> Caused by: java.lang.NoSuchMethodError:
> org.apache.commons.pool2.impl.GenericObjectPool.setTestOnCreate(Z)V
> at
> org.apache.commons.dbcp2.BasicDataSource.createConnectionPool(BasicDataSource.java:2224)
> ~[commons-dbcp2-2.4.0.jar:2.4.0]
> at
> org.apache.commons.dbcp2.BasicDataSource.createDataSource(BasicDataSource.java:2085)
> ~[commons-dbcp2-2.4.0.jar:2.4.0]
> at
> org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:1525)
> ~[commons-dbcp2-2.4.0.jar:2.4.0]
> at
> org.apache.calcite.adapter.jdbc.JdbcUtils$DialectPool.get(JdbcUtils.java:78)
> ~[calcite-core-1.17.0-drill-r0.jar:1.17.0-drill-r0]
> at
> org.apache.calcite.adapter.jdbc.JdbcSchema.createDialect(JdbcSchema.java:184)
> ~[calcite-core-1.17.0-drill-r0.jar:1.17.0-drill-r0]
> at
> org.apache.drill.exec.store.jdbc.JdbcStoragePlugin.(JdbcStoragePlugin.java:101)
> ~[drill-jdbc-storage-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT]
> Reason:
> bundled dependency commons-pool2:2.1
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (DRILL-6712) Creation of jdbc storage plugin fails with NoSuchMethod
[
https://issues.apache.org/jira/browse/DRILL-6712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16593760#comment-16593760
]
Oleg Zinoviev edited comment on DRILL-6712 at 8/27/18 2:39 PM:
---
I just try to create a new jdbc storage plugin:
{
"type": "jdbc",
"driver": "oracle.jdbc.OracleDriver",
"url": "jdbc:oracle:thin:@***",
"username": "***",
"password": "***",
"enabled": true
}
Drill response was "Please retry: Error while creating/ updating storage :
org.apache.commons.pool2.impl.GenericObjectPool.setTestOnCreate(Z)V"
*All unit tests pass, so I am wondering how it slipped through.*
Test classpath differs from bundled jars. Calcite 1.17.0 has it's own
commons-pool2 dependency (2.5.0 version:
https://mvnrepository.com/artifact/org.apache.calcite/calcite-core/1.17.0), but
drill java-exec has a commons-pool2:2.1 dependency.
was (Author: le.louch):
I just try to create a new jdbc storage plugin:
{
"type": "jdbc",
"driver": "oracle.jdbc.OracleDriver",
"url": "jdbc:oracle:thin:@***",
"username": "***",
"password": "***",
"enabled": true
}
Drill response was "Please retry: Error while creating/ updating storage :
org.apache.commons.pool2.impl.GenericObjectPool.setTestOnCreate(Z)V"
All unit tests pass, so I am wondering how it slipped through.
Test classpath differs from bundled jars. Calcite 1.17.0 has it's own
commons-pool2 dependency (2.5.0 version:
https://mvnrepository.com/artifact/org.apache.calcite/calcite-core/1.17.0), but
drill java-exec has a commons-pool2:2.1 dependency.
> Creation of jdbc storage plugin fails with NoSuchMethod
> ---
>
> Key: DRILL-6712
> URL: https://issues.apache.org/jira/browse/DRILL-6712
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - JDBC
>Affects Versions: 1.15.0
>Reporter: Oleg Zinoviev
>Priority: Major
>
> Creation of jdbc-storage-plugin falis with:
> Caused by: java.lang.NoSuchMethodError:
> org.apache.commons.pool2.impl.GenericObjectPool.setTestOnCreate(Z)V
> at
> org.apache.commons.dbcp2.BasicDataSource.createConnectionPool(BasicDataSource.java:2224)
> ~[commons-dbcp2-2.4.0.jar:2.4.0]
> at
> org.apache.commons.dbcp2.BasicDataSource.createDataSource(BasicDataSource.java:2085)
> ~[commons-dbcp2-2.4.0.jar:2.4.0]
> at
> org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:1525)
> ~[commons-dbcp2-2.4.0.jar:2.4.0]
> at
> org.apache.calcite.adapter.jdbc.JdbcUtils$DialectPool.get(JdbcUtils.java:78)
> ~[calcite-core-1.17.0-drill-r0.jar:1.17.0-drill-r0]
> at
> org.apache.calcite.adapter.jdbc.JdbcSchema.createDialect(JdbcSchema.java:184)
> ~[calcite-core-1.17.0-drill-r0.jar:1.17.0-drill-r0]
> at
> org.apache.drill.exec.store.jdbc.JdbcStoragePlugin.(JdbcStoragePlugin.java:101)
> ~[drill-jdbc-storage-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT]
> Reason:
> bundled dependency commons-pool2:2.1
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (DRILL-6712) Creation of jdbc storage plugin fails with NoSuchMethod
Oleg Zinoviev created DRILL-6712: Summary: Creation of jdbc storage plugin fails with NoSuchMethod Key: DRILL-6712 URL: https://issues.apache.org/jira/browse/DRILL-6712 Project: Apache Drill Issue Type: Bug Components: Storage - JDBC Affects Versions: 1.15.0 Reporter: Oleg Zinoviev Creation of jdbc-storage-plugin falis with: Caused by: java.lang.NoSuchMethodError: org.apache.commons.pool2.impl.GenericObjectPool.setTestOnCreate(Z)V at org.apache.commons.dbcp2.BasicDataSource.createConnectionPool(BasicDataSource.java:2224) ~[commons-dbcp2-2.4.0.jar:2.4.0] at org.apache.commons.dbcp2.BasicDataSource.createDataSource(BasicDataSource.java:2085) ~[commons-dbcp2-2.4.0.jar:2.4.0] at org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:1525) ~[commons-dbcp2-2.4.0.jar:2.4.0] at org.apache.calcite.adapter.jdbc.JdbcUtils$DialectPool.get(JdbcUtils.java:78) ~[calcite-core-1.17.0-drill-r0.jar:1.17.0-drill-r0] at org.apache.calcite.adapter.jdbc.JdbcSchema.createDialect(JdbcSchema.java:184) ~[calcite-core-1.17.0-drill-r0.jar:1.17.0-drill-r0] at org.apache.drill.exec.store.jdbc.JdbcStoragePlugin.(JdbcStoragePlugin.java:101) ~[drill-jdbc-storage-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] Reason: bundled dependency commons-pool2:2.1 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6318) Push down limit past flatten is incorrect
[
https://issues.apache.org/jira/browse/DRILL-6318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleg Zinoviev updated DRILL-6318:
-
Summary: Push down limit past flatten is incorrect (was: Push down limit
into flatten is incorrect)
> Push down limit past flatten is incorrect
> -
>
> Key: DRILL-6318
> URL: https://issues.apache.org/jira/browse/DRILL-6318
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.13.0, 1.14.0
> Environment: Ubuntu Mate 18.04
> Apache Drill 1.14.0-SNAPSHOT
>Reporter: Oleg Zinoviev
>Priority: Major
> Attachments: sample.json
>
>
> Sample data: [^sample.json]
> Result of
> {code:sql}
> select flatten(data) as d from dfs.root.`sample.json`
> {code}
> is
> ||d||
> |Honored Boy Scout|
> |Yawning Wolf|
> |Closed Queen|
> |Innocent Volunteer|
> |Junior Wing|
> |Lame Mantis|
> |Old Master|
> |Numb Pawn|
> |Hollow Guardian|
> |Twin Hurricane|
> |Helpless Avalange|
> Let's try to get first 3 rows:
> {code:sql}
> select flatten(data) as d from dfs.root.`sample.json` limit 3
> {code}
> Result has only 2 rows:
> ||d||
> |Honored Boy Scout|
> |Yawning Wolf|
> *Reason:* Limit was pushed down below flatten and only 3 top rows from json
> was selected. In this 3 rows only 2nd has items in "data" field.
> Let's try to get 3 rows from the middle:
> {code:sql}
> select flatten(data) as d from dfs.root.`sample.json` limit 3 offset 5
> {code}
> Result is empty.
> *Reason:* Limit and offset was pushed down below flatten and only 6, 7 and 8
> row from json was selected. This 3 rows contains only 3 items in "data"
> field. After flatten limit and offset applies second time and reject all
> select items.
> Error in org/apache/drill/exec/planner/logical/DrillPushLimitToScanRule.java
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-6318) Push down limit into flatten is incorrect
[
https://issues.apache.org/jira/browse/DRILL-6318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleg Zinoviev updated DRILL-6318:
-
Affects Version/s: 1.14.0
1.13.0
Fix Version/s: (was: 1.14.0)
(was: 1.13.0)
> Push down limit into flatten is incorrect
> -
>
> Key: DRILL-6318
> URL: https://issues.apache.org/jira/browse/DRILL-6318
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.13.0, 1.14.0
> Environment: Ubuntu Mate 18.04
> Apache Drill 1.14.0-SNAPSHOT
>Reporter: Oleg Zinoviev
>Priority: Major
> Attachments: sample.json
>
>
> Sample data: [^sample.json]
> Result of
> {code:sql}
> select flatten(data) as d from dfs.root.`sample.json`
> {code}
> is
> ||d||
> |Honored Boy Scout|
> |Yawning Wolf|
> |Closed Queen|
> |Innocent Volunteer|
> |Junior Wing|
> |Lame Mantis|
> |Old Master|
> |Numb Pawn|
> |Hollow Guardian|
> |Twin Hurricane|
> |Helpless Avalange|
> Let's try to get first 3 rows:
> {code:sql}
> select flatten(data) as d from dfs.root.`sample.json` limit 3
> {code}
> Result has only 2 rows:
> ||d||
> |Honored Boy Scout|
> |Yawning Wolf|
> *Reason:* Limit was pushed down below flatten and only 3 top rows from json
> was selected. In this 3 rows only 2nd has items in "data" field.
> Let's try to get 3 rows from the middle:
> {code:sql}
> select flatten(data) as d from dfs.root.`sample.json` limit 3 offset 5
> {code}
> Result is empty.
> *Reason:* Limit and offset was pushed down below flatten and only 6, 7 and 8
> row from json was selected. This 3 rows contains only 3 items in "data"
> field. After flatten limit and offset applies second time and reject all
> select items.
> Error in org/apache/drill/exec/planner/logical/DrillPushLimitToScanRule.java
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6318) Push down limit into flatten is incorrect
[
https://issues.apache.org/jira/browse/DRILL-6318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431802#comment-16431802
]
Oleg Zinoviev commented on DRILL-6318:
--
P.S. Sorry for my terrible English
> Push down limit into flatten is incorrect
> -
>
> Key: DRILL-6318
> URL: https://issues.apache.org/jira/browse/DRILL-6318
> Project: Apache Drill
> Issue Type: Bug
> Environment: Ubuntu Mate 18.04
> Apache Drill 1.14.0-SNAPSHOT
>Reporter: Oleg Zinoviev
>Priority: Major
> Fix For: 1.13.0, 1.14.0
>
> Attachments: sample.json
>
>
> Sample data: [^sample.json]
> Result of
> {code:sql}
> select flatten(data) as d from dfs.root.`sample.json`
> {code}
> is
> ||d||
> |Honored Boy Scout|
> |Yawning Wolf|
> |Closed Queen|
> |Innocent Volunteer|
> |Junior Wing|
> |Lame Mantis|
> |Old Master|
> |Numb Pawn|
> |Hollow Guardian|
> |Twin Hurricane|
> |Helpless Avalange|
> Let's try to get first 3 rows:
> {code:sql}
> select flatten(data) as d from dfs.root.`sample.json` limit 3
> {code}
> Result has only 2 rows:
> ||d||
> |Honored Boy Scout|
> |Yawning Wolf|
> *Reason:* Limit was pushed down below flatten and only 3 top rows from json
> was selected. In this 3 rows only 2nd has items in "data" field.
> Let's try to get 3 rows from the middle:
> {code:sql}
> select flatten(data) as d from dfs.root.`sample.json` limit 3 offset 5
> {code}
> Result is empty.
> *Reason:* Limit and offset was pushed down below flatten and only 6, 7 and 8
> row from json was selected. This 3 rows contains only 3 items in "data"
> field. After flatten limit and offset applies second time and reject all
> select items.
> Error in org/apache/drill/exec/planner/logical/DrillPushLimitToScanRule.java
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (DRILL-6318) Push down limit into flatten is incorrect
Oleg Zinoviev created DRILL-6318:
Summary: Push down limit into flatten is incorrect
Key: DRILL-6318
URL: https://issues.apache.org/jira/browse/DRILL-6318
Project: Apache Drill
Issue Type: Bug
Environment: Ubuntu Mate 18.04
Apache Drill 1.14.0-SNAPSHOT
Reporter: Oleg Zinoviev
Fix For: 1.14.0, 1.13.0
Attachments: sample.json
Sample data: [^sample.json]
Result of
{code:sql}
select flatten(data) as d from dfs.root.`sample.json`
{code}
is
||d||
|Honored Boy Scout|
|Yawning Wolf|
|Closed Queen|
|Innocent Volunteer|
|Junior Wing|
|Lame Mantis|
|Old Master|
|Numb Pawn|
|Hollow Guardian|
|Twin Hurricane|
|Helpless Avalange|
Let's try to get first 3 rows:
{code:sql}
select flatten(data) as d from dfs.root.`sample.json` limit 3
{code}
Result has only 2 rows:
||d||
|Honored Boy Scout|
|Yawning Wolf|
*Reason:* Limit was pushed down below flatten and only 3 top rows from json was
selected. In this 3 rows only 2nd has items in "data" field.
Let's try to get 3 rows from the middle:
{code:sql}
select flatten(data) as d from dfs.root.`sample.json` limit 3 offset 5
{code}
Result is empty.
*Reason:* Limit and offset was pushed down below flatten and only 6, 7 and 8
row from json was selected. This 3 rows contains only 3 items in "data" field.
After flatten limit and offset applies second time and reject all select items.
Error in org/apache/drill/exec/planner/logical/DrillPushLimitToScanRule.java
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
