[jira] [Created] (DRILL-7527) DROP METADATA doesn't work with table name starting with '/' inside workspace
Denys Ordynskiy created DRILL-7527: -- Summary: DROP METADATA doesn't work with table name starting with '/' inside workspace Key: DRILL-7527 URL: https://issues.apache.org/jira/browse/DRILL-7527 Project: Apache Drill Issue Type: Bug Affects Versions: 1.17.0 Reporter: Denys Ordynskiy *Description:* - create parquet table with CTAS: CREATE TABLE dfs.tmp.`folder/file` AS SELECT * FROM cp.`employee.json`; - refresh metadata for table: ANALYZE TABLE dfs.tmp.`folder/file` REFRESH METADATA; - drop metadata: ANALYZE TABLE dfs.tmp.`/folder/file` DROP METADATA [IF EXISTS]; *Expexted result:* Metadata for table [folder/file] dropped. *Actual result:* Error: VALIDATION ERROR: Metadata for table [/folder/file] not found. Shoud work as DROP TABLE: DROP TABLE dfs.tmp.`/folder/file`; Table [folder/file] dropped -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (DRILL-7484) Malware found with some antiviruses in the Drill test resources folder
Denys Ordynskiy created DRILL-7484: -- Summary: Malware found with some antiviruses in the Drill test resources folder Key: DRILL-7484 URL: https://issues.apache.org/jira/browse/DRILL-7484 Project: Apache Drill Issue Type: Bug Affects Versions: 1.17.0 Reporter: Denys Ordynskiy Attachments: photo_2019-12-11_17-07-45.jpg Avast antivirus found Threat "MPPT97:ShellCode-O [Expl]" in the Apache Drill sources. File with virus alert: https://github.com/apache/drill/commits/master/exec/java-exec/src/test/resources/store/pcap/attack-trace.pcap OS Windows 10. Free Avast antivirus v. Steps to reproduce: Download archive with Drill sources from GitHub - open Chrome browser; - go to https://github.com/apache/drill link; - open the "Clone or download" menu; - click the "Download ZIP" button and save archive on a disk. *Expected result* - archive successfully downloaded. *Actual result* - Chrome browser canceled the download and deleted Drill sources archive fron the disk. Avast showed an alert window with virus description. Screenshot is in the attachments. Clone Drill sources from GitHub using git console version - open CMD; - type "git clone https://github.com/apache/drill.git;; - scan clonned Drill sources folder using Avast antivirus. *Expected result* - Viruses were absent. *Actual result* - Avast showed an alert window with virus description. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (DRILL-7342) Drill replacing spaces with underlines in the column names of text files with headers
Denys Ordynskiy created DRILL-7342:
--
Summary: Drill replacing spaces with underlines in the column
names of text files with headers
Key: DRILL-7342
URL: https://issues.apache.org/jira/browse/DRILL-7342
Project: Apache Drill
Issue Type: Bug
Reporter: Denys Ordynskiy
Drill doesn't allow to query csvh columns with spaces.
*Description:*
Update Drill ctas format option to generate text file with header:
{noformat}
set `store.format` = 'csvh';
{noformat}
Create table with column names having spaces:
{noformat}
create table dfs.tmp.`csv table with spaces` (`Full Name`) as select 'James
Bond' from (values(1));
{noformat}
Drill wrote column name with space:
{noformat}
hadoop fs -cat '/tmp/csv table with spaces/0_0_0.csvh'
{noformat}
|Full Name|
|James Bond|
Try to query this table without column name:
{noformat}
select * from dfs.tmp.`csv table with spaces`;
{noformat}
|{color:#ff}*Full_Name*{color}|
|James Bond|
{color:#ff}*Drill replaced space with underline.*{color}
Try to select `Full Name` column with space:
{noformat}
select `Full Name` from dfs.tmp.`csv table with spaces`;
{noformat}
Drill
|Full Name|
| |
When I changed space to underline, query returned the data:
{noformat}
select `Full_Name` from dfs.tmp.`csv table with spaces`;
{noformat}
|Full_Name|
|James Bond|
Drill can create csvh text files with spaces in the column names. But it's
impossible to request data using the original column name.
*Expected result*
Space should be available character in the column names.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Created] (DRILL-7336) `cast_empty_string_to_null` option doesn't work when text file has > 1 column
Denys Ordynskiy created DRILL-7336:
--
Summary: `cast_empty_string_to_null` option doesn't work when text
file has > 1 column
Key: DRILL-7336
URL: https://issues.apache.org/jira/browse/DRILL-7336
Project: Apache Drill
Issue Type: Bug
Reporter: Denys Ordynskiy
*Description:*
1 - create 2 nullable csv files with 1 and 2 columns:
_one_col.csv_
{noformat}
1
2
4
{noformat}
_two_col.csv_
{noformat}
1,1
2,
,3
4,4
{noformat}
2 - enable option:
{noformat}
alter system set `drill.exec.functions.cast_empty_string_to_null`=true;
{noformat}
3 - query file with 1 column:
{noformat}
select columns[0] from dfs.tmp.`one_col.csv`;
{noformat}
| EXPR$0 |
| 1 |
| 2 |
| null|
| 4 |
4 - query file with 2 columns:
{noformat}
select columns[0] from dfs.tmp.`two_col.csv`;
{noformat}
*Expected result:*
Table with NULL in the 3-rd row:
| EXPR$0 |
| 1 |
| 2 |
| null|
| 4 |
*Actual result:*
{color:#d04437}Drill returns an empty string in the 3-rd row:{color}
| EXPR$0 |
| 1 |
| 2 |
| |
| 4 |
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Created] (DRILL-7332) Drill requires parentheses in the empty file for 'LOAD' argument in the 'CREATE SCHEMA' command
Denys Ordynskiy created DRILL-7332:
--
Summary: Drill requires parentheses in the empty file for 'LOAD'
argument in the 'CREATE SCHEMA' command
Key: DRILL-7332
URL: https://issues.apache.org/jira/browse/DRILL-7332
Project: Apache Drill
Issue Type: Bug
Affects Versions: 1.16.0
Reporter: Denys Ordynskiy
Assignee: Arina Ielchiieva
Fix For: 1.17.0
Description:
- created csvh table: {color:#205081}create table dfs.tmp.`test_table` (col1)
as select full_name from cp.`employee.json` limit 3;{color}
- created text file on the local file system `for_load` with text without
parentheses: {color:#14892c}col1 varchar not null{color}
- created some schema file: {color:#205081}create or replace schema *LOAD*
'file:///home/user/for_load' for table dfs.tmp.`test_table` properties
('drill.strict' = 'false');{color} - schema was created successfully;
- remove all text from the `for_load` file and try to create the same schema:
{color:#205081}create or replace schema *LOAD* 'file:///home/user/for_load' for
table dfs.tmp.`test_table` properties ('drill.strict' = 'false');{color}
*Actual result:*
Drill throws an error:
{color:#d04437}Error: RESOURCE ERROR: Unable to parse schema []: Line [1],
position [0], offending symbol [@0,0:-1='',<-1>,1:0]: mismatched input
'' expecting {'(', ID, QUOTED_ID}
Error while preparing / creating schema for [%s] dfs.tmp.test_table
[Error Id: faad9c09-2e3e-428c-bd7a-6da8832a943b ] (state=,code=0){color}
*Expected result:*
Since Drill doesn't require parentheses for non empty `for_load` file (with
some columns),
It couldn't require parentheses for the empty file, used in the `LOAD` argument.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Created] (DRILL-7323) Error scaling query results table in Drill Web UI
Denys Ordynskiy created DRILL-7323: -- Summary: Error scaling query results table in Drill Web UI Key: DRILL-7323 URL: https://issues.apache.org/jira/browse/DRILL-7323 Project: Apache Drill Issue Type: Bug Reporter: Denys Ordynskiy Attachments: 1_expanded_Drill_tab.png, 2_resized_Drill_tab.png, 3_maximized_Drill_tab.png *Steps to reproduce:* 1. In the Chrome browser open Drill Web UI "Query" page. 2. Maximize the browser window. 3. Run the sample query "SELECT employee_id FROM cp.`employee.json` LIMIT 20". 4. Query results table expands on all window "1_expanded_Drill_tab.png". 5. Resize the browser window "2_resized_Drill_tab.png". 6. Open a new browser tab and open some web page there. 7. Expand the browser window to the fullscreen. 8. Open Drill tab with query results table. *Expected result:* Query results table body should expand like the table header. *Actual result:* Query results table body didn't expand "3_maximized_Drill_tab.png". *Environment details:* OS - Ubuntu 16.04 Browser - Chrome v.75.0.3770.100 (Official Build) (64-bit) Apache Drill version: commit 66e8dd9f05ea068e27b272ed1b787004213c29e6 Author: Paul Rogers Date: Sun Jun 23 22:10:54 2019 -0700 -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (DRILL-7322) Align cast boolean and schema boolean conversion
Denys Ordynskiy created DRILL-7322:
--
Summary: Align cast boolean and schema boolean conversion
Key: DRILL-7322
URL: https://issues.apache.org/jira/browse/DRILL-7322
Project: Apache Drill
Issue Type: Bug
Affects Versions: 1.16.0
Reporter: Denys Ordynskiy
Information schema file allows converting any string to the boolean data type.
But "case(.. as boolean)" statement throws an error:
{color:#d04437}UserRemoteException :SYSTEM ERROR: IllegalArgumentException:
Invalid value for boolean: a
org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR:
IllegalArgumentException: Invalid value for boolean: a{color}
*Information Schema file should allow using the same range of boolean literals
as cast statement.*
*Steps to reproduce:*
Upload text file all_types.csvh to the DFS /tmp/ischema/all_types:
{noformat}
boolean_col,boolean_col_for_cast
true,true
1,1
t,t
y,y
yes,yes
on,on
false,false
0,0
f,f
n,n
no,no
off,off
a,a
-,-
!,!
`,`
7,7
@,@
^,^
*,*
{noformat}
*Create schema:*
{noformat}
create schema (boolean_col boolean, boolean_col_for_cast varchar) for table
dfs.tmp.`ischema/all_types`
{noformat}
*Run the query without cast:*
select boolean_col, sqlTypeOf(boolean_col) boolean_col_type,
boolean_col_for_cast, sqlTypeOf(boolean_col_for_cast) boolean_col_for_cast_type
from dfs.tmp.`ischema/all_types`
|boolean_col|boolean_col_type|boolean_col_for_cast|boolean_col_for_cast_type|
|true|BOOLEAN|true|CHARACTER VARYING|
|true|BOOLEAN|1|CHARACTER VARYING|
|true|BOOLEAN|t|CHARACTER VARYING|
|true|BOOLEAN|y|CHARACTER VARYING|
|true|BOOLEAN|yes|CHARACTER VARYING|
|true|BOOLEAN|on|CHARACTER VARYING|
|false|BOOLEAN|false|CHARACTER VARYING|
|false|BOOLEAN|0|CHARACTER VARYING|
|false|BOOLEAN|f|CHARACTER VARYING|
|false|BOOLEAN|n|CHARACTER VARYING|
|false|BOOLEAN|no|CHARACTER VARYING|
|false|BOOLEAN|off|CHARACTER VARYING|
|false|BOOLEAN|a|CHARACTER VARYING|
|false|BOOLEAN|-|CHARACTER VARYING|
|false|BOOLEAN|!|CHARACTER VARYING|
|false|BOOLEAN|`|CHARACTER VARYING|
|false|BOOLEAN|7|CHARACTER VARYING|
|false|BOOLEAN|@|CHARACTER VARYING|
|false|BOOLEAN|^|CHARACTER VARYING|
|false|BOOLEAN|*|CHARACTER VARYING|
*Run the query with cast:*
select boolean_col, sqlTypeOf(boolean_col) boolean_col_type,
cast(boolean_col_for_cast as boolean) boolean_col_for_cast,
sqlTypeOf(cast(boolean_col_for_cast as boolean)) boolean_col_for_cast_type from
dfs.tmp.`ischema/all_types`
{color:#d04437}UserRemoteException :SYSTEM ERROR: IllegalArgumentException:
Invalid value for boolean: a
org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR:
IllegalArgumentException: Invalid value for boolean: *a*
Fragment 0:0
Please, refer to logs for more information.
[Error Id: b9deab6f-7fd4-40c0-acdf-b2e31747e16f on cv1:31010]{color}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (DRILL-7320) DATA_READ ERROR when querying TIMESTAMP data from csvh using Schema Provision file
Denys Ordynskiy created DRILL-7320:
--
Summary: DATA_READ ERROR when querying TIMESTAMP data from csvh
using Schema Provision file
Key: DRILL-7320
URL: https://issues.apache.org/jira/browse/DRILL-7320
Project: Apache Drill
Issue Type: Bug
Affects Versions: 1.16.0
Reporter: Denys Ordynskiy
Assignee: Arina Ielchiieva
*Description:*
Set CTAS output format to 'csvh':
{noformat}
set `store.format` = 'csvh'
{noformat}
Create csvh table with timestamp string:
{noformat}
drop table if exists dfs.tmp.`employee_test`
create table dfs.tmp.`employee_test` as select
hire_date
from cp.`employee.json` LIMIT 2
{noformat}
Run the query without INFORMATION SCHEMA file using cast():
{noformat}
set `store.table.use_schema_file` = false
select
cast(hire_date as TIMESTAMP) as hire_date
from dfs.tmp.`employee_test`
{noformat}
|hire_date|
|1994-12-01T00:00|
|1994-12-01T00:00|
Create Schema Provision file:
{noformat}
create schema (
hire_date TIMESTAMP
) for table dfs.tmp.`employee_test`
{noformat}
Run the query using Schema Provision file:
{noformat}
alter system set `store.table.use_schema_file` = true
select
cast(hire_date as TIMESTAMP) as hire_date
from dfs.tmp.`employee_test`
{noformat}
*Expected result:*
|hire_date|
|1994-12-01T00:00|
|1994-12-01T00:00|
*Actual result:*
{color:#d04437}UserRemoteException :DATA_READ ERROR: Invalid format:
"1994-12-01 00:00:00.0" is malformed at " 00:00:00.0"
org.apache.drill.common.exceptions.UserRemoteException: DATA_READ ERROR:
Invalid format: "1994-12-01 00:00:00.0" is malformed at " 00:00:00.0"
Line 2
Record 0
Fragment 0:0
[Error Id: 06965d3f-55ba-4c76-9c41-c4095f0c4445 on localhost:31010]{color}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (DRILL-7237) IllegalStateException in aggregation function 'single_value' when there is a varchar datatype in the subquery results
Denys Ordynskiy created DRILL-7237:
--
Summary: IllegalStateException in aggregation function
'single_value' when there is a varchar datatype in the subquery results
Key: DRILL-7237
URL: https://issues.apache.org/jira/browse/DRILL-7237
Project: Apache Drill
Issue Type: Bug
Affects Versions: 1.15.0, 1.14.0
Reporter: Denys Ordynskiy
Assignee: Volodymyr Vysotskyi
Attachments: drillbit.log
*Description:*
The following issue can be reproduced on the EBF for the
[DRILL-7050|https://issues.apache.org/jira/browse/DRILL-7050].
_For the query with > 1 row in subquery results where the data type of these
results *is not varchar*:_
{noformat}
SELECT
e.full_name,
(
SELECT
ine.employee_id
FROM
cp.`employee.json` ine
WHERE
ine.position_id = e.position_id
) as empl_id
FROM
cp.`employee.json` e
LIMIT 20
{noformat}
_We have the following correct and informative error:_
{noformat}
Query Failed: An Error Occurred
org.apache.drill.common.exceptions.UserRemoteException: FUNCTION ERROR: Input
for single_value function has more than one row Fragment 0:0 [Error Id:
b770098f-b1c7-4647-9f41-9e986a0e47b7 on maprhost:31010]
{noformat}
_But when in the result set of the subquery we have *a varchar data type*:_
{noformat}
SELECT
e.full_name,
(
SELECT
ine.first_name
FROM
cp.`employee.json` ine
WHERE
ine.position_id = e.position_id
) as empl_id
FROM
cp.`employee.json` e
LIMIT 20
{noformat}
*Actual result:*
_Drill throws the following error:_
{noformat}
org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR:
IllegalStateException: Workspace variable 'value' in aggregation function
'single_value' is not allowed to have variable length type. Fragment 0:0
Please, refer to logs for more information. [Error Id:
32325ba9-d2b3-4216-acf6-8e80dfe4a56a on maprhost:31010]
{noformat}
Log file is in the attachment "drillbit.log"
*Expected result:*
Drill should return the same informative error to any of a data types in the
subquery result set.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (DRILL-7184) Set the IDs for the unique HTML tags in the Drill Web UI
Denys Ordynskiy created DRILL-7184: -- Summary: Set the IDs for the unique HTML tags in the Drill Web UI Key: DRILL-7184 URL: https://issues.apache.org/jira/browse/DRILL-7184 Project: Apache Drill Issue Type: Task Reporter: Denys Ordynskiy Assignee: Denys Ordynskiy Selenium web pages automation requires identifiers on HTML tags. We need to find all HTML tags without ID's that are useful for Drill Web UI automation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (DRILL-7079) Drill can't query views from the S3 storage when plain authentication is enabled
Denys Ordynskiy created DRILL-7079:
--
Summary: Drill can't query views from the S3 storage when plain
authentication is enabled
Key: DRILL-7079
URL: https://issues.apache.org/jira/browse/DRILL-7079
Project: Apache Drill
Issue Type: Bug
Reporter: Denys Ordynskiy
Enable plain authentication in Drill.
Create the view on the S3 storage:
create view s3.tmp.`testview` as select * from cp.`employee.json` limit 20;
Try to select data from the created view:
select * from s3.tmp.`testview`;
*Actual result*:
{noformat}
2019-02-27 17:01:09,202 [Client-1] INFO o.a.d.j.i.DrillCursor$ResultsListener
- [#4] Query failed:
org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR:
IllegalArgumentException: A valid userName is expected
Please, refer to logs for more information.
[Error Id: 2271c3aa-6d09-4b51-a585-0e0e954b46eb on maprhost:31010]
at
org.apache.drill.exec.rpc.user.QueryResultHandler.resultArrived(QueryResultHandler.java:123)
[drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
at
org.apache.drill.exec.rpc.user.UserClient.handle(UserClient.java:422)
[drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
at org.apache.drill.exec.rpc.user.UserClient.handle(UserClient.java:96)
[drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
at
org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:273)
[drill-rpc-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
at
org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:243)
[drill-rpc-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
at
io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88)
[netty-codec-4.0.48.Final.jar:4.0.48.Final]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
[netty-transport-4.0.48.Final.jar:4.0.48.Final]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
[netty-transport-4.0.48.Final.jar:4.0.48.Final]
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335)
[netty-transport-4.0.48.Final.jar:4.0.48.Final]
at
io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287)
[netty-handler-4.0.48.Final.jar:4.0.48.Final]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
[netty-transport-4.0.48.Final.jar:4.0.48.Final]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
[netty-transport-4.0.48.Final.jar:4.0.48.Final]
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335)
[netty-transport-4.0.48.Final.jar:4.0.48.Final]
at
io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
[netty-codec-4.0.48.Final.jar:4.0.48.Final]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
[netty-transport-4.0.48.Final.jar:4.0.48.Final]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
[netty-transport-4.0.48.Final.jar:4.0.48.Final]
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335)
[netty-transport-4.0.48.Final.jar:4.0.48.Final]
at
io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:312)
[netty-codec-4.0.48.Final.jar:4.0.48.Final]
at
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:286)
[netty-codec-4.0.48.Final.jar:4.0.48.Final]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
[netty-transport-4.0.48.Final.jar:4.0.48.Final]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
[netty-transport-4.0.48.Final.jar:4.0.48.Final]
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335)
[netty-transport-4.0.48.Final.jar:4.0.48.Final]
at
io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
[netty-transport-4.0.48.Final.jar:4.0.48.Final]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356)
[netty-transport-4.0.48.Final.jar:4.0.48.Final]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342)
[netty-transport-4.0.48.Final.jar:4.0.48.Final]
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335)
[jira] [Created] (DRILL-6995) Count() function is not working with columns alias
Denys Ordynskiy created DRILL-6995:
--
Summary: Count() function is not working with columns alias
Key: DRILL-6995
URL: https://issues.apache.org/jira/browse/DRILL-6995
Project: Apache Drill
Issue Type: Bug
Affects Versions: 1.15.0
Reporter: Denys Ordynskiy
The query works correctly if I use column name in the count() function:
{code:java}
select department_id c from cp.`employee.json` group by c having
count(department_id) > 1
{code}
|c|
|1|
|2|
|3|
|11|
|5|
|4|
|19|
|14|
|15|
|16|
But when I set column alias "c" in the count() function, Drill returns *an
empty result set*:
{code:java}
select department_id c from cp.`employee.json` group by c having count(c) > 1
{code}
|c|
| |
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (DRILL-6932) Drill can't submit Physical plan, returned by submitting Logical plan
Denys Ordynskiy created DRILL-6932:
--
Summary: Drill can't submit Physical plan, returned by submitting
Logical plan
Key: DRILL-6932
URL: https://issues.apache.org/jira/browse/DRILL-6932
Project: Apache Drill
Issue Type: Bug
Affects Versions: 1.14.0
Reporter: Denys Ordynskiy
*Description:*
Drill WebUI allows to submit Logical plan and returns 2 types of a result:
- "resultMode" : "EXEC" - to get query result set
- "resultMode" : "PHYSICAL" - to get Physical plan of the query
If I get Physical plan by submiting Logical plan with "resultMode" : "PHYSICAL",
this Physical plan throws an error after submiting it.
*Steps to reproduce issue in a Drill WebUI:*
- get Logical plan by running query "EXPLAIN PLAN WITHOUT IMPLEMENTATION FOR
SELECT * FROM cp.`employee.json` LIMIT 2";
- change "resultMode" from "LOGICAL" to "PHYSICAL";
- select "Query Type" - "LOGICAL" and submit Logical plan and get Physical plan;
- select "Query Type" - "PHYSICAL" and submit Physical plan;
*Actual result:*
{color:#d04437}*Query Failed: An Error Occurred*
org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR:
IllegalStateException: Already had POJO for id (java.lang.Integer) [[ObjectId:
key=0,
type=com.fasterxml.jackson.databind.deser.impl.PropertyBasedObjectIdGenerator,
scope=java.lang.Object]] Please, refer to logs for more information. [Error Id:
f7a2422f-55e8-4478-9e22-1008175ce9b6 on maprhost:31010]{color}
*Expected result:*
result set for the original query "SELECT * FROM cp.`employee.json` LIMIT 2"
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (DRILL-6931) Drill "SHOW FILES" command duplicates empty S3 folders as subfolders
Denys Ordynskiy created DRILL-6931: -- Summary: Drill "SHOW FILES" command duplicates empty S3 folders as subfolders Key: DRILL-6931 URL: https://issues.apache.org/jira/browse/DRILL-6931 Project: Apache Drill Issue Type: Bug Affects Versions: 1.14.0 Reporter: Denys Ordynskiy *Description:* If folder was created by pressing "+ Create folder" button on the S3 management console, Drill "SHOW FILES" command showing in this folder another subfolder with the same name. But this folder doesn't exist. *Example:* - login to the S3 management console; - open your bucket `some_bucket`; - create an empty folder `my_test_folder` by pressing "+ Create folder" button; - run Drill and open sqlline client; - exec query "show files in s3.tmp.`my_test_folder`;" *Actual result:* | name | isDirectory | isFile | length | owner | group | permissions | accessTime |modificationTime| | my_test_folder | true | false | 0 ||| rwxrwxrwx| 1970-01-01 03:00:00.0 | 1970-01-01 03:00:00.0 | 1 row selected (1.318 seconds) *Expected result:* an empty result set. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (DRILL-6919) Error: cannot find symbol in class ServerSocketUtil
Denys Ordynskiy created DRILL-6919:
--
Summary: Error: cannot find symbol in class ServerSocketUtil
Key: DRILL-6919
URL: https://issues.apache.org/jira/browse/DRILL-6919
Project: Apache Drill
Issue Type: Bug
Affects Versions: 1.15.0
Reporter: Denys Ordynskiy
Assignee: Volodymyr Vysotskyi
Compilation error when building Drill with mapr profile:
{color:#d04437}[ERROR]
/home/mapr/git_repo/drill/exec/java-exec/src/test/java/org/apache/drill/test/TestGracefulShutdown.java:[24,28]
error: cannot find symbol
symbol: class ServerSocketUtil
location: package org.apache.hadoop.net{color}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (DRILL-6911) Documentation issue - Hadoop core-site.xml is not supported by Drill to read S3 credentials
Denys Ordynskiy created DRILL-6911:
--
Summary: Documentation issue - Hadoop core-site.xml is not
supported by Drill to read S3 credentials
Key: DRILL-6911
URL: https://issues.apache.org/jira/browse/DRILL-6911
Project: Apache Drill
Issue Type: Bug
Affects Versions: 1.14.0
Reporter: Denys Ordynskiy
Assignee: Bridget Bevens
In the Drill S3 documentation https://drill.apache.org/docs/s3-storage-plugin/
Section "Providing AWS Credentials" describing 3 ways to setup AWS S3
credentials in Drill:
- storage plugin;
- Drill-specific core-site.xml;
- existing S3 configuration for Hadoop.
Third item is not supported by Drill. Hadoop core-site.xml config file may
contains S3 credentials, but Drill doesn't read any S3 parameters directly from
Hadoop config file.
Third item
{code:java}
In a Hadoop environment, you can use the existing S3 configuration for Hadoop.
The AWS credentials should already be defined. All you need to do is configure
the S3 storage plugin.
{code}
should be removed from the document
https://drill.apache.org/docs/s3-storage-plugin/
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (DRILL-6894) CTAS and CTTAS are not working on S3 storage when cache is disabled
Denys Ordynskiy created DRILL-6894:
--
Summary: CTAS and CTTAS are not working on S3 storage when cache
is disabled
Key: DRILL-6894
URL: https://issues.apache.org/jira/browse/DRILL-6894
Project: Apache Drill
Issue Type: Bug
Affects Versions: 1.15.0
Reporter: Denys Ordynskiy
Attachments: CTAS_drillbit.log, CTAS_drillbit_queries.json,
CTAS_sqlline.log, CTTAS_drillbit.log, CTTAS_drillbit_queries.json,
CTTAS_sqlline.log, s3.json
When S3 storage plugin option "fs.s3a.impl.disable.cache" is true in the config
section,
CTAS and CTTAS queries throwing error in Sqlline:
*create temporary table s3.tmp.`cttastblwithcache2` as select * from
cp.`employee.json`;*
{color:#d04437}Error: SYSTEM ERROR: SdkClientException: Unable to load AWS
credentials from any provider in the chain
Fragment 0:0
Please, refer to logs for more information.
[Error Id: 8e386b68-d4fb-4cc6-ba1c-fb41ae0cc9ed on maprhost:31010]
(state=,code=0){color}
*create table s3.tmp.`ctastblwithcache` as select * from cp.`employee.json`;*
{color:#d04437}Error: SYSTEM ERROR: SdkClientException: Unable to load AWS
credentials from any provider in the chain
Fragment 0:0
Please, refer to logs for more information.
[Error Id: 4346d300-44be-4f17-90b6-4f3a0db0a148 on maprhost:31010]
(state=,code=0){color}
Logs and my storage plugin are in the attachments.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (DRILL-6875) Drill doesn't try to update connection for S3 after session expired
Denys Ordynskiy created DRILL-6875:
--
Summary: Drill doesn't try to update connection for S3 after
session expired
Key: DRILL-6875
URL: https://issues.apache.org/jira/browse/DRILL-6875
Project: Apache Drill
Issue Type: Bug
Affects Versions: 1.15.0
Reporter: Denys Ordynskiy
Attachments: drillbit.log
*Steps to reproduce:*
- Drill has S3 storage plugin.
- Open sqlline and run query to S3.
- Leave sqlline opened for more than 12 hours.
- In opened sqlline run query to S3.
*Expected result:*
Drill should update authorization session and successfully execute query.
*Actual result:*
Sqlline returns an error:
*{color:#d04437}Error: VALIDATION ERROR: Forbidden (Service: Amazon S3; Status
Code: 403; Error Code: 403 Forbidden; Request ID: 4A94DD331A035625; S3 Extended
Request ID:
uy94YdRpQ3ZriCz9xbnDi0yinB4O9kGrH7XPAURhjh8WZoxsbawojQA6v7mfvu920yOYbEI5WP8=)
[Error Id: 4b44a83b-0e47-45a4-92e3-75f94f5a70cb on maprhost:31010]
(state=,code=0){color}*
*Reopening sqlline doesn't help to get S3 access.*
*Access problem can be solved only by restarting Drill.*
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (DRILL-6874) CTAS from json to parquet is not working on S3 storage
Denys Ordynskiy created DRILL-6874:
--
Summary: CTAS from json to parquet is not working on S3 storage
Key: DRILL-6874
URL: https://issues.apache.org/jira/browse/DRILL-6874
Project: Apache Drill
Issue Type: Bug
Affects Versions: 1.15.0
Reporter: Denys Ordynskiy
Attachments: ctasjsontoparquet.zip, drillbit.log,
drillbit_queries.json, s3src.json, sqlline.log
Json file "s3src.json" was uploaded to the s3 storage.
Query from Json works fine:
select * from s3.tmp.`s3src.json`;
| id | first_name | last_name |
| 1 | first_name1 | last_name1 |
| 2 | first_name2 | last_name2 |
| 3 | first_name3 | last_name3 |
| 4 | first_name4 | last_name4 |
| 5 | first_name5 | last_name5 |
5 rows selected (2.803 seconds)
CTAS from this json file returns successfully result:
create table s3.tmp.`ctasjsontoparquet` as select * from s3.tmp.`s3src.json`;
| Fragment | Number of records written |
| 0_0 | 5 |
1 row selected (9.264 seconds)
*Query from the created parquet table {color:#d04437}throws an error:{color}*
select * from s3.tmp.`ctasjsontoparquet`;
{code:java}
Error: INTERNAL_ERROR ERROR: Error in parquet record reader.
Message: Failure in setting up reader
Parquet Metadata: ParquetMetaData{FileMetaData{schema: message root {
optional int64 id;
optional binary first_name (UTF8);
optional binary last_name (UTF8);
}
, metadata: {drill-writer.version=2, drill.version=1.15.0-SNAPSHOT}}, blocks:
[BlockMetaData{5, 360 [ColumnMetaData{UNCOMPRESSED [id] optional int64 id
[BIT_PACKED, RLE, PLAIN], 4}, ColumnMetaData{UNCOMPRESSED [first_name] optional
binary first_name (UTF8) [BIT_PACKED, RLE, PLAIN], 111},
ColumnMetaData{UNCOMPRESSED [last_name] optional binary last_name (UTF8)
[BIT_PACKED, RLE, PLAIN], 241}]}]}
Fragment 0:0
Please, refer to logs for more information.
[Error Id: 885723e4-8385-4fb0-87dd-c08b0570db95 on maprhost:31010]
(state=,code=0)
{code}
The same CTAS query works fine on MapRFS and FileSystem storages.
Log files, json file and created parquet file from S3 are in the attachments.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (DRILL-6869) Drill allows to create views outside workspace
Denys Ordynskiy created DRILL-6869:
--
Summary: Drill allows to create views outside workspace
Key: DRILL-6869
URL: https://issues.apache.org/jira/browse/DRILL-6869
Project: Apache Drill
Issue Type: Bug
Affects Versions: 1.15.0
Reporter: Denys Ordynskiy
Attachments: Amazon_S3_FS_stor_plugin.json,
FileSystem_stor_plugin.json, MapR_FS_stor_plugin.json
Parameter 'allowAccessOutsideWorkspace' is false for tested workspaces.
On MaprFS and S3 storages Drill allows to create views outside workspace.
*Example on MapRFS:*
create view dfs.tmp.`{color:#d04437}*/*{color}testbugonmfs` as SELECT * FROM
cp.`employee.json` LIMIT 20;
|ok|summary|
|true|View '/testbugonmfs' *created successfully in 'dfs.tmp' schema*|
1 row selected (0.93 seconds)
The file "testbugonmfs.view.drill" was *created* in the *root "/" folder,* but
not in used workspace "/tmp" folder.
Select query works with root "/" folder {color:#d04437}*outside*{color} the
dfs.tmp workspace:
select count * from dfs.tmp.`{color:#d04437}*/*{color}testbugonmfs`;
|EXPR$0|
|20|
1 row selected (1.813 seconds)
*Example on S3*:
create view s3.tmp.`{color:#d04437}*/*{color}testbugons3` as SELECT * FROM
cp.`employee.json` LIMIT 20;
|ok|summary|
|true|View '/testbugons3' *created successfully in 's3.tmp' schema*|
1 row selected (3.455 seconds)
The file "testbugons3.view.drill" was *created* in the *root "/" bucket
folder*, but not in used workspace "/tmp" folder.
Select query also works with root "/" bucket folder
{color:#d04437}*outside*{color} the s3.tmp workspace:
select count * from s3.tmp.`/testbugons3`;
|EXPR$0|
|20|
1 row selected (3.209 seconds)
*Expected result* should be like on FileSystem storage:
On FileSystem storage plugin Drill doesn't allow to create views outside
workspace.
Query "create view dfs.tmp.`/testbugonfs` as SELECT * FROM cp.`employee.json`
LIMIT 20;"
Returns an error: "{color:#d04437}Error: SYSTEM ERROR: FileNotFoundException:
/testbugonfs.view.drill (Permission denied){color}".
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (DRILL-6863) Drop table is not working on amazon S3
Denys Ordynskiy created DRILL-6863:
--
Summary: Drop table is not working on amazon S3
Key: DRILL-6863
URL: https://issues.apache.org/jira/browse/DRILL-6863
Project: Apache Drill
Issue Type: Bug
Affects Versions: 1.15.0
Reporter: Denys Ordynskiy
Parquet table was created using CTAS on S3.
Request "drop table
s3.tmp.`/drill/transitive_closure/DRILL_6173_filterPushdown/tab1`"
returns successfully response:
"Table [/drill/transitive_closure/DRILL_6173_filterPushdown/tab1] dropped"
*Actual result:*
Drill did not delete the table `tab1`, files still present on S3 storage.
In "drillbit.out":
{code:java}
23:54:49.661 [2416095b-6544-fc80-0dfa-2fc19c4dee0e:foreman] ERROR
o.apache.hadoop.fs.s3a.S3AFileSystem - rename: src not found
/drill/transitive_closure/DRILL_6173_filterPushdown/tab1
{code}
*Expected result:*
Drill should delete table from S3 storage.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (DRILL-6858) INFORMATION_SCHEMA.`FILES` table not show any files if storage contains "write only" folder
Denys Ordynskiy created DRILL-6858:
--
Summary: INFORMATION_SCHEMA.`FILES` table not show any files if
storage contains "write only" folder
Key: DRILL-6858
URL: https://issues.apache.org/jira/browse/DRILL-6858
Project: Apache Drill
Issue Type: Bug
Affects Versions: 1.15.0
Reporter: Denys Ordynskiy
*Steps to reproduce bug:*
* storage contains folder with "write only" permission;
* Drill option "storage.list_files_recursively" is false;
* query result for "SELECT * FROM INFORMATION_SCHEMA.`FILES`;" shows the list
of the files and folders for every workspace for each storage plugin;
* set Drill option "storage.list_files_recursively" to true;
* run query "SELECT * FROM INFORMATION_SCHEMA.`FILES`;"
*Actual result:*
Drill returns empty results table if workspace has a folder with "write only"
permission. Even if this workspace contains files and folders, that have "read"
access right.
*Expected result:*
Drill returns list of files for each folder with "read" permission and hide
files list for "write only" folders. Showing only folders name when permission
is "write only".
This bug can be reproduced on the "Amazon S3" and "File System" Drill storage
plugins.
In "drillbit.log" for "*File System*" Drill storage plugin:
{code:java}
2018-11-15 20:20:19,760 [2412471b-a4f5-d247-0db6-5d50cdcef634:frag:0:0] WARN
o.a.d.e.s.i.InfoSchemaRecordGenerator - Failure while trying to list files
java.io.FileNotFoundException: file:///home/mapr/test/writeonly: null file
list{code}
In "drillbit.log" for "*Amazon S3*" Drill storage plugin:
{code:java}
2018-11-15 16:50:04,735 [24127862-a0e3-02ad-6b66-d649a62e8b8a:frag:0:0] WARN
o.a.d.e.s.i.InfoSchemaRecordGenerator - Failure while trying to list files
java.io.FileNotFoundException: No such file or directory:
s3a://bucket/writeonly{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (DRILL-6819) Remove invisible "back" link on query results page in Drill WebUI
Denys Ordynskiy created DRILL-6819: -- Summary: Remove invisible "back" link on query results page in Drill WebUI Key: DRILL-6819 URL: https://issues.apache.org/jira/browse/DRILL-6819 Project: Apache Drill Issue Type: Bug Affects Versions: 1.14.0 Reporter: Denys Ordynskiy Assignee: Denys Ordynskiy Attachments: image.png In Drill WebUI on page with query results and some other pages we have invisible link with text - "back" and URL - "/queries" Since we have visible link "Query" on navigation tab with the same URL, we don't need "back" link on query results page. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (DRILL-6769) Sqlline doesn't see line endings when it prints first row fo the explain plan results table
Denys Ordynskiy created DRILL-6769: -- Summary: Sqlline doesn't see line endings when it prints first row fo the explain plan results table Key: DRILL-6769 URL: https://issues.apache.org/jira/browse/DRILL-6769 Project: Apache Drill Issue Type: Bug Affects Versions: 1.15.0 Reporter: Denys Ordynskiy Assignee: Arina Ielchiieva Attachments: Sqlline.png After Sqlline was upgraded in DRILL-3853, in the query "EXPLAIN PLAN FOR .." results table has a very long first row with titles. After I run the query "EXPLAIN PLAN FOR SELECT 1;", width of the table column is calculated by the number of symbols in the biggest cell of columns. If there are new lines in the data, Sqlline should set width of this column by the longest line of the cells data instead of number of all symbols of that cell. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (DRILL-6761) Documentation - broken table of contents on the REST-API page
Denys Ordynskiy created DRILL-6761: -- Summary: Documentation - broken table of contents on the REST-API page Key: DRILL-6761 URL: https://issues.apache.org/jira/browse/DRILL-6761 Project: Apache Drill Issue Type: Bug Affects Versions: 1.15.0 Reporter: Denys Ordynskiy Assignee: Denys Ordynskiy Page [https://drill.apache.org/docs/rest-api-introduction/] has a broken links on the table of contents: [https://drill.apache.org/docs/rest-api/#query] [https://drill.apache.org/docs/rest-api-introduction/(/docs/rest-api/#profiles)] [https://drill.apache.org/docs/rest-api/#storage] [https://drill.apache.org/docs/rest-api-introduction/(/docs/rest-api/#threads)] [https://drill.apache.org/docs/rest-api/#options] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (DRILL-6716) NullPointerException for select query with alias on * symbol
Denys Ordynskiy created DRILL-6716: -- Summary: NullPointerException for select query with alias on * symbol Key: DRILL-6716 URL: https://issues.apache.org/jira/browse/DRILL-6716 Project: Apache Drill Issue Type: Bug Affects Versions: 1.14.0 Reporter: Denys Ordynskiy Fix For: 1.15.0 When I added alias for * in SELECT query in Drill WebUI and in sqlline: SELECT * as testAlias FROM cp.`employee.json` LIMIT 2; I got an error: -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (DRILL-6472) Drill allows to use decimal zero precision in CAST function for CTAS
Denys Ordynskiy created DRILL-6472:
--
Summary: Drill allows to use decimal zero precision in CAST
function for CTAS
Key: DRILL-6472
URL: https://issues.apache.org/jira/browse/DRILL-6472
Project: Apache Drill
Issue Type: Bug
Affects Versions: 1.14.0
Reporter: Denys Ordynskiy
I can run query with casting data for decimal with zero precision in SELECT
statement
{code:java}
select cast(.05 as decimal(0,5));
{code}
{code:java}
+-+
| EXPR$0 |
+-+
| .05000 |
+-+
{code}
Also I can use this query for CTAS
{code:java}
create table dfs.tmp.`zero_pr` as select cast(.05 as decimal(0,5));
{code}
{code:java}
+--+---+
| Fragment | Number of records written |
+--+---+
| 0_0 | 1 |
+--+---+
{code}
But when I try to select data from this table, I get an error:
{code:java}
create table dfs.tmp.`zero_pr` as select cast(.05 as decimal(0,5));
{code}
{code:java}
Error: SYSTEM ERROR: IllegalArgumentException: Invalid DECIMAL precision: 0
[Error Id: 2e320f55-aa65-4127-82be-6256e0744bdc on maprhost:31010]
(state=,code=0){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (DRILL-6471) Different result for CAST String and Decimal literals as Decimal
Denys Ordynskiy created DRILL-6471:
--
Summary: Different result for CAST String and Decimal literals as
Decimal
Key: DRILL-6471
URL: https://issues.apache.org/jira/browse/DRILL-6471
Project: Apache Drill
Issue Type: Bug
Affects Versions: 1.14.0
Reporter: Denys Ordynskiy
If I cast String and Decimal literals as Decimal and literal precision is more
than I casting to,
I see different results for the same literal data in String and Decimal formats
SELECT cast('1234.5678' as DECIMAL(2, 2));
{code:java}
+--+
| EXPR$0 |
+--+
| 1200.00 |
+--+
{code}
SELECT cast(1234.5678 as DECIMAL(2, 2));
{code:java}
+-+
| EXPR$0 |
+-+
| 0.12|
+-+
{code}
CAST result for one number in String and Decimal formats should be the same.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
