[jira] [Commented] (DRILL-2408) CTAS should not create empty folders when underlying query returns no results

2015-05-08 Thread Rahul Challapalli (JIRA) 

[ 
https://issues.apache.org/jira/browse/DRILL-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14535451#comment-14535451
 ] 

Rahul Challapalli commented on DRILL-2408:
--

Verified!

> CTAS should not create empty folders when underlying query returns no results
> -
>
> Key: DRILL-2408
> URL: https://issues.apache.org/jira/browse/DRILL-2408
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Storage - Writer
>Affects Versions: 0.8.0
>Reporter: Aman Sinha
>Assignee: Aman Sinha
> Fix For: 1.0.0
>
> Attachments: DRILL-2408.1.patch.txt, DRILL-2408.2.patch.txt, 
> DRILL-2408.3.patch.txt, DRILL-2408.4.patch.txt, DRILL-2408.5.patch.txt, 
> DRILL-2408.6.patch.txt, DRILL-2408.7.patch.txt, DRILL-2408.8.patch.txt
>
>
> {noformat}
> 0: jdbc:drill:schema=dfs> select c_integer, c_bigint, c_date, c_time, 
> c_varchar from j4 where c_bigint is null;
> ++++++
> | c_integer  |  c_bigint  |   c_date   |   c_time   | c_varchar  |
> ++++++
> ++++++
> No rows selected (0.126 seconds)
> 0: jdbc:drill:schema=dfs> create table ctas_t6(c1,c2,c3,c4,c5) as select 
> c_integer, c_bigint, c_date, c_time, c_varchar from j4 where c_bigint is null;
> ++---+
> |  Fragment  | Number of records written |
> ++---+
> | 0_0| 0 |
> ++---+
> 1 row selected (0.214 seconds)
> 0: jdbc:drill:schema=dfs> select * from ctas_t6;
> Query failed: IndexOutOfBoundsException: Index: 0, Size: 0
> Error: exception while executing query: Failure while executing query. 
> (state=,code=0)
> {noformat}
> parquet file was not created, but directory was:
> {noformat}
> [Mon Apr 06 09:03:41 
> root@/mapr/vmarkman.cluster.com/drill/testdata/joins/ctas_t6 ] # pwd
> /mapr/vmarkman.cluster.com/drill/testdata/joins/ctas_t6
> [Mon Apr 06 09:03:45 
> root@/mapr/vmarkman.cluster.com/drill/testdata/joins/ctas_t6 ] # ls -l
> total 0
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-2408) CTAS should not create empty folders when underlying query returns no results

2015-05-05 Thread Deneche A. Hakim (JIRA) 

[ 
https://issues.apache.org/jira/browse/DRILL-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14528681#comment-14528681
 ] 

Deneche A. Hakim commented on DRILL-2408:
-

I changed this to 1.0.0 as it is in a reviewable state. If it doesn't pass the 
review we can change it back to 1.2.0

> CTAS should not create empty folders when underlying query returns no results
> -
>
> Key: DRILL-2408
> URL: https://issues.apache.org/jira/browse/DRILL-2408
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Storage - Writer
>Affects Versions: 0.8.0
>Reporter: Aman Sinha
>Assignee: Aman Sinha
> Fix For: 1.0.0
>
> Attachments: DRILL-2408.1.patch.txt, DRILL-2408.2.patch.txt, 
> DRILL-2408.3.patch.txt, DRILL-2408.4.patch.txt, DRILL-2408.5.patch.txt, 
> DRILL-2408.6.patch.txt, DRILL-2408.7.patch.txt
>
>
> {noformat}
> 0: jdbc:drill:schema=dfs> select c_integer, c_bigint, c_date, c_time, 
> c_varchar from j4 where c_bigint is null;
> ++++++
> | c_integer  |  c_bigint  |   c_date   |   c_time   | c_varchar  |
> ++++++
> ++++++
> No rows selected (0.126 seconds)
> 0: jdbc:drill:schema=dfs> create table ctas_t6(c1,c2,c3,c4,c5) as select 
> c_integer, c_bigint, c_date, c_time, c_varchar from j4 where c_bigint is null;
> ++---+
> |  Fragment  | Number of records written |
> ++---+
> | 0_0| 0 |
> ++---+
> 1 row selected (0.214 seconds)
> 0: jdbc:drill:schema=dfs> select * from ctas_t6;
> Query failed: IndexOutOfBoundsException: Index: 0, Size: 0
> Error: exception while executing query: Failure while executing query. 
> (state=,code=0)
> {noformat}
> parquet file was not created, but directory was:
> {noformat}
> [Mon Apr 06 09:03:41 
> root@/mapr/vmarkman.cluster.com/drill/testdata/joins/ctas_t6 ] # pwd
> /mapr/vmarkman.cluster.com/drill/testdata/joins/ctas_t6
> [Mon Apr 06 09:03:45 
> root@/mapr/vmarkman.cluster.com/drill/testdata/joins/ctas_t6 ] # ls -l
> total 0
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-2408) CTAS should not create empty folders when underlying query returns no results

2015-04-21 Thread Aman Sinha (JIRA) 

[ 
https://issues.apache.org/jira/browse/DRILL-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14506031#comment-14506031
 ] 

Aman Sinha commented on DRILL-2408:
---

Can you test your patch with a CTAS that has parallelism where some parquet 
writer minor fragments don't have any input data ?  Those writers would have 
normally created 0 length files.  Here's an example test case: 
  - create a table T1 (a1) with 100 rows where a1 has only 2 unique values
  - alter session set `planner.slice_target` = 1 
  - CREATE TABLE T2 AS SELECT a1, count(*) FROM T1 GROUP BY a1

> CTAS should not create empty folders when underlying query returns no results
> -
>
> Key: DRILL-2408
> URL: https://issues.apache.org/jira/browse/DRILL-2408
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Storage - Writer
>Affects Versions: 0.8.0
>Reporter: Aman Sinha
>Assignee: Aman Sinha
>Priority: Critical
> Fix For: 1.0.0
>
> Attachments: DRILL-2408.1.patch.txt, DRILL-2408.2.patch.txt, 
> DRILL-2408.3.patch.txt, DRILL-2408.4.patch.txt
>
>
> {noformat}
> 0: jdbc:drill:schema=dfs> select c_integer, c_bigint, c_date, c_time, 
> c_varchar from j4 where c_bigint is null;
> ++++++
> | c_integer  |  c_bigint  |   c_date   |   c_time   | c_varchar  |
> ++++++
> ++++++
> No rows selected (0.126 seconds)
> 0: jdbc:drill:schema=dfs> create table ctas_t6(c1,c2,c3,c4,c5) as select 
> c_integer, c_bigint, c_date, c_time, c_varchar from j4 where c_bigint is null;
> ++---+
> |  Fragment  | Number of records written |
> ++---+
> | 0_0| 0 |
> ++---+
> 1 row selected (0.214 seconds)
> 0: jdbc:drill:schema=dfs> select * from ctas_t6;
> Query failed: IndexOutOfBoundsException: Index: 0, Size: 0
> Error: exception while executing query: Failure while executing query. 
> (state=,code=0)
> {noformat}
> parquet file was not created, but directory was:
> {noformat}
> [Mon Apr 06 09:03:41 
> root@/mapr/vmarkman.cluster.com/drill/testdata/joins/ctas_t6 ] # pwd
> /mapr/vmarkman.cluster.com/drill/testdata/joins/ctas_t6
> [Mon Apr 06 09:03:45 
> root@/mapr/vmarkman.cluster.com/drill/testdata/joins/ctas_t6 ] # ls -l
> total 0
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-2408) CTAS should not create empty folders when underlying query returns no results

2015-04-21 Thread Aman Sinha (JIRA) 

[ 
https://issues.apache.org/jira/browse/DRILL-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14505562#comment-14505562
 ] 

Aman Sinha commented on DRILL-2408:
---

Could you post the patch on review board ? thanks. 

> CTAS should not create empty folders when underlying query returns no results
> -
>
> Key: DRILL-2408
> URL: https://issues.apache.org/jira/browse/DRILL-2408
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Storage - Writer
>Affects Versions: 0.8.0
>Reporter: Aman Sinha
>Assignee: Aman Sinha
>Priority: Critical
> Fix For: 1.0.0
>
> Attachments: DRILL-2408.1.patch.txt, DRILL-2408.2.patch.txt, 
> DRILL-2408.3.patch.txt, DRILL-2408.4.patch.txt
>
>
> {noformat}
> 0: jdbc:drill:schema=dfs> select c_integer, c_bigint, c_date, c_time, 
> c_varchar from j4 where c_bigint is null;
> ++++++
> | c_integer  |  c_bigint  |   c_date   |   c_time   | c_varchar  |
> ++++++
> ++++++
> No rows selected (0.126 seconds)
> 0: jdbc:drill:schema=dfs> create table ctas_t6(c1,c2,c3,c4,c5) as select 
> c_integer, c_bigint, c_date, c_time, c_varchar from j4 where c_bigint is null;
> ++---+
> |  Fragment  | Number of records written |
> ++---+
> | 0_0| 0 |
> ++---+
> 1 row selected (0.214 seconds)
> 0: jdbc:drill:schema=dfs> select * from ctas_t6;
> Query failed: IndexOutOfBoundsException: Index: 0, Size: 0
> Error: exception while executing query: Failure while executing query. 
> (state=,code=0)
> {noformat}
> parquet file was not created, but directory was:
> {noformat}
> [Mon Apr 06 09:03:41 
> root@/mapr/vmarkman.cluster.com/drill/testdata/joins/ctas_t6 ] # pwd
> /mapr/vmarkman.cluster.com/drill/testdata/joins/ctas_t6
> [Mon Apr 06 09:03:45 
> root@/mapr/vmarkman.cluster.com/drill/testdata/joins/ctas_t6 ] # ls -l
> total 0
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-2408) CTAS should not create empty folders when underlying query returns no results

2015-04-21 Thread Deneche A. Hakim (JIRA) 

[ 
https://issues.apache.org/jira/browse/DRILL-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14505190#comment-14505190
 ] 

Deneche A. Hakim commented on DRILL-2408:
-

All unit tests are passing along with functional, customer and tpch100

> CTAS should not create empty folders when underlying query returns no results
> -
>
> Key: DRILL-2408
> URL: https://issues.apache.org/jira/browse/DRILL-2408
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Storage - Writer
>Affects Versions: 0.8.0
>Reporter: Aman Sinha
>Assignee: Deneche A. Hakim
>Priority: Critical
> Fix For: 1.0.0
>
> Attachments: DRILL-2408.1.patch.txt, DRILL-2408.2.patch.txt, 
> DRILL-2408.3.patch.txt, DRILL-2408.4.patch.txt
>
>
> {noformat}
> 0: jdbc:drill:schema=dfs> select c_integer, c_bigint, c_date, c_time, 
> c_varchar from j4 where c_bigint is null;
> ++++++
> | c_integer  |  c_bigint  |   c_date   |   c_time   | c_varchar  |
> ++++++
> ++++++
> No rows selected (0.126 seconds)
> 0: jdbc:drill:schema=dfs> create table ctas_t6(c1,c2,c3,c4,c5) as select 
> c_integer, c_bigint, c_date, c_time, c_varchar from j4 where c_bigint is null;
> ++---+
> |  Fragment  | Number of records written |
> ++---+
> | 0_0| 0 |
> ++---+
> 1 row selected (0.214 seconds)
> 0: jdbc:drill:schema=dfs> select * from ctas_t6;
> Query failed: IndexOutOfBoundsException: Index: 0, Size: 0
> Error: exception while executing query: Failure while executing query. 
> (state=,code=0)
> {noformat}
> parquet file was not created, but directory was:
> {noformat}
> [Mon Apr 06 09:03:41 
> root@/mapr/vmarkman.cluster.com/drill/testdata/joins/ctas_t6 ] # pwd
> /mapr/vmarkman.cluster.com/drill/testdata/joins/ctas_t6
> [Mon Apr 06 09:03:45 
> root@/mapr/vmarkman.cluster.com/drill/testdata/joins/ctas_t6 ] # ls -l
> total 0
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-2408) CTAS should not create empty folders when underlying query returns no results

2015-04-14 Thread Deneche A. Hakim (JIRA) 

[ 
https://issues.apache.org/jira/browse/DRILL-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14494225#comment-14494225
 ] 

Deneche A. Hakim commented on DRILL-2408:
-

Fixing DRILL-2618 should resolve the issue when querying empty folders. I don't 
know about empty parquet files, when I tried creating empty parquet files with 
valid footers I still got reading errors from Drill and the parquet tools.

> CTAS should not create empty folders when underlying query returns no results
> -
>
> Key: DRILL-2408
> URL: https://issues.apache.org/jira/browse/DRILL-2408
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Storage - Writer
>Affects Versions: 0.8.0
>Reporter: Aman Sinha
>Assignee: Deneche A. Hakim
>Priority: Critical
> Fix For: 0.9.0
>
> Attachments: DRILL-2408.1.patch.txt, DRILL-2408.2.patch.txt, 
> DRILL-2408.3.patch.txt
>
>
> {noformat}
> 0: jdbc:drill:schema=dfs> select c_integer, c_bigint, c_date, c_time, 
> c_varchar from j4 where c_bigint is null;
> ++++++
> | c_integer  |  c_bigint  |   c_date   |   c_time   | c_varchar  |
> ++++++
> ++++++
> No rows selected (0.126 seconds)
> 0: jdbc:drill:schema=dfs> create table ctas_t6(c1,c2,c3,c4,c5) as select 
> c_integer, c_bigint, c_date, c_time, c_varchar from j4 where c_bigint is null;
> ++---+
> |  Fragment  | Number of records written |
> ++---+
> | 0_0| 0 |
> ++---+
> 1 row selected (0.214 seconds)
> 0: jdbc:drill:schema=dfs> select * from ctas_t6;
> Query failed: IndexOutOfBoundsException: Index: 0, Size: 0
> Error: exception while executing query: Failure while executing query. 
> (state=,code=0)
> {noformat}
> parquet file was not created, but directory was:
> {noformat}
> [Mon Apr 06 09:03:41 
> root@/mapr/vmarkman.cluster.com/drill/testdata/joins/ctas_t6 ] # pwd
> /mapr/vmarkman.cluster.com/drill/testdata/joins/ctas_t6
> [Mon Apr 06 09:03:45 
> root@/mapr/vmarkman.cluster.com/drill/testdata/joins/ctas_t6 ] # ls -l
> total 0
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-2408) CTAS should not create empty folders when underlying query returns no results

2015-04-13 Thread Rahul Challapalli (JIRA) 

[ 
https://issues.apache.org/jira/browse/DRILL-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14493316#comment-14493316
 ] 

Rahul Challapalli commented on DRILL-2408:
--

[~amansinha100] 
CTAS operations could be part of a daily workflow job. On some days these 
queries might return empty results. If we do not create empty tables (with 
empty parquet files), any subsequent automated jobs which query this table 
would fail. Thoughts?

> CTAS should not create empty folders when underlying query returns no results
> -
>
> Key: DRILL-2408
> URL: https://issues.apache.org/jira/browse/DRILL-2408
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Storage - Writer
>Affects Versions: 0.8.0
>Reporter: Aman Sinha
>Assignee: Deneche A. Hakim
>Priority: Critical
> Fix For: 0.9.0
>
> Attachments: DRILL-2408.1.patch.txt, DRILL-2408.2.patch.txt, 
> DRILL-2408.3.patch.txt
>
>
> {noformat}
> 0: jdbc:drill:schema=dfs> select c_integer, c_bigint, c_date, c_time, 
> c_varchar from j4 where c_bigint is null;
> ++++++
> | c_integer  |  c_bigint  |   c_date   |   c_time   | c_varchar  |
> ++++++
> ++++++
> No rows selected (0.126 seconds)
> 0: jdbc:drill:schema=dfs> create table ctas_t6(c1,c2,c3,c4,c5) as select 
> c_integer, c_bigint, c_date, c_time, c_varchar from j4 where c_bigint is null;
> ++---+
> |  Fragment  | Number of records written |
> ++---+
> | 0_0| 0 |
> ++---+
> 1 row selected (0.214 seconds)
> 0: jdbc:drill:schema=dfs> select * from ctas_t6;
> Query failed: IndexOutOfBoundsException: Index: 0, Size: 0
> Error: exception while executing query: Failure while executing query. 
> (state=,code=0)
> {noformat}
> parquet file was not created, but directory was:
> {noformat}
> [Mon Apr 06 09:03:41 
> root@/mapr/vmarkman.cluster.com/drill/testdata/joins/ctas_t6 ] # pwd
> /mapr/vmarkman.cluster.com/drill/testdata/joins/ctas_t6
> [Mon Apr 06 09:03:45 
> root@/mapr/vmarkman.cluster.com/drill/testdata/joins/ctas_t6 ] # ls -l
> total 0
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-2408) CTAS should not create empty folders when underlying query returns no results

2015-04-06 Thread Deneche A. Hakim (JIRA) 

[ 
https://issues.apache.org/jira/browse/DRILL-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14481385#comment-14481385
 ] 

Deneche A. Hakim commented on DRILL-2408:
-

True. I updated the issue to describe the new problem

> CTAS should not create empty folders when underlying query returns no results
> -
>
> Key: DRILL-2408
> URL: https://issues.apache.org/jira/browse/DRILL-2408
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Storage - Writer
>Affects Versions: 0.8.0
>Reporter: Aman Sinha
>Assignee: Deneche A. Hakim
>Priority: Critical
> Fix For: 0.9.0
>
> Attachments: DRILL-2408.1.patch.txt, DRILL-2408.2.patch.txt, 
> DRILL-2408.3.patch.txt
>
>
> {noformat}
> 0: jdbc:drill:schema=dfs> select c_integer, c_bigint, c_date, c_time, 
> c_varchar from j4 where c_bigint is null;
> ++++++
> | c_integer  |  c_bigint  |   c_date   |   c_time   | c_varchar  |
> ++++++
> ++++++
> No rows selected (0.126 seconds)
> 0: jdbc:drill:schema=dfs> create table ctas_t6(c1,c2,c3,c4,c5) as select 
> c_integer, c_bigint, c_date, c_time, c_varchar from j4 where c_bigint is null;
> ++---+
> |  Fragment  | Number of records written |
> ++---+
> | 0_0| 0 |
> ++---+
> 1 row selected (0.214 seconds)
> 0: jdbc:drill:schema=dfs> select * from ctas_t6;
> Query failed: IndexOutOfBoundsException: Index: 0, Size: 0
> Error: exception while executing query: Failure while executing query. 
> (state=,code=0)
> {noformat}
> parquet file was not created, but directory was:
> {noformat}
> [Mon Apr 06 09:03:41 
> root@/mapr/vmarkman.cluster.com/drill/testdata/joins/ctas_t6 ] # pwd
> /mapr/vmarkman.cluster.com/drill/testdata/joins/ctas_t6
> [Mon Apr 06 09:03:45 
> root@/mapr/vmarkman.cluster.com/drill/testdata/joins/ctas_t6 ] # ls -l
> total 0
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)