[jira] [Commented] (DRILL-2765) Throwing a NPE on exceuting query against parquet files
[ https://issues.apache.org/jira/browse/DRILL-2765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14498538#comment-14498538 ] Steven Phillips commented on DRILL-2765: What I am trying to say is that you don't have the latest master. You need to pull the latest changes from the apache_drill git repository. Run these steps: 1. git pull g...@github.com:apache/drill.git 2. git checkout master 3. mvn clean install -DskipTests That should produce a Tarball in the distribution/target directory that has the fix for your issue. > Throwing a NPE on exceuting query against parquet files > > > Key: DRILL-2765 > URL: https://issues.apache.org/jira/browse/DRILL-2765 > Project: Apache Drill > Issue Type: Sub-task > Components: Storage - Parquet >Affects Versions: 0.8.0 >Reporter: Mohsen Zainalpour >Assignee: Steven Phillips > Fix For: 1.0.0 > > Attachments: commit-id.png, master-branch commit-id.png, > url_column_completely_null.parquet, url_column_onetime_value.parquet > > > Executing a query on a parquet file that one of it`s columns haven`t any > value at all(but field exists in the schema) thorws a NPE -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-2765) Throwing a NPE on exceuting query against parquet files
[ https://issues.apache.org/jira/browse/DRILL-2765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14497910#comment-14497910 ] Masood Parvari commented on DRILL-2765: --- [~sphillips] No Steven. I compiled myself that drill and that was a master branch drill. But for reassurance I just again clone drill master branch (--branch master ) and it was pre compiled. i just introduce jars dependencies by coping them from "jars" directory in 0.8.0 version to the branch directory and I again run that example and again it throws a null pointer exception. screen shot of commit-id screen is attached. I am really amazed how your query doesn't return a null pointer exception. any clue please? > Throwing a NPE on exceuting query against parquet files > > > Key: DRILL-2765 > URL: https://issues.apache.org/jira/browse/DRILL-2765 > Project: Apache Drill > Issue Type: Sub-task > Components: Storage - Parquet >Affects Versions: 0.8.0 >Reporter: Mohsen Zainalpour >Assignee: Steven Phillips > Fix For: 1.0.0 > > Attachments: commit-id.png, master-branch commit-id.png, > url_column_completely_null.parquet, url_column_onetime_value.parquet > > > Executing a query on a parquet file that one of it`s columns haven`t any > value at all(but field exists in the schema) thorws a NPE -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-2765) Throwing a NPE on exceuting query against parquet files
[ https://issues.apache.org/jira/browse/DRILL-2765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14497856#comment-14497856 ] Steven Phillips commented on DRILL-2765: I see from screen shot that you are running 0.8. If you checkout and build the latest master, you should be able to run the query. > Throwing a NPE on exceuting query against parquet files > > > Key: DRILL-2765 > URL: https://issues.apache.org/jira/browse/DRILL-2765 > Project: Apache Drill > Issue Type: Sub-task > Components: Storage - Parquet >Affects Versions: 0.8.0 >Reporter: Mohsen Zainalpour >Assignee: Steven Phillips > Fix For: 1.0.0 > > Attachments: commit-id.png, url_column_completely_null.parquet, > url_column_onetime_value.parquet > > > Executing a query on a parquet file that one of it`s columns haven`t any > value at all(but field exists in the schema) thorws a NPE -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-2765) Throwing a NPE on exceuting query against parquet files
[ https://issues.apache.org/jira/browse/DRILL-2765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14497833#comment-14497833 ] Masood Parvari commented on DRILL-2765: --- [~sphillips] I attached the screen capture for commit id but I couldn't find any log. I just saw a sqlline.log in /var/log/drill but that was empty > Throwing a NPE on exceuting query against parquet files > > > Key: DRILL-2765 > URL: https://issues.apache.org/jira/browse/DRILL-2765 > Project: Apache Drill > Issue Type: Sub-task > Components: Storage - Parquet >Affects Versions: 0.8.0 >Reporter: Mohsen Zainalpour >Assignee: Steven Phillips > Fix For: 1.0.0 > > Attachments: commit-id.png, url_column_completely_null.parquet, > url_column_onetime_value.parquet > > > Executing a query on a parquet file that one of it`s columns haven`t any > value at all(but field exists in the schema) thorws a NPE -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-2765) Throwing a NPE on exceuting query against parquet files
[ https://issues.apache.org/jira/browse/DRILL-2765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14497698#comment-14497698 ] Steven Phillips commented on DRILL-2765: In response to your previous message, I am saying that the error occurs in the 0.8 release, but that it is fixed in current master branch. I don't know about any previous versions. > Throwing a NPE on exceuting query against parquet files > > > Key: DRILL-2765 > URL: https://issues.apache.org/jira/browse/DRILL-2765 > Project: Apache Drill > Issue Type: Sub-task > Components: Storage - Parquet >Affects Versions: 0.8.0 >Reporter: Mohsen Zainalpour >Assignee: Steven Phillips > Fix For: 1.0.0 > > Attachments: url_column_completely_null.parquet, > url_column_onetime_value.parquet > > > Executing a query on a parquet file that one of it`s columns haven`t any > value at all(but field exists in the schema) thorws a NPE -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-2765) Throwing a NPE on exceuting query against parquet files
[
https://issues.apache.org/jira/browse/DRILL-2765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14497692#comment-14497692
]
Steven Phillips commented on DRILL-2765:
I can't make a patch because I don't have a fix. But whatever was causing the
problem it is fixed now. I just verified with latest master:
{code}
0: jdbc:drill:> select * from dfs.tmp.nulls;
+++
|date|url |
+++
| 14 april 2015 | null |
| 14 april 2015 | null |
| 14 april 2015 | null |
| 14 april 2015 | null |
| 14 april 2015 | null |
| 14 april 2015 | null |
| 14 april 2015 | null |
| 14 april 2015 | null |
| 14 april 2015 | null |
| 14 april 2015 | null |
| 14 april 2015 | null |
| 14 april 2015 | null |
| 14 april 2015 | null |
| 14 april 2015 | null |
| 14 april 2015 | null |
| 14 april 2015 | null |
| 14 april 2015 | null |
| 14 april 2015 | null |
| 14 april 2015 | null |
| 14 april 2015 | null |
| 14 april 2015 | null |
| 14 april 2015 | null |
| 14 april 2015 | null |
+++
23 rows selected (0.599 seconds)
0: jdbc:drill:> select * from sys.version;
+++-+-++
| commit_id | commit_message | commit_time | build_email | build_time |
+++-+-++
| cb47df0c87676c5e0c0f94e5346fcd28f73080d7 | DRILL-2798 - don't print message
about Drill log dir unless environment variable DRILL_LOG_DEBUG=1 | 15.04.2015
@ 14:08:46 PDT | s...@apache.org | 16.04.2015 @ 00:48:52 PDT |
{code}
What is the git commit id you are building from?
Can you include the error message and stack trace from drillbit.log?
> Throwing a NPE on exceuting query against parquet files
>
>
> Key: DRILL-2765
> URL: https://issues.apache.org/jira/browse/DRILL-2765
> Project: Apache Drill
> Issue Type: Sub-task
> Components: Storage - Parquet
>Affects Versions: 0.8.0
>Reporter: Mohsen Zainalpour
>Assignee: Steven Phillips
> Fix For: 1.0.0
>
> Attachments: url_column_completely_null.parquet,
> url_column_onetime_value.parquet
>
>
> Executing a query on a parquet file that one of it`s columns haven`t any
> value at all(but field exists in the schema) thorws a NPE
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-2765) Throwing a NPE on exceuting query against parquet files
[ https://issues.apache.org/jira/browse/DRILL-2765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14497678#comment-14497678 ] Masood Parvari commented on DRILL-2765: --- [~sphillips] Steven I compiled drill from master branch and still getting null pointer exception. I am wondering with which version you got the result. Can you please make a patch to apply to 0.8.0 version ? > Throwing a NPE on exceuting query against parquet files > > > Key: DRILL-2765 > URL: https://issues.apache.org/jira/browse/DRILL-2765 > Project: Apache Drill > Issue Type: Sub-task > Components: Storage - Parquet >Affects Versions: 0.8.0 >Reporter: Mohsen Zainalpour >Assignee: Steven Phillips > Fix For: 1.0.0 > > Attachments: url_column_completely_null.parquet, > url_column_onetime_value.parquet > > > Executing a query on a parquet file that one of it`s columns haven`t any > value at all(but field exists in the schema) thorws a NPE -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-2765) Throwing a NPE on exceuting query against parquet files
[ https://issues.apache.org/jira/browse/DRILL-2765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14493892#comment-14493892 ] Masood Parvari commented on DRILL-2765: --- [~sphillips] Thank you for testing steven. So you mean it has been working in versions prior to 0.8.0 ? what was the drill version that you got the answer? And can you please make a patch to apply to 0.8.0 to solve the problem? > Throwing a NPE on exceuting query against parquet files > > > Key: DRILL-2765 > URL: https://issues.apache.org/jira/browse/DRILL-2765 > Project: Apache Drill > Issue Type: Sub-task > Components: Storage - Parquet >Affects Versions: 0.8.0 >Reporter: Mohsen Zainalpour >Assignee: Steven Phillips > Fix For: 1.0.0 > > Attachments: url_column_completely_null.parquet, > url_column_onetime_value.parquet > > > Executing a query on a parquet file that one of it`s columns haven`t any > value at all(but field exists in the schema) thorws a NPE -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-2765) Throwing a NPE on exceuting query against parquet files
[
https://issues.apache.org/jira/browse/DRILL-2765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14493837#comment-14493837
]
Steven Phillips commented on DRILL-2765:
This seems to already be fixed:
{code}
0: jdbc:drill:> select * from
dfs.`/tmp/nulls/url_column_completely_null.parquet`;
+++
|date|url |
+++
| 14 april 2015 | null |
| 14 april 2015 | null |
| 14 april 2015 | null |
| 14 april 2015 | null |
| 14 april 2015 | null |
| 14 april 2015 | null |
| 14 april 2015 | null |
| 14 april 2015 | null |
| 14 april 2015 | null |
| 14 april 2015 | null |
| 14 april 2015 | null |
| 14 april 2015 | null |
| 14 april 2015 | null |
| 14 april 2015 | null |
| 14 april 2015 | null |
| 14 april 2015 | null |
| 14 april 2015 | null |
| 14 april 2015 | null |
| 14 april 2015 | null |
| 14 april 2015 | null |
| 14 april 2015 | null |
| 14 april 2015 | null |
| 14 april 2015 | null |
+++
{code}
When I checkout 0.8 and run again, I hit the NPE:
{code}
0: jdbc:drill:> select * from
dfs.`/tmp/nulls/url_column_completely_null.parquet`;
Query failed: NullPointerException:
Error: exception while executing query: Failure while executing query.
(state=,code=0)
{code}
> Throwing a NPE on exceuting query against parquet files
>
>
> Key: DRILL-2765
> URL: https://issues.apache.org/jira/browse/DRILL-2765
> Project: Apache Drill
> Issue Type: Sub-task
> Components: Storage - Parquet
>Affects Versions: 0.8.0
>Reporter: Mohsen Zainalpour
>Assignee: Steven Phillips
> Fix For: 1.0.0
>
> Attachments: url_column_completely_null.parquet,
> url_column_onetime_value.parquet
>
>
> Executing a query on a parquet file that one of it`s columns haven`t any
> value at all(but field exists in the schema) thorws a NPE
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-2765) Throwing a NPE on exceuting query against parquet files
[ https://issues.apache.org/jira/browse/DRILL-2765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14493660#comment-14493660 ] Steven Phillips commented on DRILL-2765: [~masood] [~mohsenz] Could you attach the problematic files? That will make it quicker to debug and fix. > Throwing a NPE on exceuting query against parquet files > > > Key: DRILL-2765 > URL: https://issues.apache.org/jira/browse/DRILL-2765 > Project: Apache Drill > Issue Type: Sub-task > Components: Storage - Parquet >Affects Versions: 0.8.0 >Reporter: Mohsen Zainalpour >Assignee: Steven Phillips > Fix For: 1.0.0 > > > Executing a query on a parquet file that one of it`s columns haven`t any > value at all(but field exists in the schema) thorws a NPE -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-2765) Throwing a NPE on exceuting query against parquet files
[ https://issues.apache.org/jira/browse/DRILL-2765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14493642#comment-14493642 ] Masood Parvari commented on DRILL-2765: --- Also I posted the question on stackoverflow and still no answer : http://stackoverflow.com/questions/29607396/why-does-apache-drill-return-null-pointer-exception-when-querying-a-parquet-file too strange that any body else doesn't have experienced the same issue. > Throwing a NPE on exceuting query against parquet files > > > Key: DRILL-2765 > URL: https://issues.apache.org/jira/browse/DRILL-2765 > Project: Apache Drill > Issue Type: Sub-task > Components: Storage - Parquet >Affects Versions: 0.8.0 >Reporter: Mohsen Zainalpour >Assignee: Steven Phillips > Fix For: 1.0.0 > > > Executing a query on a parquet file that one of it`s columns haven`t any > value at all(but field exists in the schema) thorws a NPE -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-2765) Throwing a NPE on exceuting query against parquet files
[
https://issues.apache.org/jira/browse/DRILL-2765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14493633#comment-14493633
]
Masood Parvari commented on DRILL-2765:
---
Mohsen is right. it is a very annoying bug. I wish you guys could release a
patch as soon as possible instead of waiting till version 1.0
here I explain the problem with more details
Assume we are writing to parquet file using protobuf (or Avro). The proto file
looks like this:
message Log {
optional string date = 1;
optional string url = 2;
}
it is a reduced version of my problem. Now when writing to a parquet file
(Non-hadoop), I want to have a case in which for one of the columns there exist
no data at all (I am working on schema evolution). so I am just writing value
for 'date' column in parquet file and column 'url' is completely null through
all records. File is created successfully but when I am trying to query it by
apache drill, It throws a null pointer exception. If at least for one of
written records, I set 'url' field to a value, then it is ok and all other
'url' values can be null and drill can query it. But I need the case in which
in a parquet file, a whole column is null (but other parquet files may have
values for that column). please help me guys. my parquet version is : 1.6.0rc7
and apache drill version is 0.8.0.
> Throwing a NPE on exceuting query against parquet files
>
>
> Key: DRILL-2765
> URL: https://issues.apache.org/jira/browse/DRILL-2765
> Project: Apache Drill
> Issue Type: Sub-task
> Components: Storage - Parquet
>Affects Versions: 0.8.0
>Reporter: Mohsen Zainalpour
>Assignee: Steven Phillips
> Fix For: 1.0.0
>
>
> Executing a query on a parquet file that one of it`s columns haven`t any
> value at all(but field exists in the schema) thorws a NPE
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
