[jira] [Assigned] (DRILL-3827) Empty metadata file causes queries on the table to fail
[ https://issues.apache.org/jira/browse/DRILL-3827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vitalii Diravka reassigned DRILL-3827: -- Assignee: Vitalii Diravka (was: Parth Chandra) > Empty metadata file causes queries on the table to fail > --- > > Key: DRILL-3827 > URL: https://issues.apache.org/jira/browse/DRILL-3827 > Project: Apache Drill > Issue Type: Bug > Components: Query Planning & Optimization >Affects Versions: 1.2.0 >Reporter: Victoria Markman >Assignee: Vitalii Diravka >Priority: Critical > Fix For: Future > > > I ran into a situation where drill created an empty metadata file (which is a > separate issue and I will try to narrow it down. Suspicion is that this > happens when "refresh table metada x" fails with "permission denied" error). > However, we need to guard against situation where metadata file is empty or > corrupted. We probably should skip reading it if we encounter unexpected > result and continue with query planning without that information. In the same > fashion as partition pruning failure. It's also important to log this > information somewhere, drillbit.log as a start. It would be really nice to > have a flag in the query profile that tells a user if we used metadata file > for planning or not. Will help in debugging performance issues. > Very confusing exception is thrown if you have zero length meta data file in > the directory: > {code} > [Wed Sep 23 07:45:28] # ls -la > total 2 > drwxr-xr-x 2 root root 2 Sep 10 14:55 . > drwxr-xr-x 16 root root 35 Sep 15 12:54 .. > -rwxr-xr-x 1 root root 483 Jul 1 11:29 0_0_0.parquet > -rwxr-xr-x 1 root root 0 Sep 10 14:55 .drill.parquet_metadata > 0: jdbc:drill:schema=dfs> select * from t1; > Error: SYSTEM ERROR: JsonMappingException: No content to map due to > end-of-input > at [Source: com.mapr.fs.MapRFsDataInputStream@342bd88d; line: 1, column: 1] > [Error Id: c97574f6-b3e8-4183-8557-c30df6ca675f on atsqa4-133.qa.lab:31010] > (state=,code=0) > {code} > Workaround is trivial, remove the file. Marking it as critical, since we > don't have any concurrency control in place and this file can get corrupted > as well. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (DRILL-3827) Empty metadata file causes queries on the table to fail
[ https://issues.apache.org/jira/browse/DRILL-3827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra reassigned DRILL-3827: Assignee: Parth Chandra (was: Jinfeng Ni) > Empty metadata file causes queries on the table to fail > --- > > Key: DRILL-3827 > URL: https://issues.apache.org/jira/browse/DRILL-3827 > Project: Apache Drill > Issue Type: Bug > Components: Query Planning & Optimization >Affects Versions: 1.2.0 >Reporter: Victoria Markman >Assignee: Parth Chandra >Priority: Critical > > I ran into a situation where drill created an empty metadata file (which is a > separate issue and I will try to narrow it down. Suspicion is that this > happens when "refresh table metada x" fails with "permission denied" error). > However, we need to guard against situation where metadata file is empty or > corrupted. We probably should skip reading it if we encounter unexpected > result and continue with query planning without that information. In the same > fashion as partition pruning failure. It's also important to log this > information somewhere, drillbit.log as a start. It would be really nice to > have a flag in the query profile that tells a user if we used metadata file > for planning or not. Will help in debugging performance issues. > Very confusing exception is thrown if you have zero length meta data file in > the directory: > {code} > [Wed Sep 23 07:45:28] # ls -la > total 2 > drwxr-xr-x 2 root root 2 Sep 10 14:55 . > drwxr-xr-x 16 root root 35 Sep 15 12:54 .. > -rwxr-xr-x 1 root root 483 Jul 1 11:29 0_0_0.parquet > -rwxr-xr-x 1 root root 0 Sep 10 14:55 .drill.parquet_metadata > 0: jdbc:drill:schema=dfs> select * from t1; > Error: SYSTEM ERROR: JsonMappingException: No content to map due to > end-of-input > at [Source: com.mapr.fs.MapRFsDataInputStream@342bd88d; line: 1, column: 1] > [Error Id: c97574f6-b3e8-4183-8557-c30df6ca675f on atsqa4-133.qa.lab:31010] > (state=,code=0) > {code} > Workaround is trivial, remove the file. Marking it as critical, since we > don't have any concurrency control in place and this file can get corrupted > as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)