[jira] [Commented] (IMPALA-7877) Support Hive GenericUDF

2019-12-16 Thread Quanlong Huang (Jira)


[ 
https://issues.apache.org/jira/browse/IMPALA-7877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16997957#comment-16997957
 ] 

Quanlong Huang commented on IMPALA-7877:


[~vihangk1] Looks like we haven't supported GenericUDF in IMPALA-8369 yet. I 
think we need to plan for it.

> Support Hive GenericUDF
> ---
>
> Key: IMPALA-7877
> URL: https://issues.apache.org/jira/browse/IMPALA-7877
> Project: IMPALA
>  Issue Type: Improvement
>Affects Versions: Impala 3.0
>Reporter: eugen yushin
>Priority: Major
>
> Running Hive UDF extending GenericUDF interface results in class cast 
> exception. Relevant [code 
> block|https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/hive/executor/UdfExecutor.java#L586]:
> {code}
> LOG.debug("Loading UDF '" + udfPath + "' from " + jarPath);
> loader = getClassLoader(jarPath);
> Class c = Class.forName(udfPath, true, loader);
> Class udfClass = c.asSubclass(UDF.class);
> {code}
> Reproduce steps:
> {code}
> create function my_lower(string) returns string location 
> '/path/to/hive-exec-1.1.0-cdh5.15.0.jar' 
> symbol='org.apache.hadoop.hive.ql.udf.generic.GenericUDFLower';
> select my_lower('Some String NOT ALREADY LOWERCASE');
> {code}
> Stack trace:
> {code}
> I1121 11:58:29.509138 29092 Frontend.java:952] Analyzing query: select 
> my_lower('Some String NOT ALREADY LOWERCASE')
> I1121 11:58:29.513121 29092 UdfExecutor.java:581] Loading UDF 
> 'org.apache.hadoop.hive.ql.udf.generic.GenericUDFLower' from 
> /var/lib/impala/udfs/hive-exec-1.1.0-cdh5.15.0.83728.2.jar
> I1121 11:58:29.515535 29092 jni-util.cc:230] java.lang.ClassCastException: 
> class org.apache.hadoop.hive.ql.udf.generic.GenericUDFLower
> at java.lang.Class.asSubclass(Class.java:3404)
> at 
> org.apache.impala.hive.executor.UdfExecutor.init(UdfExecutor.java:584)
> at 
> org.apache.impala.hive.executor.UdfExecutor.(UdfExecutor.java:217)
> at 
> org.apache.impala.service.FeSupport.NativeEvalExprsWithoutRow(Native Method)
> at 
> org.apache.impala.service.FeSupport.EvalExprsWithoutRow(FeSupport.java:208)
> at 
> org.apache.impala.service.FeSupport.EvalExprWithoutRow(FeSupport.java:163)
> at org.apache.impala.analysis.LiteralExpr.create(LiteralExpr.java:184)
> at 
> org.apache.impala.rewrite.FoldConstantsRule.apply(FoldConstantsRule.java:68)
> at 
> org.apache.impala.rewrite.ExprRewriter.applyRuleBottomUp(ExprRewriter.java:85)
> at 
> org.apache.impala.rewrite.ExprRewriter.applyRuleRepeatedly(ExprRewriter.java:71)
> at 
> org.apache.impala.rewrite.ExprRewriter.rewrite(ExprRewriter.java:55)
> at 
> org.apache.impala.analysis.SelectList.rewriteExprs(SelectList.java:97)
> at 
> org.apache.impala.analysis.SelectStmt.rewriteExprs(SelectStmt.java:894)
> at 
> org.apache.impala.analysis.AnalysisContext.analyze(AnalysisContext.java:432)
> at 
> org.apache.impala.analysis.AnalysisContext.analyzeAndAuthorize(AnalysisContext.java:393)
> at 
> org.apache.impala.service.Frontend.createExecRequest(Frontend.java:962)
> at 
> org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:156)
> I1121 11:58:29.523166 29092 status.cc:125] ClassCastException: class 
> org.apache.hadoop.hive.ql.udf.generic.GenericUDFLower
> @   0x96663a  impala::Status::Status()
> @   0xcedfdd  impala::JniUtil::GetJniExceptionMsg()
> @  0x109457f  impala::HiveUdfCall::OpenEvaluator()
> @   0x96d757  impala::ScalarExprEvaluator::Open()
> @   0xbedc2d  
> Java_org_apache_impala_service_FeSupport_NativeEvalExprsWithoutRow
> @ 0x7fc705b49e6d  (unknown)
> {code}
> Marked as bug because there're no any notes related to this behaviour in docs 
> (while it claims Impala supports Hive UDF, it should support all possible 
> Hive UDF formats if other is not specified).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Commented] (IMPALA-9089) Failed to link impalad on SUSE12

2019-12-16 Thread Donghui Xu (Jira)


[ 
https://issues.apache.org/jira/browse/IMPALA-9089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16997819#comment-16997819
 ] 

Donghui Xu commented on IMPALA-9089:


[~stigahuang] Thanks.
I have found libpthread-2.19.so in /usr/lib64 and added this path to 
$LD_LIBRARY_PATH. Errors still exist.
I found that the Red Hat operating system had similar errors, but the 
compilation was successful.
I have uploaded the cmake log file with compilation error.


> Failed to link impalad on SUSE12
> 
>
> Key: IMPALA-9089
> URL: https://issues.apache.org/jira/browse/IMPALA-9089
> Project: IMPALA
>  Issue Type: Bug
>  Components: Infrastructure
>Affects Versions: Impala 3.3.0
>Reporter: Donghui Xu
>Priority: Minor
> Attachments: CMakeError.log, CMakeOutput.log
>
>
> Failed to link impalad on SUSE12, as follows:
> [100%] Linking CXX executable ../../build/release/service/impalad
> /toolchain/binutils-2.26.1/bin/ld.gold: internal error in find_view, at 
> fileread.cc:336
> collect2: error: ld returned 1 exit status
> CMakeError.log content is as following:
> /toolchain/cmake-3.14.3/bin/cmake -E cmake_link_script 
> CMakeFiles/cmTC_31214.dir/link.txt --verbose=1
> /toolchain/gcc-4.9.2/bin/gcc  -DCHECK_FUNCTION_EXISTS=pthread_create
> -rdynamic CMakeFiles/cmTC_31214.dir/CheckFunctionExists.c.o  -o cmTC_31214 
> -lpthreads 
> /usr/bin/ld: cannot find -lpthreads
> collect2: error: ld returned 1 exit status



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Updated] (IMPALA-9089) Failed to link impalad on SUSE12

2019-12-16 Thread Donghui Xu (Jira)


 [ 
https://issues.apache.org/jira/browse/IMPALA-9089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Donghui Xu updated IMPALA-9089:
---
Attachment: CMakeOutput.log

> Failed to link impalad on SUSE12
> 
>
> Key: IMPALA-9089
> URL: https://issues.apache.org/jira/browse/IMPALA-9089
> Project: IMPALA
>  Issue Type: Bug
>  Components: Infrastructure
>Affects Versions: Impala 3.3.0
>Reporter: Donghui Xu
>Priority: Minor
> Attachments: CMakeError.log, CMakeOutput.log
>
>
> Failed to link impalad on SUSE12, as follows:
> [100%] Linking CXX executable ../../build/release/service/impalad
> /toolchain/binutils-2.26.1/bin/ld.gold: internal error in find_view, at 
> fileread.cc:336
> collect2: error: ld returned 1 exit status
> CMakeError.log content is as following:
> /toolchain/cmake-3.14.3/bin/cmake -E cmake_link_script 
> CMakeFiles/cmTC_31214.dir/link.txt --verbose=1
> /toolchain/gcc-4.9.2/bin/gcc  -DCHECK_FUNCTION_EXISTS=pthread_create
> -rdynamic CMakeFiles/cmTC_31214.dir/CheckFunctionExists.c.o  -o cmTC_31214 
> -lpthreads 
> /usr/bin/ld: cannot find -lpthreads
> collect2: error: ld returned 1 exit status



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Updated] (IMPALA-9089) Failed to link impalad on SUSE12

2019-12-16 Thread Donghui Xu (Jira)


 [ 
https://issues.apache.org/jira/browse/IMPALA-9089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Donghui Xu updated IMPALA-9089:
---
Attachment: CMakeError.log

> Failed to link impalad on SUSE12
> 
>
> Key: IMPALA-9089
> URL: https://issues.apache.org/jira/browse/IMPALA-9089
> Project: IMPALA
>  Issue Type: Bug
>  Components: Infrastructure
>Affects Versions: Impala 3.3.0
>Reporter: Donghui Xu
>Priority: Minor
> Attachments: CMakeError.log, CMakeOutput.log
>
>
> Failed to link impalad on SUSE12, as follows:
> [100%] Linking CXX executable ../../build/release/service/impalad
> /toolchain/binutils-2.26.1/bin/ld.gold: internal error in find_view, at 
> fileread.cc:336
> collect2: error: ld returned 1 exit status
> CMakeError.log content is as following:
> /toolchain/cmake-3.14.3/bin/cmake -E cmake_link_script 
> CMakeFiles/cmTC_31214.dir/link.txt --verbose=1
> /toolchain/gcc-4.9.2/bin/gcc  -DCHECK_FUNCTION_EXISTS=pthread_create
> -rdynamic CMakeFiles/cmTC_31214.dir/CheckFunctionExists.c.o  -o cmTC_31214 
> -lpthreads 
> /usr/bin/ld: cannot find -lpthreads
> collect2: error: ld returned 1 exit status



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Updated] (IMPALA-8981) Support column masking in Impala

2019-12-16 Thread Quanlong Huang (Jira)


 [ 
https://issues.apache.org/jira/browse/IMPALA-8981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Quanlong Huang updated IMPALA-8981:
---
Epic Name: Ranger Column Masking

> Support column masking in Impala
> 
>
> Key: IMPALA-8981
> URL: https://issues.apache.org/jira/browse/IMPALA-8981
> Project: IMPALA
>  Issue Type: Epic
>Affects Versions: Impala 3.4.0
>Reporter: Kurt Deschler
>Assignee: Quanlong Huang
>Priority: Critical
> Attachments: impala_column_masking.patch
>
>
> Related Hive Jira https://issues.apache.org/jira/browse/HIVE-13125
> Column Masking High-level design:
> [https://docs.google.com/document/d/1GC7au6F5Snp8zQisRopOhKSjKsI1XPPg8S2foQxfJrA/edit?usp=sharing]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Comment Edited] (IMPALA-9089) Failed to link impalad on SUSE12

2019-12-16 Thread Quanlong Huang (Jira)


[ 
https://issues.apache.org/jira/browse/IMPALA-9089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16997760#comment-16997760
 ] 

Quanlong Huang edited comment on IMPALA-9089 at 12/17/19 1:51 AM:
--

[~davidxdh] Could you check whether pthread is installed? For example, does 
something like {{/lib/x86_64-linux-gnu/libpthread-2.23.so}} exists? If so, is 
it included in {{$LD_LIBRARY_PATH}}?


was (Author: stiga-huang):
[~davidxdh] Could you check whether pthread is installed? For example, does 
something like /lib/x86_64-linux-gnu/libpthread-2.23.so exists?

> Failed to link impalad on SUSE12
> 
>
> Key: IMPALA-9089
> URL: https://issues.apache.org/jira/browse/IMPALA-9089
> Project: IMPALA
>  Issue Type: Bug
>  Components: Infrastructure
>Affects Versions: Impala 3.3.0
>Reporter: Donghui Xu
>Priority: Minor
>
> Failed to link impalad on SUSE12, as follows:
> [100%] Linking CXX executable ../../build/release/service/impalad
> /toolchain/binutils-2.26.1/bin/ld.gold: internal error in find_view, at 
> fileread.cc:336
> collect2: error: ld returned 1 exit status
> CMakeError.log content is as following:
> /toolchain/cmake-3.14.3/bin/cmake -E cmake_link_script 
> CMakeFiles/cmTC_31214.dir/link.txt --verbose=1
> /toolchain/gcc-4.9.2/bin/gcc  -DCHECK_FUNCTION_EXISTS=pthread_create
> -rdynamic CMakeFiles/cmTC_31214.dir/CheckFunctionExists.c.o  -o cmTC_31214 
> -lpthreads 
> /usr/bin/ld: cannot find -lpthreads
> collect2: error: ld returned 1 exit status



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Commented] (IMPALA-9089) Failed to link impalad on SUSE12

2019-12-16 Thread Quanlong Huang (Jira)


[ 
https://issues.apache.org/jira/browse/IMPALA-9089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16997760#comment-16997760
 ] 

Quanlong Huang commented on IMPALA-9089:


[~davidxdh] Could you check whether pthread is installed? For example, does 
something like /lib/x86_64-linux-gnu/libpthread-2.23.so exists?

> Failed to link impalad on SUSE12
> 
>
> Key: IMPALA-9089
> URL: https://issues.apache.org/jira/browse/IMPALA-9089
> Project: IMPALA
>  Issue Type: Bug
>  Components: Infrastructure
>Affects Versions: Impala 3.3.0
>Reporter: Donghui Xu
>Priority: Minor
>
> Failed to link impalad on SUSE12, as follows:
> [100%] Linking CXX executable ../../build/release/service/impalad
> /toolchain/binutils-2.26.1/bin/ld.gold: internal error in find_view, at 
> fileread.cc:336
> collect2: error: ld returned 1 exit status
> CMakeError.log content is as following:
> /toolchain/cmake-3.14.3/bin/cmake -E cmake_link_script 
> CMakeFiles/cmTC_31214.dir/link.txt --verbose=1
> /toolchain/gcc-4.9.2/bin/gcc  -DCHECK_FUNCTION_EXISTS=pthread_create
> -rdynamic CMakeFiles/cmTC_31214.dir/CheckFunctionExists.c.o  -o cmTC_31214 
> -lpthreads 
> /usr/bin/ld: cannot find -lpthreads
> collect2: error: ld returned 1 exit status



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Work started] (IMPALA-9197) Hash table lookup should be read-only

2019-12-16 Thread Tim Armstrong (Jira)


 [ 
https://issues.apache.org/jira/browse/IMPALA-9197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on IMPALA-9197 started by Tim Armstrong.
-
> Hash table lookup should be read-only
> -
>
> Key: IMPALA-9197
> URL: https://issues.apache.org/jira/browse/IMPALA-9197
> Project: IMPALA
>  Issue Type: Improvement
>  Components: Backend
>Reporter: Tim Armstrong
>Assignee: Tim Armstrong
>Priority: Major
>  Labels: multithreading
>
> For IMPALA-9156, we need concurrent lookups to the hash table to be thread 
> safe. We are pretty close to that, except a few stats are maintained in 
> HashTable and would be modified concurrently from multiple threads.
> We should modify those  places to update the stats in HashTableCtx instead.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Commented] (IMPALA-9257) Events processor last synced event does not advance if all the events are filtered out

2019-12-16 Thread Vihang Karajgaonkar (Jira)


[ 
https://issues.apache.org/jira/browse/IMPALA-9257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16997750#comment-16997750
 ] 

Vihang Karajgaonkar commented on IMPALA-9257:
-

http://gerrit.cloudera.org:8080/14916

> Events processor last synced event does not advance if all the events are 
> filtered out
> --
>
> Key: IMPALA-9257
> URL: https://issues.apache.org/jira/browse/IMPALA-9257
> Project: IMPALA
>  Issue Type: Bug
>Affects Versions: Impala 3.4.0
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Minor
>
> If the metastore events processor filters out all the events from a batch, it 
> does not advance the {{lastSyncedEventId}}. This is causes unnecessary logs 
> to be printed in catalogd. This does not affect events processor since as 
> soon as a unfiltered event is received it proceeds further. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Updated] (IMPALA-9257) Events processor last synced event does not advance if all the events are filtered out

2019-12-16 Thread Vihang Karajgaonkar (Jira)


 [ 
https://issues.apache.org/jira/browse/IMPALA-9257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vihang Karajgaonkar updated IMPALA-9257:

Affects Version/s: Impala 3.4.0

> Events processor last synced event does not advance if all the events are 
> filtered out
> --
>
> Key: IMPALA-9257
> URL: https://issues.apache.org/jira/browse/IMPALA-9257
> Project: IMPALA
>  Issue Type: Bug
>Affects Versions: Impala 3.4.0
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Minor
>
> If the metastore events processor filters out all the events from a batch, it 
> does not advance the {{lastSyncedEventId}}. This is causes unnecessary logs 
> to be printed in catalogd. This does not affect events processor since as 
> soon as a unfiltered event is received it proceeds further. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Created] (IMPALA-9257) Events processor last synced event does not advance if all the events are filtered out

2019-12-16 Thread Vihang Karajgaonkar (Jira)
Vihang Karajgaonkar created IMPALA-9257:
---

 Summary: Events processor last synced event does not advance if 
all the events are filtered out
 Key: IMPALA-9257
 URL: https://issues.apache.org/jira/browse/IMPALA-9257
 Project: IMPALA
  Issue Type: Bug
Reporter: Vihang Karajgaonkar
Assignee: Vihang Karajgaonkar


If the metastore events processor filters out all the events from a batch, it 
does not advance the {{lastSyncedEventId}}. This is causes unnecessary logs to 
be printed in catalogd. This does not affect events processor since as soon as 
a unfiltered event is received it proceeds further. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Created] (IMPALA-9256) Refactor constraint information into a separate class.

2019-12-16 Thread Anurag Mantripragada (Jira)
Anurag Mantripragada created IMPALA-9256:


 Summary: Refactor constraint information into a separate class.
 Key: IMPALA-9256
 URL: https://issues.apache.org/jira/browse/IMPALA-9256
 Project: IMPALA
  Issue Type: Sub-task
  Components: Frontend
Reporter: Anurag Mantripragada
Assignee: Anurag Mantripragada


We recently added support for primary keys and foreign keys information for 
tables. However, it is cleaner to have an SQLConstraint class as a container 
for these constraints just like hive does here: 
[https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/ddl/table/constraint/Constraints.java.|https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/ddl/table/constraint/Constraints.java]

This can be extended to support other kinds of constraints in the future.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Created] (IMPALA-9255) Take plan dependencies into account for resource consumption

2019-12-16 Thread Tim Armstrong (Jira)
Tim Armstrong created IMPALA-9255:
-

 Summary: Take plan dependencies into account for resource 
consumption
 Key: IMPALA-9255
 URL: https://issues.apache.org/jira/browse/IMPALA-9255
 Project: IMPALA
  Issue Type: Improvement
  Components: Frontend
Reporter: Tim Armstrong


In Planner.computeResourceReqs(), we assume that all fragment's peak resource 
consumption occurs at the same time. In actually, with the separate join build, 
there are dependencies between fragments and we could take those into account 
to reduce the estimated resource requirements.





--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Created] (IMPALA-9254) Queries should only be retried if all fragments fail with retryable errors

2019-12-16 Thread Sahil Takiar (Jira)
Sahil Takiar created IMPALA-9254:


 Summary: Queries should only be retried if all fragments fail with 
retryable errors
 Key: IMPALA-9254
 URL: https://issues.apache.org/jira/browse/IMPALA-9254
 Project: IMPALA
  Issue Type: Sub-task
Reporter: Sahil Takiar


Currently, Impala only propagates an {{overall_status}} from an executor to the 
coordinator. The {{overall_status}} is set in the {{QueryState}} and "If 
multiple fragments have errors, the first fragment to hit an error is given 
preference.".

The issue is that if multiple fragments fail, it is possible some of the errors 
should trigger a retry, while other errors shouldn't. For example, one fragment 
could fail due to faulty disks, but others could fail due to mem limit 
exceptions. These types of queries shouldn't be retried because it is likely 
the query will just fail again.

This can only happen if the non-retryable error occurs in a specific time 
window: [when the retryable error occurs, the query is cancelled]. Since any 
fragment failure causes the entire query to be cancelled, this can only occur 
if the non-retryable error occurs after the retryable error, but before the 
query is cancelled.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Created] (IMPALA-9253) Blacklist additional posix error codes for failed DataStreamService RPCs

2019-12-16 Thread Sahil Takiar (Jira)
Sahil Takiar created IMPALA-9253:


 Summary: Blacklist additional posix error codes for failed 
DataStreamService RPCs
 Key: IMPALA-9253
 URL: https://issues.apache.org/jira/browse/IMPALA-9253
 Project: IMPALA
  Issue Type: Improvement
Reporter: Sahil Takiar


Filing as a follow up to 
[IMPALA-9137|http://issues.cloudera.org/browse/IMPALA-9137], 
[IMPALA-9137|http://issues.cloudera.org/browse/IMPALA-9137] blacklists a node 
if a RPC fails with specific posix error codes:
 * 107 = ENOTCONN: Transport endpoint is not connected
 * 108 = ESHUTDOWN: Cannot send after transport endpoint shutdown
 * 111 = ECONNREFUSED: Connection refused

These codes were produced by running a query, killing a node running that 
query, and then seeing what error codes the query failed with.

There may be other error codes that are worth using for node blacklisting as 
well. One way to come up with more error codes is to use iptables to introduce 
network faults between Impala processes and see how RPCs fail.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Resolved] (IMPALA-9104) Add support for retrieval of primary keys and foreign keys in impala-hs2-server.

2019-12-16 Thread Anurag Mantripragada (Jira)


 [ 
https://issues.apache.org/jira/browse/IMPALA-9104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anurag Mantripragada resolved IMPALA-9104.
--
Fix Version/s: Impala 3.4.0
   Resolution: Fixed

> Add support for retrieval of primary keys and foreign keys in 
> impala-hs2-server.
> 
>
> Key: IMPALA-9104
> URL: https://issues.apache.org/jira/browse/IMPALA-9104
> Project: IMPALA
>  Issue Type: Sub-task
>Reporter: Anurag Mantripragada
>Assignee: Anurag Mantripragada
>Priority: Critical
> Fix For: Impala 3.4.0
>
>
> Impala's TCLIService.thrift files was taken from Hive-1 days and does not 
> contain APIs related to primary keys and foreign keys that was added to Hive 
> jdbc driver start Hive-2.1. We need to port these APIs inorder to be able to 
> use them in Impala JDBC clients.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Assigned] (IMPALA-9249) HdfsOrcScanner crash on resolving file schema

2019-12-16 Thread Csaba Ringhofer (Jira)


 [ 
https://issues.apache.org/jira/browse/IMPALA-9249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Csaba Ringhofer reassigned IMPALA-9249:
---

Assignee: Csaba Ringhofer

> HdfsOrcScanner crash on resolving file schema
> -
>
> Key: IMPALA-9249
> URL: https://issues.apache.org/jira/browse/IMPALA-9249
> Project: IMPALA
>  Issue Type: Bug
>  Components: Backend
>Reporter: Quanlong Huang
>Assignee: Csaba Ringhofer
>Priority: Blocker
>  Labels: impala-orc-experimental
>
>  Encounter a crash when running test_scanners_fuzz test on orc format:
> {code:java}
> F1214 12:35:18.365988  4945 orc-metadata-utils.cc:29] 
> c749de778e550dcb:50f6705f0001] Check failed: root.getKind() == 
> orc::TypeKind::STRUCT (0 vs. 12)
> *** Check failure stack trace: ***
> @  0x4d864fc  google::LogMessage::Fail()
> @  0x4d87da1  google::LogMessage::SendToLog()
> @  0x4d85ed6  google::LogMessage::Flush()
> @  0x4d8949d  google::LogMessageFatal::~LogMessageFatal()
> @  0x2622c73  impala::OrcMetadataUtils::BuildSchemaPaths()
> @  0x25e49ff  impala::HdfsOrcScanner::Open()
> @  0x2570e06  
> impala::HdfsScanNodeBase::CreateAndOpenScannerHelper()
> @  0x256418f  impala::HdfsScanNode::ProcessSplit()
> @  0x25636f4  impala::HdfsScanNode::ScannerThread()
> @  0x2562a7d  
> _ZZN6impala12HdfsScanNode22ThreadTokenAvailableCbEPNS_18ThreadResourcePoolEENKUlvE_clEv
> @  0x2565045  
> _ZN5boost6detail8function26void_function_obj_invoker0IZN6impala12HdfsScanNode22ThreadTokenAvailableCbEPNS3_18ThreadResourcePoolEEUlvE_vE6invokeERNS1_15function_bufferE
> @  0x1f1e469  boost::function0<>::operator()()
> @  0x24aaec8  impala::Thread::SuperviseThread()
> @  0x24b324c  boost::_bi::list5<>::operator()<>()
> @  0x24b3170  boost::_bi::bind_t<>::operator()()
> @  0x24b3133  boost::detail::thread_data<>::run()
> @  0x3caf6e9  thread_proxy
> @ 0x7f4b5be7a6b9  start_thread
> @ 0x7f4b5863641c  clone{code}
> Code snipper:
> {code:c++}
> void OrcMetadataUtils::BuildSchemaPaths(const orc::Type& root, int 
> num_partition_keys,
> vector* paths) {
>   SchemaPath path;
>   paths->push_back(path);
>   DCHECK_EQ(root.getKind(), orc::TypeKind::STRUCT); <-- It could be wrong for 
> corrupt files
>   int num_columns = root.getSubtypeCount();
>   for (int i = 0; i < num_columns; ++i) {
> path.push_back(i + num_partition_keys);
> BuildSchemaPath(*root.getSubtype(i), , paths);
> path.pop_back();
>   }
> }
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Commented] (IMPALA-9243) Coordinator Web UI should list which executors have been blacklisted

2019-12-16 Thread Lars Volker (Jira)


[ 
https://issues.apache.org/jira/browse/IMPALA-9243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16997452#comment-16997452
 ] 

Lars Volker commented on IMPALA-9243:
-

I think it would be good to have a counter for the number of times that a node 
was blacklisted (or total time) in addition to a flag. That way it's possible 
to spot bad nodes and check for thresholds.

> Coordinator Web UI should list which executors have been blacklisted
> 
>
> Key: IMPALA-9243
> URL: https://issues.apache.org/jira/browse/IMPALA-9243
> Project: IMPALA
>  Issue Type: Improvement
>Reporter: Sahil Takiar
>Priority: Major
>
> Currently, information about which nodes are blacklisted only shows up in 
> runtime profiles and Coordinator logs. It would be nice to display 
> blacklisting information in the Web UI as well so that a user can view which 
> nodes are blacklisted at any given time.
> One potential place to put the blacklisting information is in the /backends 
> page, which already lists out all the backends part of the cluster. A new 
> column called "Status" which can have values of either "Active" or 
> "Blacklisted" would be nice (perhaps we should re-factor the "Quiescing" 
> column to use the new "Status" column as well). This is similar to what the 
> Spark Web UI does for blacklisted nodes: 
> [https://ndu0e1pobsf1dobtvj5nls3q-wpengine.netdna-ssl.com/wp-content/uploads/2019/08/BLACKLIST-SCHEDULING.png]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Commented] (IMPALA-8577) Crash during OpenSSLSocket.read

2019-12-16 Thread Sahil Takiar (Jira)


[ 
https://issues.apache.org/jira/browse/IMPALA-8577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16997434#comment-16997434
 ] 

Sahil Takiar commented on IMPALA-8577:
--

The new WildFly release + AWS SDK release seemed to do the trick, with the 
updated versions I can no longer reproduce this bug in Impala.

> Crash during OpenSSLSocket.read
> ---
>
> Key: IMPALA-8577
> URL: https://issues.apache.org/jira/browse/IMPALA-8577
> Project: IMPALA
>  Issue Type: Bug
>  Components: Backend
>Affects Versions: Impala 3.3.0
>Reporter: David Rorke
>Assignee: Sahil Takiar
>Priority: Major
> Attachments: 5ca78771-ad78-4a29-31f88aa6-9bfac38c.dmp, 
> hs_err_pid6313.log, 
> impalad.drorke-impala-r5d2xl2-30w-17.vpc.cloudera.com.impala.log.ERROR.20190521-103105.6313,
>  
> impalad.drorke-impala-r5d2xl2-30w-17.vpc.cloudera.com.impala.log.INFO.20190521-103105.6313
>
>
> Impalad crashed while running a TPC-DS 10 TB run against S3.   Excerpt from 
> the stack trace (hs_err log file attached with more complete stack):
> {noformat}
> Stack: [0x7f3d095bc000,0x7f3d09dbc000],  sp=0x7f3d09db9050,  free 
> space=8180k
> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native 
> code)
> C  [impalad+0x2528a33]  
> tcmalloc::ThreadCache::ReleaseToCentralCache(tcmalloc::ThreadCache::FreeList*,
>  unsigned long, int)+0x133
> C  [impalad+0x2528e0f]  tcmalloc::ThreadCache::Scavenge()+0x3f
> C  [impalad+0x266468a]  operator delete(void*)+0x32a
> C  [libcrypto.so.10+0x6e70d]  CRYPTO_free+0x1d
> J 5709  org.wildfly.openssl.SSLImpl.freeBIO0(J)V (0 bytes) @ 
> 0x7f3d4dadf9f9 [0x7f3d4dadf940+0xb9]
> J 5708 C1 org.wildfly.openssl.SSLImpl.freeBIO(J)V (5 bytes) @ 
> 0x7f3d4dfd0dfc [0x7f3d4dfd0d80+0x7c]
> J 5158 C1 org.wildfly.openssl.OpenSSLEngine.shutdown()V (78 bytes) @ 
> 0x7f3d4de4fe2c [0x7f3d4de4f720+0x70c]
> J 5758 C1 org.wildfly.openssl.OpenSSLEngine.closeInbound()V (51 bytes) @ 
> 0x7f3d4de419cc [0x7f3d4de417c0+0x20c]
> J 2994 C2 
> org.wildfly.openssl.OpenSSLEngine.unwrap(Ljava/nio/ByteBuffer;[Ljava/nio/ByteBuffer;II)Ljavax/net/ssl/SSLEngineResult;
>  (892 bytes) @ 0x7f3d4db8da34 [0x7f3d4db8c900+0x1134]
> J 3161 C2 org.wildfly.openssl.OpenSSLSocket.read([BII)I (810 bytes) @ 
> 0x7f3d4dd64cb0 [0x7f3d4dd646c0+0x5f0]
> J 5090 C2 
> com.amazonaws.thirdparty.apache.http.impl.io.SessionInputBufferImpl.fillBuffer()I
>  (97 bytes) @ 0x7f3d4ddd9ee0 [0x7f3d4ddd9e40+0xa0]
> J 5846 C1 
> com.amazonaws.thirdparty.apache.http.impl.BHttpConnectionBase.fillInputBuffer(I)I
>  (48 bytes) @ 0x7f3d4d7acb24 [0x7f3d4d7ac7a0+0x384]
> J 5845 C1 
> com.amazonaws.thirdparty.apache.http.impl.BHttpConnectionBase.isStale()Z (31 
> bytes) @ 0x7f3d4d7ad49c [0x7f3d4d7ad220+0x27c]
> {noformat}
> The crash may not be easy to reproduce.  I've run this test multiple times 
> and only crashed once.   I have a core file if needed.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Updated] (IMPALA-9252) Add authorization tests that use Ranger's deny policies

2019-12-16 Thread Csaba Ringhofer (Jira)


 [ 
https://issues.apache.org/jira/browse/IMPALA-9252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Csaba Ringhofer updated IMPALA-9252:

Labels: ranger  (was: )

> Add authorization tests that use Ranger's deny policies
> ---
>
> Key: IMPALA-9252
> URL: https://issues.apache.org/jira/browse/IMPALA-9252
> Project: IMPALA
>  Issue Type: Test
>  Components: Security
>Reporter: Csaba Ringhofer
>Priority: Major
>  Labels: ranger
>
> Currently no test is run with deny policies enabled.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Updated] (IMPALA-9252) Add authorization tests that use Ranger's deny policies

2019-12-16 Thread Csaba Ringhofer (Jira)


 [ 
https://issues.apache.org/jira/browse/IMPALA-9252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Csaba Ringhofer updated IMPALA-9252:

Component/s: Security

> Add authorization tests that use Ranger's deny policies
> ---
>
> Key: IMPALA-9252
> URL: https://issues.apache.org/jira/browse/IMPALA-9252
> Project: IMPALA
>  Issue Type: Test
>  Components: Security
>Reporter: Csaba Ringhofer
>Priority: Major
>
> Currently no test is run with deny policies enabled.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Created] (IMPALA-9252) Add authorization tests that use Ranger's deny policies

2019-12-16 Thread Csaba Ringhofer (Jira)
Csaba Ringhofer created IMPALA-9252:
---

 Summary: Add authorization tests that use Ranger's deny policies
 Key: IMPALA-9252
 URL: https://issues.apache.org/jira/browse/IMPALA-9252
 Project: IMPALA
  Issue Type: Test
Reporter: Csaba Ringhofer


Currently no test is run with deny policies enabled.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Updated] (IMPALA-9249) HdfsOrcScanner crash on resolving file schema

2019-12-16 Thread Quanlong Huang (Jira)


 [ 
https://issues.apache.org/jira/browse/IMPALA-9249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Quanlong Huang updated IMPALA-9249:
---
Labels: impala-orc-experimental  (was: )

> HdfsOrcScanner crash on resolving file schema
> -
>
> Key: IMPALA-9249
> URL: https://issues.apache.org/jira/browse/IMPALA-9249
> Project: IMPALA
>  Issue Type: Bug
>  Components: Backend
>Reporter: Quanlong Huang
>Priority: Blocker
>  Labels: impala-orc-experimental
>
>  Encounter a crash when running test_scanners_fuzz test on orc format:
> {code:java}
> F1214 12:35:18.365988  4945 orc-metadata-utils.cc:29] 
> c749de778e550dcb:50f6705f0001] Check failed: root.getKind() == 
> orc::TypeKind::STRUCT (0 vs. 12)
> *** Check failure stack trace: ***
> @  0x4d864fc  google::LogMessage::Fail()
> @  0x4d87da1  google::LogMessage::SendToLog()
> @  0x4d85ed6  google::LogMessage::Flush()
> @  0x4d8949d  google::LogMessageFatal::~LogMessageFatal()
> @  0x2622c73  impala::OrcMetadataUtils::BuildSchemaPaths()
> @  0x25e49ff  impala::HdfsOrcScanner::Open()
> @  0x2570e06  
> impala::HdfsScanNodeBase::CreateAndOpenScannerHelper()
> @  0x256418f  impala::HdfsScanNode::ProcessSplit()
> @  0x25636f4  impala::HdfsScanNode::ScannerThread()
> @  0x2562a7d  
> _ZZN6impala12HdfsScanNode22ThreadTokenAvailableCbEPNS_18ThreadResourcePoolEENKUlvE_clEv
> @  0x2565045  
> _ZN5boost6detail8function26void_function_obj_invoker0IZN6impala12HdfsScanNode22ThreadTokenAvailableCbEPNS3_18ThreadResourcePoolEEUlvE_vE6invokeERNS1_15function_bufferE
> @  0x1f1e469  boost::function0<>::operator()()
> @  0x24aaec8  impala::Thread::SuperviseThread()
> @  0x24b324c  boost::_bi::list5<>::operator()<>()
> @  0x24b3170  boost::_bi::bind_t<>::operator()()
> @  0x24b3133  boost::detail::thread_data<>::run()
> @  0x3caf6e9  thread_proxy
> @ 0x7f4b5be7a6b9  start_thread
> @ 0x7f4b5863641c  clone{code}
> Code snipper:
> {code:c++}
> void OrcMetadataUtils::BuildSchemaPaths(const orc::Type& root, int 
> num_partition_keys,
> vector* paths) {
>   SchemaPath path;
>   paths->push_back(path);
>   DCHECK_EQ(root.getKind(), orc::TypeKind::STRUCT); <-- It could be wrong for 
> corrupt files
>   int num_columns = root.getSubtypeCount();
>   for (int i = 0; i < num_columns; ++i) {
> path.push_back(i + num_partition_keys);
> BuildSchemaPath(*root.getSubtype(i), , paths);
> path.pop_back();
>   }
> }
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Updated] (IMPALA-9250) HdfsOrcScanner crash in calling orc::createReader

2019-12-16 Thread Quanlong Huang (Jira)


 [ 
https://issues.apache.org/jira/browse/IMPALA-9250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Quanlong Huang updated IMPALA-9250:
---
Labels: impala-orc-experimental  (was: )

> HdfsOrcScanner crash in calling orc::createReader
> -
>
> Key: IMPALA-9250
> URL: https://issues.apache.org/jira/browse/IMPALA-9250
> Project: IMPALA
>  Issue Type: Bug
>Reporter: Quanlong Huang
>Priority: Blocker
>  Labels: impala-orc-experimental
> Attachments: alltypes_year=2010_month=1_05_0.orc
>
>
> Encounter a crash when running the test_scanners_fuzz test for orc:
> {code}
> Crash reason:  SIGSEGV
> Crash address: 0x0
> Process uptime: not available
> Thread 313 (crashed)
>  0  libstdc++.so.6.0.21 + 0xd0328
>  1  impalad!orc::TypeImpl::addStructField(std::string const&, 
> std::unique_ptr >) [new_allocator.h 
> : 120 + 0xd]
>  2  impalad!orc::convertType(orc::proto::Type const&, orc::proto::Footer 
> const&) [TypeImpl.cc : 414 + 0x10]
>  3  impalad!orc::ReaderImpl::ReaderImpl(std::shared_ptr, 
> orc::ReaderOptions const&, unsigned long, unsigned long) [Reader.cc : 413 + 
> 0x9]
>  4  impalad!orc::createReader(std::unique_ptr std::default_delete >, orc::ReaderOptions const&) 
> [Reader.cc : 1135 + 0x20]
>  5  impalad!impala::HdfsOrcScanner::ProcessFileTail() [hdfs-orc-scanner.cc : 
> 250 + 0x4e]
>  6  impalad!impala::HdfsOrcScanner::Open(impala::ScannerContext*) 
> [hdfs-orc-scanner.cc : 168 + 0x19]
>  7  
> impalad!impala::HdfsScanNodeBase::CreateAndOpenScannerHelper(impala::HdfsPartitionDescriptor*,
>  impala::ScannerContext*, boost::scoped_ptr*) 
> [hdfs-scan-node-base.cc : 730 + 0x29]
>  8  
> impalad!impala::HdfsScanNode::ProcessSplit(std::vector std::allocator > const&, impala::MemPool*, 
> impala::io::ScanRange*, long*) [hdfs-scan-node.cc : 494 + 0x2b]
>  9  impalad!impala::HdfsScanNode::ScannerThread(bool, long) 
> [hdfs-scan-node.cc : 416 + 0x2a]
> 10  
> impalad!impala::HdfsScanNode::ThreadTokenAvailableCb(impala::ThreadResourcePool*)::{lambda()#1}::operator()()
>  const + 0x30
> 11  
> impalad!boost::detail::function::void_function_obj_invoker0,
>  void>::invoke [function_template.hpp : 153 + 0xc]
> 12  impalad!boost::function0::operator()() const [function_template.hpp 
> : 767 + 0x11]
> 13  impalad!impala::Thread::SuperviseThread(std::string const&, std::string 
> const&, boost::function, impala::ThreadDebugInfo const*, 
> impala::Promise*) [thread.cc : 360 + 0xf]
> 14  impalad!void boost::_bi::list5, 
> boost::_bi::value, boost::_bi::value >, 
> boost::_bi::value, 
> boost::_bi::value*> 
> >::operator() boost::function, impala::ThreadDebugInfo const*, 
> impala::Promise*), 
> boost::_bi::list0>(boost::_bi::type, void (*&)(std::string const&, 
> std::string const&, boost::function, impala::ThreadDebugInfo const*, 
> impala::Promise*), boost::_bi::list0&, int) 
> [bind.hpp : 525 + 0x15]
> 15  impalad!boost::_bi::bind_t const&, boost::function, impala::ThreadDebugInfo const*, 
> impala::Promise*), 
> boost::_bi::list5, 
> boost::_bi::value, boost::_bi::value >, 
> boost::_bi::value, 
> boost::_bi::value*> > 
> >::operator()() [bind_template.hpp : 20 + 0x22]
> 16  impalad!boost::detail::thread_data (*)(std::string const&, std::string const&, boost::function, 
> impala::ThreadDebugInfo const*, impala::Promise (impala::PromiseMode)0>*), boost::_bi::list5, 
> boost::_bi::value, boost::_bi::value >, 
> boost::_bi::value, 
> boost::_bi::value*> > > 
> >::run() [thread.hpp : 116 + 0x12]
> 17  impalad!thread_proxy + 0xda
> 18  libpthread-2.23.so + 0x76ba
> 19  libc-2.23.so + 0x10741d
> {code}
> Another similar crash:
> {code}
> Thread 302 (crashed)
>  0  libstdc++.so.6.0.21 + 0xd0328
>  1  impalad!orc::TypeImpl::addStructField(std::string const&, 
> std::unique_ptr >) [new_allocator.h 
> : 120 + 0xd]
>  2  impalad!orc::convertType(orc::proto::Type const&, orc::proto::Footer 
> const&) [TypeImpl.cc : 414 + 0x10]
>  3  impalad!orc::ReaderImpl::ReaderImpl(std::shared_ptr, 
> orc::ReaderOptions const&, unsigned long, unsigned long) [Reader.cc : 413 + 
> 0x9]
>  4  impalad!orc::createReader(std::unique_ptr std::default_delete >, orc::ReaderOptions const&) 
> [Reader.cc : 1135 + 0x20]
>  5  impalad!impala::HdfsOrcScanner::ProcessFileTail() [hdfs-orc-scanner.cc : 
> 250 + 0x4e]
>  6  impalad!impala::HdfsOrcScanner::Open(impala::ScannerContext*) 
> [hdfs-orc-scanner.cc : 168 + 0x19]
>  7  
> impalad!impala::HdfsScanNodeBase::CreateAndOpenScannerHelper(impala::HdfsPartitionDescriptor*,
>  impala::ScannerContext*, boost::scoped_ptr*) 
> [hdfs-scan-node-base.cc : 730 + 0x29]
>  8  
> impalad!impala::HdfsScanNode::ProcessSplit(std::vector std::allocator > const&, impala::MemPool*, 
> impala::io::ScanRange*, long*) [hdfs-scan-node.cc : 494 + 0x2b]
>  9  impalad!impala::HdfsScanNode::ScannerThread(bool, long) 
> 

[jira] [Updated] (IMPALA-9251) HdfsOrcScanner crash in AssembleRows

2019-12-16 Thread Quanlong Huang (Jira)


 [ 
https://issues.apache.org/jira/browse/IMPALA-9251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Quanlong Huang updated IMPALA-9251:
---
Labels: impala-orc-experimental  (was: )

> HdfsOrcScanner crash in AssembleRows
> 
>
> Key: IMPALA-9251
> URL: https://issues.apache.org/jira/browse/IMPALA-9251
> Project: IMPALA
>  Issue Type: Bug
>  Components: Backend
>Reporter: Quanlong Huang
>Priority: Blocker
>  Labels: impala-orc-experimental
>
> Encounter a crash when running test_scanner_fuzz test on ORC. 
> {code}
> Crash reason:  SIGSEGV
> Crash address: 0x8
> Process uptime: not available
> Thread 319 (crashed)
>  0  impalad!orc::StripeStreamsImpl::getEncoding(unsigned long) const 
> [StripeStream.cc : 67 + 0x0]
>  1  impalad!orc::StructColumnReader::StructColumnReader(orc::Type const&, 
> orc::StripeStreams&) [ColumnReader.cc : 869 + 0x12]
>  2  impalad!orc::buildReader(orc::Type const&, orc::StripeStreams&) 
> [ColumnReader.cc : 1802 + 0x1b]
>  3  impalad!orc::RowReaderImpl::startNextStripe() [Reader.cc : 910 + 0x12]
>  4  impalad!orc::RowReaderImpl::next(orc::ColumnVectorBatch&) [Reader.cc : 
> 925 + 0x5]
>  5  impalad!impala::HdfsOrcScanner::AssembleRows(impala::RowBatch*) 
> [hdfs-orc-scanner.cc : 611 + 0x40]
>  6  impalad!impala::HdfsOrcScanner::GetNextInternal(impala::RowBatch*) 
> [hdfs-orc-scanner.cc : 509 + 0x20]
>  7  impalad!impala::HdfsOrcScanner::ProcessSplit() [hdfs-orc-scanner.cc : 428 
> + 0x39]
>  8  
> impalad!impala::HdfsScanNode::ProcessSplit(std::vector std::allocator > const&, impala::MemPool*, 
> impala::io::ScanRange*, long*) [hdfs-scan-node.cc : 514 + 0x28]
>  9  impalad!impala::HdfsScanNode::ScannerThread(bool, long) 
> [hdfs-scan-node.cc : 416 + 0x2a]
> 10  
> impalad!impala::HdfsScanNode::ThreadTokenAvailableCb(impala::ThreadResourcePool*)::{lambda()#1}::operator()()
>  const + 0x30
> 11  
> impalad!boost::detail::function::void_function_obj_invoker0,
>  void>::invoke [function_template.hpp : 153 + 0xc]
> 12  impalad!boost::function0::operator()() const [function_template.hpp 
> : 767 + 0x11]
> 13  impalad!impala::Thread::SuperviseThread(std::string const&, std::string 
> const&, boost::function, impala::ThreadDebugInfo const*, 
> impala::Promise*) [thread.cc : 360 + 0xf]
> 14  impalad!void boost::_bi::list5, 
> boost::_bi::value, boost::_bi::value >, 
> boost::_bi::value, 
> boost::_bi::value*> 
> >::operator() boost::function, impala::ThreadDebugInfo const*, 
> impala::Promise*), 
> boost::_bi::list0>(boost::_bi::type, void (*&)(std::string const&, 
> std::string const&, boost::function, impala::ThreadDebugInfo const*, 
> impala::Promise*), boost::_bi::list0&, int) 
> [bind.hpp : 525 + 0x15]
> 15  impalad!boost::_bi::bind_t const&, boost::function, impala::ThreadDebugInfo const*, 
> impala::Promise*), 
> boost::_bi::list5, 
> boost::_bi::value, boost::_bi::value >, 
> boost::_bi::value, 
> boost::_bi::value*> > 
> >::operator()() [bind_template.hpp : 20 + 0x22]
> 16  impalad!boost::detail::thread_data (*)(std::string const&, std::string const&, boost::function, 
> impala::ThreadDebugInfo const*, impala::Promise (impala::PromiseMode)0>*), boost::_bi::list5, 
> boost::_bi::value, boost::_bi::value >, 
> boost::_bi::value, 
> boost::_bi::value*> > > 
> >::run() [thread.hpp : 116 + 0x12]
> 17  impalad!thread_proxy + 0xda
> 18  libpthread-2.23.so + 0x76ba
> 19  libc-2.23.so + 0x10741d
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org



[jira] [Resolved] (IMPALA-8706) SQL:2016 datetime patterns - Milestone 4

2019-12-16 Thread Attila Jeges (Jira)


 [ 
https://issues.apache.org/jira/browse/IMPALA-8706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Jeges resolved IMPALA-8706.
--
Fix Version/s: Impala 3.4.0
   Resolution: Fixed

> SQL:2016 datetime patterns - Milestone 4
> 
>
> Key: IMPALA-8706
> URL: https://issues.apache.org/jira/browse/IMPALA-8706
> Project: IMPALA
>  Issue Type: Sub-task
>  Components: Backend
>Affects Versions: Impala 2.2.4
>Reporter: Gabor Kaszab
>Assignee: Attila Jeges
>Priority: Major
>  Labels: ramp-up
> Fix For: Impala 3.4.0
>
>
> Design doc for SQL:2016 datetime patterns:
>  
> [https://docs.google.com/document/d/1V7k6-lrPGW7_uhqM-FhKl3QsxwCRy69v2KIxPsGjc1k/]
> Milestone 4 content:
>  - ISO8601 week numbering tokens:
>  ** IYYY
>  ** IYY
>  ** IY
>  ** I
>  ** IW



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org