[jira] [Commented] (IMPALA-12468) Add the ability to update EventProcessorStatus

2024-03-03 Thread Maxwell Guo (Jira) 


[ 
https://issues.apache.org/jira/browse/IMPALA-12468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17823015#comment-17823015
 ] 

Maxwell Guo commented on IMPALA-12468:
--

Hi  [~mylogi...@gmail.com] , 
Yeah , what you said is another solution for this problem, but I want to make 
this feature more general, as there are other event statuses  for EP , not only 
NEEDD_INVALIDATE. What I want to do is let this state have room for manual 
adjustment, you know that any status that is not ACTIVE will stop the process 
of EP, and then the only way we can do is restart or invalidate all for 
catalogd.



> Add the ability to update EventProcessorStatus
> --
>
> Key: IMPALA-12468
> URL: https://issues.apache.org/jira/browse/IMPALA-12468
> Project: IMPALA
>  Issue Type: Improvement
>  Components: be, Catalog, fe
>Reporter: Maxwell Guo
>Assignee: Maxwell Guo
>Priority: Minor
>
> Once the impala and hive's status is missmatched , and the 
> EventProcessorStatus become NEED_INVALIDATEļ¼Œ we usually use invalidate 
> metadata to reset the catalog instance. And then impala will update the 
> status to ACTIVE . 
> But if impala contains many tables , the cost of invalidate is a bit high for 
> a global invalidate. So we may invalidate metadata for tables one by one for 
> these incremental changed table. For example , we have 1000,000,000,000 
> tables but only some of the table event process occurs CatalogException and 
> MetastoreNotificationNeedsInvalidateException was thrown. I think there is no 
> need to invalidate all table caches in order to reset the catalog instance 
> see [here 
> |https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java#L2088].
>  
> MetaStoresProcessor 's async update process will not update the currentStatus 
> when the status is not ACTIVE, see 
> [here|https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java#L876]
> So what about add a new SQL grammar : SET EVENT STATUS ${status}  ?  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org

[jira] [Commented] (IMPALA-12815) Support timestamp data type for predicates for external data source table

2024-03-03 Thread ASF subversion and git services (Jira) 


[ 
https://issues.apache.org/jira/browse/IMPALA-12815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17822922#comment-17822922
 ] 

ASF subversion and git services commented on IMPALA-12815:
--

Commit 96964be7a31a78058dea08b43c3b4115fb93dea7 in impala's branch 
refs/heads/master from gaurav1086
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=96964be7a ]

IMPALA-12815: Support timestamp for scan predicates
for external data source table.

Binary SCAN predicates involving timestamp literals are pushed down
to remote Database. The current logic assumes ISO 8601 (SQL standard)
format for timestamp literals - '-mm-dd hh:mm:ss.ms'

Testing:
- Added custom cluster tests for timestamp predicates with operators:
  '=', '>', '<', '>=', '<=', '!=', 'BETWEEN' for postgres, mysql
  and remote impala.
- Added coverage for timestamp with/without time in the timestamp
- Added coverage for timestamp with/without milliseconds in timestamp.
- Added Planner tests to check predicate pushdown for date/timestamp
  literals, date/timestamp functions and CASTs

Change-Id: If6ffe672b4027e2cee094cec4f99b9df9308e441
Reviewed-on: http://gerrit.cloudera.org:8080/21015
Tested-by: Impala Public Jenkins 
Reviewed-by: Wenzhe Zhou 


> Support timestamp data type for predicates for external data source table
> -
>
> Key: IMPALA-12815
> URL: https://issues.apache.org/jira/browse/IMPALA-12815
> Project: IMPALA
>  Issue Type: Sub-task
>  Components: Backend, Frontend
>Reporter: gaurav singh
>Assignee: gaurav singh
>Priority: Major
> Fix For: Impala 4.4.0
>
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org