[jira] [Resolved] (IMPALA-7717) Partition id does not exist exception - Catalog V2

2018-10-19 Thread bharath v (JIRA) 


 [ 
https://issues.apache.org/jira/browse/IMPALA-7717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

bharath v resolved IMPALA-7717.
---
   Resolution: Fixed
Fix Version/s: Impala 3.1.0

> Partition id does not exist exception - Catalog V2
> --
>
> Key: IMPALA-7717
> URL: https://issues.apache.org/jira/browse/IMPALA-7717
> Project: IMPALA
>  Issue Type: Bug
>  Components: Catalog
>Reporter: bharath v
>Assignee: bharath v
>Priority: Critical
> Fix For: Impala 3.1.0
>
> Attachments: IMPALA-7717-repro.patch
>
>
> Concurrent invalidates with partial RPC on partitioned tables can throw this 
> exception.
> {noformat}
> I1016 15:49:03.438048 30197 jni-util.cc:256] 
> java.lang.IllegalArgumentException: Partition id 162 does not exist
>   at 
> com.google.common.base.Preconditions.checkArgument(Preconditions.java:119)
>   at org.apache.impala.catalog.HdfsTable.getPartialInfo(HdfsTable.java:1711)
>   at 
> org.apache.impala.catalog.CatalogServiceCatalog.doGetPartialCatalogObject(CatalogServiceCatalog.java:2202)
>   at 
> org.apache.impala.catalog.CatalogServiceCatalog.getPartialCatalogObject(CatalogServiceCatalog.java:2141)
>   at 
> org.apache.impala.service.JniCatalog.getPartialCatalogObject(JniCatalog.java:237)
> I1016 15:49:03.440939 30197 status.cc:129] IllegalArgumentException: 
> Partition id 162 does not exist
> {noformat}
> {noformat}
>  @Override
>   public TGetPartialCatalogObjectResponse getPartialInfo(
>   TGetPartialCatalogObjectRequest req) throws TableLoadingException {
> 
> if (partIds != null) {
>   resp.table_info.partitions = 
> Lists.newArrayListWithCapacity(partIds.size());
>   for (long partId : partIds) {
> HdfsPartition part = partitionMap_.get(partId);
> Preconditions.checkArgument(part != null, "Partition id %s does not 
> exist",
> partId); <
> {noformat}
> The issue is that the invalidate command can reset the partition IDs and the 
> RPCs could look up with older IDs. 
> We should wrap this into an inconsistent metadata fetch exception and retry 
> rather than throwing a RTE.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org

[jira] [Resolved] (IMPALA-7717) Partition id does not exist exception - Catalog V2

2018-10-19 Thread bharath v (JIRA) 


 [ 
https://issues.apache.org/jira/browse/IMPALA-7717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

bharath v resolved IMPALA-7717.
---
   Resolution: Fixed
Fix Version/s: Impala 3.1.0

> Partition id does not exist exception - Catalog V2
> --
>
> Key: IMPALA-7717
> URL: https://issues.apache.org/jira/browse/IMPALA-7717
> Project: IMPALA
>  Issue Type: Bug
>  Components: Catalog
>Reporter: bharath v
>Assignee: bharath v
>Priority: Critical
> Fix For: Impala 3.1.0
>
> Attachments: IMPALA-7717-repro.patch
>
>
> Concurrent invalidates with partial RPC on partitioned tables can throw this 
> exception.
> {noformat}
> I1016 15:49:03.438048 30197 jni-util.cc:256] 
> java.lang.IllegalArgumentException: Partition id 162 does not exist
>   at 
> com.google.common.base.Preconditions.checkArgument(Preconditions.java:119)
>   at org.apache.impala.catalog.HdfsTable.getPartialInfo(HdfsTable.java:1711)
>   at 
> org.apache.impala.catalog.CatalogServiceCatalog.doGetPartialCatalogObject(CatalogServiceCatalog.java:2202)
>   at 
> org.apache.impala.catalog.CatalogServiceCatalog.getPartialCatalogObject(CatalogServiceCatalog.java:2141)
>   at 
> org.apache.impala.service.JniCatalog.getPartialCatalogObject(JniCatalog.java:237)
> I1016 15:49:03.440939 30197 status.cc:129] IllegalArgumentException: 
> Partition id 162 does not exist
> {noformat}
> {noformat}
>  @Override
>   public TGetPartialCatalogObjectResponse getPartialInfo(
>   TGetPartialCatalogObjectRequest req) throws TableLoadingException {
> 
> if (partIds != null) {
>   resp.table_info.partitions = 
> Lists.newArrayListWithCapacity(partIds.size());
>   for (long partId : partIds) {
> HdfsPartition part = partitionMap_.get(partId);
> Preconditions.checkArgument(part != null, "Partition id %s does not 
> exist",
> partId); <
> {noformat}
> The issue is that the invalidate command can reset the partition IDs and the 
> RPCs could look up with older IDs. 
> We should wrap this into an inconsistent metadata fetch exception and retry 
> rather than throwing a RTE.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)