[
https://issues.apache.org/jira/browse/ATLAS-5424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
SONALI SAKSENA updated ATLAS-5424:
----------------------------------
Description:
{*}Issue{*}{*}{{{}*{}}} : Basic search has approximate count in the response.
The count is approximate because, that count is what we get from the index
query. we don't update the count, if we have any in-memory filtering above it.
*Approach* : We can atleast start indicating user, when incase there is no
in-memory filtering and count is exact.
{{curl
'https://console.cdp.mow-int.cloudera.com/dss/api/atlas/crn:cdp:datalake:us-west-1:c8dbde4b-ccce-4f8d-a581-830970ba4908:datalake:6701b390-4243-4dd9-a4e7-d709e9bbabb7/basic-search'
\ -H 'Accept: application/json, text/plain, */*' \ -H 'Accept-Language:
en-GB,en;q=0.9' \ -H 'Cache-Control: no-cache, no-store, max-age=0,
must-revalidate' \ -H 'Connection: keep-alive' \ -H 'Content-Type:
application/json' \ -b 'AMCVS_97C7898555F6819F7F000101%40AdobeOrg=1;
s_ecid=MCMID%7C33738302293673960010520587691847326584; s_cc=true;
_TRAEFIK_BACKEND=588bf34c7a19d105;
AMCV_97C7898555F6819F7F000101%40AdobeOrg=359503849%7CMCIDTS%7C20166%7CMCMID%7C33738302293673960010520587691847326584%7CMCAAMLH-1742971357%7C7%7CMCAAMB-1742971357%7C6G1ynYcLPuiQxYZrsz_pkqfLG9yMXBpb2zX5dvJdYQJzPXImdj0y%7CMCOPTOUT-1742373757s%7CNONE%7CMCAID%7CNONE%7CMCSYNCSOP%7C411-20174%7CvVersion%7C5.0.1;
RT="z=1&dm=cloudera.com&si=ebbd50da-e922-424d-8729-ce0130ab402f&ss=m8fk24m7&sl=1&tt=11g&bcn=%2F%2F17de4c19.akstat.io%2F&ul=n28g&hd=n28m";
cdp-session-token=eyJraWQiOm51bGwsImFsZyI6IkVTMjU2In0.eyJpc3MiOiJBbHR1cyBJQU0iLCJhdWQiOiJBbHR1cyBJQU0iLCJqdGkiOiJKV2wwZGtiT3FaN0dnNUZTclI3WU9BIiwiaWF0IjoxNzQyNDY5ODgyLCJleHAiOjE3NDI1MTMwODIsInN1YiI6ImNybjphbHR1czppYW06dXMtd2VzdC0xOmM4ZGJkZTRiLWNjY2UtNGY4ZC1hNTgxLTgzMDk3MGJhNDkwODp1c2VyOmIyMjhiMWRjLTIwZWQtNDdlMy1hODMyLTcwZWQ4YzY0NzlhYyJ9.E5OTgFh_e1tF56l-fE6_yJr-zprQKq3dj36uSxrDqy1gfUjQwetuKOz2kJe838QlDjF5hOBPorv9YQMcUo8MzA;
cdp-csrf-token=51b24d4f-f2e0-45d6-9736-a957e66c167b' \ -H 'Origin:
https://console.cdp.mow-int.cloudera.com' \ -H 'Sec-Fetch-Dest: empty' \ -H
'Sec-Fetch-Mode: cors' \ -H 'Sec-Fetch-Site: same-origin' \ -H 'User-Agent:
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like
Gecko) Chrome/133.0.0.0 Safari/537.36' \ -H 'X-Requested-With: XMLHttpRequest'
\ -H 'sec-ch-ua: "Not(A:Brand";v="99", "Google Chrome";v="133",
"Chromium";v="133"' \ -H 'sec-ch-ua-mobile: ?0' \ -H 'sec-ch-ua-platform:
"macOS"' \ --data-raw
'\{"excludeDeletedEntities":true,"includeSubClassifications":false,"includeClassificationAttributes":true,"includeSubTypes":false,"tagFilters":null,"attributes":["type","position","comment"],"typeName":"hive_column","entityFilters":{"condition":"AND","criterion":[{"attributeName":"qualifiedName","operator":"startsWith","attributeValue":"test123.us_customers_csv113."},\{"attributeName":"qualifiedName","operator":"endsWith","attributeValue":"@cm"},\{"condition":"OR","criterion":[{"attributeName":"__isIncomplete","operator":"isNull","attributeValue":""},\{"attributeName":"__isIncomplete","operator":"eq","attributeValue":"0"}]}]},"classification":null,"limit":20,"offset":0,"sortBy":"name","sortOrder":"ASCENDING"}'}}
{{"isIncomplete": false, "labels": [] } ],
"approximateCount": 72}}}
approximateCount is getting incremented on creating new tables
was:
*Description:*
If an invalid or unknown {{typeName}} is encountered during notification
processing, we should validate it upfront and throw an exception before
processing the notification. This would prevent the notification from being
processed, failing later, and being unnecessarily retried.
*Logs:*
{{2026-09-24 06:27:31,201 [NotificationHookConsumer thread-0] INFO
[NotificationHookConsumer.java:585] Message type:
ENTITY_PARTIAL_UPDATE_V22026-09-24 06:27:31,203 [NotificationHookConsumer
thread-0] INFO [GraphTransactionAdvisor.java:43] GraphTransaction intercept for
org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2.updateEntity2026-09-24
06:27:31,203 [NotificationHookConsumer thread-0] ERROR
[GraphTransactionInterceptor.java:280] graph rollback due to
exceptionorg.apache.atlas.exception.AtlasBaseException: trino_table:
Unknown/invalid typename at
org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2.updateEntity(AtlasEntityStoreV2.java:366)
at
org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2$$FastClassBySpringCGLIB$$6861dca9.invoke(<generated>)
at
org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:792)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:762)
at
org.apache.atlas.GraphTransactionInterceptor.invoke(GraphTransactionInterceptor.java:190)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:762)
at
org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:707)
at
org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2$$EnhancerBySpringCGLIB$$a46f3b8f.updateEntity(<generated>)
at
org.apache.atlas.notification.SerialEntityProcessor.handleMessage(SerialEntityProcessor.java:584)
at
org.apache.atlas.notification.SerialEntityProcessor.handleMessage(SerialEntityProcessor.java:324)
at
org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.handleMessage(NotificationHookConsumer.java:587)
at
org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.run(NotificationHookConsumer.java:501)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:840) }}
> Basic Search : Add count feature
> --------------------------------
>
> Key: ATLAS-5424
> URL: https://issues.apache.org/jira/browse/ATLAS-5424
> Project: Atlas
> Issue Type: Bug
> Reporter: SONALI SAKSENA
> Assignee: SONALI SAKSENA
> Priority: Major
> Fix For: 3.0.0, 2.6.0
>
>
> {*}Issue{*}{*}{{{}*{}}} : Basic search has approximate count in the response.
> The count is approximate because, that count is what we get from the index
> query. we don't update the count, if we have any in-memory filtering above
> it.
> *Approach* : We can atleast start indicating user, when incase there is no
> in-memory filtering and count is exact.
>
>
> {{curl
> 'https://console.cdp.mow-int.cloudera.com/dss/api/atlas/crn:cdp:datalake:us-west-1:c8dbde4b-ccce-4f8d-a581-830970ba4908:datalake:6701b390-4243-4dd9-a4e7-d709e9bbabb7/basic-search'
> \ -H 'Accept: application/json, text/plain, */*' \ -H 'Accept-Language:
> en-GB,en;q=0.9' \ -H 'Cache-Control: no-cache, no-store, max-age=0,
> must-revalidate' \ -H 'Connection: keep-alive' \ -H 'Content-Type:
> application/json' \ -b 'AMCVS_97C7898555F6819F7F000101%40AdobeOrg=1;
> s_ecid=MCMID%7C33738302293673960010520587691847326584; s_cc=true;
> _TRAEFIK_BACKEND=588bf34c7a19d105;
> AMCV_97C7898555F6819F7F000101%40AdobeOrg=359503849%7CMCIDTS%7C20166%7CMCMID%7C33738302293673960010520587691847326584%7CMCAAMLH-1742971357%7C7%7CMCAAMB-1742971357%7C6G1ynYcLPuiQxYZrsz_pkqfLG9yMXBpb2zX5dvJdYQJzPXImdj0y%7CMCOPTOUT-1742373757s%7CNONE%7CMCAID%7CNONE%7CMCSYNCSOP%7C411-20174%7CvVersion%7C5.0.1;
>
> RT="z=1&dm=cloudera.com&si=ebbd50da-e922-424d-8729-ce0130ab402f&ss=m8fk24m7&sl=1&tt=11g&bcn=%2F%2F17de4c19.akstat.io%2F&ul=n28g&hd=n28m";
>
> cdp-session-token=eyJraWQiOm51bGwsImFsZyI6IkVTMjU2In0.eyJpc3MiOiJBbHR1cyBJQU0iLCJhdWQiOiJBbHR1cyBJQU0iLCJqdGkiOiJKV2wwZGtiT3FaN0dnNUZTclI3WU9BIiwiaWF0IjoxNzQyNDY5ODgyLCJleHAiOjE3NDI1MTMwODIsInN1YiI6ImNybjphbHR1czppYW06dXMtd2VzdC0xOmM4ZGJkZTRiLWNjY2UtNGY4ZC1hNTgxLTgzMDk3MGJhNDkwODp1c2VyOmIyMjhiMWRjLTIwZWQtNDdlMy1hODMyLTcwZWQ4YzY0NzlhYyJ9.E5OTgFh_e1tF56l-fE6_yJr-zprQKq3dj36uSxrDqy1gfUjQwetuKOz2kJe838QlDjF5hOBPorv9YQMcUo8MzA;
> cdp-csrf-token=51b24d4f-f2e0-45d6-9736-a957e66c167b' \ -H 'Origin:
> https://console.cdp.mow-int.cloudera.com' \ -H 'Sec-Fetch-Dest: empty' \ -H
> 'Sec-Fetch-Mode: cors' \ -H 'Sec-Fetch-Site: same-origin' \ -H 'User-Agent:
> Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML,
> like Gecko) Chrome/133.0.0.0 Safari/537.36' \ -H 'X-Requested-With:
> XMLHttpRequest' \ -H 'sec-ch-ua: "Not(A:Brand";v="99", "Google
> Chrome";v="133", "Chromium";v="133"' \ -H 'sec-ch-ua-mobile: ?0' \ -H
> 'sec-ch-ua-platform: "macOS"' \ --data-raw
> '\{"excludeDeletedEntities":true,"includeSubClassifications":false,"includeClassificationAttributes":true,"includeSubTypes":false,"tagFilters":null,"attributes":["type","position","comment"],"typeName":"hive_column","entityFilters":{"condition":"AND","criterion":[{"attributeName":"qualifiedName","operator":"startsWith","attributeValue":"test123.us_customers_csv113."},\{"attributeName":"qualifiedName","operator":"endsWith","attributeValue":"@cm"},\{"condition":"OR","criterion":[{"attributeName":"__isIncomplete","operator":"isNull","attributeValue":""},\{"attributeName":"__isIncomplete","operator":"eq","attributeValue":"0"}]}]},"classification":null,"limit":20,"offset":0,"sortBy":"name","sortOrder":"ASCENDING"}'}}
>
> {{"isIncomplete": false, "labels": [] } ],
> "approximateCount": 72}}}
> approximateCount is getting incremented on creating new tables
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
