[jira] [Created] (ATLAS-4712) atlas-hive image shows "No such file or directory error"
Snehal Ambavkar created ATLAS-4712:
--
Summary: atlas-hive image shows "No such file or directory error"
Key: ATLAS-4712
URL: https://issues.apache.org/jira/browse/ATLAS-4712
Project: Atlas
Issue Type: Bug
Reporter: Snehal Ambavkar
Assignee: Snehal Ambavkar
Fix For: 3.0.0, 2.3.0
Running docker-compose
{code:java}
docker-compose -f docker-compose.atlas-base.yml -f docker-compose.atlas.yml -f
docker-compose.atlas-hadoop.yml -f docker-compose.atlas-hbase.yml -f
docker-compose.atlas-kafka.yml -f docker-compose.atlas-hive.yml up -d
{code}
shows with:
{code:java}
/home/atlas/scripts/atlas-hive-setup.sh: line 44: cd:
/opt/atlas/atlas-hive-plugin: No such file or directory
/home/atlas/scripts/atlas-hive-setup.sh: line 45: ./enable-hive-plugin.sh: No
such file or directory {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (ATLAS-4571) Impakla Hook : Indexed string field (solr.StrField) which is too large ERROR
[ https://issues.apache.org/jira/browse/ATLAS-4571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Snehal Ambavkar resolved ATLAS-4571. Fix Version/s: 3.0.0 Resolution: Fixed > Impakla Hook : Indexed string field (solr.StrField) which is too large ERROR > > > Key: ATLAS-4571 > URL: https://issues.apache.org/jira/browse/ATLAS-4571 > Project: Atlas > Issue Type: Bug > Components: atlas-core > Reporter: Snehal Ambavkar > Assignee: Snehal Ambavkar >Priority: Major > Fix For: 3.0.0 > > > ERROR : > Exception writing document id test1 to the index; possible analysis error: > Document contains at least one immense term in field="test_s" (whose UTF8 > encoding is longer than the max length 32766), all of which were skipped. > Please correct the analyzer to not produce such terms. -- This message was sent by Atlassian Jira (v8.20.1#820001)
Re: Review Request 73926: ATLAS-4571 : Impala Hook : Indexed string field (solr.StrField) which is too large ERROR
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73926/ --- (Updated April 12, 2022, 7:03 a.m.) Review request for atlas, Jayendra Parab, Mandar Ambawane, Pinal Shah, Radhika Kundam, and Sidharth Mishra. Bugs: ATLAS-4571 https://issues.apache.org/jira/browse/ATLAS-4571 Repository: atlas Description --- ERROR : Exception writing document id test1 to the index; possible analysis error: Document contains at least one immense term in field="test_s" (whose UTF8 encoding is longer than the max length 32766), all of which were skipped. Please correct the analyzer to not produce such terms. RCA : Imapala process entities created by ImpalaHook saves query-string in name field. Since query-string can be large, we are getting the longer than the max error. Fix : To store qualifiedName in name field instead of query-string Diffs (updated) - addons/impala-bridge/src/main/java/org/apache/atlas/impala/hook/events/BaseImpalaEvent.java 32efb8321 addons/impala-bridge/src/test/java/org/apache/atlas/impala/ImpalaLineageToolIT.java 53e9b1224 addons/impala-bridge/src/test/java/org/apache/atlas/impala/hook/ImpalaLineageHookIT.java 56d74fee3 Diff: https://reviews.apache.org/r/73926/diff/3/ Changes: https://reviews.apache.org/r/73926/diff/2-3/ Testing --- Created tables and inserted data as per jira scenario Created smaller hive tables through impala-shell and verified presence in Atlas through hook Applied and removed classifications on impala-generated entities and hive tables Assigned terms to impala-generated entities and hive tables Created and assigned business meta data to impala-generated entities and hive tables. Deleted and purged impala-generated entities (Checked the above in both old and new UI) Eg : Created two hive tables with 4000 columns. Performed query as follows insert into table_2 select from <4000 column names> from table_1; Also tested with other queries that would intiate creation of impala_process and impala_process_execution entities. Precommit : https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1068/ Thanks, Snehal Ambavkar
Re: Review Request 73926: ATLAS-4571 : Impala Hook : Indexed string field (solr.StrField) which is too large ERROR
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73926/ --- (Updated April 12, 2022, 5:22 a.m.) Review request for atlas, Jayendra Parab, Mandar Ambawane, Pinal Shah, Radhika Kundam, and Sidharth Mishra. Bugs: ATLAS-4571 https://issues.apache.org/jira/browse/ATLAS-4571 Repository: atlas Description --- ERROR : Exception writing document id test1 to the index; possible analysis error: Document contains at least one immense term in field="test_s" (whose UTF8 encoding is longer than the max length 32766), all of which were skipped. Please correct the analyzer to not produce such terms. RCA : Imapala process entities created by ImpalaHook saves query-string in name field. Since query-string can be large, we are getting the longer than the max error. Fix : To store qualifiedName in name field instead of query-string Diffs (updated) - addons/impala-bridge/src/main/java/org/apache/atlas/impala/hook/events/BaseImpalaEvent.java 32efb8321 addons/impala-bridge/src/test/java/org/apache/atlas/impala/ImpalaLineageToolIT.java 53e9b1224 addons/impala-bridge/src/test/java/org/apache/atlas/impala/hook/ImpalaLineageHookIT.java 56d74fee3 Diff: https://reviews.apache.org/r/73926/diff/2/ Changes: https://reviews.apache.org/r/73926/diff/1-2/ Testing --- Created tables and inserted data as per jira scenario Created smaller hive tables through impala-shell and verified presence in Atlas through hook Applied and removed classifications on impala-generated entities and hive tables Assigned terms to impala-generated entities and hive tables Created and assigned business meta data to impala-generated entities and hive tables. Deleted and purged impala-generated entities (Checked the above in both old and new UI) Eg : Created two hive tables with 4000 columns. Performed query as follows insert into table_2 select from <4000 column names> from table_1; Also tested with other queries that would intiate creation of impala_process and impala_process_execution entities. Precommit : https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1068/ Thanks, Snehal Ambavkar
[jira] [Commented] (ATLAS-4571) Impakla Hook : Indexed string field (solr.StrField) which is too large ERROR
[ https://issues.apache.org/jira/browse/ATLAS-4571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17515972#comment-17515972 ] Snehal Ambavkar commented on ATLAS-4571: RCA : Imapala process entities created by ImpalaHook saves query-string in name field. Since query-string can be large, we are getting the longer than the max error. Fix : To store qualifiedName in name field instead of query-string > Impakla Hook : Indexed string field (solr.StrField) which is too large ERROR > > > Key: ATLAS-4571 > URL: https://issues.apache.org/jira/browse/ATLAS-4571 > Project: Atlas > Issue Type: Bug > Components: atlas-core > Reporter: Snehal Ambavkar > Assignee: Snehal Ambavkar >Priority: Major > > ERROR : > Exception writing document id test1 to the index; possible analysis error: > Document contains at least one immense term in field="test_s" (whose UTF8 > encoding is longer than the max length 32766), all of which were skipped. > Please correct the analyzer to not produce such terms. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (ATLAS-4571) Impakla Hook : Indexed string field (solr.StrField) which is too large ERROR
[ https://issues.apache.org/jira/browse/ATLAS-4571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Snehal Ambavkar updated ATLAS-4571: --- Summary: Impakla Hook : Indexed string field (solr.StrField) which is too large ERROR (was: Indexed string field (solr.StrField) which is too large ERROR) > Impakla Hook : Indexed string field (solr.StrField) which is too large ERROR > > > Key: ATLAS-4571 > URL: https://issues.apache.org/jira/browse/ATLAS-4571 > Project: Atlas > Issue Type: Bug > Components: atlas-core > Reporter: Snehal Ambavkar > Assignee: Snehal Ambavkar >Priority: Major > > ERROR : > Exception writing document id test1 to the index; possible analysis error: > Document contains at least one immense term in field="test_s" (whose UTF8 > encoding is longer than the max length 32766), all of which were skipped. > Please correct the analyzer to not produce such terms. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (ATLAS-4571) Indexed string field (solr.StrField) which is too large ERROR
[ https://issues.apache.org/jira/browse/ATLAS-4571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Snehal Ambavkar reassigned ATLAS-4571: -- Assignee: Snehal Ambavkar > Indexed string field (solr.StrField) which is too large ERROR > - > > Key: ATLAS-4571 > URL: https://issues.apache.org/jira/browse/ATLAS-4571 > Project: Atlas > Issue Type: Bug > Components: atlas-core > Reporter: Snehal Ambavkar > Assignee: Snehal Ambavkar >Priority: Major > > ERROR : > Exception writing document id test1 to the index; possible analysis error: > Document contains at least one immense term in field="test_s" (whose UTF8 > encoding is longer than the max length 32766), all of which were skipped. > Please correct the analyzer to not produce such terms. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (ATLAS-4571) Indexed string field (solr.StrField) which is too large ERROR
Snehal Ambavkar created ATLAS-4571: -- Summary: Indexed string field (solr.StrField) which is too large ERROR Key: ATLAS-4571 URL: https://issues.apache.org/jira/browse/ATLAS-4571 Project: Atlas Issue Type: Bug Components: atlas-core Reporter: Snehal Ambavkar ERROR : Exception writing document id test1 to the index; possible analysis error: Document contains at least one immense term in field="test_s" (whose UTF8 encoding is longer than the max length 32766), all of which were skipped. Please correct the analyzer to not produce such terms. -- This message was sent by Atlassian Jira (v8.20.1#820001)
Re: Review Request 73883: ATLAS-4560 : DSL Search : Error when keyword 'limit' is present in the search value
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73883/#review224139 --- Ship it! Ship It! - Snehal Ambavkar On March 4, 2022, 10:35 a.m., Snehal Ambavkar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73883/ > --- > > (Updated March 4, 2022, 10:35 a.m.) > > > Review request for atlas, Jayendra Parab, Mandar Ambawane, and Pinal Shah. > > > Bugs: ATLAS-4560 > https://issues.apache.org/jira/browse/ATLAS-4560 > > > Repository: atlas > > > Description > --- > > Root cause : > > There is a check in the code, if the query contains 'limit' keyword and does > not contain 'offset' then add 'offset 0' to the query. > eg : where name="test_column" limit 6, the updated query would be, where > name="test_column" limit 6 offset 0 > > In the query, where name="test_limit" > Since the query contains 'limit' word, offset 0 is added to the query without > the 'limit' keyword. > As per the grammar, there has to be 'limit' before 'offset', hence the error. > > > Solution: > It is not required to have offset when limit is specified.so the check is not > required. > Removed the above mentioned check, so no offset is added when there is > 'limit' word in the query. > Search works as expected with just limit specified > If the query does not contain 'limit', default limit value is used. > > > Diffs > - > > > repository/src/main/java/org/apache/atlas/query/executors/TraversalBasedExecutor.java > ecd82e3fd > repository/src/test/java/org/apache/atlas/BasicTestSetup.java a1d7b62a5 > > repository/src/test/java/org/apache/atlas/discovery/AtlasDiscoveryServiceTest.java > 8c257d866 > repository/src/test/java/org/apache/atlas/query/BaseDSLComposer.java > 419050ec8 > repository/src/test/java/org/apache/atlas/query/DSLQueriesTest.java > 74cc4e0a2 > repository/src/test/java/org/apache/atlas/query/TraversalComposerTest.java > 60645771d > > > Diff: https://reviews.apache.org/r/73883/diff/3/ > > > Testing > --- > > 1. created hive table with column test_limit and did basic search and > advanced search > 2. created hive table with 'limit' in name and did basic search and advanced > search > 3. used quick search entity types to create entities with limit in their name > and searched for them > 4. Tested for 'limit' string in classifications and glossaries. > 5. Small sanity in basic search and advanced search. > > > Precommit : > https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1052/console > > > Thanks, > > Snehal Ambavkar > >
Re: Review Request 73883: ATLAS-4560 : DSL Search : Error when keyword 'limit' is present in the search value
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73883/#review224138 --- Ship it! Ship It! - Snehal Ambavkar On March 4, 2022, 10:35 a.m., Snehal Ambavkar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73883/ > --- > > (Updated March 4, 2022, 10:35 a.m.) > > > Review request for atlas, Jayendra Parab, Mandar Ambawane, and Pinal Shah. > > > Bugs: ATLAS-4560 > https://issues.apache.org/jira/browse/ATLAS-4560 > > > Repository: atlas > > > Description > --- > > Root cause : > > There is a check in the code, if the query contains 'limit' keyword and does > not contain 'offset' then add 'offset 0' to the query. > eg : where name="test_column" limit 6, the updated query would be, where > name="test_column" limit 6 offset 0 > > In the query, where name="test_limit" > Since the query contains 'limit' word, offset 0 is added to the query without > the 'limit' keyword. > As per the grammar, there has to be 'limit' before 'offset', hence the error. > > > Solution: > It is not required to have offset when limit is specified.so the check is not > required. > Removed the above mentioned check, so no offset is added when there is > 'limit' word in the query. > Search works as expected with just limit specified > If the query does not contain 'limit', default limit value is used. > > > Diffs > - > > > repository/src/main/java/org/apache/atlas/query/executors/TraversalBasedExecutor.java > ecd82e3fd > repository/src/test/java/org/apache/atlas/BasicTestSetup.java a1d7b62a5 > > repository/src/test/java/org/apache/atlas/discovery/AtlasDiscoveryServiceTest.java > 8c257d866 > repository/src/test/java/org/apache/atlas/query/BaseDSLComposer.java > 419050ec8 > repository/src/test/java/org/apache/atlas/query/DSLQueriesTest.java > 74cc4e0a2 > repository/src/test/java/org/apache/atlas/query/TraversalComposerTest.java > 60645771d > > > Diff: https://reviews.apache.org/r/73883/diff/3/ > > > Testing > --- > > 1. created hive table with column test_limit and did basic search and > advanced search > 2. created hive table with 'limit' in name and did basic search and advanced > search > 3. used quick search entity types to create entities with limit in their name > and searched for them > 4. Tested for 'limit' string in classifications and glossaries. > 5. Small sanity in basic search and advanced search. > > > Precommit : > https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1052/console > > > Thanks, > > Snehal Ambavkar > >
[jira] [Commented] (ATLAS-4560) DSL Search : Error when keyword 'limit' is present in the search value
[
https://issues.apache.org/jira/browse/ATLAS-4560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17501246#comment-17501246
]
Snehal Ambavkar commented on ATLAS-4560:
Review Request : https://reviews.apache.org/r/73883/
> DSL Search : Error when keyword 'limit' is present in the search value
> --
>
> Key: ATLAS-4560
> URL: https://issues.apache.org/jira/browse/ATLAS-4560
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Reporter: Snehal Ambavkar
>Assignee: Snehal Ambavkar
>Priority: Critical
>
> When executing the following query {color:#172b4d}for hive column{color}:
> {color:#172b4d}where name = "test_limit"
> Getting the following error :
> Invalid DSL query: `hive_column` where name="test_limit" offset 0 | Reason:
> mismatched input 'offset' expecting \{, K_ORDERBY, K_GROUPBY, K_LIMIT,
> K_SELECT}. Please refer to Atlas DSL grammar for more information{color}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
Re: Review Request 73883: ATLAS-4560 : DSL Search : Error when keyword 'limit' is present in the search value
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73883/ --- (Updated March 4, 2022, 10:35 a.m.) Review request for atlas, Jayendra Parab, Mandar Ambawane, and Pinal Shah. Bugs: ATLAS-4560 https://issues.apache.org/jira/browse/ATLAS-4560 Repository: atlas Description --- Root cause : There is a check in the code, if the query contains 'limit' keyword and does not contain 'offset' then add 'offset 0' to the query. eg : where name="test_column" limit 6, the updated query would be, where name="test_column" limit 6 offset 0 In the query, where name="test_limit" Since the query contains 'limit' word, offset 0 is added to the query without the 'limit' keyword. As per the grammar, there has to be 'limit' before 'offset', hence the error. Solution: It is not required to have offset when limit is specified.so the check is not required. Removed the above mentioned check, so no offset is added when there is 'limit' word in the query. Search works as expected with just limit specified If the query does not contain 'limit', default limit value is used. Diffs - repository/src/main/java/org/apache/atlas/query/executors/TraversalBasedExecutor.java ecd82e3fd repository/src/test/java/org/apache/atlas/BasicTestSetup.java a1d7b62a5 repository/src/test/java/org/apache/atlas/discovery/AtlasDiscoveryServiceTest.java 8c257d866 repository/src/test/java/org/apache/atlas/query/BaseDSLComposer.java 419050ec8 repository/src/test/java/org/apache/atlas/query/DSLQueriesTest.java 74cc4e0a2 repository/src/test/java/org/apache/atlas/query/TraversalComposerTest.java 60645771d Diff: https://reviews.apache.org/r/73883/diff/3/ Testing --- 1. created hive table with column test_limit and did basic search and advanced search 2. created hive table with 'limit' in name and did basic search and advanced search 3. used quick search entity types to create entities with limit in their name and searched for them 4. Tested for 'limit' string in classifications and glossaries. 5. Small sanity in basic search and advanced search. Precommit : https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1052/console Thanks, Snehal Ambavkar
[jira] [Assigned] (ATLAS-4560) DSL Search : Error when keyword 'limit' is present in the search value
[
https://issues.apache.org/jira/browse/ATLAS-4560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Snehal Ambavkar reassigned ATLAS-4560:
--
Assignee: Snehal Ambavkar (was: Jayendra Parab)
> DSL Search : Error when keyword 'limit' is present in the search value
> --
>
> Key: ATLAS-4560
> URL: https://issues.apache.org/jira/browse/ATLAS-4560
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Reporter: Snehal Ambavkar
>Assignee: Snehal Ambavkar
>Priority: Critical
>
> When executing the following query {color:#172b4d}for hive column{color}:
> {color:#172b4d}where name = "test_limit"
> Getting the following error :
> Invalid DSL query: `hive_column` where name="test_limit" offset 0 | Reason:
> mismatched input 'offset' expecting \{, K_ORDERBY, K_GROUPBY, K_LIMIT,
> K_SELECT}. Please refer to Atlas DSL grammar for more information{color}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
Request to join Apache Atlas as contributor
Hi Atlas Team, I am interested in contributing to the Apache Atlas project, please add me as a contributor. My Apache Jira id is snehal.ambavkar Thank you, Snehal Ambavkar
[jira] [Updated] (ATLAS-4560) DSL Search : Error when keyword 'limit' is present in the search value
[
https://issues.apache.org/jira/browse/ATLAS-4560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Snehal Ambavkar updated ATLAS-4560:
---
Summary: DSL Search : Error when keyword 'limit' is present in the search
value (was: Getting error when keyword 'limit' present in Advanced Search)
> DSL Search : Error when keyword 'limit' is present in the search value
> --
>
> Key: ATLAS-4560
> URL: https://issues.apache.org/jira/browse/ATLAS-4560
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Reporter: Snehal Ambavkar
>Priority: Critical
>
> When executing the following query {color:#172b4d}for hive column{color}:
> {color:#172b4d}where name = "test_limit"
> Getting the following error :
> Invalid DSL query: `hive_column` where name="test_limit" offset 0 | Reason:
> mismatched input 'offset' expecting \{, K_ORDERBY, K_GROUPBY, K_LIMIT,
> K_SELECT}. Please refer to Atlas DSL grammar for more information{color}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (ATLAS-4560) Getting error when keyword 'limit' present in Advanced Search
[
https://issues.apache.org/jira/browse/ATLAS-4560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Snehal Ambavkar updated ATLAS-4560:
---
Description:
When executing the following query {color:#172b4d}for hive column{color}:
{color:#172b4d}where name = "test_limit"
Getting the following error :
Invalid DSL query: `hive_column` where name="test_limit" offset 0 | Reason:
mismatched input 'offset' expecting \{, K_ORDERBY, K_GROUPBY, K_LIMIT,
K_SELECT}. Please refer to Atlas DSL grammar for more information{color}
was:
When executing the following query :
{color:#172b4d}where name = "test_limit" for hive column
Getting the following error :
Invalid DSL query: `hive_column` where name="test_limit" offset 0 | Reason:
mismatched input 'offset' expecting \{, K_ORDERBY, K_GROUPBY, K_LIMIT,
K_SELECT}. Please refer to Atlas DSL grammar for more information{color}
> Getting error when keyword 'limit' present in Advanced Search
> -
>
> Key: ATLAS-4560
> URL: https://issues.apache.org/jira/browse/ATLAS-4560
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Reporter: Snehal Ambavkar
>Priority: Critical
>
> When executing the following query {color:#172b4d}for hive column{color}:
> {color:#172b4d}where name = "test_limit"
> Getting the following error :
> Invalid DSL query: `hive_column` where name="test_limit" offset 0 | Reason:
> mismatched input 'offset' expecting \{, K_ORDERBY, K_GROUPBY, K_LIMIT,
> K_SELECT}. Please refer to Atlas DSL grammar for more information{color}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Created] (ATLAS-4560) Getting error when keyword 'limit' present in Advanced Search
Snehal Ambavkar created ATLAS-4560:
--
Summary: Getting error when keyword 'limit' present in Advanced
Search
Key: ATLAS-4560
URL: https://issues.apache.org/jira/browse/ATLAS-4560
Project: Atlas
Issue Type: Bug
Components: atlas-core
Reporter: Snehal Ambavkar
When executing the following query :
{color:#172b4d}where name = "test_limit" for hive column
Getting the following error :
Invalid DSL query: `hive_column` where name="test_limit" offset 0 | Reason:
mismatched input 'offset' expecting \{, K_ORDERBY, K_GROUPBY, K_LIMIT,
K_SELECT}. Please refer to Atlas DSL grammar for more information{color}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
