[
https://issues.apache.org/jira/browse/METRON-1811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16642365#comment-16642365
]
ASF GitHub Bot commented on METRON-1811:
----------------------------------------
GitHub user merrimanr opened a pull request:
https://github.com/apache/metron/pull/1231
METRON-1811: Alert Search Fails When Sorting by Alert Status
## Contributor Comments
This PR fixes sorting on the `alert_status` field in the Alerts UI by
defining the field in ES templates as a `keyword` type. The change was applied
to the sensor templates that ship with Metron: bro, snort and yaf. This field
was added to the Solr schemas as well.
I also updated our documentation to give users guidance when defining their
own templates or upgrading their templates. I expanded this to include other
internal fields like `source:type` and `metron_alert`. I did not include
dynamic fields but I can add documentation for that here if it makes sense.
### Testing
This has been tested in full dev:
1. Spin up full dev and navigate to the Alerts UI.
2. Change the status of a couple alerts by opening up their details panel
and clicking a different status (OPEN for example).
3. Sort by `alert_status`. The Alerts UI should properly display alerts by
`alert_status` and no errors should be reported in the console.
4. Enable Solr and verify data is visible in the Alerts UI. Repeat steps 2
and 3.
## Pull Request Checklist
Thank you for submitting a contribution to Apache Metron.
Please refer to our [Development
Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235)
for the complete guide to follow for contributions.
Please refer also to our [Build Verification
Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview)
for complete smoke testing guides.
In order to streamline the review of the contribution we ask you follow
these guidelines and ask you to double check the following:
### For all changes:
- [x] Is there a JIRA ticket associated with this PR? If not one needs to
be created at [Metron
Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
- [x] Does your PR title start with METRON-XXXX where XXXX is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [x] Has your PR been rebased against the latest commit within the target
branch (typically master)?
### For code changes:
- [x] Have you included steps to reproduce the behavior or problem that is
being changed or addressed?
- [x] Have you included steps or a guide to how the change may be verified
and tested manually?
- [x] Have you ensured that the full suite of tests and checks have been
executed in the root metron folder via:
```
mvn -q clean integration-test install &&
dev-utilities/build-utils/verify_licenses.sh
```
- [x] Have you written or updated unit tests and or integration tests to
verify your changes?
- [x] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [x] Have you verified the basic functionality of the build by building
and running locally with Vagrant full-dev environment or the equivalent?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in
which it is rendered by building and verifying the site-book? If not then run
the following commands and the verify changes via
`site-book/target/site/index.html`:
```
cd site-book
mvn site
```
#### Note:
Please ensure that once the PR is submitted, you check travis-ci for build
issues and submit an update to your PR as soon as possible.
It is also recommended that [travis-ci](https://travis-ci.org) is set up
for your personal repository such that your branches are built there before
submitting a pull request.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/merrimanr/incubator-metron METRON-1811
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/metron/pull/1231.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1231
----
commit 7a707bfbb1c6339f5891763c82e611eb080c4af7
Author: merrimanr <merrimanr@...>
Date: 2018-10-08T14:40:37Z
initial commit
----
> Alert Search Fails When Sorting by Alert Status
> -----------------------------------------------
>
> Key: METRON-1811
> URL: https://issues.apache.org/jira/browse/METRON-1811
> Project: Metron
> Issue Type: Bug
> Reporter: Ryan Merriman
> Assignee: Ryan Merriman
> Priority: Major
>
> Searching for alerts does not work when sorting by Alert Status. When this
> happens, no error is indicated in the UI, but the REST calls fails.
> Request:
> {{{"indices":[],"facetFields":[],"query":"*","from":0,"size":25,"sort":[\{"field":"alert_status","sortOrder":"desc"}]}
> }}
> Response:
> {{500 Internal Server Error }}
> The following is logged in the REST logs @ /var/log/metron/metron-rest.log
> {{18/09/26 20:38:24 ERROR controller.RestExceptionHandler: Encountered error:
> Failed to execute search; error='IllegalArgumentException: Fielddata is
> disabled on text fields by default. Set fielddata=true on [__anonymous_text]
> in order to load fielddata in memory by uninverting the inverted index. Note
> that this can however use significant memory. Alternatively use a keyword
> field instead.',
> search='\{"from":0,"size":25,"query":{"constant_score":{"filter":{"bool":{"must":[{"bool":{"should":[{"query_string":{"query":"*","fields":[],"use_dis_max":true,"tie_breaker":0.0,"default_operator":"or","auto_generate_phrase_queries":false,"max_determinized_states":10000,"enable_position_increments":true,"fuzziness":"AUTO","fuzzy_prefix_length":0,"fuzzy_max_expansions":50,"phrase_slop":0,"escape":false,"split_on_whitespace":true,"boost":1.0}},\{"nested":{"query":{"query_string":{"query":"*","fields":[],"use_dis_max":true,"tie_breaker":0.0,"default_operator":"or","auto_generate_phrase_queries":false,"max_determinized_states":10000,"enable_position_increments":true,"fuzziness":"AUTO","fuzzy_prefix_length":0,"fuzzy_max_expansions":50,"phrase_slop":0,"escape":false,"split_on_whitespace":true,"boost":1.0}},"path":"metron_alert","ignore_unmapped":false,"score_mode":"none","boost":1.0}}],"disable_coord":false,"adjust_pure_negative":true,"boost":1.0}},\{"bool":{"should":[{"term":{"status":{"value":"active","boost":1.0}}},\{"bool":{"must_not":[{"exists":{"field":"status","boost":1.0}}],"disable_coord":false,"adjust_pure_negative":true,"boost":1.0}}],"disable_coord":false,"adjust_pure_negative":true,"boost":1.0}}],"must_not":[\{"exists":{"field":"metaalerts","boost":1.0}}],"disable_coord":false,"adjust_pure_negative":true,"boost":1.0}},"boost":1.0}},"_source":\{"includes":[],"excludes":[]},"sort":[\{"alert_status":{"order":"desc","missing":"_last","unmapped_type":"text"}}],"track_scores":true,"aggregations":\{"source:type_count":{"terms":{"field":"source:type","size":10,"min_doc_count":1,"shard_min_doc_count":0,"show_term_doc_count_error":false,"order":[{"_count":"desc"},\{"_term":"asc"}]}},"ip_src_addr_count":\{"terms":{"field":"ip_src_addr","size":10,"min_doc_count":1,"shard_min_doc_count":0,"show_term_doc_count_error":false,"order":[{"_count":"desc"},\{"_term":"asc"}]}},"ip_dst_addr_count":\{"terms":{"field":"ip_dst_addr","size":10,"min_doc_count":1,"shard_min_doc_count":0,"show_term_doc_count_error":false,"order":[{"_count":"desc"},\{"_term":"asc"}]}},"enrichments:geo:ip_dst_addr:country_count":\{"terms":{"field":"enrichments:geo:ip_dst_addr:country","size":10,"min_doc_count":1,"shard_min_doc_count":0,"show_term_doc_count_error":false,"order":[{"_count":"desc"},\{"_term":"asc"}]}}}}'
> org.apache.metron.rest.RestException: Failed to execute search;
> error='IllegalArgumentException: Fielddata is disabled on text fields by
> default. Set fielddata=true on [__anonymous_text] in order to load fielddata
> in memory by uninverting the inverted index. Note that this can however use
> significant memory. Alternatively use a keyword field instead.',
> search='\{"from":0,"size":25,"query":{"constant_score":{"filter":{"bool":{"must":[{"bool":{"should":[{"query_string":{"query":"*","fields":[],"use_dis_max":true,"tie_breaker":0.0,"default_operator":"or","auto_generate_phrase_queries":false,"max_determinized_states":10000,"enable_position_increments":true,"fuzziness":"AUTO","fuzzy_prefix_length":0,"fuzzy_max_expansions":50,"phrase_slop":0,"escape":false,"split_on_whitespace":true,"boost":1.0}},\{"nested":{"query":{"query_string":{"query":"*","fields":[],"use_dis_max":true,"tie_breaker":0.0,"default_operator":"or","auto_generate_phrase_queries":false,"max_determinized_states":10000,"enable_position_increments":true,"fuzziness":"AUTO","fuzzy_prefix_length":0,"fuzzy_max_expansions":50,"phrase_slop":0,"escape":false,"split_on_whitespace":true,"boost":1.0}},"path":"metron_alert","ignore_unmapped":false,"score_mode":"none","boost":1.0}}],"disable_coord":false,"adjust_pure_negative":true,"boost":1.0}},\{"bool":{"should":[{"term":{"status":{"value":"active","boost":1.0}}},\{"bool":{"must_not":[{"exists":{"field":"status","boost":1.0}}],"disable_coord":false,"adjust_pure_negative":true,"boost":1.0}}],"disable_coord":false,"adjust_pure_negative":true,"boost":1.0}}],"must_not":[\{"exists":{"field":"metaalerts","boost":1.0}}],"disable_coord":false,"adjust_pure_negative":true,"boost":1.0}},"boost":1.0}},"_source":\{"includes":[],"excludes":[]},"sort":[\{"alert_status":{"order":"desc","missing":"_last","unmapped_type":"text"}}],"track_scores":true,"aggregations":\{"source:type_count":{"terms":{"field":"source:type","size":10,"min_doc_count":1,"shard_min_doc_count":0,"show_term_doc_count_error":false,"order":[{"_count":"desc"},\{"_term":"asc"}]}},"ip_src_addr_count":\{"terms":{"field":"ip_src_addr","size":10,"min_doc_count":1,"shard_min_doc_count":0,"show_term_doc_count_error":false,"order":[{"_count":"desc"},\{"_term":"asc"}]}},"ip_dst_addr_count":\{"terms":{"field":"ip_dst_addr","size":10,"min_doc_count":1,"shard_min_doc_count":0,"show_term_doc_count_error":false,"order":[{"_count":"desc"},\{"_term":"asc"}]}},"enrichments:geo:ip_dst_addr:country_count":\{"terms":{"field":"enrichments:geo:ip_dst_addr:country","size":10,"min_doc_count":1,"shard_min_doc_count":0,"show_term_doc_count_error":false,"order":[{"_count":"desc"},\{"_term":"asc"}]}}}}'
> at
> org.apache.metron.rest.service.impl.SearchServiceImpl.search(SearchServiceImpl.java:95)
> at
> org.apache.metron.rest.controller.SearchController.search(SearchController.java:54)
> at sun.reflect.GeneratedMethodAccessor89.invoke(Unknown Source) at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:209)
> at
> org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:136)
> at
> org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:102)
> at
> org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:877)
> at
> org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:783)
> at
> org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
> at
> org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:991)
> at
> org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:925)
> at
> org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:974)
> at
> org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:877)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:661) at
> org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:851)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:742) at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
> at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:320)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
> at
> org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
> at
> org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:119)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
> at
> org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
> at
> org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
> at
> org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:170)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
> at
> org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
> at
> org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:158)
> at
> org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
> at
> org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
> at
> org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:66)
> at
> org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
> at
> org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
> at
> org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)
> at
> org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
> at
> org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:215)
> at
> org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:178)
> at
> org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:357)
> at
> org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:270)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
> at
> org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99)
> at
> org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
> at
> org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:109)
> at
> org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
> at
> org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:81)
> at
> org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
> at
> org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:200)
> at
> org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198)
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
> at
> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:496)
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)
> at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:803)
> at
> org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
> at
> org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:790)
> at
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1459)
> at
> org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:745) Caused by:
> org.apache.metron.indexing.dao.search.InvalidSearchException: Failed to
> execute search; error='IllegalArgumentException: Fielddata is disabled on
> text fields by default. Set fielddata=true on [__anonymous_text] in order to
> load fielddata in memory by uninverting the inverted index. Note that this
> can however use significant memory. Alternatively use a keyword field
> instead.',
> search='\{"from":0,"size":25,"query":{"constant_score":{"filter":{"bool":{"must":[{"bool":{"should":[{"query_string":{"query":"*","fields":[],"use_dis_max":true,"tie_breaker":0.0,"default_operator":"or","auto_generate_phrase_queries":false,"max_determinized_states":10000,"enable_position_increments":true,"fuzziness":"AUTO","fuzzy_prefix_length":0,"fuzzy_max_expansions":50,"phrase_slop":0,"escape":false,"split_on_whitespace":true,"boost":1.0}},\{"nested":{"query":{"query_string":{"query":"*","fields":[],"use_dis_max":true,"tie_breaker":0.0,"default_operator":"or","auto_generate_phrase_queries":false,"max_determinized_states":10000,"enable_position_increments":true,"fuzziness":"AUTO","fuzzy_prefix_length":0,"fuzzy_max_expansions":50,"phrase_slop":0,"escape":false,"split_on_whitespace":true,"boost":1.0}},"path":"metron_alert","ignore_unmapped":false,"score_mode":"none","boost":1.0}}],"disable_coord":false,"adjust_pure_negative":true,"boost":1.0}},\{"bool":{"should":[{"term":{"status":{"value":"active","boost":1.0}}},\{"bool":{"must_not":[{"exists":{"field":"status","boost":1.0}}],"disable_coord":false,"adjust_pure_negative":true,"boost":1.0}}],"disable_coord":false,"adjust_pure_negative":true,"boost":1.0}}],"must_not":[\{"exists":{"field":"metaalerts","boost":1.0}}],"disable_coord":false,"adjust_pure_negative":true,"boost":1.0}},"boost":1.0}},"_source":\{"includes":[],"excludes":[]},"sort":[\{"alert_status":{"order":"desc","missing":"_last","unmapped_type":"text"}}],"track_scores":true,"aggregations":\{"source:type_count":{"terms":{"field":"source:type","size":10,"min_doc_count":1,"shard_min_doc_count":0,"show_term_doc_count_error":false,"order":[{"_count":"desc"},\{"_term":"asc"}]}},"ip_src_addr_count":\{"terms":{"field":"ip_src_addr","size":10,"min_doc_count":1,"shard_min_doc_count":0,"show_term_doc_count_error":false,"order":[{"_count":"desc"},\{"_term":"asc"}]}},"ip_dst_addr_count":\{"terms":{"field":"ip_dst_addr","size":10,"min_doc_count":1,"shard_min_doc_count":0,"show_term_doc_count_error":false,"order":[{"_count":"desc"},\{"_term":"asc"}]}},"enrichments:geo:ip_dst_addr:country_count":\{"terms":{"field":"enrichments:geo:ip_dst_addr:country","size":10,"min_doc_count":1,"shard_min_doc_count":0,"show_term_doc_count_error":false,"order":[{"_count":"desc"},\{"_term":"asc"}]}}}}'
> at
> org.apache.metron.elasticsearch.dao.ElasticsearchRequestSubmitter.submitSearch(ElasticsearchRequestSubmitter.java:74)
> at
> org.apache.metron.elasticsearch.dao.ElasticsearchSearchDao.search(ElasticsearchSearchDao.java:139)
> at
> org.apache.metron.elasticsearch.dao.ElasticsearchDao.search(ElasticsearchDao.java:197)
> at
> org.apache.metron.elasticsearch.dao.ElasticsearchMetaAlertSearchDao.search(ElasticsearchMetaAlertSearchDao.java:79)
> at
> org.apache.metron.elasticsearch.dao.ElasticsearchMetaAlertDao.search(ElasticsearchMetaAlertDao.java:210)
> at
> org.apache.metron.rest.service.impl.SearchServiceImpl.search(SearchServiceImpl.java:92)
> ... 87 more Caused by: Failed to execute phase [query], all shards failed;
> shardFailures \{[51Fb8bqzTZCwDpxkSC7IlQ][bro_index_2018.09.26.19][0]:
> RemoteTransportException[[node1][192.168.66.121:9300][indices:data/read/search[phase/query]]];
> nested: IllegalArgumentException[Fielddata is disabled on text fields by
> default. Set fielddata=true on [__anonymous_text] in order to load fielddata
> in memory by uninverting the inverted index. Note that this can however use
> significant memory. Alternatively use a keyword field instead.];
> }{[51Fb8bqzTZCwDpxkSC7IlQ][bro_index_2018.09.26.20][0]:
> RemoteTransportException[[node1][192.168.66.121:9300][indices:data/read/search[phase/query]]];
> nested: IllegalArgumentException[Fielddata is disabled on text fields by
> default. Set fielddata=true on [__anonymous_text] in order to load fielddata
> in memory by uninverting the inverted index. Note that this can however use
> significant memory. Alternatively use a keyword field instead.];
> }\{[51Fb8bqzTZCwDpxkSC7IlQ][snort_index_2018.09.26.19][0]:
> RemoteTransportException[[node1][192.168.66.121:9300][indices:data/read/search[phase/query]]];
> nested: IllegalArgumentException[Fielddata is disabled on text fields by
> default. Set fielddata=true on [alert_status] in order to load fielddata in
> memory by uninverting the inverted index. Note that this can however use
> significant memory. Alternatively use a keyword field instead.];
> }{[51Fb8bqzTZCwDpxkSC7IlQ][snort_index_2018.09.26.20][0]:
> RemoteTransportException[[node1][192.168.66.121:9300][indices:data/read/search[phase/query]]];
> nested: IllegalArgumentException[Fielddata is disabled on text fields by
> default. Set fielddata=true on [alert_status] in order to load fielddata in
> memory by uninverting the inverted index. Note that this can however use
> significant memory. Alternatively use a keyword field instead.];
> }\{[51Fb8bqzTZCwDpxkSC7IlQ][bro_index_2018.09.26.19][1]:
> RemoteTransportException[[node1][192.168.66.121:9300][indices:data/read/search[phase/query]]];
> nested: IllegalArgumentException[Fielddata is disabled on text fields by
> default. Set fielddata=true on [__anonymous_text] in order to load fielddata
> in memory by uninverting the inverted index. Note that this can however use
> significant memory. Alternatively use a keyword field instead.];
> }{[51Fb8bqzTZCwDpxkSC7IlQ][bro_index_2018.09.26.20][1]:
> RemoteTransportException[[node1][192.168.66.121:9300][indices:data/read/search[phase/query]]];
> nested: IllegalArgumentException[Fielddata is disabled on text fields by
> default. Set fielddata=true on [__anonymous_text] in order to load fielddata
> in memory by uninverting the inverted index. Note that this can however use
> significant memory. Alternatively use a keyword field instead.];
> }\{[51Fb8bqzTZCwDpxkSC7IlQ][snort_index_2018.09.26.19][1]:
> RemoteTransportException[[node1][192.168.66.121:9300][indices:data/read/search[phase/query]]];
> nested: IllegalArgumentException[Fielddata is disabled on text fields by
> default. Set fielddata=true on [alert_status] in order to load fielddata in
> memory by uninverting the inverted index. Note that this can however use
> significant memory. Alternatively use a keyword field instead.];
> }{[51Fb8bqzTZCwDpxkSC7IlQ][snort_index_2018.09.26.20][1]:
> RemoteTransportException[[node1][192.168.66.121:9300][indices:data/read/search[phase/query]]];
> nested: IllegalArgumentException[Fielddata is disabled on text fields by
> default. Set fielddata=true on [alert_status] in order to load fielddata in
> memory by uninverting the inverted index. Note that this can however use
> significant memory. Alternatively use a keyword field instead.];
> }\{[51Fb8bqzTZCwDpxkSC7IlQ][bro_index_2018.09.26.19][2]:
> RemoteTransportException[[node1][192.168.66.121:9300][indices:data/read/search[phase/query]]];
> nested: IllegalArgumentException[Fielddata is disabled on text fields by
> default. Set fielddata=true on [__anonymous_text] in order to load fielddata
> in memory by uninverting the inverted index. Note that this can however use
> significant memory. Alternatively use a keyword field instead.];
> }{[51Fb8bqzTZCwDpxkSC7IlQ][bro_index_2018.09.26.20][2]:
> RemoteTransportException[[node1][192.168.66.121:9300][indices:data/read/search[phase/query]]];
> nested: IllegalArgumentException[Fielddata is disabled on text fields by
> default. Set fielddata=true on [__anonymous_text] in order to load fielddata
> in memory by uninverting the inverted index. Note that this can however use
> significant memory. Alternatively use a keyword field instead.];
> }\{[51Fb8bqzTZCwDpxkSC7IlQ][snort_index_2018.09.26.19][2]:
> RemoteTransportException[[node1][192.168.66.121:9300][indices:data/read/search[phase/query]]];
> nested: IllegalArgumentException[Fielddata is disabled on text fields by
> default. Set fielddata=true on [alert_status] in order to load fielddata in
> memory by uninverting the inverted index. Note that this can however use
> significant memory. Alternatively use a keyword field instead.];
> }{[51Fb8bqzTZCwDpxkSC7IlQ][snort_index_2018.09.26.20][2]:
> RemoteTransportException[[node1][192.168.66.121:9300][indices:data/read/search[phase/query]]];
> nested: IllegalArgumentException[Fielddata is disabled on text fields by
> default. Set fielddata=true on [alert_status] in order to load fielddata in
> memory by uninverting the inverted index. Note that this can however use
> significant memory. Alternatively use a keyword field instead.];
> }\{[51Fb8bqzTZCwDpxkSC7IlQ][bro_index_2018.09.26.19][3]:
> RemoteTransportException[[node1][192.168.66.121:9300][indices:data/read/search[phase/query]]];
> nested: IllegalArgumentException[Fielddata is disabled on text fields by
> default. Set fielddata=true on [__anonymous_text] in order to load fielddata
> in memory by uninverting the inverted index. Note that this can however use
> significant memory. Alternatively use a keyword field instead.];
> }{[51Fb8bqzTZCwDpxkSC7IlQ][bro_index_2018.09.26.20][3]:
> RemoteTransportException[[node1][192.168.66.121:9300][indices:data/read/search[phase/query]]];
> nested: IllegalArgumentException[Fielddata is disabled on text fields by
> default. Set fielddata=true on [__anonymous_text] in order to load fielddata
> in memory by uninverting the inverted index. Note that this can however use
> significant memory. Alternatively use a keyword field instead.];
> }\{[51Fb8bqzTZCwDpxkSC7IlQ][snort_index_2018.09.26.19][3]:
> RemoteTransportException[[node1][192.168.66.121:9300][indices:data/read/search[phase/query]]];
> nested: IllegalArgumentException[Fielddata is disabled on text fields by
> default. Set fielddata=true on [alert_status] in order to load fielddata in
> memory by uninverting the inverted index. Note that this can however use
> significant memory. Alternatively use a keyword field instead.];
> }{[51Fb8bqzTZCwDpxkSC7IlQ][snort_index_2018.09.26.20][3]:
> RemoteTransportException[[node1][192.168.66.121:9300][indices:data/read/search[phase/query]]];
> nested: IllegalArgumentException[Fielddata is disabled on text fields by
> default. Set fielddata=true on [alert_status] in order to load fielddata in
> memory by uninverting the inverted index. Note that this can however use
> significant memory. Alternatively use a keyword field instead.];
> }\{[51Fb8bqzTZCwDpxkSC7IlQ][bro_index_2018.09.26.19][4]:
> RemoteTransportException[[node1][192.168.66.121:9300][indices:data/read/search[phase/query]]];
> nested: IllegalArgumentException[Fielddata is disabled on text fields by
> default. Set fielddata=true on [__anonymous_text] in order to load fielddata
> in memory by uninverting the inverted index. Note that this can however use
> significant memory. Alternatively use a keyword field instead.];
> }{[51Fb8bqzTZCwDpxkSC7IlQ][bro_index_2018.09.26.20][4]:
> RemoteTransportException[[node1][192.168.66.121:9300][indices:data/read/search[phase/query]]];
> nested: IllegalArgumentException[Fielddata is disabled on text fields by
> default. Set fielddata=true on [__anonymous_text] in order to load fielddata
> in memory by uninverting the inverted index. Note that this can however use
> significant memory. Alternatively use a keyword field instead.];
> }\{[51Fb8bqzTZCwDpxkSC7IlQ][snort_index_2018.09.26.19][4]:
> RemoteTransportException[[node1][192.168.66.121:9300][indices:data/read/search[phase/query]]];
> nested: IllegalArgumentException[Fielddata is disabled on text fields by
> default. Set fielddata=true on [alert_status] in order to load fielddata in
> memory by uninverting the inverted index. Note that this can however use
> significant memory. Alternatively use a keyword field instead.];
> }{[51Fb8bqzTZCwDpxkSC7IlQ][snort_index_2018.09.26.20][4]:
> RemoteTransportException[[node1][192.168.66.121:9300][indices:data/read/search[phase/query]]];
> nested: IllegalArgumentException[Fielddata is disabled on text fields by
> default. Set fielddata=true on [alert_status] in order to load fielddata in
> memory by uninverting the inverted index. Note that this can however use
> significant memory. Alternatively use a keyword field instead.]; } at
> org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseFailure(AbstractSearchAsyncAction.java:272)
> at
> org.elasticsearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:130)
> at
> org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseDone(AbstractSearchAsyncAction.java:241)
> at
> org.elasticsearch.action.search.InitialSearchPhase.onShardFailure(InitialSearchPhase.java:90)
> at
> org.elasticsearch.action.search.InitialSearchPhase.access$100(InitialSearchPhase.java:46)
> at
> org.elasticsearch.action.search.InitialSearchPhase$1.onFailure(InitialSearchPhase.java:169)
> at
> org.elasticsearch.action.ActionListenerResponseHandler.handleException(ActionListenerResponseHandler.java:51)
> at
> org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1067)
> at
> org.elasticsearch.transport.TransportService$DirectResponseChannel.processException(TransportService.java:1171)
> at
> org.elasticsearch.transport.TransportService$DirectResponseChannel.sendResponse(TransportService.java:1149)
> at
> org.elasticsearch.transport.TransportService$7.onFailure(TransportService.java:655)
> at
> org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.onFailure(ThreadContext.java:623)
> at
> org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:39)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> ... 1 more Caused by: NotSerializableExceptionWrapper[: Fielddata is
> disabled on text fields by default. Set fielddata=true on [__anonymous_text]
> in order to load fielddata in memory by uninverting the inverted index. Note
> that this can however use significant memory. Alternatively use a keyword
> field instead.]; nested: IllegalArgumentException[Fielddata is disabled on
> text fields by default. Set fielddata=true on [__anonymous_text] in order to
> load fielddata in memory by uninverting the inverted index. Note that this
> can however use significant memory. Alternatively use a keyword field
> instead.]; at
> org.elasticsearch.ElasticsearchException.guessRootCauses(ElasticsearchException.java:618)
> at
> org.elasticsearch.action.search.SearchPhaseExecutionException.guessRootCauses(SearchPhaseExecutionException.java:170)
> at
> org.elasticsearch.action.search.SearchPhaseExecutionException.getCause(SearchPhaseExecutionException.java:111)
> at
> org.elasticsearch.ElasticsearchException.writeTo(ElasticsearchException.java:285)
> at
> org.elasticsearch.action.search.SearchPhaseExecutionException.writeTo(SearchPhaseExecutionException.java:61)
> at
> org.elasticsearch.common.io.stream.StreamOutput.writeException(StreamOutput.java:838)
> at
> org.elasticsearch.ElasticsearchException.writeTo(ElasticsearchException.java:285)
> at
> org.elasticsearch.transport.ActionTransportException.writeTo(ActionTransportException.java:64)
> at
> org.elasticsearch.common.io.stream.StreamOutput.writeException(StreamOutput.java:838)
> at
> org.elasticsearch.transport.TcpTransport.sendErrorResponse(TcpTransport.java:1136)
> at
> org.elasticsearch.transport.TcpTransportChannel.sendResponse(TcpTransportChannel.java:76)
> at
> org.elasticsearch.transport.DelegatingTransportChannel.sendResponse(DelegatingTransportChannel.java:70)
> at
> org.elasticsearch.transport.RequestHandlerRegistry$TransportChannelWrapper.sendResponse(RequestHandlerRegistry.java:123)
> at
> org.elasticsearch.action.support.HandledTransportAction$TransportHandler$1.onFailure(HandledTransportAction.java:77)
> at
> org.elasticsearch.action.search.AbstractSearchAsyncAction.raisePhaseFailure(AbstractSearchAsyncAction.java:220)
> ... 16 more Caused by: java.lang.IllegalArgumentException: Fielddata is
> disabled on text fields by default. Set fielddata=true on [__anonymous_text]
> in order to load fielddata in memory by uninverting the inverted index. Note
> that this can however use significant memory. Alternatively use a keyword
> field instead. at
> org.elasticsearch.index.mapper.TextFieldMapper$TextFieldType.fielddataBuilder(TextFieldMapper.java:336)
> at
> org.elasticsearch.index.fielddata.IndexFieldDataService.getForField(IndexFieldDataService.java:111)
> at
> org.elasticsearch.index.query.QueryShardContext.getForField(QueryShardContext.java:166)
> at
> org.elasticsearch.search.sort.FieldSortBuilder.build(FieldSortBuilder.java:277)
> at org.elasticsearch.search.sort.SortBuilder.buildSort(SortBuilder.java:156)
> at org.elasticsearch.search.SearchService.parseSource(SearchService.java:634)
> at
> org.elasticsearch.search.SearchService.createContext(SearchService.java:485)
> at
> org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:461)
> at
> org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:257)
> at
> org.elasticsearch.action.search.SearchTransportService$6.messageReceived(SearchTransportService.java:340)
> at
> org.elasticsearch.action.search.SearchTransportService$6.messageReceived(SearchTransportService.java:337)
> at
> org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:69)
> at
> org.elasticsearch.transport.TransportService$7.doRun(TransportService.java:644)
> at
> org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:638)
> at
> org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
> ... 3 more }}
> Steps to Replicate
> 1. Spin-up the development environment.
> 2. Open the Alerts UI
> 3. Click on "alert_status" in the table to sort by Alert Status.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
