[ https://issues.apache.org/jira/browse/METRON-1526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16448774#comment-16448774 ]
ASF GitHub Bot commented on METRON-1526: ---------------------------------------- Github user merrimanr commented on the issue: https://github.com/apache/metron/pull/995 To give an idea of how I arrived at the conclusion that other fields are not a problem, this is how I tested it. I will use currency as an example. First I added this to our bro schema: ``` <dynamicField name="*.c" type="currency" indexed="true" stored="true"/> <fieldType name="currency" class="solr.CurrencyField" precisionStep="8" defaultCurrency="USD" currencyConfig="currency.xml" /> ``` I also added the currency.xml file from the example configs that ship with Solr. Then I added that field to the integration test added in this PR (SolrUpdateIntegrationTest.suppress_expanded_fields). It ends up looking like: ``` @Test public void suppress_expanded_fields() throws Exception { dao = new MultiIndexDao(createDao()); dao.init(accessConfig); Map<String, Object> fields = new HashMap<>(); fields.put("guid", "bro_1"); fields.put("source.type", SENSOR_NAME); fields.put("ip_src_port", 8010); fields.put("long_field", 10000); fields.put("latitude", 48.5839); fields.put("score", 10.0); fields.put("is_alert", true); fields.put("field.location_point", "48.5839,7.7455"); fields.put("field.c", "1000,EUR"); Document document = new Document(fields, "bro_1", SENSOR_NAME, 0L); dao.update(document, Optional.of(SENSOR_NAME)); Document indexedDocument = dao.getLatest("bro_1", SENSOR_NAME); // assert no extra expanded fields are included Assert.assertEquals(9, indexedDocument.getDocument().size()); } ``` If we get the same number of fields back that we indexed originally, we know no extra fields are being returned. > Location field types cause DocValuesField appear more than once error > --------------------------------------------------------------------- > > Key: METRON-1526 > URL: https://issues.apache.org/jira/browse/METRON-1526 > Project: Metron > Issue Type: Bug > Reporter: Ryan Merriman > Assignee: Ryan Merriman > Priority: Major > > While testing [https://github.com/apache/metron/pull/970] I get this error > when creating a meta alert: > {code:java} > Error from server at http://10.0.2.15:8983/solr/bro: Exception writing > document id bbc150f5-92f8-485d-93cc-11730c1edf31 to the index; possible > analysis error: DocValuesField > \"enrichments.geo.ip_dst_addr.location_point_0_coordinate\" appears more than > once in this document (only one value is allowed per field){code} > I tracked it down to the fact that multiple fields are returned for a > location field. For example when a field named > "enrichments.geo.ip_dst_addr.location_point" is configured in a schema, these > fields are returned in a query: > {code:java} > { > "enrichments.geo.ip_dst_addr.location_point_0_coordinate": "33.4499", > "enrichments.geo.ip_dst_addr.location_point_1_coordinate": "-112.0712", > "enrichments.geo.ip_dst_addr.location_point": "33.4499,-112.0712" > } > {code} > We need a way to either suppress these extra fields when querying or remove > them before updating a document. -- This message was sent by Atlassian JIRA (v7.6.3#76005)