This is an automated email from the ASF dual-hosted git repository.
epugh pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/solr-mcp.git
The following commit(s) were added to refs/heads/main by this push:
new 621b25a8 refactor(indexing): remove FieldNameSanitizer; use field
names as given (#235)
621b25a8 is described below
commit 621b25a80eda5a5b3c4e6a46f843d5c2534064a6
Author: Aditya Parikh <[email protected]>
AuthorDate: Tue Oct 6 14:32:27 2026 -0400
refactor(indexing): remove FieldNameSanitizer; use field names as given
(#235)
JSON keys and Markdown front-matter keys were lowercased, stripped of
non-word characters and trimmed of underscores before indexing. Solr
does not require any of that: its reference guide only recommends
alphanumeric/underscore names, and says this is not enforced. The
rewriting also did harm and was inconsistent:
- Lowercasing breaks collections with an explicit schema. Solr field
names are case-sensitive, so a key "Title" or "firstName" was
renamed to a field the schema does not define.
- Solr's own schemaless chain (the _default configset) already
replaces characters outside [\w.-]; testSpecialCharactersInFieldNames
passes unchanged because Solr now does the renaming itself.
- CSV and XML field names already reach Solr as given.
Nested JSON objects are still flattened with underscores
("user": {"name": ...} -> user_name), and the indexing response still
lists the indexed names.
Signed-off-by: Aditya Parikh <[email protected]>
Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
---
AGENTS.md | 1 -
dev-docs/ARCHITECTURE.md | 4 +-
docs/FAQ.md | 3 +-
.../solr/mcp/server/indexing/IndexingService.java | 20 ++--
.../documentcreator/FieldNameSanitizer.java | 124 ---------------------
.../documentcreator/IndexingDocumentCreator.java | 2 -
.../documentcreator/JsonDocumentCreator.java | 15 +--
.../documentcreator/MarkdownDocumentCreator.java | 6 +-
.../documentcreator/SolrDocumentCreator.java | 9 +-
.../indexing/IndexingServiceIntegrationTest.java | 119 +++++---------------
.../mcp/server/indexing/MarkdownIndexingTest.java | 6 +-
.../documentcreator/JsonDocumentCreatorTest.java | 2 +-
12 files changed, 49 insertions(+), 262 deletions(-)
diff --git a/AGENTS.md b/AGENTS.md
index 53fd90c6..cbe4cd91 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -110,7 +110,6 @@ in `indexing/documentcreator/`:
- `SolrDocumentCreator` - Common interface
- `JsonDocumentCreator`, `MarkdownDocumentCreator` - Format implementations
(Jackson, commonmark)
- `IndexingDocumentCreator` - Orchestrator that delegates to the
format-specific creator
-- `FieldNameSanitizer` - Automatic field name validation for Solr compatibility
CSV and XML are forwarded unchanged to Solr's own update handlers (`/update`
with
`text/csv` or `application/xml`); the server does not parse them and field
names
diff --git a/dev-docs/ARCHITECTURE.md b/dev-docs/ARCHITECTURE.md
index f379e152..49a30724 100644
--- a/dev-docs/ARCHITECTURE.md
+++ b/dev-docs/ARCHITECTURE.md
@@ -23,7 +23,6 @@ src/main/java/org/apache/solr/mcp/server/
│ ├── JsonDocumentCreator.java # JSON document parser (implements
SolrDocumentCreator)
│ ├── MarkdownDocumentCreator.java # Markdown document parser
(implements SolrDocumentCreator)
│ ├── SolrDocumentCreator.java # Common interface for document
creators
-│ ├── FieldNameSanitizer.java # Field name sanitization utility
│ └── DocumentProcessingException.java # Indexing exceptions
└── collection/ # Collection management functionality
├── CollectionService.java # MCP tools for collection operations
@@ -56,7 +55,7 @@ Spring Boot configuration using properties files:
Server-side parsing exists only for JSON and Markdown, the formats Solr's
update
handlers do not accept directly:
-- Automatically sanitizes field names to comply with Solr schema requirements
+- Uses field names as given, as the CSV and XML pass-throughs do
- Supports nested JSON structures and multi-valued fields
- Delegation via service composition (IndexingDocumentCreator) to the
appropriate format-specific creator
@@ -114,7 +113,6 @@ Additional Jib benefits:
The document creator pattern allows for:
- **Extensibility**: Easy to add new format parsers
- **Testability**: Each creator can be tested independently
-- **Field Sanitization**: Automatic conversion of field names to
Solr-compatible format
- **Explicit Format Selection**: Separate methods for JSON/CSV/XML (no
automatic detection)
### Error Handling
diff --git a/docs/FAQ.md b/docs/FAQ.md
index 9c38e662..c56939db 100644
--- a/docs/FAQ.md
+++ b/docs/FAQ.md
@@ -32,8 +32,7 @@ re-derive it imperfectly every call:
- **Indexing resilience** — 1000-doc batches, single commit, per-doc retry to
salvage valid docs from a failed batch.
-- **Format hardening** — nested-object flattening and field sanitization for
- JSON and markdown; for XML, an **XXE-hardened `<add>`-only pre-check** so
+- **Format hardening** — nested-object flattening for JSON; for XML, an
**XXE-hardened `<add>`-only pre-check** so
`<delete>`/`<commit>` cannot ride in on an indexing call.
- **Metric aggregation** — `get-collection-stats` folds Luke + Metrics APIs,
normalizes shard names, and degrades gracefully on Solr 10.
diff --git
a/src/main/java/org/apache/solr/mcp/server/indexing/IndexingService.java
b/src/main/java/org/apache/solr/mcp/server/indexing/IndexingService.java
index e7230ed3..e5ae85f2 100644
--- a/src/main/java/org/apache/solr/mcp/server/indexing/IndexingService.java
+++ b/src/main/java/org/apache/solr/mcp/server/indexing/IndexingService.java
@@ -72,9 +72,8 @@ import org.springframework.stereotype.Service;
* configurable batch sizes
* <li><strong>Error Resilience</strong>: Individual document fallback when
* batch operations fail
- * <li><strong>Field Sanitization</strong>: Automatic cleanup of JSON and
- * markdown field names for Solr compatibility; CSV and XML field names are
used
- * as given
+ * <li><strong>Field Names</strong>: Used as given in every format; nested JSON
+ * objects are flattened with underscores
* </ul>
*
* <p>
@@ -220,8 +219,7 @@ public class IndexingService {
annotations = @McpTool.McpAnnotations(idempotentHint =
true),
description = "Index documents passed as a JSON array
of objects into Solr collection; one object"
+ " per document, multi-valued fields
as arrays, nested objects flattened with underscores."
- + " Pass the array itself, not a JSON
string. Field names are sanitized for Solr"
- + " compatibility (lowercased, special
characters replaced with underscores); the response"
+ + " Pass the array itself, not a JSON
string. Field names are used as given; the response"
+ " lists the field names as indexed")
public String indexJsonDocuments(@McpToolParam(description = "Solr
collection to index into") String collection,
@McpToolParam(
@@ -339,7 +337,7 @@ public class IndexingService {
*
* <ul>
* <li><strong>YAML Front Matter</strong>: Each entry becomes a
document field
- * with a sanitized name (multi-valued where applicable)
+ * under its own name (multi-valued where applicable)
* <li><strong>title</strong>: From the {@code title} front matter
entry, or the
* first level-1 heading
* <li><strong>headings</strong>: Multi-valued field with the text of
every
@@ -407,11 +405,9 @@ public class IndexingService {
private static final int MAX_REPORTED_FIELDS = 50;
/**
- * Summarizes the field names that were actually indexed. Document
creators
- * sanitize input field names for Solr compatibility (lowercasing,
replacing
- * special characters with underscores), so the indexed names can
differ from
- * the input; reporting them lets MCP clients query the right fields
instead of
- * assuming the input names survived.
+ * Summarizes the field names that were indexed. Nested JSON objects are
+ * flattened with underscores, so the indexed names can differ from the
input;
+ * reporting them lets MCP clients query the right fields.
*
* @param documents
* the documents that were submitted for indexing
@@ -428,7 +424,7 @@ public class IndexingService {
String elided = fieldNames.size() > MAX_REPORTED_FIELDS
? " and " + (fieldNames.size() -
MAX_REPORTED_FIELDS) + " more"
: "";
- return ". Indexed field names (input names are sanitized for
Solr compatibility): " + listed + elided;
+ return ". Indexed field names: " + listed + elided;
}
/**
diff --git
a/src/main/java/org/apache/solr/mcp/server/indexing/documentcreator/FieldNameSanitizer.java
b/src/main/java/org/apache/solr/mcp/server/indexing/documentcreator/FieldNameSanitizer.java
deleted file mode 100644
index 7b636718..00000000
---
a/src/main/java/org/apache/solr/mcp/server/indexing/documentcreator/FieldNameSanitizer.java
+++ /dev/null
@@ -1,124 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.solr.mcp.server.indexing.documentcreator;
-
-import java.util.Locale;
-import java.util.regex.Pattern;
-
-/**
- * Utility class for sanitizing field names to ensure compatibility with Solr's
- * field naming requirements.
- *
- * <p>
- * This class provides shared regex patterns and sanitization logic that can be
- * used across all document creators to ensure consistent field name handling.
- *
- * <p>
- * Solr has specific requirements for field names that must be met to ensure
- * proper indexing and searching functionality. This utility transforms
- * arbitrary field names into Solr-compliant identifiers.
- */
-public final class FieldNameSanitizer {
-
- /**
- * Pattern to match invalid characters in field names. Matches any
character
- * that is not alphanumeric or underscore.
- */
- private static final Pattern INVALID_CHARACTERS_PATTERN =
Pattern.compile("[\\W]");
-
- /**
- * Pattern to match leading and trailing underscores. Uses explicit
grouping to
- * make operator precedence clear.
- */
- private static final Pattern LEADING_TRAILING_UNDERSCORES_PATTERN =
Pattern.compile("(^_+)|(_+$)");
-
- /**
- * Pattern to match multiple consecutive underscores. Matches two or
more
- * consecutive underscores to collapse them into one.
- */
- private static final Pattern MULTIPLE_UNDERSCORES_PATTERN =
Pattern.compile("_{2,}");
-
- // Private constructor to prevent instantiation
- private FieldNameSanitizer() {
- throw new UnsupportedOperationException("Utility class cannot
be instantiated");
- }
-
- /**
- * Sanitizes field names to ensure they are compatible with Solr's
field naming
- * requirements.
- *
- * <p>
- * <strong>Sanitization Rules:</strong>
- *
- * <ul>
- * <li><strong>Case Conversion</strong>: All characters converted to
lowercase
- * <li><strong>Character Replacement</strong>: Non-alphanumeric
characters
- * replaced with underscores
- * <li><strong>Edge Trimming</strong>: Leading and trailing underscores
removed
- * <li><strong>Duplicate Compression</strong>: Multiple consecutive
underscores
- * collapsed to single
- * <li><strong>Numeric Prefix</strong>: Field names starting with
numbers get
- * "field_" prefix
- * </ul>
- *
- * <p>
- * <strong>Example Transformations:</strong>
- *
- * <ul>
- * <li>"User-Name" → "user_name"
- * <li>"product.price" → "product_price"
- * <li>"__field__name__" → "field_name"
- * <li>"Field123@Test" → "field123_test"
- * <li>"123field" → "field_123field"
- * </ul>
- *
- * @param fieldName
- * the original field name to sanitize
- * @return sanitized field name compatible with Solr requirements, or
"field" if
- * the input is empty or sanitizes away to nothing
- * @see <a href=
- *
"https://solr.apache.org/guide/solr/latest/indexing-guide/fields.html";>Solr
- * Field Guide</a>
- */
- public static String sanitizeFieldName(String fieldName) {
-
- // No null guard: this package is @NullMarked and NullAway
enforces the
- // non-null contract at compile time. An empty input falls
through to the
- // isEmpty() check below and yields the default name.
-
- // Convert to lowercase and replace invalid characters with
underscores.
- // Locale.ROOT keeps this deterministic - the default locale
would map
- // 'I' to a dotless 'ı' under a Turkish locale and produce a
different field.
- String sanitized =
INVALID_CHARACTERS_PATTERN.matcher(fieldName.toLowerCase(Locale.ROOT)).replaceAll("_");
-
- // Remove leading/trailing underscores and collapse multiple
underscores
- sanitized =
LEADING_TRAILING_UNDERSCORES_PATTERN.matcher(sanitized).replaceAll("");
- sanitized =
MULTIPLE_UNDERSCORES_PATTERN.matcher(sanitized).replaceAll("_");
-
- // If the result is empty after sanitization, provide a default
name
- if (sanitized.isEmpty()) {
- return "field";
- }
-
- // Ensure the field name doesn't start with a number (Solr
requirement)
- if (Character.isDigit(sanitized.charAt(0))) {
- sanitized = "field_" + sanitized;
- }
-
- return sanitized;
- }
-}
diff --git
a/src/main/java/org/apache/solr/mcp/server/indexing/documentcreator/IndexingDocumentCreator.java
b/src/main/java/org/apache/solr/mcp/server/indexing/documentcreator/IndexingDocumentCreator.java
index 8c12d881..2772b4c2 100644
---
a/src/main/java/org/apache/solr/mcp/server/indexing/documentcreator/IndexingDocumentCreator.java
+++
b/src/main/java/org/apache/solr/mcp/server/indexing/documentcreator/IndexingDocumentCreator.java
@@ -43,8 +43,6 @@ import org.springframework.stereotype.Service;
* documents
* <li><strong>Markdown Processing</strong>: Support for markdown documents
with
* front matter, title, and heading extraction
- * <li><strong>Field Sanitization</strong>: Automatic cleanup of field names
for
- * Solr compatibility
* </ul>
*
* @see SolrInputDocument
diff --git
a/src/main/java/org/apache/solr/mcp/server/indexing/documentcreator/JsonDocumentCreator.java
b/src/main/java/org/apache/solr/mcp/server/indexing/documentcreator/JsonDocumentCreator.java
index e770826c..ec3ad10f 100644
---
a/src/main/java/org/apache/solr/mcp/server/indexing/documentcreator/JsonDocumentCreator.java
+++
b/src/main/java/org/apache/solr/mcp/server/indexing/documentcreator/JsonDocumentCreator.java
@@ -87,14 +87,6 @@ public class JsonDocumentCreator implements
SolrDocumentCreator {
* </ul>
*
* <p>
- * <strong>Field Name Sanitization:</strong>
- *
- * <p>
- * Field names are automatically sanitized to ensure Solr compatibility
by
- * removing special characters and converting to lowercase with
underscore
- * separators.
- *
- * <p>
* <strong>Example Transformations:</strong>
*
* <pre>{@code
@@ -110,7 +102,6 @@ public class JsonDocumentCreator implements
SolrDocumentCreator {
* is invalid
* @see SolrInputDocument
* @see #addAllFieldsFlat(SolrInputDocument, JsonNode, String)
- * @see FieldNameSanitizer#sanitizeFieldName(String)
*/
public List<SolrInputDocument> create(String json) throws
DocumentProcessingException {
if (json.isBlank()) {
@@ -138,7 +129,7 @@ public class JsonDocumentCreator implements
SolrDocumentCreator {
* <p>
* The maps are converted to a {@link JsonNode} tree and handed to the
same
* {@link #flatten(JsonNode)} walk {@link #create(String)} uses, so
both entry
- * points flatten and sanitize identically by construction.
+ * points flatten identically by construction.
*
* @param documents
* the documents, each a map of field name to value
@@ -219,12 +210,10 @@ public class JsonDocumentCreator implements
SolrDocumentCreator {
* @param prefix
* current field name prefix for nested object flattening
* @see #convertJsonValue(JsonNode)
- * @see FieldNameSanitizer#sanitizeFieldName(String)
*/
private void addAllFieldsFlat(SolrInputDocument doc, JsonNode node,
String prefix) {
Set<Map.Entry<String, JsonNode>> fields = node.properties();
- fields.forEach(field -> processFieldValue(doc, field.getValue(),
- FieldNameSanitizer.sanitizeFieldName(prefix +
field.getKey())));
+ fields.forEach(field -> processFieldValue(doc,
field.getValue(), prefix + field.getKey()));
}
/**
diff --git
a/src/main/java/org/apache/solr/mcp/server/indexing/documentcreator/MarkdownDocumentCreator.java
b/src/main/java/org/apache/solr/mcp/server/indexing/documentcreator/MarkdownDocumentCreator.java
index 3cd0b3ad..20e008ca 100644
---
a/src/main/java/org/apache/solr/mcp/server/indexing/documentcreator/MarkdownDocumentCreator.java
+++
b/src/main/java/org/apache/solr/mcp/server/indexing/documentcreator/MarkdownDocumentCreator.java
@@ -52,7 +52,7 @@ import org.springframework.stereotype.Component;
*
* <ul>
* <li><strong>YAML Front Matter</strong>: Each front matter entry becomes a
- * document field with a sanitized name. Entries with multiple values become
+ * document field under its own name. Entries with multiple values become
* multi-valued fields.
* <li><strong>id</strong>: Taken from the {@code id} front matter entry when
* present, otherwise derived deterministically from a SHA-256 hash of the
input
@@ -85,7 +85,6 @@ import org.springframework.stereotype.Component;
* }</pre>
*
* @see SolrInputDocument
- * @see FieldNameSanitizer#sanitizeFieldName(String)
*/
@Component
public class MarkdownDocumentCreator implements SolrDocumentCreator {
@@ -179,10 +178,9 @@ public class MarkdownDocumentCreator implements
SolrDocumentCreator {
document.accept(frontMatterVisitor);
frontMatterVisitor.getData().forEach((key, values) -> {
- String fieldName =
FieldNameSanitizer.sanitizeFieldName(key);
for (String value : flattenFlowSequences(values)) {
if (!value.isEmpty()) {
- doc.addField(fieldName, value);
+ doc.addField(key, value);
}
}
});
diff --git
a/src/main/java/org/apache/solr/mcp/server/indexing/documentcreator/SolrDocumentCreator.java
b/src/main/java/org/apache/solr/mcp/server/indexing/documentcreator/SolrDocumentCreator.java
index 5761fed3..10f07fbe 100644
---
a/src/main/java/org/apache/solr/mcp/server/indexing/documentcreator/SolrDocumentCreator.java
+++
b/src/main/java/org/apache/solr/mcp/server/indexing/documentcreator/SolrDocumentCreator.java
@@ -26,8 +26,7 @@ import org.apache.solr.common.SolrInputDocument;
* <p>
* This interface provides a unified abstraction for converting different
* document formats (JSON, CSV, XML, etc.) into Solr-compatible
- * SolrInputDocument objects. Implementations handle format-specific parsing
and
- * field sanitization to ensure proper Solr indexing.
+ * SolrInputDocument objects. Implementations handle format-specific parsing.
*
* <p>
* <strong>Design Principles:</strong>
@@ -38,8 +37,6 @@ import org.apache.solr.common.SolrInputDocument;
* without predefined schema
* <li><strong>Error Handling</strong>: Consistent exception handling across
* implementations
- * <li><strong>Field Sanitization</strong>: Automatic cleanup of field names
for
- * Solr compatibility
* </ul>
*
* <p>
@@ -47,7 +44,6 @@ import org.apache.solr.common.SolrInputDocument;
*
* <ul>
* <li>Handle null or empty input gracefully
- * <li>Sanitize field names using {@link FieldNameSanitizer}
* <li>Preserve original data types where possible
* <li>Throw {@link DocumentProcessingException} for processing errors
* </ul>
@@ -63,7 +59,6 @@ import org.apache.solr.common.SolrInputDocument;
*
* @see SolrInputDocument
* @see DocumentProcessingException
- * @see FieldNameSanitizer
*/
public interface SolrDocumentCreator {
@@ -79,8 +74,6 @@ public interface SolrDocumentCreator {
* <strong>Processing Behavior:</strong>
*
* <ul>
- * <li><strong>Field Sanitization</strong>: All field names are
sanitized for
- * Solr compatibility
* <li><strong>Type Preservation</strong>: Original data types are
maintained
* where possible
* <li><strong>Multiple Documents</strong>: Single content string may
produce
diff --git
a/src/test/java/org/apache/solr/mcp/server/indexing/IndexingServiceIntegrationTest.java
b/src/test/java/org/apache/solr/mcp/server/indexing/IndexingServiceIntegrationTest.java
index c9f10840..b1b07000 100644
---
a/src/test/java/org/apache/solr/mcp/server/indexing/IndexingServiceIntegrationTest.java
+++
b/src/test/java/org/apache/solr/mcp/server/indexing/IndexingServiceIntegrationTest.java
@@ -145,26 +145,24 @@ class IndexingServiceIntegrationTest {
}
@Test
- void indexJsonDocuments_reportsSanitizedFieldNames() throws Exception {
+ void indexJsonDocuments_reportsFieldNamesAsIndexed() throws Exception {
String json = """
[
{
- "id": "sanitize001",
+ "id": "names001",
"User-Name": "Jane Doe",
- "product.price": 9.99
+ "product": {"price": 9.99}
}
]
""";
String result =
indexingService.indexJsonDocuments(COLLECTION_NAME, TestDocuments.json(json));
- // The response must list the names as indexed, not as
submitted, so MCP
- // clients query the fields that actually exist.
- assertTrue(result.contains("user_name"));
- assertTrue(result.contains("product_price"));
- assertFalse(result.contains("User-Name"));
+ // Names are kept as given; nested objects are flattened with
underscores,
+ // which is why the response lists the indexed names.
+ assertTrue(result.contains("User-Name"), result);
+ assertTrue(result.contains("product_price"), result);
}
-
@Test
void testIndexJsonDocuments() throws Exception {
@@ -346,69 +344,30 @@ class IndexingServiceIntegrationTest {
}
@Test
- void testSanitizeFieldName() throws Exception {
-
- // Test JSON string with field names that need sanitizing
+ void testFieldNamesAreUsedAsGiven() throws Exception {
String json = """
[
{
"id": "test005",
- "invalid-field": "Value with hyphen",
- "another.invalid": "Value with dot",
- "UPPERCASE": "Value with uppercase",
- "multiple__underscores": "Value with
multiple underscores"
+ "first-name": "Value with hyphen",
+ "product.price": "Value with dot",
+ "Title": "Value with uppercase"
}
]
""";
- // Index documents
indexingService.indexJsonDocuments(COLLECTION_NAME,
TestDocuments.json(json));
- // Verify documents were indexed with sanitized field names
SearchResponse result = searchService.search(COLLECTION_NAME,
"id:test005", null, null, null, null, null);
-
- assertNotNull(result);
- List<Map<String, Object>> documents = result.documents();
- assertEquals(1, documents.size());
-
- Map<String, Object> doc = documents.getFirst();
-
- // Check that field names were sanitized
- assertNotNull(doc.get("invalid_field"));
- Object invalidFieldValue = doc.get("invalid_field");
- if (invalidFieldValue instanceof List) {
- assertEquals("Value with hyphen", ((List<?>)
invalidFieldValue).getFirst());
- } else {
- assertEquals("Value with hyphen", invalidFieldValue);
- }
-
- assertNotNull(doc.get("another_invalid"));
- Object anotherInvalidValue = doc.get("another_invalid");
- if (anotherInvalidValue instanceof List) {
- assertEquals("Value with dot", ((List<?>)
anotherInvalidValue).getFirst());
- } else {
- assertEquals("Value with dot", anotherInvalidValue);
- }
-
- // Should be lowercase
- assertNotNull(doc.get("uppercase"));
- Object uppercaseValue = doc.get("uppercase");
- if (uppercaseValue instanceof List) {
- assertEquals("Value with uppercase", ((List<?>)
uppercaseValue).getFirst());
- } else {
- assertEquals("Value with uppercase", uppercaseValue);
- }
-
- // Multiple underscores should be collapsed
- assertNotNull(doc.get("multiple_underscores"));
- Object multipleUnderscoresValue =
doc.get("multiple_underscores");
- if (multipleUnderscoresValue instanceof List) {
- assertEquals("Value with multiple underscores",
((List<?>) multipleUnderscoresValue).getFirst());
- } else {
- assertEquals("Value with multiple underscores",
multipleUnderscoresValue);
- }
+ assertEquals(1, result.documents().size());
+ Map<String, Object> doc = result.documents().getFirst();
+
+ // Solr accepts these names as they are; field names are
case-sensitive
+ assertEquals("Value with hyphen", getFieldValue(doc,
"first-name"));
+ assertEquals("Value with dot", getFieldValue(doc,
"product.price"));
+ assertEquals("Value with uppercase", getFieldValue(doc,
"Title"));
+ assertNull(doc.get("title"));
}
-
@Test
void testDeeplyNestedJsonStructures() throws Exception {
@@ -546,8 +505,8 @@ class IndexingServiceIntegrationTest {
Map<String, Object> doc = documents.getFirst();
- // Check that field names with special characters were sanitized
- // All special characters should be replaced with underscores
+ // The server sends these names as given; Solr's schemaless
update chain
+ // (the _default configset) replaces the special characters
with underscores
assertNotNull(doc.get("field_with_at"));
assertEquals("Value with @ symbols", getFieldValue(doc,
"field_with_at"));
@@ -727,44 +686,26 @@ class IndexingServiceIntegrationTest {
}
@Test
- void testDirectSanitizeFieldName() throws Exception {
- // Test sanitizing field names directly
- // Create a document with field names that need sanitizing
+ void testDirectFieldNamesAreUsedAsGiven() throws Exception {
String json = """
[
{
"id": "field_names_001",
"field-with-hyphens": "Value 1",
- "field.with.dots": "Value 2",
- "field with spaces": "Value 3",
- "UPPERCASE_FIELD": "Value 4",
- "__leading_underscores__": "Value 5",
- "trailing_underscores___": "Value 6",
- "multiple___underscores": "Value 7"
+ "field with spaces": "Value 2",
+ "UPPERCASE_FIELD": "Value 3",
+ "nested": {"Inner-Key": "Value 4"}
}
]
""";
- // Create documents
- List<SolrInputDocument> documents =
indexingDocumentCreator.createSchemalessDocumentsFromJson(json);
+ SolrInputDocument doc =
indexingDocumentCreator.createSchemalessDocumentsFromJson(json).getFirst();
- // Verify documents were created correctly
- assertNotNull(documents);
- assertEquals(1, documents.size());
-
- SolrInputDocument doc = documents.getFirst();
-
- // Verify field names were sanitized correctly
- assertEquals("field_names_001", doc.getFieldValue("id"));
- assertEquals("Value 1",
doc.getFieldValue("field_with_hyphens"));
- assertEquals("Value 2", doc.getFieldValue("field_with_dots"));
- assertEquals("Value 3", doc.getFieldValue("field_with_spaces"));
- assertEquals("Value 4", doc.getFieldValue("uppercase_field"));
- assertEquals("Value 5",
doc.getFieldValue("leading_underscores"));
- assertEquals("Value 6",
doc.getFieldValue("trailing_underscores"));
- assertEquals("Value 7",
doc.getFieldValue("multiple_underscores"));
+ assertEquals("Value 1",
doc.getFieldValue("field-with-hyphens"));
+ assertEquals("Value 2", doc.getFieldValue("field with spaces"));
+ assertEquals("Value 3", doc.getFieldValue("UPPERCASE_FIELD"));
+ assertEquals("Value 4", doc.getFieldValue("nested_Inner-Key"));
}
-
@Test
void
indexCsvDocuments_goesThroughSolrsCsvHandlerWithColumnNamesAsGiven() throws
Exception {
String result =
indexingService.indexCsvDocuments(COLLECTION_NAME, """
diff --git
a/src/test/java/org/apache/solr/mcp/server/indexing/MarkdownIndexingTest.java
b/src/test/java/org/apache/solr/mcp/server/indexing/MarkdownIndexingTest.java
index 2ec3a7c5..ab802c9a 100644
---
a/src/test/java/org/apache/solr/mcp/server/indexing/MarkdownIndexingTest.java
+++
b/src/test/java/org/apache/solr/mcp/server/indexing/MarkdownIndexingTest.java
@@ -153,7 +153,7 @@ class MarkdownIndexingTest {
}
@Test
- void testFrontMatterFieldNamesAreSanitized() throws Exception {
+ void testFrontMatterFieldNamesAreUsedAsGiven() throws Exception {
// Given
String markdown = """
---
@@ -172,8 +172,8 @@ class MarkdownIndexingTest {
assertThat(documents).hasSize(1);
SolrInputDocument doc = documents.getFirst();
- assertThat(doc.getFieldValue("created_by")).isEqualTo("Jane
Doe");
-
assertThat(doc.getFieldValue("last_updated")).isEqualTo("2026-01-01");
+ assertThat(doc.getFieldValue("Created-By")).isEqualTo("Jane
Doe");
+
assertThat(doc.getFieldValue("last.updated")).isEqualTo("2026-01-01");
}
@Test
diff --git
a/src/test/java/org/apache/solr/mcp/server/indexing/documentcreator/JsonDocumentCreatorTest.java
b/src/test/java/org/apache/solr/mcp/server/indexing/documentcreator/JsonDocumentCreatorTest.java
index 3172146f..bb294e66 100644
---
a/src/test/java/org/apache/solr/mcp/server/indexing/documentcreator/JsonDocumentCreatorTest.java
+++
b/src/test/java/org/apache/solr/mcp/server/indexing/documentcreator/JsonDocumentCreatorTest.java
@@ -65,7 +65,7 @@ class JsonDocumentCreatorTest {
SolrInputDocument result =
creator.create(List.of(doc)).getFirst();
assertThat(result.getFieldValue("id")).isEqualTo("1");
-
assertThat(result.getFieldValue("author_name_first")).isEqualTo("Ann");
+
assertThat(result.getFieldValue("Author.Name_first")).isEqualTo("Ann");
assertThat(result.getFieldValues("tags")).containsExactly("a",
"b");
assertThat(result.getFieldNames()).doesNotContain("missing");
}
