This is an automated email from the ASF dual-hosted git repository.
epugh pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/solr-mcp.git
The following commit(s) were added to refs/heads/main by this push:
new 589a4b82 feat(indexing): index-json-documents takes a typed documents
array (#202)
589a4b82 is described below
commit 589a4b82ad4302f12c22f07261a23c2d7db23996
Author: Aditya Parikh <[email protected]>
AuthorDate: Wed Sep 16 10:44:41 2026 -0400
feat(indexing): index-json-documents takes a typed documents array (#202)
* feat(indexing): index-json-documents takes a typed documents array
The tool took its documents as a JSON string, so the model had to escape
every quote and newline of the payload inside a JSON string argument. On a
61-document sample that escaping is about 12% of the output tokens, and a
single mis-escaped quote fails the whole call after the payload has been
generated. The parameter is now List<Map<String, Object>>, advertised as an
array of objects, so the model emits native JSON and the SDK parses it once.
JsonDocumentCreator gains a create(List<Map>) entry point that runs the same
flattening as the string path; JsonDocumentCreatorTest pins that both
produce
identical documents. The index-data prompt tells the model to pass the array
itself, and its opening line now names the format rather than the parameter.
Tests keep their JSON text blocks and parse them through TestDocuments.json.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01CiUHyyXLTo9ATdgg8eRFZJ
Signed-off-by: Aditya Parikh <[email protected]>
* docs(indexing): javadoc param name for the documents argument
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01CiUHyyXLTo9ATdgg8eRFZJ
Signed-off-by: Aditya Parikh <[email protected]>
* refactor(indexing): share one JSON flattening path between both entry
points
Cleanup pass over the typed-documents change. No behaviour change beyond
the markdown prompt fix noted below.
JsonDocumentCreator: create(List) was inserted between the create(String)
javadoc and the method it documents, so javac bound only the trailing
block -- create(String) lost its docs and create(List) inherited a
@param json it does not have. Moving it below reunites them. Both entry
points now share flatten(JsonNode)/toDocument(JsonNode) instead of
maintaining parallel loops, so objectsAndStringProduceTheSameDocuments
guards a shared path rather than being the only thing keeping two
hand-written loops in step. create(List) converts the whole list with one
valueToTree call instead of one per document.
Null moves down to the creator, which already owns the "nothing to index"
policy, so it reports null the same way it reports empty. That drops the
one-off IllegalArgumentException from indexJsonDocuments, which none of
the three sibling index tools had.
IndexTool gains the canonical format. The prompt's first %s was
indexTool.paramName(), which worked only because paramName happened to
equal the format keyword; renaming the JSON parameter to `documents`
broke that, and re-normalising the raw argument lost the md -> markdown
canonicalisation, so format=md rendered "You are indexing md data".
Also: createSchemalessDocuments -> createSchemalessDocumentsFromJson, so
the orchestrator keeps its ...From<Format> family; the stale @see on
indexJsonDocuments; TestDocuments catches JsonProcessingException, which
is what readValue declares and what survives Jackson 3, and hoists its
TypeReference; and CollectionServiceIntegrationTest passes the documents
it already holds instead of serialising and reparsing them, retiring a
now-unused autowired ObjectMapper.
Signed-off-by: Aditya Parikh <[email protected]>
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01JZoVjWoHU315uEngMQdhWE
Signed-off-by: Aditya Parikh <[email protected]>
* refactor(indexing): drop the 10 MB input cap from JSON indexing
The typed `documents` array has no string form to measure, so the
`index-json-documents` path was already uncapped. Rather than add a
second limit to match, remove the one on the string entry point so both
JSON paths behave the same way.
Payload size is now bounded by the heap and by whatever limit the
transport in front of the server imposes, not by this creator. CSV and
markdown keep their own caps; this changes JSON only.
No test covered the limit, so nothing else moves.
Signed-off-by: Aditya Parikh <[email protected]>
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01JZoVjWoHU315uEngMQdhWE
Signed-off-by: Aditya Parikh <[email protected]>
* refactor(indexing): drop the 10 MB input caps from CSV and markdown
Follows the JSON change: all three creators carried the same copy-pasted
10 MB constant and the same getBytes() length check, which materialised a
second full copy of the payload purely to measure it, immediately before
the parser read the string again.
Payload size is now bounded by the heap and by whatever limit the
transport in front of the server imposes, not by the creators. The XML
path keeps its own "XML document too large" check in
IndexingDocumentCreator, which is a separate limit with test coverage.
No test covered either cap, so nothing else moves.
Signed-off-by: Aditya Parikh <[email protected]>
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01JZoVjWoHU315uEngMQdhWE
Signed-off-by: Aditya Parikh <[email protected]>
* refactor(indexing): drop the 10 MB XML input limit and its test
Completes the removal across all four formats. The XML limit lived in the
orchestrator rather than in XmlDocumentCreator, and was the last of the
copy-pasted 10 MB checks; like the others it allocated a full byte[] copy
of the payload purely to measure it before the parser read the string.
Removes MAX_XML_SIZE_BYTES, the check, the now-unused StandardCharsets
import, and testCreateSchemalessDocumentsFromXmlWithLargeDocument, which
built a 12 MB document to assert the limit fired. The null/blank
validation on the XML path is unchanged and still covered.
Payload size is now bounded by the heap and by whatever limit the
transport in front of the server imposes.
Signed-off-by: Aditya Parikh <[email protected]>
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01JZoVjWoHU315uEngMQdhWE
Signed-off-by: Aditya Parikh <[email protected]>
---------
Signed-off-by: Aditya Parikh <[email protected]>
Co-authored-by: Claude Fable 5.1 <[email protected]>
---
README.md | 2 +-
.../solr/mcp/server/indexing/IndexingService.java | 52 ++++++----
.../documentcreator/CsvDocumentCreator.java | 8 --
.../documentcreator/IndexingDocumentCreator.java | 25 +++--
.../documentcreator/JsonDocumentCreator.java | 94 ++++++++++++-----
.../documentcreator/MarkdownDocumentCreator.java | 9 +-
.../mcp/server/McpClientIntegrationTestBase.java | 13 ++-
.../org/apache/solr/mcp/server/TestDocuments.java | 54 ++++++++++
.../CollectionServiceIntegrationTest.java | 7 +-
.../ConferenceEndToEndIntegrationTest.java | 3 +-
.../indexing/IndexingServiceIntegrationTest.java | 15 +--
.../mcp/server/indexing/IndexingServiceTest.java | 7 +-
.../solr/mcp/server/indexing/XmlIndexingTest.java | 28 -----
.../documentcreator/JsonDocumentCreatorTest.java | 113 +++++++++++++++++++++
.../observability/OtlpExportIntegrationTest.java | 7 +-
.../search/SearchServiceIntegrationTest.java | 5 +-
16 files changed, 313 insertions(+), 129 deletions(-)
diff --git a/README.md b/README.md
index 0d0c1f7c..1f33223e 100644
--- a/README.md
+++ b/README.md
@@ -96,7 +96,7 @@ Using a different client, or want STDIO/HTTP/Docker options?
See the per-client
| Tool | Description |
|------|-------------|
| `search` | Full-text search with filtering, faceting, sorting, and
pagination |
-| `index-json-documents` | Index documents from a JSON string into a
collection |
+| `index-json-documents` | Index documents passed as a JSON array of objects
into a collection |
| `index-csv-documents` | Index documents from a CSV string into a collection |
| `index-xml-documents` | Index documents from an XML string into a collection
|
| `index-markdown-documents` | Index a markdown document into a collection,
extracting front matter, title, headings, and body text |
diff --git
a/src/main/java/org/apache/solr/mcp/server/indexing/IndexingService.java
b/src/main/java/org/apache/solr/mcp/server/indexing/IndexingService.java
index c186bca6..4a61dd13 100644
--- a/src/main/java/org/apache/solr/mcp/server/indexing/IndexingService.java
+++ b/src/main/java/org/apache/solr/mcp/server/indexing/IndexingService.java
@@ -19,6 +19,7 @@ package org.apache.solr.mcp.server.indexing;
import io.micrometer.observation.annotation.Observed;
import java.io.IOException;
import java.util.List;
+import java.util.Map;
import java.util.Set;
import java.util.TreeSet;
import java.util.stream.Collectors;
@@ -197,8 +198,8 @@ public class IndexingService {
*
* @param collection
* the name of the Solr collection to index documents into
- * @param json
- * JSON string containing an array of documents to index
+ * @param documents
+ * the documents to index, one map per document
* @return a human-readable summary reporting how many documents were
* successfully indexed
* @throws IOException
@@ -206,20 +207,23 @@ public class IndexingService {
* communication
* @throws SolrServerException
* if Solr server encounters errors during indexing
- * @see
IndexingDocumentCreator#createSchemalessDocumentsFromJson(String)
+ * @see IndexingDocumentCreator#createSchemalessDocumentsFromJson(List)
* @see #indexDocuments(String, List)
*/
@PreAuthorize("isAuthenticated()")
@McpTool(
name = "index-json-documents",
annotations = @McpTool.McpAnnotations(idempotentHint =
true),
- description = "Index documents from json String into
Solr collection. Field names are"
- + " sanitized for Solr compatibility
(lowercased, special characters replaced"
- + " with underscores); the response
lists the field names as indexed")
+ description = "Index documents passed as a JSON array
of objects into Solr collection; one object"
+ + " per document, multi-valued fields
as arrays, nested objects flattened with underscores."
+ + " Pass the array itself, not a JSON
string. Field names are sanitized for Solr"
+ + " compatibility (lowercased, special
characters replaced with underscores); the response"
+ + " lists the field names as indexed")
public String indexJsonDocuments(@McpToolParam(description = "Solr
collection to index into") String collection,
- @McpToolParam(description = "JSON string containing
documents to index") String json)
+ @McpToolParam(
+ description = "Documents to index: a
JSON array with one object per document") List<Map<String, Object>> documents)
throws IOException, SolrServerException {
- List<SolrInputDocument> schemalessDoc =
indexingDocumentCreator.createSchemalessDocumentsFromJson(json);
+ List<SolrInputDocument> schemalessDoc =
indexingDocumentCreator.createSchemalessDocumentsFromJson(documents);
int successCount = indexDocuments(collection, schemalessDoc);
return "Successfully indexed " + successCount + " of " +
schemalessDoc.size() + " documents into collection '"
+ collection + "'" +
describeIndexedFields(schemalessDoc);
@@ -611,19 +615,31 @@ public class IndexingService {
}
/**
- * Maps an input-format keyword to the MCP tool and payload parameter
for that
- * format.
+ * Maps an input-format keyword to the canonical format name, the MCP
tool, and
+ * the payload parameter for that format.
+ *
+ * @param format
+ * canonical format name, so the prompt reads "markdown"
even when
+ * the caller passed the {@code md} alias
+ * @param name
+ * the MCP tool that indexes this format
+ * @param paramName
+ * the tool's payload parameter name
+ * @param payload
+ * prose describing what to pass for {@code paramName}
*/
- private record IndexTool(String name, String paramName) {
+ private record IndexTool(String format, String name, String paramName,
String payload) {
}
private static IndexTool resolveIndexTool(String format) {
String normalized = (format == null) ? "" :
format.trim().toLowerCase();
return switch (normalized) {
- case "json" -> new IndexTool("index-json-documents",
"json");
- case "csv" -> new IndexTool("index-csv-documents",
"csv");
- case "xml" -> new IndexTool("index-xml-documents",
"xml");
- case "markdown", "md" -> new
IndexTool("index-markdown-documents", "markdown");
+ case "json" -> new IndexTool("json",
"index-json-documents", "documents",
+ "the documents as a JSON array of
objects, not as a string");
+ case "csv" -> new IndexTool("csv",
"index-csv-documents", "csv", "the CSV text");
+ case "xml" -> new IndexTool("xml",
"index-xml-documents", "xml", "the XML text");
+ case "markdown", "md" ->
+ new IndexTool("markdown",
"index-markdown-documents", "markdown", "the markdown text");
default ->
throw new IllegalArgumentException("format must
be one of json/csv/xml/markdown, got: " + format);
};
@@ -680,7 +696,7 @@ public class IndexingService {
%s
3. Index the documents.
- - Call `%s` with `collection=%s` and
`%s=<the document payload>`.
+ - Call `%s` with `collection=%s` and
`%s=<%s>`.
- The tool batches internally and commits at
the end. The return value is the count
of successfully indexed documents.
- On error, read the message carefully: an
"unknown field" error means the schema is
@@ -694,7 +710,7 @@ public class IndexingService {
Next step suggestion: once data is indexed, the
`search-collection` prompt drives
searching it.
- """.formatted(indexTool.paramName(),
collection, collection, sampleSection, indexTool.name(),
- collection, indexTool.paramName(), collection);
+ """.formatted(indexTool.format(), collection,
collection, sampleSection, indexTool.name(), collection,
+ indexTool.paramName(), indexTool.payload(),
collection);
}
}
diff --git
a/src/main/java/org/apache/solr/mcp/server/indexing/documentcreator/CsvDocumentCreator.java
b/src/main/java/org/apache/solr/mcp/server/indexing/documentcreator/CsvDocumentCreator.java
index 6c0a6642..00ade443 100644
---
a/src/main/java/org/apache/solr/mcp/server/indexing/documentcreator/CsvDocumentCreator.java
+++
b/src/main/java/org/apache/solr/mcp/server/indexing/documentcreator/CsvDocumentCreator.java
@@ -18,7 +18,6 @@ package org.apache.solr.mcp.server.indexing.documentcreator;
import java.io.IOException;
import java.io.StringReader;
-import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.List;
import org.apache.commons.csv.CSVFormat;
@@ -39,8 +38,6 @@ import org.springframework.stereotype.Component;
@Component
public class CsvDocumentCreator implements SolrDocumentCreator {
- private static final int MAX_INPUT_SIZE_BYTES = 10 * 1024 * 1024;
-
/** Default constructor used by Spring to instantiate this component. */
public CsvDocumentCreator() {
}
@@ -112,11 +109,6 @@ public class CsvDocumentCreator implements
SolrDocumentCreator {
if (csv.isBlank()) {
throw new DocumentProcessingException("CSV input cannot
be empty");
}
- if (csv.getBytes(StandardCharsets.UTF_8).length >
MAX_INPUT_SIZE_BYTES) {
- throw new DocumentProcessingException(
- "Input too large: exceeds maximum size
of " + MAX_INPUT_SIZE_BYTES + " bytes");
- }
-
List<SolrInputDocument> documents = new ArrayList<>();
try (CSVParser parser = new CSVParser(new StringReader(csv),
diff --git
a/src/main/java/org/apache/solr/mcp/server/indexing/documentcreator/IndexingDocumentCreator.java
b/src/main/java/org/apache/solr/mcp/server/indexing/documentcreator/IndexingDocumentCreator.java
index d1801714..4dc90089 100644
---
a/src/main/java/org/apache/solr/mcp/server/indexing/documentcreator/IndexingDocumentCreator.java
+++
b/src/main/java/org/apache/solr/mcp/server/indexing/documentcreator/IndexingDocumentCreator.java
@@ -16,8 +16,8 @@
*/
package org.apache.solr.mcp.server.indexing.documentcreator;
-import java.nio.charset.StandardCharsets;
import java.util.List;
+import java.util.Map;
import org.apache.solr.common.SolrInputDocument;
import org.apache.solr.mcp.server.indexing.IndexingService;
import org.springframework.stereotype.Service;
@@ -56,8 +56,6 @@ import org.springframework.stereotype.Service;
@Service
public class IndexingDocumentCreator {
- private static final int MAX_XML_SIZE_BYTES = 10 * 1024 * 1024; // 10MB
limit
-
private final XmlDocumentCreator xmlDocumentCreator;
private final CsvDocumentCreator csvDocumentCreator;
@@ -104,6 +102,21 @@ public class IndexingDocumentCreator {
return jsonDocumentCreator.create(json);
}
+ /**
+ * Creates schema-less documents from already-parsed JSON objects, one
per map.
+ *
+ * @param documents
+ * the documents as field-name-to-value maps
+ * @return list of SolrInputDocument objects ready for indexing
+ * @throws DocumentProcessingException
+ * if the list is {@code null} or empty
+ * @see JsonDocumentCreator#create(List)
+ */
+ public List<SolrInputDocument>
createSchemalessDocumentsFromJson(List<Map<String, Object>> documents)
+ throws DocumentProcessingException {
+ return jsonDocumentCreator.create(documents);
+ }
+
/**
* Creates a list of schema-less SolrInputDocument objects from a CSV
string.
*
@@ -143,12 +156,6 @@ public class IndexingDocumentCreator {
throw new DocumentProcessingException("XML input cannot
be null or empty");
}
- byte[] xmlBytes = xml.getBytes(StandardCharsets.UTF_8);
- if (xmlBytes.length > MAX_XML_SIZE_BYTES) {
- throw new DocumentProcessingException(
- "XML document too large: " +
xmlBytes.length + " bytes (max: " + MAX_XML_SIZE_BYTES + ")");
- }
-
return xmlDocumentCreator.create(xml);
}
diff --git
a/src/main/java/org/apache/solr/mcp/server/indexing/documentcreator/JsonDocumentCreator.java
b/src/main/java/org/apache/solr/mcp/server/indexing/documentcreator/JsonDocumentCreator.java
index 605e5204..e770826c 100644
---
a/src/main/java/org/apache/solr/mcp/server/indexing/documentcreator/JsonDocumentCreator.java
+++
b/src/main/java/org/apache/solr/mcp/server/indexing/documentcreator/JsonDocumentCreator.java
@@ -19,7 +19,6 @@ package org.apache.solr.mcp.server.indexing.documentcreator;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import java.io.IOException;
-import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
@@ -39,8 +38,6 @@ import org.springframework.stereotype.Component;
@Component
public class JsonDocumentCreator implements SolrDocumentCreator {
- private static final int MAX_INPUT_SIZE_BYTES = 10 * 1024 * 1024;
-
private final ObjectMapper objectMapper;
/**
@@ -119,40 +116,81 @@ public class JsonDocumentCreator implements
SolrDocumentCreator {
if (json.isBlank()) {
throw new DocumentProcessingException("JSON input
cannot be empty");
}
- if (json.getBytes(StandardCharsets.UTF_8).length >
MAX_INPUT_SIZE_BYTES) {
- throw new DocumentProcessingException(
- "Input too large: exceeds maximum size
of " + MAX_INPUT_SIZE_BYTES + " bytes");
- }
-
- List<SolrInputDocument> documents = new ArrayList<>();
+ JsonNode rootNode;
try {
- JsonNode rootNode = this.objectMapper.readTree(json);
-
- if (rootNode.isArray()) {
- for (JsonNode item : rootNode) {
- SolrInputDocument doc = new
SolrInputDocument();
-
- // Add all fields without type suffixes
- let Solr figure it out
- addAllFieldsFlat(doc, item, "");
- documents.add(doc);
- }
- } else if (rootNode.isObject()) {
- // A single document. Previously fell through
and returned an empty
- // list, so indexing one object silently
indexed nothing.
- SolrInputDocument doc = new SolrInputDocument();
- addAllFieldsFlat(doc, rootNode, "");
- documents.add(doc);
- } else {
- throw new DocumentProcessingException("JSON
input must be an object or an array of objects");
- }
+ rootNode = this.objectMapper.readTree(json);
} catch (IOException e) {
throw new DocumentProcessingException("Failed to parse
JSON document", e);
}
+ if (!rootNode.isArray() && !rootNode.isObject()) {
+ throw new DocumentProcessingException("JSON input must
be an object or an array of objects");
+ }
+ return flatten(rootNode);
+ }
+
+ /**
+ * Creates schema-less documents from already-parsed JSON objects, one
document
+ * per map. This is the entry point for the {@code
index-json-documents} tool,
+ * whose {@code documents} argument is a typed JSON array: the MCP
client parses
+ * it, so the model emits native JSON instead of JSON escaped inside a
string.
+ *
+ * <p>
+ * The maps are converted to a {@link JsonNode} tree and handed to the
same
+ * {@link #flatten(JsonNode)} walk {@link #create(String)} uses, so
both entry
+ * points flatten and sanitize identically by construction.
+ *
+ * @param documents
+ * the documents, each a map of field name to value
+ * @return list of SolrInputDocument objects ready for indexing
+ * @throws DocumentProcessingException
+ * if the list is {@code null} or empty
+ */
+ public List<SolrInputDocument> create(List<Map<String, Object>>
documents) throws DocumentProcessingException {
+ if (documents == null || documents.isEmpty()) {
+ throw new DocumentProcessingException("JSON input
cannot be empty");
+ }
+ return flatten(objectMapper.valueToTree(documents));
+ }
+ /**
+ * Builds one document per element of an array node, or a single
document from
+ * an object node.
+ *
+ * <p>
+ * A lone object is a single document: it previously fell through and
returned
+ * an empty list, so indexing one object silently indexed nothing.
+ *
+ * @param rootNode
+ * an array of document objects, or one document object
+ * @return list of SolrInputDocument objects ready for indexing
+ */
+ private List<SolrInputDocument> flatten(JsonNode rootNode) {
+ List<SolrInputDocument> documents = new
ArrayList<>(rootNode.size());
+ if (rootNode.isArray()) {
+ for (JsonNode item : rootNode) {
+ documents.add(toDocument(item));
+ }
+ } else {
+ documents.add(toDocument(rootNode));
+ }
return documents;
}
+ /**
+ * Flattens one JSON object node into a SolrInputDocument. Field names
are added
+ * without type suffixes; Solr infers the types.
+ *
+ * @param node
+ * the JSON object to flatten
+ * @return the document, with nested objects flattened and arrays
multi-valued
+ */
+ private SolrInputDocument toDocument(JsonNode node) {
+ SolrInputDocument doc = new SolrInputDocument();
+ addAllFieldsFlat(doc, node, "");
+ return doc;
+ }
+
/**
* Recursively flattens JSON nodes and adds them as fields to a
* SolrInputDocument.
diff --git
a/src/main/java/org/apache/solr/mcp/server/indexing/documentcreator/MarkdownDocumentCreator.java
b/src/main/java/org/apache/solr/mcp/server/indexing/documentcreator/MarkdownDocumentCreator.java
index db560202..3cd0b3ad 100644
---
a/src/main/java/org/apache/solr/mcp/server/indexing/documentcreator/MarkdownDocumentCreator.java
+++
b/src/main/java/org/apache/solr/mcp/server/indexing/documentcreator/MarkdownDocumentCreator.java
@@ -90,8 +90,6 @@ import org.springframework.stereotype.Component;
@Component
public class MarkdownDocumentCreator implements SolrDocumentCreator {
- private static final int MAX_INPUT_SIZE_BYTES = 10 * 1024 * 1024;
-
/** Solr field holding the document's unique key. */
public static final String FIELD_ID = "id";
@@ -128,15 +126,10 @@ public class MarkdownDocumentCreator implements
SolrDocumentCreator {
* @return a single-element list containing the created document, or an
empty
* list if the input is blank
* @throws DocumentProcessingException
- * if the input exceeds the size limit or parsing fails
+ * if parsing fails
*/
@Override
public List<SolrInputDocument> create(String markdown) throws
DocumentProcessingException {
- if (markdown.getBytes(StandardCharsets.UTF_8).length >
MAX_INPUT_SIZE_BYTES) {
- throw new DocumentProcessingException(
- "Input too large: exceeds maximum size
of " + MAX_INPUT_SIZE_BYTES + " bytes");
- }
-
if (markdown.trim().isEmpty()) {
return List.of();
}
diff --git
a/src/test/java/org/apache/solr/mcp/server/McpClientIntegrationTestBase.java
b/src/test/java/org/apache/solr/mcp/server/McpClientIntegrationTestBase.java
index 3b9f2302..0a2a4c8d 100644
--- a/src/test/java/org/apache/solr/mcp/server/McpClientIntegrationTestBase.java
+++ b/src/test/java/org/apache/solr/mcp/server/McpClientIntegrationTestBase.java
@@ -192,8 +192,8 @@ public abstract class McpClientIntegrationTestBase {
]
""";
- CallToolResult result = mcpClient
- .callTool(new
CallToolRequest("index-json-documents", Map.of("collection", COLLECTION,
"json", json)));
+ CallToolResult result = mcpClient.callTool(new
CallToolRequest("index-json-documents",
+ Map.of("collection", COLLECTION, "documents",
TestDocuments.json(json))));
assertNotNull(result);
assertNotError(result);
@@ -377,8 +377,8 @@ public abstract class McpClientIntegrationTestBase {
]
""";
- CallToolResult result = mcpClient
- .callTool(new
CallToolRequest("index-json-documents", Map.of("collection", COLLECTION,
"json", json)));
+ CallToolResult result = mcpClient.callTool(new
CallToolRequest("index-json-documents",
+ Map.of("collection", COLLECTION, "documents",
TestDocuments.json(json))));
assertNotNull(result);
assertNotError(result);
@@ -539,8 +539,8 @@ public abstract class McpClientIntegrationTestBase {
String showsJson = loadClasspathResource("/shows.json");
assertFalse(showsJson.isBlank(), "shows.json resource must not
be blank");
- CallToolResult result = mcpClient.callTool(
- new CallToolRequest("index-json-documents",
Map.of("collection", SHOWS_COLLECTION, "json", showsJson)));
+ CallToolResult result = mcpClient.callTool(new
CallToolRequest("index-json-documents",
+ Map.of("collection", SHOWS_COLLECTION,
"documents", TestDocuments.json(showsJson))));
assertNotNull(result);
assertNotError(result);
@@ -877,5 +877,4 @@ public abstract class McpClientIntegrationTestBase {
assertNotNull(value, "documents should be present in response");
return (List<Map<String, Object>>) value;
}
-
}
diff --git a/src/test/java/org/apache/solr/mcp/server/TestDocuments.java
b/src/test/java/org/apache/solr/mcp/server/TestDocuments.java
new file mode 100644
index 00000000..1f3c224e
--- /dev/null
+++ b/src/test/java/org/apache/solr/mcp/server/TestDocuments.java
@@ -0,0 +1,54 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.mcp.server;
+
+import com.fasterxml.jackson.core.JsonProcessingException;
+import com.fasterxml.jackson.core.type.TypeReference;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import java.util.List;
+import java.util.Map;
+
+/**
+ * Turns a JSON array literal into the typed {@code documents} argument of the
+ * {@code index-json-documents} tool, so tests can keep readable JSON text
+ * blocks while the tool takes parsed objects.
+ */
+public final class TestDocuments {
+
+ private static final ObjectMapper OBJECT_MAPPER = new ObjectMapper();
+
+ private static final TypeReference<List<Map<String, Object>>> DOCUMENTS
= new TypeReference<>() {
+ };
+
+ private TestDocuments() {
+ }
+
+ /**
+ * Parses a JSON array of objects.
+ *
+ * @param json
+ * a JSON array literal
+ * @return one map per document
+ */
+ public static List<Map<String, Object>> json(String json) {
+ try {
+ return OBJECT_MAPPER.readValue(json, DOCUMENTS);
+ } catch (JsonProcessingException e) {
+ throw new IllegalArgumentException("not a JSON array of
objects: " + json, e);
+ }
+ }
+}
diff --git
a/src/test/java/org/apache/solr/mcp/server/collection/CollectionServiceIntegrationTest.java
b/src/test/java/org/apache/solr/mcp/server/collection/CollectionServiceIntegrationTest.java
index 6560af9d..300f0198 100644
---
a/src/test/java/org/apache/solr/mcp/server/collection/CollectionServiceIntegrationTest.java
+++
b/src/test/java/org/apache/solr/mcp/server/collection/CollectionServiceIntegrationTest.java
@@ -22,7 +22,6 @@ import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertNull;
import static org.junit.jupiter.api.Assertions.assertTrue;
-import com.fasterxml.jackson.databind.ObjectMapper;
import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.List;
@@ -64,9 +63,6 @@ class CollectionServiceIntegrationTest {
@Autowired
private SearchService searchService;
- @Autowired
- private ObjectMapper objectMapper;
-
@BeforeAll
void setupCollectionWithData() throws Exception {
// 1. Create collection via CollectionService MCP tool
@@ -84,8 +80,7 @@ class CollectionServiceIntegrationTest {
doc.put("count_i", i);
docs.add(doc);
}
- String json = objectMapper.writeValueAsString(docs);
- indexingService.indexJsonDocuments(TEST_COLLECTION, json);
+ indexingService.indexJsonDocuments(TEST_COLLECTION, docs);
log.debug("Indexed {} documents via IndexingService",
DOC_COUNT);
// 3. Run searches via SearchService MCP tool to populate
caches and handler
diff --git
a/src/test/java/org/apache/solr/mcp/server/collection/ConferenceEndToEndIntegrationTest.java
b/src/test/java/org/apache/solr/mcp/server/collection/ConferenceEndToEndIntegrationTest.java
index ca9cb7d0..5ba65a11 100644
---
a/src/test/java/org/apache/solr/mcp/server/collection/ConferenceEndToEndIntegrationTest.java
+++
b/src/test/java/org/apache/solr/mcp/server/collection/ConferenceEndToEndIntegrationTest.java
@@ -22,6 +22,7 @@ import java.nio.file.Files;
import java.nio.file.Path;
import java.util.List;
import java.util.Map;
+import org.apache.solr.mcp.server.TestDocuments;
import org.apache.solr.mcp.server.TestcontainersConfiguration;
import org.apache.solr.mcp.server.indexing.IndexingService;
import org.apache.solr.mcp.server.search.SearchResponse;
@@ -69,7 +70,7 @@ class ConferenceEndToEndIntegrationTest {
assertTrue(result.success(), "Collection creation should
succeed: " + result.message());
String json =
Files.readString(Path.of("mydata/devnexus-2026.json"));
- indexingService.indexJsonDocuments(COLLECTION, json);
+ indexingService.indexJsonDocuments(COLLECTION,
TestDocuments.json(json));
}
@Test
diff --git
a/src/test/java/org/apache/solr/mcp/server/indexing/IndexingServiceIntegrationTest.java
b/src/test/java/org/apache/solr/mcp/server/indexing/IndexingServiceIntegrationTest.java
index ae291468..739a3190 100644
---
a/src/test/java/org/apache/solr/mcp/server/indexing/IndexingServiceIntegrationTest.java
+++
b/src/test/java/org/apache/solr/mcp/server/indexing/IndexingServiceIntegrationTest.java
@@ -23,6 +23,7 @@ import java.util.Map;
import org.apache.solr.client.solrj.SolrClient;
import org.apache.solr.client.solrj.request.CollectionAdminRequest;
import org.apache.solr.common.SolrInputDocument;
+import org.apache.solr.mcp.server.TestDocuments;
import org.apache.solr.mcp.server.TestcontainersConfiguration;
import org.apache.solr.mcp.server.indexing.documentcreator.CsvDocumentCreator;
import
org.apache.solr.mcp.server.indexing.documentcreator.IndexingDocumentCreator;
@@ -171,7 +172,7 @@ class IndexingServiceIntegrationTest {
]
""";
- String result =
indexingService.indexJsonDocuments(COLLECTION_NAME, json);
+ String result =
indexingService.indexJsonDocuments(COLLECTION_NAME, TestDocuments.json(json));
// The response must list the names as indexed, not as
submitted, so MCP
// clients query the fields that actually exist.
@@ -208,7 +209,7 @@ class IndexingServiceIntegrationTest {
""";
// Index documents
- indexingService.indexJsonDocuments(COLLECTION_NAME, json);
+ indexingService.indexJsonDocuments(COLLECTION_NAME,
TestDocuments.json(json));
// Verify documents were indexed by searching for them
SearchResponse result = searchService.search(COLLECTION_NAME,
"id:test002 OR id:test003", null, null, null,
@@ -313,7 +314,7 @@ class IndexingServiceIntegrationTest {
""";
// Index documents
- indexingService.indexJsonDocuments(COLLECTION_NAME, json);
+ indexingService.indexJsonDocuments(COLLECTION_NAME,
TestDocuments.json(json));
// Verify documents were indexed by searching for them
SearchResponse result = searchService.search(COLLECTION_NAME,
"id:test004", null, null, null, null, null);
@@ -377,7 +378,7 @@ class IndexingServiceIntegrationTest {
""";
// Index documents
- indexingService.indexJsonDocuments(COLLECTION_NAME, json);
+ indexingService.indexJsonDocuments(COLLECTION_NAME,
TestDocuments.json(json));
// Verify documents were indexed with sanitized field names
SearchResponse result = searchService.search(COLLECTION_NAME,
"id:test005", null, null, null, null, null);
@@ -469,7 +470,7 @@ class IndexingServiceIntegrationTest {
""";
// Index documents
- indexingService.indexJsonDocuments(COLLECTION_NAME, json);
+ indexingService.indexJsonDocuments(COLLECTION_NAME,
TestDocuments.json(json));
// Verify documents were indexed by searching for them
SearchResponse result = searchService.search(COLLECTION_NAME,
"id:nested001", null, null, null, null, null);
@@ -549,7 +550,7 @@ class IndexingServiceIntegrationTest {
""";
// Index documents
- indexingService.indexJsonDocuments(COLLECTION_NAME, json);
+ indexingService.indexJsonDocuments(COLLECTION_NAME,
TestDocuments.json(json));
// Verify documents were indexed by searching for them
SearchResponse result = searchService.search(COLLECTION_NAME,
"id:special_fields_001", null, null, null, null,
@@ -638,7 +639,7 @@ class IndexingServiceIntegrationTest {
""";
// Index documents
- indexingService.indexJsonDocuments(COLLECTION_NAME, json);
+ indexingService.indexJsonDocuments(COLLECTION_NAME,
TestDocuments.json(json));
// Verify documents were indexed by searching for them
SearchResponse result = searchService.search(COLLECTION_NAME,
"id:array_objects_001", null, null, null, null,
diff --git
a/src/test/java/org/apache/solr/mcp/server/indexing/IndexingServiceTest.java
b/src/test/java/org/apache/solr/mcp/server/indexing/IndexingServiceTest.java
index 9b94a0be..9b633742 100644
--- a/src/test/java/org/apache/solr/mcp/server/indexing/IndexingServiceTest.java
+++ b/src/test/java/org/apache/solr/mcp/server/indexing/IndexingServiceTest.java
@@ -24,6 +24,7 @@ import java.io.IOException;
import java.util.ArrayList;
import java.util.Collection;
import java.util.List;
+import java.util.Map;
import org.apache.solr.client.solrj.SolrClient;
import org.apache.solr.client.solrj.SolrServerException;
import org.apache.solr.common.SolrInputDocument;
@@ -63,7 +64,7 @@ class IndexingServiceTest {
@Test
void indexJsonDocuments_WithValidJson_ShouldIndexDocuments() throws
Exception {
- String json = "[{\"id\":\"1\",\"title\":\"Test\"}]";
+ List<Map<String, Object>> json = List.of(Map.of("id", "1",
"title", "Test"));
List<SolrInputDocument> mockDocs = createMockDocuments(1);
when(indexingDocumentCreator.createSchemalessDocumentsFromJson(json)).thenReturn(mockDocs);
when(solrClient.add(eq("test_collection"),
any(Collection.class))).thenReturn(null);
@@ -78,7 +79,7 @@ class IndexingServiceTest {
@Test
void
indexJsonDocuments_WhenDocumentCreatorThrowsException_ShouldPropagateException()
throws Exception {
- String invalidJson = "not valid json";
+ List<Map<String, Object>> invalidJson = List.of();
when(indexingDocumentCreator.createSchemalessDocumentsFromJson(invalidJson)).thenThrow(
new
org.apache.solr.mcp.server.indexing.documentcreator.DocumentProcessingException("Invalid
JSON"));
@@ -279,7 +280,7 @@ class IndexingServiceTest {
@Test
void
indexJsonDocuments_WhenSolrClientThrowsException_ShouldPropagateException()
throws Exception {
- String json = "[{\"id\":\"1\"}]";
+ List<Map<String, Object>> json = List.of(Map.of("id", "1"));
List<SolrInputDocument> mockDocs = createMockDocuments(1);
when(indexingDocumentCreator.createSchemalessDocumentsFromJson(json)).thenReturn(mockDocs);
when(solrClient.add(eq("test_collection"), any(List.class)))
diff --git
a/src/test/java/org/apache/solr/mcp/server/indexing/XmlIndexingTest.java
b/src/test/java/org/apache/solr/mcp/server/indexing/XmlIndexingTest.java
index 35129e8a..63927f39 100644
--- a/src/test/java/org/apache/solr/mcp/server/indexing/XmlIndexingTest.java
+++ b/src/test/java/org/apache/solr/mcp/server/indexing/XmlIndexingTest.java
@@ -368,34 +368,6 @@ class XmlIndexingTest {
.hasMessageContaining("XML input cannot be null
or empty");
}
- @Test
- void testCreateSchemalessDocumentsFromXmlWithLargeDocument() {
- // Given
-
- // Create a large XML document (over 10MB)
- StringBuilder largeXml = new StringBuilder();
- largeXml.append("<books>");
-
- // Add enough data to exceed the 10MB limit
- String bookTemplate = """
- <book id="%d">
- <title>%s</title>
- <content>%s</content>
- </book>
- """;
-
- // Create approximately 11MB of XML data
- String longContent = "A".repeat(10000); // 10KB per book
- for (int i = 0; i < 1200; i++) { // 1200 * 10KB = 12MB
- largeXml.append(String.format(bookTemplate, i, "Title "
+ i, longContent));
- }
- largeXml.append("</books>");
-
- // When/Then
- assertThatThrownBy(() ->
indexingDocumentCreator.createSchemalessDocumentsFromXml(largeXml.toString()))
-
.isInstanceOf(DocumentProcessingException.class).hasMessageContaining("XML
document too large");
- }
-
@Test
void testCreateSchemalessDocumentsFromXmlWithComplexNestedStructure()
throws Exception {
// Given
diff --git
a/src/test/java/org/apache/solr/mcp/server/indexing/documentcreator/JsonDocumentCreatorTest.java
b/src/test/java/org/apache/solr/mcp/server/indexing/documentcreator/JsonDocumentCreatorTest.java
new file mode 100644
index 00000000..3172146f
--- /dev/null
+++
b/src/test/java/org/apache/solr/mcp/server/indexing/documentcreator/JsonDocumentCreatorTest.java
@@ -0,0 +1,113 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.mcp.server.indexing.documentcreator;
+
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.assertj.core.api.Assertions.assertThatThrownBy;
+
+import com.fasterxml.jackson.databind.ObjectMapper;
+import java.util.ArrayList;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.TreeMap;
+import org.apache.solr.common.SolrInputDocument;
+import org.junit.jupiter.api.Test;
+
+/**
+ * The JSON creator has two entry points: a JSON string (files, tests) and a
+ * list of already-parsed objects (the {@code index-json-documents} tool, whose
+ * {@code documents} argument is a typed JSON array so the model never has to
+ * escape JSON inside a string). Both must produce identical documents.
+ */
+class JsonDocumentCreatorTest {
+
+ private final ObjectMapper objectMapper = new ObjectMapper();
+
+ private final JsonDocumentCreator creator = new
JsonDocumentCreator(objectMapper);
+
+ @Test
+ void objectsAndStringProduceTheSameDocuments() throws Exception {
+ List<Map<String, Object>> objects = List.of(
+ show("netflix-001", "Stranger Things", 8.7,
true, "Sci-Fi", "Horror"),
+ show("hbo-001", "Game of Thrones", 9.2, false,
"Fantasy"));
+ String json = objectMapper.writeValueAsString(objects);
+
+ List<SolrInputDocument> fromObjects = creator.create(objects);
+ List<SolrInputDocument> fromString = creator.create(json);
+
+ assertThat(fromObjects).hasSize(2);
+ assertThat(fields(fromObjects)).isEqualTo(fields(fromString));
+ }
+
+ @Test
+ void nestedObjectsFlattenAndArraysBecomeMultiValued() {
+ Map<String, Object> doc = new LinkedHashMap<>();
+ doc.put("id", "1");
+ doc.put("Author.Name", Map.of("first", "Ann", "last", "Lee"));
+ doc.put("tags", List.of("a", "b"));
+ doc.put("missing", null);
+
+ SolrInputDocument result =
creator.create(List.of(doc)).getFirst();
+
+ assertThat(result.getFieldValue("id")).isEqualTo("1");
+
assertThat(result.getFieldValue("author_name_first")).isEqualTo("Ann");
+ assertThat(result.getFieldValues("tags")).containsExactly("a",
"b");
+ assertThat(result.getFieldNames()).doesNotContain("missing");
+ }
+
+ @Test
+ void emptyListIsRejected() {
+ assertThatThrownBy(() ->
creator.create(List.of())).isInstanceOf(DocumentProcessingException.class)
+ .hasMessage("JSON input cannot be empty");
+ }
+
+ /**
+ * A missing {@code documents} argument arrives as null. The creator
owns the
+ * "nothing to index" policy for both entry points, so it reports null
the same
+ * way it reports empty rather than letting the service NPE.
+ */
+ @Test
+ void nullListIsRejected() {
+ assertThatThrownBy(() -> creator.create((List<Map<String,
Object>>) null))
+
.isInstanceOf(DocumentProcessingException.class).hasMessage("JSON input cannot
be empty");
+ }
+
+ private static Map<String, Object> show(String id, String title, double
rating, boolean ongoing, String... genres) {
+ Map<String, Object> show = new LinkedHashMap<>();
+ show.put("id", id);
+ show.put("title", title);
+ show.put("imdb_rating", rating);
+ show.put("ongoing", ongoing);
+ show.put("seasons", 3);
+ show.put("genres", List.of(genres));
+ show.put("end_year", null);
+ return show;
+ }
+
+ private static List<Map<String, List<String>>>
fields(List<SolrInputDocument> docs) {
+ List<Map<String, List<String>>> all = new ArrayList<>();
+ for (SolrInputDocument doc : docs) {
+ Map<String, List<String>> fields = new TreeMap<>();
+ for (String name : doc.getFieldNames()) {
+ fields.put(name,
doc.getFieldValues(name).stream().map(String::valueOf).toList());
+ }
+ all.add(fields);
+ }
+ return all;
+ }
+}
diff --git
a/src/test/java/org/apache/solr/mcp/server/observability/OtlpExportIntegrationTest.java
b/src/test/java/org/apache/solr/mcp/server/observability/OtlpExportIntegrationTest.java
index 2001521f..894fae0e 100644
---
a/src/test/java/org/apache/solr/mcp/server/observability/OtlpExportIntegrationTest.java
+++
b/src/test/java/org/apache/solr/mcp/server/observability/OtlpExportIntegrationTest.java
@@ -24,6 +24,7 @@ import com.fasterxml.jackson.databind.ObjectMapper;
import java.util.concurrent.TimeUnit;
import org.apache.solr.client.solrj.SolrClient;
import org.apache.solr.client.solrj.request.CollectionAdminRequest;
+import org.apache.solr.mcp.server.TestDocuments;
import org.apache.solr.mcp.server.TestcontainersConfiguration;
import org.apache.solr.mcp.server.indexing.IndexingService;
import org.apache.solr.mcp.server.search.SearchService;
@@ -147,7 +148,7 @@ class OtlpExportIntegrationTest {
// When: We perform operations that create spans
// Then: Operations should execute without throwing exceptions
- indexingService.indexJsonDocuments(COLLECTION_NAME, testData);
+ indexingService.indexJsonDocuments(COLLECTION_NAME,
TestDocuments.json(testData));
solrClient.commit(COLLECTION_NAME);
searchService.search(COLLECTION_NAME, "*:*", null, null, null,
null, null);
@@ -176,7 +177,7 @@ class OtlpExportIntegrationTest {
""";
// Then: All operations should succeed
- indexingService.indexJsonDocuments(COLLECTION_NAME, testData);
+ indexingService.indexJsonDocuments(COLLECTION_NAME,
TestDocuments.json(testData));
solrClient.commit(COLLECTION_NAME);
// Verify we can search for the documents
@@ -190,7 +191,7 @@ class OtlpExportIntegrationTest {
String testData = """
[{"id": "metrics_test_1", "name": "Metrics
Test"}]
""";
- indexingService.indexJsonDocuments(COLLECTION_NAME, testData);
+ indexingService.indexJsonDocuments(COLLECTION_NAME,
TestDocuments.json(testData));
solrClient.commit(COLLECTION_NAME);
searchService.search(COLLECTION_NAME, "*:*", null, null, null,
null, null);
diff --git
a/src/test/java/org/apache/solr/mcp/server/search/SearchServiceIntegrationTest.java
b/src/test/java/org/apache/solr/mcp/server/search/SearchServiceIntegrationTest.java
index af19b907..c7f7d7c0 100644
---
a/src/test/java/org/apache/solr/mcp/server/search/SearchServiceIntegrationTest.java
+++
b/src/test/java/org/apache/solr/mcp/server/search/SearchServiceIntegrationTest.java
@@ -27,6 +27,7 @@ import org.apache.solr.client.solrj.SolrClient;
import org.apache.solr.client.solrj.SolrServerException;
import org.apache.solr.client.solrj.request.CollectionAdminRequest;
import org.apache.solr.common.SolrException;
+import org.apache.solr.mcp.server.TestDocuments;
import org.apache.solr.mcp.server.TestcontainersConfiguration;
import org.apache.solr.mcp.server.indexing.IndexingService;
import org.junit.jupiter.api.BeforeEach;
@@ -170,7 +171,7 @@ class SearchServiceIntegrationTest {
]
""";
- indexingService.indexJsonDocuments(COLLECTION_NAME,
sampleData);
+ indexingService.indexJsonDocuments(COLLECTION_NAME,
TestDocuments.json(sampleData));
solrClient.commit(COLLECTION_NAME);
initialized = true;
}
@@ -492,7 +493,7 @@ class SearchServiceIntegrationTest {
}
]
""";
- indexingService.indexJsonDocuments(COLLECTION_NAME,
specialJson);
+ indexingService.indexJsonDocuments(COLLECTION_NAME,
TestDocuments.json(specialJson));
solrClient.commit(COLLECTION_NAME);
String query = "id:special001";
SearchResponse result = searchService.search(COLLECTION_NAME,
query, null, null, null, null, null);
