Hi Eric,
What a caller sees is that add-field, add-dynamic-field and add-copy-field
no longer reload the core. A document sent after the command returns can
use the new field. In SolrCloud a query sees the field after the next
commit or soft commit. In standalone a query can use it when the command
returns. Delete, replace and mixed requests still reload.
I compared the same standalone Solr 9.9 core, test1, before and after the
change. The check is startTime from the core status. A reload replaces the
core, so startTime changes and uptime goes back to a few seconds.
Before the change:
curl -s "
http://localhost:8983/solr/admin/cores?action=STATUS&core=test1&wt=json";
startTime: 2026-10-22T02:28:55.803Z
uptime: 73 seconds
curl -s "http://localhost:8983/solr/test1/schema"; -H 'Content-type:
application/json' -d '{
"add-field":{"name":"discuss_live_s","type":"string","stored":true}
}'
QTime: 5981
startTime: 2026-10-22T02:30:09.262Z
uptime: 5 seconds
The core was reloaded. On this small core the reload took about 6 seconds.
After the change, same core, restarted from the patched build:
startTime: 2026-10-22T02:33:14.261Z
uptime: 71 seconds
curl -s "http://localhost:8983/solr/test1/schema"; -H 'Content-type:
application/json' -d '{
"add-field":{"name":"discuss_live_s2","type":"string","stored":true}
}'
QTime: 30
startTime: 2026-10-22T02:33:14.261Z
uptime: 71 seconds
startTime did not change. A document using discuss_live_s2 was then indexed
and committed, and select found it (numFound 1).
On a small index the reload this removes is short. It lasts as long as the
merges already running, so it is longer when those merges are expensive,
such as vector fields. TestSchemaChangeDuringMerge holds a merge open
during add-field. That test fails on main and passes with the change.
Thanks,
Li Enlai
This message was drafted with AI assistance and reviewed by me.
On Wed, Oct 21, 2026 at 8:27 PM Eric Pugh <[email protected]> wrote:
>
>
> On 2026/10/16 03:20:25 Enlai Li wrote:
> > Hi Eric,
> >
> > Thanks for looking at this.
> >
> > The cluster reload you describe comes from touchConfDir(). The current
> > Schema API writes the managed schema and then bumps the configset znode,
> > and the listener on that znode reloads every core using the configset.
> >
> > This change does not do that for a request containing only add-field,
> > add-dynamic-field and add-copy-field. It calls persistManagedSchema(),
> > which updates the managed-schema znode and leaves the configset znode
> > alone. The schema file in ZooKeeper is updated. Replicas already watch
> the
> > managed-schema znode in ZkIndexSchemaReader and install it with
> > setLatestSchema(), without a reload. That is the path schemaless mode
> uses,
> > so the replica that received the request and the others behave the same
> way.
> >
> > One change from my first mail: I am no longer opening a searcher on every
> > replica. In SolrCloud, queries see the new field after the next commit or
> > soft commit. Opening one only on the replica that handled the request
> would
> > make a distributed query succeed or fail depending on which replica it
> > hits. Standalone still opens a searcher before responding, which keeps
> > SOLR-13795.
> >
> > On similarity: the IndexWriter keeps the Similarity instance it was given
> > when it was constructed. The default SchemaSimilarityFactory returns a
> > PerFieldSimilarityWrapper that reads core.getLatestSchema() for each
> field,
> > so a new field of an existing field type picks up that type's similarity
> > without a new writer. A custom SimilarityFactory that snapshots its
> > similarities at construction time would not see the new field until the
> > writer is replaced. This change does not cover that case. Delete, replace
> > and mixed requests still reload.
> >
> > The change is one commit on schema-api-additive-no-reload:
> >
> > https://github.com/linslee75/solr/commit/d692a66dbf
> >
> > TestSchemaChangeDuringMerge holds a merge open and checks that an update
> > submitted during add-field returns. On main the same test fails, because
> > add-field reloads the core and the update stays blocked until the merge
> > finishes. TestManagedSchemaAPI checks the cluster case: after add-field,
> > add-dynamic-field and add-copy-field, no replica's core uniqueId changes.
> >
> > Thanks,
> > Li Enlai
> >
> > This message was drafted with AI assistance and reviewed by me.
> >
> > On Fri, Oct 16, 2026 at 1:31 AM Eric Pugh <[email protected]> wrote:
> >
> > >
> > >
> > > On 2026-10-12 12:01 UTC Enlai Li wrote:
> > > > Hi,
> > > >
> > > > Schema API add-field, add-dynamic-field, and add-copy-field currently
> > > > persist the managed schema and reload every core using that
> configset.
> > > > Reload calls newIndexWriter(), which closes the old IndexWriter and
> waits
> > > > for running merges while holding the writer lock. Under continuous
> > > > indexing this blocks updates until those merges finish. Vector fields
> > > make
> > > > the merges expensive, so the stall can last for minutes.
> > > >
> > > > Schemaless mode already adds fields without a reload: persist the
> managed
> > > > schema and install it with setLatestSchema(). Writes see the new
> field
> > > > because the IndexWriter reads the core's latest schema. I would like
> the
> > > > same for these three Schema API operations only. Delete and replace
> > > > operations would still reload.
> > > >
> > > > One difference from schemaless: after installing the schema, open a
> new
> > > > searcher without committing, so queries can use the new field
> > > immediately.
> > > > The current searcher keeps the old schema.
> > > >
> > > > Any objection to this approach?
> > > >
> > > > Thanks,
> > > > Li Enlai
> > > >
> > > > This message was drafted with AI assistance and reviewed by me.
> > > >
> > >
> > > Hi Li,
> > >
> > > I am not an expert in this area, but wanted to chime in to give some
> > > feedback.
> > >
> > > The idea is simpler to do in a single node setup, but if we are
> > > distributed then an issue may be that a Schema API change propagates to
> > > other replicas via a ZK watch on the configset, and that watch handler
> > > currently does an unconditional full core.reload() whenever the schema
> > > version bumps — it doesn't distinguish additive from destructive
> changes.
> > > So unless that path also gets updated, this would only skip the reload
> on
> > > the node that received the API call; every other replica in the
> collection
> > > would still reload and stall the same way. Given the problem is
> > > specifically about continuous indexing across a cluster, that seems
> like
> > > it'd need to be part of the change too.
> > >
> > > Smaller thing: the IndexWriter's analyzer is re-fetched from the
> schema on
> > > every lookup, so it naturally picks up new fields — but Similarity is
> > > captured once at writer-construction time, not re-fetched. Probably
> fine
> > > for most setups, but worth a second look if anyone's using custom
> per-field
> > > similarity. Or something that might need to trigger the full .reload?
> > >
> > > I like the idea of making Solr more robust and friendlier to changing
> > > schemas. Lots of cool use cases where you are getting new fields
> added to
> > > your setup as new data comes in!
> > >
> > > Thanks,
> > > Eric
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: [email protected]
> > > For additional commands, e-mail: [email protected]
> > >
> > >
> >
>
> So.. I guess a little bit is "what does this all mean?" What are the
> ramifications of this change? One thing that helps me sometimes is a
> bash/curl based reproduction script that shows how things work from a end
> user perspective.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [email protected]
> For additional commands, e-mail: [email protected]
>
>
