Hi Sergiu,
Are you using Jackrabbit Oak underneath?
If yes, Oak has a limit on how many observation event it can keep in memory. Once it goes beyond
that limit it will collapse events into an external event that looses individual commit granularity.
That could be one explanation that would come to mind for what you are
describing.
Cheers,
Stefan
On 24.02.23 16:35, Sergiu Dumitriu wrote:
We noticed that under heavy load a ResourceChangeListener doesn't receive notifications about all
the changes in the repository. Is this expected behavior?
More details:
We have a batch import scheduled job that pulls in a few thousand records from an external database
and creates a new node for each of them (a Visit node). For each of the new nodes, we have a
ResourceChangeListener that creates additional nodes as needed (one or two Surveys associated with
the Visit), and changes the Visit node to mark that it has surveys. All the new Visit nodes
correctly trigger a ResourceChange event which is correctly processed by the survey assignment
ResourceChangeListener. The subsequent change of the hasSurveys property in the Visit node triggers
a new ResourceChange event, which is then caught by another ResourceChangeListener that is
monitoring the status of the visits. And this second listener doesn't always receive events.
Some debugging:
We only noticed the issue only when our import went from tens of records in our test environment to
thousands of records in production. The first few hundreds of records don't seem to be affected,
then more and more events are missed, with a total of between 5 and 10% event loss. When I added a
500ms delay between each imported record, no events were lost. With a 100ms delay between each
imported record, 2% of the events were lost.
There's nothing in the logs (at the INFO level) to indicate that the events
were intentionally dropped.
Any ideas?
Thanks,
