I believe that I’ve found the problem. It could be two different problems
actually. One was that, during some experimentation locally, I ended up
running a VERY LONG running update script…which never actually finished. This
seems like it could have been the cause for things not running smoothly
locally. As for another environment, I’ve found that if I have updates that are
too large, the sync from the delta server runs out of memory. Some of these
patch files were about 30 MB. And like you mentioned, this causes very long
running updates…which cause the memory to run out…but strangely doesn’t crash
the server or throw any errors. It consistently stopped at the exact same point
(according to the debug logs) in its update every single time I restarted the
server. To remedy this, I’ve split up the manual updates that I’m applying into
smaller batches, things seem to be running smoothly again. But this does bring
up a concern as to what workarounds I would need if I ever needed to do a large
scale dynamic insert/delete via an update script.
> On Aug 12, 2021, at 7:37 AM, Andy Seaborne <a...@apache.org> wrote:
>
> "EXTERNAL EMAIL" - This email originated from outside of the organization. Do
> not click or open attachments unless you recognize the sender and know the
> content is safe. If you are unsure, please contact CTS at
> hel...@pointclickcare.com.
>
>
>
> On 11/08/2021 21:21, Brandon Sara wrote:
>>> 10s of millions triples of RDFS schema and no instance data?
>> Yeah, it’s kinda weird. I inherited this project and am working on fixing
>> much of the structuring, but in the mean time, need to keep it going as is.
>> We are loading ICD-10 CM, SNOMED CT, and many other medical
>> ontologies/thesauri…hence the large ontology. Pretty much every concept is
>> treated as a class. At this point in time, we are using ontology itself for
>> some inference and mapping. Eventually, we will be bringing instance data
>> into the KG to do more powerful inference using the medical ontologies I
>> mentioned.
>
> Try running without it as a test.
>
> The transitive reasoner fires up either as the when the server starts or
> first request (can't remember which).
>
>>> custom:id has super properties?
>> No
>
> From what you've said, that takes not much memory - at very worse, it
> populates the node cache which is an LRU cache and usually 2G is enough.
> (unless you have a lot of very large literals - many lines of text).
>
>>> is the request causing the database to be sync'ed before the request starts?
>> Yes
>
> That's a source of RAM use if there are large pending updates.
>
> Also try the query
>
> SELECT * {} or ASK{}
>
> which does all the end-to-end stuff for setup and sync but does not touch the
> data.
>
> The other thing to try is point VisualVM at the process and look for the
> memory usage and heap usage.
>
> Andy
>
No PHI in Email: PointClickCare and Collective Medical, A PointClickCare
Company, policies prohibit sending protected health information (PHI) by email,
which may violate regulatory requirements. If sending PHI is necessary, please
contact the sender for secure delivery instructions.
Confidentiality Notice: This email message, including any attachments, is for
the sole use of the intended recipient(s) and may contain confidential and
privileged information. Any unauthorized review, use, disclosure or
distribution is prohibited. If you are not the intended recipient, please
contact the sender by reply email and destroy all copies of the original
message.
