And you use TDB1?
TDB1 can use more memory - and between all the other components it might
all amount to 6G if the server isn't able to do some of the background
tidy-up work for a while.
TDB2 does not have this effect.
Andy
On 12/08/2021 17:19, Brandon Sara wrote:
I believe that I’ve found the problem. It could be two different problems
actually. One was that, during some experimentation locally, I ended up
running a VERY LONG running update script…which never actually finished. This
seems like it could have been the cause for things not running smoothly
locally. As for another environment, I’ve found that if I have updates that are
too large, the sync from the delta server runs out of memory. Some of these
patch files were about 30 MB. And like you mentioned, this causes very long
running updates…which cause the memory to run out…but strangely doesn’t crash
the server or throw any errors. It consistently stopped at the exact same point
(according to the debug logs) in its update every single time I restarted the
server. To remedy this, I’ve split up the manual updates that I’m applying into
smaller batches, things seem to be running smoothly again. But this does bring
up a concern as to what workarounds I would need if I ever needed to do a large
scale dynamic insert/delete via an update script.
On Aug 12, 2021, at 7:37 AM, Andy Seaborne <[email protected]> wrote:
"EXTERNAL EMAIL" - This email originated from outside of the organization. Do
not click or open attachments unless you recognize the sender and know the content is
safe. If you are unsure, please contact CTS at [email protected].
On 11/08/2021 21:21, Brandon Sara wrote:
10s of millions triples of RDFS schema and no instance data?
Yeah, it’s kinda weird. I inherited this project and am working on fixing much
of the structuring, but in the mean time, need to keep it going as is. We are
loading ICD-10 CM, SNOMED CT, and many other medical ontologies/thesauri…hence
the large ontology. Pretty much every concept is treated as a class. At this
point in time, we are using ontology itself for some inference and mapping.
Eventually, we will be bringing instance data into the KG to do more powerful
inference using the medical ontologies I mentioned.
Try running without it as a test.
The transitive reasoner fires up either as the when the server starts or first
request (can't remember which).
custom:id has super properties?
No
From what you've said, that takes not much memory - at very worse, it
populates the node cache which is an LRU cache and usually 2G is enough.
(unless you have a lot of very large literals - many lines of text).
is the request causing the database to be sync'ed before the request starts?
Yes
That's a source of RAM use if there are large pending updates.
Also try the query
SELECT * {} or ASK{}
which does all the end-to-end stuff for setup and sync but does not touch the
data.
The other thing to try is point VisualVM at the process and look for the memory
usage and heap usage.
Andy
No PHI in Email: PointClickCare and Collective Medical, A PointClickCare
Company, policies prohibit sending protected health information (PHI) by email,
which may violate regulatory requirements. If sending PHI is necessary, please
contact the sender for secure delivery instructions.
Confidentiality Notice: This email message, including any attachments, is for
the sole use of the intended recipient(s) and may contain confidential and
privileged information. Any unauthorized review, use, disclosure or
distribution is prohibited. If you are not the intended recipient, please
contact the sender by reply email and destroy all copies of the original
message.
