dr0ptp4kt added a comment.
By the way, I'm attempting a run for the first 1332 munged files (one shy of the 1333 where terminated last time around) with buffer at 100000**0** and heap size at 31g and queue capacity at 4000 on the gaming-class desktop to see whether this imports smoothly and whether performance gains are noticeable. ubuntu22:~/rdf/dist/target/service-0.3.138-SNAPSHOT$ date Wed Mar 20 02:36:59 PM CDT 2024 ubuntu22:~/rdf/dist/target/service-0.3.138-SNAPSHOT$ time ./loadData.sh -n wdq -d /mnt/firehose/munge_on_later_data_set -s 1 -e 1332 ...screen'ing in to check: Processing wikidump-000000505.ttl.gz <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"><html><head><meta http-equiv="Content-Type" content="text/html;charset=UTF-8"><title>blazegraph™ by SYSTAP</title ></head ><body<p>totalElapsed=13452ms, elapsed=13452ms, connFlush=0ms, batchResolve=0, whereClause=0ms, deleteClause=0ms, insertClause=0ms</p ><hr><p>COMMIT: totalElapsed=167329ms, commitTime=1711041930967, mutationCount=4566497</p ></html >Thu Mar 21 12:25:35 PM CDT 2024 Processing wikidump-000000506.ttl.gz <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"><html><head><meta http-equiv="Content-Type" content="text/html;charset=UTF-8"><title>blazegraph™ by SYSTAP</title ></head ><body<p>totalElapsed=15405ms, elapsed=15405ms, connFlush=0ms, batchResolve=0, whereClause=0ms, deleteClause=0ms, insertClause=0ms</p ><hr><p>COMMIT: totalElapsed=203202ms, commitTime=1711042135111, mutationCount=5262167</p ></html >Thu Mar 21 12:28:58 PM CDT 2024 Processing wikidump-000000507.ttl.gz <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"><html><head><meta http-equiv="Content-Type" content="text/html;charset=UTF-8"><title>blazegraph™ by SYSTAP</title ></head ><body<p>totalElapsed=14701ms, elapsed=14700ms, connFlush=0ms, batchResolve=0, whereClause=0ms, deleteClause=0ms, insertClause=0ms</p ><hr><p>COMMIT: totalElapsed=178754ms, commitTime=1711042314114, mutationCount=5005853</p ></html >Thu Mar 21 12:31:57 PM CDT 2024 TASK DETAIL https://phabricator.wikimedia.org/T359062 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dr0ptp4kt Cc: ssingh, bking, dr0ptp4kt, Aklapper, Danny_Benjafield_WMDE, Astuthiodit_1, karapayneWMDE, Invadibot, maantietaja, ItamarWMDE, Akuckartz, Nandana, Lahi, Gq86, GoranSMilovanovic, QZanden, EBjune, KimKelting, LawExplorer, _jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, Mbch331
_______________________________________________ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org