Re: Strategies to avoid log flooding

Mikael Pesonen Wed, 29 Mar 2023 02:22:12 -0700

Below is the log, so is it possible to filter just these out?

Unfortunately I don't recall the exact regex but it was related toescaping parentheses, so maybe this or with one back slash:

...
VALUES ?class_label { " \\(häiriö\\)" " \\(löydös\\)" " \\(toimenpide\\)" }
?concept rdfs:label ?fsnl FILTER (REGEX(?fsnl, $class_label)) .
...

So this is a bug not a feature and can be corrected?

Mar 27 13:13:33 insight-terms java[2512289]: [2023-03-27 13:13:33]QueryIterFilterExpr WARN Expression Exception in (regex ?fsnl ?class_label)Mar 27 13:13:33 insight-terms java[2512289]:org.apache.jena.sparql.expr.ExprException: REGEX: Pattern is not astring: " \$häiriö\$"@fiMar 27 13:13:33 insight-terms java[2512289]: #011atorg.apache.jena.sparql.expr.E_Regex.makeRegexEngine(E_Regex.java:120)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.apache.jena.sparql.expr.E_Regex.eval(E_Regex.java:102)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.apache.jena.sparql.expr.ExprFunctionN.eval(ExprFunctionN.java:113)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.apache.jena.sparql.expr.ExprFunctionN.eval(ExprFunctionN.java:110)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.apache.jena.sparql.expr.ExprNode.isSatisfied(ExprNode.java:42)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.apache.jena.sparql.engine.iterator.QueryIterFilterExpr.accept(QueryIterFilterExpr.java:49)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.apache.jena.sparql.engine.iterator.QueryIterProcessBinding.hasNextBinding(QueryIterProcessBinding.java:81)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:116)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.apache.jena.sparql.engine.iterator.QueryIterSlice.hasNextBinding(QueryIterSlice.java:76)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:116)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.apache.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBinding(QueryIteratorWrapper.java:38)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:116)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.apache.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBinding(QueryIteratorWrapper.java:38)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:116)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.apache.jena.sparql.exec.RowSetStream.hasNext(RowSetStream.java:47)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.apache.jena.sparql.engine.ResultSetStream.hasNext(ResultSetStream.java:81)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.apache.jena.fuseki.servlets.SPARQLQueryProcessor.executeQuery(SPARQLQueryProcessor.java:378)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.apache.jena.fuseki.servlets.SPARQLQueryProcessor.execute(SPARQLQueryProcessor.java:277)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.apache.jena.fuseki.servlets.SPARQLQueryProcessor.executeWithParameter(SPARQLQueryProcessor.java:222)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.apache.jena.fuseki.servlets.SPARQLQueryProcessor.execute(SPARQLQueryProcessor.java:207)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.apache.jena.fuseki.servlets.ActionService.executeLifecycle(ActionService.java:58)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.apache.jena.fuseki.servlets.SPARQLQueryProcessor.execPost(SPARQLQueryProcessor.java:83)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.apache.jena.fuseki.servlets.ActionProcessor.process(ActionProcessor.java:34)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.apache.jena.fuseki.servlets.ActionBase.process(ActionBase.java:54)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.apache.jena.fuseki.servlets.ActionExecLib.execActionSub(ActionExecLib.java:124)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.apache.jena.fuseki.servlets.ActionExecLib.execAction(ActionExecLib.java:98)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.apache.jena.fuseki.server.Dispatcher.dispatchAction(Dispatcher.java:164)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.apache.jena.fuseki.server.Dispatcher.process(Dispatcher.java:156)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.apache.jena.fuseki.server.Dispatcher.dispatch(Dispatcher.java:83)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.apache.jena.fuseki.servlets.FusekiFilter.doFilter(FusekiFilter.java:48)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:61)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:450)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:387)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.apache.jena.fuseki.servlets.CrossOriginFilter.handle(CrossOriginFilter.java:284)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.apache.jena.fuseki.servlets.CrossOriginFilter.doFilter(CrossOriginFilter.java:247)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:210)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:527)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:131)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:578)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:223)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1571)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:221)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1378)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:176)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:484)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1544)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:174)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1300)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:129)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:822)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.eclipse.jetty.server.Server.handle(Server.java:562)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.eclipse.jetty.server.HttpChannel.lambda$handle$0(HttpChannel.java:505)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:762)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:497)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:282)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:319)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:100)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.runTask(AdaptiveExecutionStrategy.java:412)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:381)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:268)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.lambda$new$0(AdaptiveExecutionStrategy.java:138)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:407)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:894)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atorg.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1038)~[fuseki-server.jar:4.6.1]Mar 27 13:13:33 insight-terms java[2512289]: #011atjava.lang.Thread.run(Thread.java:829) ~[?:?]

apps-fileview.texmex_20230316.01_p2


On 28/03/2023 16.04, Rob @ DNR wrote:

A GitHub issue with a minimal example query that reproduces the issue would be 
a good start so we can reproduce the issue and look into a fix

In workaround terms end users control their logging configuration so you could 
create a Log4j configuration that disables logging for the specific offending 
logger (assuming that this is a sufficiently specific logger to not suppress 
actually relevant logging)

Rob

From: Mikael Pesonen <[email protected]>
Date: Tuesday, 28 March 2023 at 11:21
To: [email protected] <[email protected]>
Subject: Strategies to avoid log flooding
Hi,

there are some cases where Jena generates dozens of gigs, maybe even
terabytes, of log in one query. If you add a bad REGEX, it generates a
long warning level exception for every row in db, or atleast million of
them (disk filled up so don't know). Is there another way to avoid this
except disable warnings?


--
Lingsoft - 30 years of Leading Language Management

www.lingsoft.fi

Speech Applications - Language Management - Translation - Reader's and Writer's 
Tools - Text Tools - E-books and M-books

Mikael Pesonen
Semantic Technologies

e-mail: [email protected]
Tel. +358 2 279 3300

Time zone: GMT+2

Helsinki Office
Eteläranta 10
FI-00130 Helsinki
FINLAND

Turku Office
Kauppiaskatu 5 A
FI-20100 Turku
FINLAND

Re: Strategies to avoid log flooding

Reply via email to