Yes, you can filter these out, the logger in question is the class name shown,
the log4j configuration will need to reference that via its fully qualified
name i.e. org.apache.jena.sparql.engine.iterator.QueryIterFilterExpr and set it
to ERROR/OFF to suppress these warnings
Issuing millions of instances of the same identical warning certainly seems
like a bug to me, especially since this is elicited by query input it could
potentially be abused as a DoS attack vector.
Rob
From: Mikael Pesonen <mikael.peso...@lingsoft.fi>
Date: Wednesday, 29 March 2023 at 10:22
To: users@jena.apache.org <users@jena.apache.org>
Subject: Re: Strategies to avoid log flooding
Below is the log, so is it possible to filter just these out?
Unfortunately I don't recall the exact regex but it was related to
escaping parentheses, so maybe this or with one back slash:
...
VALUES ?class_label { " \\(häiriö\\)" " \\(löydös\\)" " \\(toimenpide\\)" }
?concept rdfs:label ?fsnl FILTER (REGEX(?fsnl, $class_label)) .
...
So this is a bug not a feature and can be corrected?
Mar 27 13:13:33 insight-terms java[2512289]: [2023-03-27 13:13:33]
QueryIterFilterExpr WARN Expression Exception in (regex ?fsnl ?class_label)
Mar 27 13:13:33 insight-terms java[2512289]:
org.apache.jena.sparql.expr.ExprException: REGEX: Pattern is not a
string: " \\(häiriö\\)"@fi
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.apache.jena.sparql.expr.E_Regex.makeRegexEngine(E_Regex.java:120)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.apache.jena.sparql.expr.E_Regex.eval(E_Regex.java:102)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.apache.jena.sparql.expr.ExprFunctionN.eval(ExprFunctionN.java:113)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.apache.jena.sparql.expr.ExprFunctionN.eval(ExprFunctionN.java:110)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.apache.jena.sparql.expr.ExprNode.isSatisfied(ExprNode.java:42)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.apache.jena.sparql.engine.iterator.QueryIterFilterExpr.accept(QueryIterFilterExpr.java:49)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.apache.jena.sparql.engine.iterator.QueryIterProcessBinding.hasNextBinding(QueryIterProcessBinding.java:81)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:116)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.apache.jena.sparql.engine.iterator.QueryIterSlice.hasNextBinding(QueryIterSlice.java:76)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:116)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.apache.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBinding(QueryIteratorWrapper.java:38)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:116)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.apache.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBinding(QueryIteratorWrapper.java:38)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:116)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.apache.jena.sparql.exec.RowSetStream.hasNext(RowSetStream.java:47)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.apache.jena.sparql.engine.ResultSetStream.hasNext(ResultSetStream.java:81)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.apache.jena.fuseki.servlets.SPARQLQueryProcessor.executeQuery(SPARQLQueryProcessor.java:378)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.apache.jena.fuseki.servlets.SPARQLQueryProcessor.execute(SPARQLQueryProcessor.java:277)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.apache.jena.fuseki.servlets.SPARQLQueryProcessor.executeWithParameter(SPARQLQueryProcessor.java:222)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.apache.jena.fuseki.servlets.SPARQLQueryProcessor.execute(SPARQLQueryProcessor.java:207)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.apache.jena.fuseki.servlets.ActionService.executeLifecycle(ActionService.java:58)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.apache.jena.fuseki.servlets.SPARQLQueryProcessor.execPost(SPARQLQueryProcessor.java:83)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.apache.jena.fuseki.servlets.ActionProcessor.process(ActionProcessor.java:34)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.apache.jena.fuseki.servlets.ActionBase.process(ActionBase.java:54)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.apache.jena.fuseki.servlets.ActionExecLib.execActionSub(ActionExecLib.java:124)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.apache.jena.fuseki.servlets.ActionExecLib.execAction(ActionExecLib.java:98)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.apache.jena.fuseki.server.Dispatcher.dispatchAction(Dispatcher.java:164)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.apache.jena.fuseki.server.Dispatcher.process(Dispatcher.java:156)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.apache.jena.fuseki.server.Dispatcher.dispatch(Dispatcher.java:83)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.apache.jena.fuseki.servlets.FusekiFilter.doFilter(FusekiFilter.java:48)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:61)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:450)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:387)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.apache.jena.fuseki.servlets.CrossOriginFilter.handle(CrossOriginFilter.java:284)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.apache.jena.fuseki.servlets.CrossOriginFilter.doFilter(CrossOriginFilter.java:247)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:210)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:527)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:131)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:578)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:223)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1571)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:221)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1378)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:176)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:484)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1544)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:174)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1300)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:129)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:822)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.eclipse.jetty.server.Server.handle(Server.java:562)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.eclipse.jetty.server.HttpChannel.lambda$handle$0(HttpChannel.java:505)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:762)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:497)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:282)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:319)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:100)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.runTask(AdaptiveExecutionStrategy.java:412)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:381)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:268)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.lambda$new$0(AdaptiveExecutionStrategy.java:138)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:407)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:894)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1038)
~[fuseki-server.jar:4.6.1]
Mar 27 13:13:33 insight-terms java[2512289]: #011at
java.lang.Thread.run(Thread.java:829) ~[?:?]
apps-fileview.texmex_20230316.01_p2
On 28/03/2023 16.04, Rob @ DNR wrote:
> A GitHub issue with a minimal example query that reproduces the issue would
> be a good start so we can reproduce the issue and look into a fix
>
> In workaround terms end users control their logging configuration so you
> could create a Log4j configuration that disables logging for the specific
> offending logger (assuming that this is a sufficiently specific logger to not
> suppress actually relevant logging)
>
> Rob
>
> From: Mikael Pesonen <mikael.peso...@lingsoft.fi>
> Date: Tuesday, 28 March 2023 at 11:21
> To: users@jena.apache.org <users@jena.apache.org>
> Subject: Strategies to avoid log flooding
> Hi,
>
> there are some cases where Jena generates dozens of gigs, maybe even
> terabytes, of log in one query. If you add a bad REGEX, it generates a
> long warning level exception for every row in db, or atleast million of
> them (disk filled up so don't know). Is there another way to avoid this
> except disable warnings?
>
--
Lingsoft - 30 years of Leading Language Management
www.lingsoft.fi<http://www.lingsoft.fi>
Speech Applications - Language Management - Translation - Reader's and Writer's
Tools - Text Tools - E-books and M-books
Mikael Pesonen
Semantic Technologies
e-mail: mikael.peso...@lingsoft.fi
Tel. +358 2 279 3300
Time zone: GMT+2
Helsinki Office
Eteläranta 10
FI-00130 Helsinki
FINLAND
Turku Office
Kauppiaskatu 5 A
FI-20100 Turku
FINLAND
