Hi, I created the ehcache file, dropped it into the same folder as jspwiki-custom.properties, stopped the service, cleared the WorkingDir, restarted the service.
The good news is that it now logs all 13,000 entries! I then had to assign more memory to Tomcat, but that’s understandable. Thank you for this! Andy From: Juan Pablo Santos Rodríguez [mailto:juanpablo.san...@gmail.com] Sent: 21 April 2022 22:21 To: user@jspwiki.apache.org Subject: Re: [External] Re: Search not working on unmodified files Alert! This is an external email sent from juanpablo.san...@gmail.com. Hi Andrew, Jim, hmm upon closer looking, seems LuceneSearchProvider requests all pages to the page manager, which in turn delegates to the caching provider, which, by default, holds up to 1.000 elements, and that's why no more pages appear indexed on a full reindex. I'll push a fix for that tomorrow / this weekend. In the meantime, you can use [#1] as an example of an ehcache file. Please see [#2] to see the appropiate parameters needed to use a custom ehcache file. Would you mind testing if the error persist while using a custom ehcache file with more elements than pages available? best regards, juan pablo [#1] https://github.com/apache/jspwiki/blob/master/jspwiki-cache/src/main/resources/ehcache-jspwiki.xml<https://github.com/apache/jspwiki/blob/master/jspwiki-cache/src/main/resources/ehcache-jspwiki.xml> [#2] https://jspwiki-wiki.apache.org/Wiki.jsp?page=NewIn2.11#section-NewIn2.11-NewInJSPWiki2.11.1ReleasedOn18122021<https://jspwiki-wiki.apache.org/Wiki.jsp?page=NewIn2.11#section-NewIn2.11-NewInJSPWiki2.11.1ReleasedOn18122021> On Thu, Apr 21, 2022 at 2:38 PM Jim Willeke <j...@willeke.com> wrote: > > I found this one: > https://jar-download.com/artifacts/org.apache.jspwiki/jspwiki-war/2.10.1/source-code/ehcache.xml<https://jar-download.com/artifacts/org.apache.jspwiki/jspwiki-war/2.10.1/source-code/ehcache.xml> > > -- > -jim > Jim Willeke > > > On Thu, Apr 21, 2022 at 6:41 AM Andrew Ducker < > andrew_duc...@thephoenixgroup.com> wrote: > > > Is there an example ehcache file somewhere? It doesn’t get created > > anywhere automatically that I can see, and it’s not in the > > https://jspwiki-wiki.apache.org/Wiki.jsp?page=Configuration<https://jspwiki-wiki.apache.org/Wiki.jsp?page=Configuration> > > documentation. > > > > Cheers, > > > > Andy > > > > From: Jim Willeke [mailto:j...@willeke.com] > > Sent: 20 April 2022 21:35 > > To: user@jspwiki.apache.org > > Subject: Re: [External] Re: Search not working on unmodified files > > > > Alert! This is an external email sent from j...@willeke.com. > > > > I have observed something similar. > > > > Active Sessions 51 > > Uptime 0d, 18h 30m 39s > > Number of pages 16390FYI: I too am seeing similar issues on 2.11.0-M8. > > > > I have NOT observed the issues on 2.10.1. > > > > The instance on version 2.10.1 > > (https://ldapwiki.com/wiki/SystemInfo<https://ldapwiki.com/wiki/SystemInfo>< > > https://ldapwiki.com/wiki/SystemInfo<https://ldapwiki.com/wiki/SystemInfo>>) > > is > > busy > > and the 2.11.0-M8 instance on has very low traffic. > > > > On the version 2.11.0-M8, Cleared the Lucene and work directory and > > restarted: > > > > Active Sessions 1 > > Uptime 0d, 9h 11m 7s > > Number of pages 16499 > > > > And one day later is shows: > > Active Sessions 1 > > Uptime 1d, 10h 15m 43s > > Number of pages 46 > > > > Then after using for a few minutes it now says: > > Active Sessions 1 > > Uptime 1d, 10h 35m 52s > > Number of pages 16500 > > (And yes one page was really added over the day) > > > > Where Number of pages is the [{$totalpages}] > > > > On the version 2.10.1 below is typical: > > > > Active Sessions 51 > > Uptime 0d, 18h 30m 39s > > Number of pages 16390 > > > > The ability to perform a search appears to be related to the > > [{$totalpages}] variable. That is when it shows 46 almost nothing can be > > found. > > > > I do see in the log: > > 2022-04-20 10:12:22,050 [main] WARN > > org.apache.wiki.providers.CachingProvider - seems > > Ldapwiki.jspwiki.pageCache can't hold all pages from your page repository, > > so we're delegating on the underlying provider instead. Please consider > > increasing your cache sizes on ehcache.xml to avoid this behaviour > > > > Which led to the finding the entry in > > > > https://github.com/apache/jspwiki/blob/master/jspwiki-main/src/main/resources/ini/jspwiki.properties<https://github.com/apache/jspwiki/blob/master/jspwiki-main/src/main/resources/ini/jspwiki.properties> > > < > > https://github.com/apache/jspwiki/blob/master/jspwiki-main/src/main/resources/ini/jspwiki.properties<https://github.com/apache/jspwiki/blob/master/jspwiki-main/src/main/resources/ini/jspwiki.properties> > > > > > > > " > > *# By default, JSPWiki caches will hold up to 1.000 elements, except the > > RSS cache, which will hold up to 250 > > elementsjspwiki.cache.custom-config-file = jspwiki-ehcache.xml*" > > > > Both are behind nginx. > > > > -- > > -jim > > Jim Willeke > > > > > > On Mon, Apr 18, 2022 at 3:40 PM Juan Pablo Santos Rodríguez < > > juanpablo.san...@gmail.com> wrote: > > > > > Hi! > > > > > > Looking at the source code, at jspwiki init, a full reindex is triggered. > > > The full reindex is only performed if the Lucene dir is empty. As this is > > > not the case, the initial full reindex isn't executed, and that's why you > > > are not seeing anything on searches. Pages saves do trigger a partial > > > reindex, though, so that explains the other side of the behaviour you're > > > seeing. > > > > > > What files are inside the Lucene dir? If there are Lucene index files, > > can > > > you open then with Luke to see where they came from, or the Lucene index > > > version, or any other information? Is there any chance other jspwiki > > > instance is using the same workDir? > > > > > > Last, emptying the lucene dir will enforce a full reindex next time the > > > background thread responsible of performing Lucene reindexes runs, but it > > > would be better to know why some files are getting created there outside > > > your jspwiki instance. > > > > > > > > > HTH, > > > juan pablo > > > > > > El lun., 18 abr. 2022 12:32, Andrew Ducker < > > > andrew_duc...@thephoenixgroup.com> escribió: > > > > > > > Hi, > > > > > > > > Thanks for the reply. > > > > > > > > We completely cleared the WorkDir, to be on the safe side, and that > > > didn’t > > > > help. > > > > > > > > It has access to the Work and wikipages folders, as it can create them > > > > from scratch, and you can edit the Wiki files. > > > > > > > > No mentions of “unable to index” in the logs. > > > > > > > > In the jspwiki-custom.properties file there’s the following: > > > > jspwiki.applicationName =IS Wiki > > > > jspwiki.baseURL = <removed> > > > > jspwiki.frontPage = Main > > > > jspwiki.pageProvider =VersioningFileProvider > > > > jspwiki.fileSystemProvider.pageDir =d:/JSPWiki/JSPWikiPages/Pages/wiki/ > > > > jspwiki.basicAttachmentProvider.storageDir > > > > =d:/JSPWiki/JSPWikiPages/Attachments/wiki/ > > > > jspwiki.interWikiRef.Notes = Notes:%s > > > > jspwiki.translatorReader.inlinePattern.1 = *.jpg > > > > jspwiki.translatorReader.inlinePattern.2 = *.png > > > > jspwiki.translatorReader.inlinePattern.3 = > > > > http://images.com/*<http://images.com/*>< > > http://images.com/*<http://images.com/*>> > > > > jspwiki.rss.generate =true > > > > jspwiki.rss.channelDescription = IS Wiki > > > > mail.from =@mail.from@ > > > > mail.smtp.host<http://mail.smtp.host><http://mail.smtp.host<http://mail.smtp.host>> > > > > =@mail.smtp.host@ > > > > jspwiki.workDir =d:/JSPWiki/WorkingDirectory/iswiki/ > > > > jspwiki.security = jaas > > > > > > > > Lucene-specific things in the logs: > > > > [INFO ] 2022-04-13 13:44:48.094 [main] o.a.w.s.LuceneSearchProvider - > > > > Lucene enabled, cache will be in: > > > d:\JSPWiki\WorkingDirectory\iswiki\lucene > > > > [WARN ] 2022-04-13 13:44:48.725 [JSPWiki Lucene Indexer] > > > > o.a.w.WikiBackgroundThread - Starting up background thread: JSPWiki > > > Lucene > > > > Indexer. > > > > [INFO ] 2022-04-13 13:45:48.732 [JSPWiki Lucene Indexer] > > > > o.a.w.s.LuceneSearchProvider - Files found in Lucene directory, not > > > > reindexing. > > > > > > > > So could it be starting to index, thinking it’s already indexed > > (despite > > > > the whole WorkingDir being cleared out) and then stopping > > near-instantly? > > > > > > > > Thanks again, > > > > > > > > Andy > > > > > > > > > > > > From: Juan Pablo Santos Rodríguez [mailto:juanpablo.san...@gmail.com] > > > > Sent: 14 April 2022 16:25 > > > > To: user@jspwiki.apache.org > > > > Subject: [External] Re: Search not working on unmodified files > > > > > > > > Alert! This is an external email sent from juanpablo.san...@gmail.com. > > > > > > > > Hi Andrew, > > > > > > > > haven't had the time to look in depth at JSPWIKI-1171, so just some > > > > questions / random thoughts: > > > > > > > > JSPWiki has changed a lot since 2.8, but the wiki pages itself > > shouldn't > > > > require any change, and should be readable by any version of JSPWiki. > > The > > > > Lucene version has been upgraded several times, so it was definitely ok > > > to > > > > clear your Lucene folder. Also there is the refmgr.ser file and > > > refmgr-attr > > > > dir inside your workDir, which contain serialized information about > > page > > > > references. I'm not 100% sure, but most probably the internal format > > has > > > > changed since 2.8 (there was a global package rename on 2.9, when > > > entering > > > > the Apache incubator, and somewhat later a public API was pushed which > > > may > > > > have affected this format too) so you should delete those too and try > > to > > > > restart your JSPWiki instance again and see if the situation persists; > > > > perhaps deleting the entire workDir would be the safest approach. > > That's > > > > what seems to be happening from the stacktrace attached at JSPWIKI-1171 > > > > (this list strips attachments, IIRC). > > > > > > > > Also, regarding the searches, would you mind confirming that the new > > > > JSPWiki instance has permissions to read/write either the work and > > > > wikipages directories/files? Also are there any errors logged when > > > starting > > > > JSPWiki? Specifically, the Lucene indexing is done on startup (if > > needed) > > > > on the background, so eventually you should get all your pages indexed > > or > > > > see log errors stating something like "Unable to index page $pageName, > > > > continuing to next". > > > > > > > > Are you using a vanilla JSPWiki installation or do you have some > > > > customizations? What parameters do you have on your > > > > jspwiki-custom.properties file? I'm assuming you're deloying the war > > > file? > > > > > > > > > > > > best regards, > > > > juan pablo > > > > > > > > On Thu, Apr 14, 2022 at 4:51 PM Andrew Ducker < > > > > andrew_duc...@thephoenixgroup.com> wrote: > > > > > > > > > Hi! > > > > > > > > > > > > > > > > > > > > I am having a problem with an upgraded JSPWiki not indexing pages. > > > > > > > > > > > > > > > > > > > > I wasn’t sure where to report it, so added an issue on Jira: > > > > > > > > > > https://issues.apache.org/jira/browse/JSPWIKI-1171<https://issues.apache.org/jira/browse/JSPWIKI-1171>< > > https://issues.apache.org/jira/browse/JSPWIKI-1171<https://issues.apache.org/jira/browse/JSPWIKI-1171>>< > > > > https://issues.apache.org/jira/browse/JSPWIKI-1171<https://issues.apache.org/jira/browse/JSPWIKI-1171>< > > https://issues.apache.org/jira/browse/JSPWIKI-1171<https://issues.apache.org/jira/browse/JSPWIKI-1171>>> > > > > > > > > > > > > > > > > > > > > I’m recreating it here, in case this is the right place to report > > > things: > > > > > > > > > > > > > > > > > > > > Lots of pages aren't turning up in search. > > > > > > > > > > > > > > > > > > > > We cleared the lucene folder from the working directory, and it > > rebuilt > > > > it > > > > > - but quite quickly for 13,000 pages. And the total size of the > > Lucene > > > > > files was around 2.5MB, which seemed far too small. > > > > > > > > > > > > > > > > > > > > Doing a search failed to bring back pages that had the given word in > > > > their > > > > > name and content. > > > > > > > > > > > > > > > > > > > > Editing one of those pages (adding a space) caused it to then appear > > in > > > > > the searches. > > > > > > > > > > > > > > > > > > > > Checking the logs I found a lot of errors in the form: > > > > > > > > > > > > > > > > > > > > [ERROR] 2022-04-12 15:50:17.274 [main] > > o.a.w.r.DefaultReferenceManager > > > - > > > > > Error while trying to fetch a page name; trying to cope with the > > > > situation. > > > > > > > > > > org.apache.wiki.api.exceptions.ProviderException: Illegal page name > > > > > > > > > > > > > > > > > > > > (See attachment for whole exception trace) > > > > > > > > > > > > > > > > > > > > Sadly, it doesn't tell me what the illegal page names are. > > > > > > > > > > > > > > > > > > > > This wiki has been upgraded from version 2.8 - are there > > compatibility > > > > > issues with page names there? > > > > > > > > > > > > > > > > > > > > Any help you can give me gratefully received! > > > > > > > > > > > > > > > > > > > > Andy > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > <https://www.thephoenixgroup.com/> > > > > > *Andrew Ducker* | Senior Systems Developer > > > > > office: +441312458823 | mobile: 07980 86 86 39 | email: > > > > > andrew_duc...@thephoenixgroup.com > > > > > P Please help save the environment, print only if necessary. > > > > > [image: logobadgefinal] > > > > > > > > > > > > > > > > > > > > Confidentiality - This email is confidential. > > > > > Not meant for you? - If you don't think this email is meant for you, > > > > > please let us know. Do not copy or forward the information it > > contains, > > > > and > > > > > delete this email from your system. > > > > > Views expressed - Any personal views or opinions expressed in this > > > email > > > > > are the sender's, and do not necessarily reflect the views of the > > > Phoenix > > > > > Group (Phoenix Group Holdings plc and its subsidiaries). > > > > > Monitoring - We filter and monitor emails to protect our systems and > > to > > > > > keep them running smoothly. > > > > > Emailing us - Email isn't a secure form of communication. If you want > > > to > > > > > send us confidential information please send it by post. However, if > > > you > > > > do > > > > > communicate with us by email on any subject, you are giving us > > > permission > > > > > to email you back. > > > > > Phoning us - Calls may be monitored and/or recorded to protect both > > you > > > > > and us and help with our training. Call charges will vary. > > > > > Standard Life Assurance Limited is registered in Scotland (SC286833) > > at > > > > > Standard Life House, 30 Lothian Road, Edinburgh EH1 2DH and > > authorised > > > by > > > > > the Prudential Regulation Authority and regulated by the Financial > > > > Conduct > > > > > Authority and the Prudential Regulation Authority. > > > > www.standardlife.co.uk<http://www.standardlife.co.uk><http://www.standardlife.co.uk<http://www.standardlife.co.uk>>< > > http://www.standardlife.co.uk<http://www.standardlife.co.uk><http://www.standardlife.co.uk<http://www.standardlife.co.uk>>> > > > > > Standard Life Assurance Limited and Standard Life Assets and Employee > > > > > Services Limited are companies in the Phoenix Group (Phoenix Group > > > > Holdings > > > > > plc and its subsidiaries). www.thephoenixgroup.com > > > > > > > > > > > > > Confidentiality - This email is confidential. > > > > Not meant for you? - If you don't think this email is meant for you, > > > > please let us know. Do not copy or forward the information it contains, > > > and > > > > delete this email from your system. > > > > Views expressed - Any personal views or opinions expressed in this > > email > > > > are the sender's, and do not necessarily reflect the views of the > > Phoenix > > > > Group (Phoenix Group Holdings plc and its subsidiaries). > > > > Monitoring - We filter and monitor emails to protect our systems and to > > > > keep them running smoothly. > > > > Emailing us - Email isn't a secure form of communication. If you want > > to > > > > send us confidential information please send it by post. However, if > > you > > > do > > > > communicate with us by email on any subject, you are giving us > > permission > > > > to email you back. > > > > Phoning us - Calls may be monitored and/or recorded to protect both you > > > > and us and help with our training. Call charges will vary. > > > > Standard Life Assurance Limited is registered in Scotland (SC286833) at > > > > Standard Life House, 30 Lothian Road, Edinburgh EH1 2DH and authorised > > by > > > > the Prudential Regulation Authority and regulated by the Financial > > > Conduct > > > > Authority and the Prudential Regulation Authority. > > > www.standardlife.co.uk<http://www.standardlife.co.uk><http://www.standardlife.co.uk<http://www.standardlife.co.uk>>< > > > > http://www.standardlife.co.uk<http://www.standardlife.co.uk><http://www.standardlife.co.uk<http://www.standardlife.co.uk>>> > > > > Standard Life Assurance Limited and Standard Life Assets and Employee > > > > Services Limited are companies in the Phoenix Group (Phoenix Group > > > Holdings > > > > plc and its subsidiaries). www.thephoenixgroup.com< > > > > http://www.thephoenixgroup.com> > > > > > > > > > > > Confidentiality - This email is confidential. > > Not meant for you? - If you don't think this email is meant for you, > > please let us know. Do not copy or forward the information it contains, and > > delete this email from your system. > > Views expressed - Any personal views or opinions expressed in this email > > are the sender's, and do not necessarily reflect the views of the Phoenix > > Group (Phoenix Group Holdings plc and its subsidiaries). > > Monitoring - We filter and monitor emails to protect our systems and to > > keep them running smoothly. > > Emailing us - Email isn't a secure form of communication. If you want to > > send us confidential information please send it by post. However, if you do > > communicate with us by email on any subject, you are giving us permission > > to email you back. > > Phoning us - Calls may be monitored and/or recorded to protect both you > > and us and help with our training. Call charges will vary. > > Standard Life Assurance Limited is registered in Scotland (SC286833) at > > Standard Life House, 30 Lothian Road, Edinburgh EH1 2DH and authorised by > > the Prudential Regulation Authority and regulated by the Financial Conduct > > Authority and the Prudential Regulation Authority. > > www.standardlife.co.uk<http://www.standardlife.co.uk>< > > http://www.standardlife.co.uk<http://www.standardlife.co.uk>> > > Standard Life Assurance Limited and Standard Life Assets and Employee > > Services Limited are companies in the Phoenix Group (Phoenix Group Holdings > > plc and its subsidiaries). www.thephoenixgroup.com< > > http://www.thephoenixgroup.com> > > Confidentiality - This email is confidential. Not meant for you? - If you don't think this email is meant for you, please let us know. Do not copy or forward the information it contains, and delete this email from your system. Views expressed - Any personal views or opinions expressed in this email are the sender's, and do not necessarily reflect the views of the Phoenix Group (Phoenix Group Holdings plc and its subsidiaries). Monitoring - We filter and monitor emails to protect our systems and to keep them running smoothly. Emailing us - Email isn't a secure form of communication. If you want to send us confidential information please send it by post. However, if you do communicate with us by email on any subject, you are giving us permission to email you back. Phoning us - Calls may be monitored and/or recorded to protect both you and us and help with our training. Call charges will vary. Standard Life Assurance Limited is registered in Scotland (SC286833) at Standard Life House, 30 Lothian Road, Edinburgh EH1 2DH and authorised by the Prudential Regulation Authority and regulated by the Financial Conduct Authority and the Prudential Regulation Authority. www.standardlife.co.uk<http://www.standardlife.co.uk> Standard Life Assurance Limited and Standard Life Assets and Employee Services Limited are companies in the Phoenix Group (Phoenix Group Holdings plc and its subsidiaries). www.thephoenixgroup.com<http://www.thephoenixgroup.com>
