Bug#919638: solr-tomcat: Permission problems after update to tomcat9
Control: severity -1 serious Control: reassign -1 solr-tomcat Am 01.03.19 um 19:58 schrieb Emmanuel Bourg: > Le 01/03/2019 à 18:29, Markus Koschany a écrit : > >> I have never extended security permissions of >> another systemd service. How is this supposed to work? > > I'm not used to this either, but I think we just have to install a > /etc/systemd/system/tomcat9.d/solr-permissions.conf file with theses lines: > > [Service] > ReadWritePaths=/var/lib/solr/ > > A call to 'systemctl daemon-reload' is probably needed in the postinst > script (but maybe there is a trigger taking care of that already). Thanks. I didn't know about those config files. Usually /etc/systemd/system is for the local administrator who can completely override specific service files, so it might be dangerous to install something in those directories. However I haven't found anything in the Debian Policy about that, I just install solr-permissions.conf and execute systemctl daemon-reload in postinst and hope that it resolves the problem. Markus signature.asc Description: OpenPGP digital signature
Bug#919638: solr-tomcat: Permission problems after update to tomcat9
Le 01/03/2019 à 18:29, Markus Koschany a écrit : > I have never extended security permissions of > another systemd service. How is this supposed to work? I'm not used to this either, but I think we just have to install a /etc/systemd/system/tomcat9.d/solr-permissions.conf file with theses lines: [Service] ReadWritePaths=/var/lib/solr/ A call to 'systemctl daemon-reload' is probably needed in the postinst script (but maybe there is a trigger taking care of that already).
Bug#919638: solr-tomcat: Permission problems after update to tomcat9
Am 01.03.19 um 18:24 schrieb Emmanuel Bourg: > Le 01/03/2019 à 18:19, Markus Koschany a écrit : > >> I think just creating /var/lib/solr with tomcat9.dirs is sufficient. I >> tested both cases, installing tomcat9 first and then solr-tomcat and >> vice versa, and the server always started correctly. SystemD just >> requires an existing directory. Do you agree with this change? >> >> https://salsa.debian.org/java-team/tomcat9/commit/fc31e79f1c5f94cfcef0c75c3133654edf00a28e > > It looks a bit odd to put solr specific stuff in tomcat9. I'd rather > investigate a solution involving a /etc/systemd/system/tomcat9.d/ file > installed by solr-tomcat and extending the write permissions. > > Emmanuel Bourg I find adding an additional empty and root owned directory to be the lesser of two evils. I have never extended security permissions of another systemd service. How is this supposed to work? signature.asc Description: OpenPGP digital signature
Bug#919638: solr-tomcat: Permission problems after update to tomcat9
Le 01/03/2019 à 18:19, Markus Koschany a écrit : > I think just creating /var/lib/solr with tomcat9.dirs is sufficient. I > tested both cases, installing tomcat9 first and then solr-tomcat and > vice versa, and the server always started correctly. SystemD just > requires an existing directory. Do you agree with this change? > > https://salsa.debian.org/java-team/tomcat9/commit/fc31e79f1c5f94cfcef0c75c3133654edf00a28e It looks a bit odd to put solr specific stuff in tomcat9. I'd rather investigate a solution involving a /etc/systemd/system/tomcat9.d/ file installed by solr-tomcat and extending the write permissions. Emmanuel Bourg
Bug#919638: solr-tomcat: Permission problems after update to tomcat9
Hi, I think just creating /var/lib/solr with tomcat9.dirs is sufficient. I tested both cases, installing tomcat9 first and then solr-tomcat and vice versa, and the server always started correctly. SystemD just requires an existing directory. Do you agree with this change? https://salsa.debian.org/java-team/tomcat9/commit/fc31e79f1c5f94cfcef0c75c3133654edf00a28e Regards, Markus signature.asc Description: OpenPGP digital signature
Bug#919638: solr-tomcat: Permission problems after update to tomcat9
Am 26.02.19 um 09:46 schrieb Emmanuel Bourg: > Control: reopen -1 > Control: notfixed -1 9.0.16-2 > > Le 17/02/2019 à 12:38, Markus Koschany a écrit : > >> Thank you for the confirmation. Then I think reassigning this issue to >> src:tomcat9 and fixing it there is sensible. > > Unfortunately the modification broke tomcat9 installations when solr > isn't installed (#923299) and I had to revert it in the version 9.0.16-3. Sorry, I didn't expected that behavior from systemd. > We have to figure out another solution. Either: > 1. abandon the idea of restricting tomcat9 write access > 2. change solr-tomcat so that it modifies the tomcat9 permissions on install > 3. drop solr-tomcat, upstream recommends using Jetty after all. I personally like the sandboxing idea of tomcat9 which improves the overall security of the server. We should keep the current settings. Making one package modify the files of another package is tricky and I bet there are thousand Debian Policy rules to follow. We don't need to drop solr-tomcat either for this release cycle because apart from this permission problem everything else seems to work. This will be the last time solr-tomcat is part of a stable distribution though. The latest solr versions embed Jetty and solr is no longer a web application but a standalone server. Still the best way to fix this bug is to add /var/lib/solr to the sandbox if the directory exists. There must be some kind of conditional solution for systemd service files so that the option is only processed if another condition is true. Tomcat 9 could also simply create /var/lib/solr which would also address this problem. signature.asc Description: OpenPGP digital signature
Bug#919638: solr-tomcat: Permission problems after update to tomcat9
Control: reopen -1 Control: notfixed -1 9.0.16-2 Le 17/02/2019 à 12:38, Markus Koschany a écrit : > Thank you for the confirmation. Then I think reassigning this issue to > src:tomcat9 and fixing it there is sensible. Unfortunately the modification broke tomcat9 installations when solr isn't installed (#923299) and I had to revert it in the version 9.0.16-3. We have to figure out another solution. Either: 1. abandon the idea of restricting tomcat9 write access 2. change solr-tomcat so that it modifies the tomcat9 permissions on install 3. drop solr-tomcat, upstream recommends using Jetty after all. Emmanuel Bourg
Bug#919638: solr-tomcat: Permission problems after update to tomcat9
Control: reassign -1 src:tomcat9 Am 17.02.19 um 06:58 schrieb Michael Welsh Duggan: [...] > Based on your prior tip, I had already done that. (Only the > /var/lib/solr/ entry seemed to be necessary.) This caused things to > work again. Thank you for the confirmation. Then I think reassigning this issue to src:tomcat9 and fixing it there is sensible. signature.asc Description: OpenPGP digital signature
Bug#919638: solr-tomcat: Permission problems after update to tomcat9
Markus Koschany writes: > Hello Michael, > > > On Fri, 18 Jan 2019 01:19:36 -0500 Michael Welsh Duggan > wrote: >> Package: solr-tomcat >> Version: 3.6.2+dfsg-16 >> Severity: important >> >> Dear Maintainer, >> >> After updating tomcat to tomcat9 and solr-tomcat to 3.6.2+dfsg-16, it >> seems to be having problems writing to its index directory. The >> problem surfaced when using dovecot to look up messages. Attached is >> the error from the catalina log. >> >> /var/lib/solr/index does look like it has the right permissions: >> /var/lib/solr/data and /var/lib/solr/data/index are owned by >> tomcat:tomcat, permissions 770, and tomcat seems to be running as user >> tomcat. I have verified that I can write to the directory as root, >> and as such it's not on a read-only filesystem. I have no idea why it >> fails to write the lock file. > > Could you try the following? > > Please copy the tomcat9.service file to /etc/systemd/system and modify > it by adding > > ReadWritePaths=/var/lib/solr/ > ReadWritePaths=/var/lib/solr/data > > to the # Security paragraph. Then execute systemctl daemon-reload. > > This should whitelist the solr directories and writing to them should be > possible again. This is caused by restrictive systemd settings like > ProtectSystem=strict. I think Debian's tomcat9 package could allow this > by default but we could probably add a NEWS and README file to > solr-tomcat too and explain the steps to make it work. Based on your prior tip, I had already done that. (Only the /var/lib/solr/ entry seemed to be necessary.) This caused things to work again. -- Michael Welsh Duggan (m...@md5i.com)
Bug#919638: solr-tomcat: Permission problems after update to tomcat9
Hello Michael, On Fri, 18 Jan 2019 01:19:36 -0500 Michael Welsh Duggan wrote: > Package: solr-tomcat > Version: 3.6.2+dfsg-16 > Severity: important > > Dear Maintainer, > > After updating tomcat to tomcat9 and solr-tomcat to 3.6.2+dfsg-16, it > seems to be having problems writing to its index directory. The > problem surfaced when using dovecot to look up messages. Attached is > the error from the catalina log. > > /var/lib/solr/index does look like it has the right permissions: > /var/lib/solr/data and /var/lib/solr/data/index are owned by > tomcat:tomcat, permissions 770, and tomcat seems to be running as user > tomcat. I have verified that I can write to the directory as root, > and as such it's not on a read-only filesystem. I have no idea why it > fails to write the lock file. Could you try the following? Please copy the tomcat9.service file to /etc/systemd/system and modify it by adding ReadWritePaths=/var/lib/solr/ ReadWritePaths=/var/lib/solr/data to the # Security paragraph. Then execute systemctl daemon-reload. This should whitelist the solr directories and writing to them should be possible again. This is caused by restrictive systemd settings like ProtectSystem=strict. I think Debian's tomcat9 package could allow this by default but we could probably add a NEWS and README file to solr-tomcat too and explain the steps to make it work. Regards, Markus signature.asc Description: OpenPGP digital signature
Bug#919638: solr-tomcat: Permission problems after update to tomcat9
Emmanuel Bourg writes: > Le 18/01/2019 à 07:19, Michael Welsh Duggan a écrit : > >> I have no idea why it fails to write the lock file. > > It probably happens because tomcat9 is sandboxed to write only into its > own directory. You'll have to tweak the systemd configuration to allow > Tomcat to write into the solr directory (with the ReadWritePaths directive). > > The solr-tomcat package is rather old now, I'm tempted to remove it. > Upstream has moved away from supporting deployments as a web > application, Solr now embeds it own web server (Jetty). We wouldn't have > this kind of issue with the latest versions. I'm not attached to the package. I use solr as a FTS backend for dovecot. If you have a suggestion for how to set this up using the debian packages without using tomcat, I'm not using tomcat for anything else. -- Michael Welsh Duggan (m...@md5i.com)
Bug#919638: solr-tomcat: Permission problems after update to tomcat9
Hi Michael, Le 18/01/2019 à 07:19, Michael Welsh Duggan a écrit : > I have no idea why it fails to write the lock file. It probably happens because tomcat9 is sandboxed to write only into its own directory. You'll have to tweak the systemd configuration to allow Tomcat to write into the solr directory (with the ReadWritePaths directive). The solr-tomcat package is rather old now, I'm tempted to remove it. Upstream has moved away from supporting deployments as a web application, Solr now embeds it own web server (Jetty). We wouldn't have this kind of issue with the latest versions. Emmanuel Bourg
Bug#919638: solr-tomcat: Permission problems after update to tomcat9
Package: solr-tomcat
Version: 3.6.2+dfsg-16
Severity: important
Dear Maintainer,
After updating tomcat to tomcat9 and solr-tomcat to 3.6.2+dfsg-16, it
seems to be having problems writing to its index directory. The
problem surfaced when using dovecot to look up messages. Attached is
the error from the catalina log.
/var/lib/solr/index does look like it has the right permissions:
/var/lib/solr/data and /var/lib/solr/data/index are owned by
tomcat:tomcat, permissions 770, and tomcat seems to be running as user
tomcat. I have verified that I can write to the directory as root,
and as such it's not on a read-only filesystem. I have no idea why it
fails to write the lock file.
17-Jan-2019 23:27:24.199 INFO [http-nio-8080-exec-6]
org.apache.solr.update.processor.LogUpdateProcessor.finish
{add=[173976/7e5de009f991854df72612cf7b9c/md5i]} 0 1002
17-Jan-2019 23:27:24.200 SEVERE [http-nio-8080-exec-6]
org.apache.solr.common.SolrException.log
org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out:
NativeFSLock@/var/lib/solr/data/index/write.lock:
java.io.FileNotFoundException: /var/lib/solr/data/index/write.lock (Read-only
file system)
at org.apache.lucene.store.Lock.obtain(Lock.java:84)
at org.apache.lucene.index.IndexWriter.(IndexWriter.java:1098)
at
org.apache.solr.update.SolrIndexWriter.(SolrIndexWriter.java:84)
at
org.apache.solr.update.UpdateHandler.createMainIndexWriter(UpdateHandler.java:101)
at
org.apache.solr.update.DirectUpdateHandler2.openWriter(DirectUpdateHandler2.java:171)
at
org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:219)
at
org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:61)
at
org.apache.solr.update.processor.LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:115)
at org.apache.solr.handler.XMLLoader.processUpdate(XMLLoader.java:157)
at org.apache.solr.handler.XMLLoader.load(XMLLoader.java:79)
at
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:58)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1376)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:365)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:260)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:490)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
at
org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:668)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
at
org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:408)
at
org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
at
org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:834)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1417)
at
org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.io.FileNotFoundException: /var/lib/solr/data/index/write.lock
(Read-only file system)
at java.base/java.io.RandomAccessFile.open0(Native Method)
at java.base/java.io.RandomAccessFile.open(RandomAccessFile.java:345)
at java.base/java.io.RandomAccessFile.(RandomAccessFile.java:259)
at java.base/java.io.RandomAccessFile.(RandomAccessFile.java:214)
at
org.apache.lucene.store.NativeFSLock.obtain(NativeFSLockFactory.java:203)
at org.apache.lucene.store.Lock.obtain(Lock.java:95)
... 33 more
17-Jan-2019 23:27:24.201 INFO [http-nio-8080-exec-6]
org.apache.solr.core.SolrCore.execute [] webapp=/solr path=/update params={}
sta
