Hi Peter, I recall seeing something similar when copying database folders from a MS Windows BaseX install to a Linux install . I found in some cases I needed to run db:optimize[1] once before any other use of the database otherwise a Java error would occur.
Now, I would always use db backup and restore to transfer databases between systems. /Andy [1] https://old.docs.basex.org/wiki/Database_Module#db:optimize On Fri, 7 Mar 2025 at 08:10, Christian Grün <[email protected]> wrote: > Hi Peter, > > Can you reproduce the problem with the latest version (11.7) and the zip > distribution of BaseX (e.g., without Docker)? > > Best, > Christian > > > > Peter Villadsen <[email protected]> schrieb am Fr., 7. März > 2025, 05:00: > >> I did some more work to capture the relevant information for the two >> crashes. >> >> >> >> As you recall, I build a container image on top of the official (but old) >> basex one. It copies the database into the right place in the container. >> >> >> >> I added the -c switch to the basexhttp command when the container starts. >> Even when I do this, the container does not have data context set when the >> first operation involving the database happens - If I do a query that >> involves the open database: >> >> >> >> <query> >> >> <text>count(/ada)</text> >> >> </query> >> >> >> >> I get: >> >> >> >> Improper use? Potential bug? Your feedback is welcome: >> >> Contact: [email protected] >> >> Version: BaseX 9.6 RC1 >> >> Java: IcedTea, >> >> 1.8.0_212 >> >> OS: Linux, amd64 >> >> Stack Trace: >> >> java.lang.NullPointerException >> >> at org.basex.data.Data.defaultNs(Data.java: 270) >> >> at org.basex.query.expr.path.NameTest.noMatches(NameTest.java: 60) >> >> at org.basex.query.expr.path.Step.optimize(Step.java: 162) >> >> at org.basex.query.expr.path.Step.optimize(Step.java: 134) >> >> at org.basex.query.expr.Preds.compile(Preds.java: 59) >> >> at org.basex.query.expr.path.Path.lambda$compile$0(Path.java: 139) >> >> at org.basex.query.CompileContext.get(CompileContext.java: 165) >> >> at org.basex.query.expr.path.Path.compile(Path.java: 134) >> >> at org.basex.query.expr.Arr.compile(Arr.java: 47) >> >> at org.basex.query.scope.MainModule.comp(MainModule.java: 81) >> >> at org.basex.query.QueryCompiler.compile(QueryCompiler.java: 119) >> >> at org.basex.query.QueryCompiler.compile(QueryCompiler.java: 106) >> >> at org.basex.query.QueryContext.compile(QueryContext.java: 306) >> >> at org.basex.query.QueryProcessor.compile(QueryProcessor.java: 79) >> >> at org.basex.core.cmd.AQuery.query(AQuery.java: 91) >> >> at org.basex.core.cmd.XQuery.run(XQuery.java: 22) >> >> at org.basex.core.Command.run(Command.java: 257) >> >> at org.basex.http.rest.RESTCmd.run(RESTCmd.java: 105) >> >> at org.basex.http.rest.RESTQuery.query(RESTQuery.java: 69) >> >> at org.basex.http.rest.RESTQuery.run0(RESTQuery.java: 37) >> >> at org.basex.http.rest.RESTCmd.run(RESTCmd.java: 70) >> >> at org.basex.core.Command.run(Command.java: 257) >> >> at org.basex.core.Command.execute(Command.java: 93) >> >> at org.basex.core.Command.execute(Command.java: 116) >> >> at org.basex.http.rest.RESTServlet.run(RESTServlet.java: 32) >> >> at org.basex.http.BaseXServlet.service(BaseXServlet.java: 65) >> >> >> >> If I remove the -c flag, and just let the container start (still with the >> database copied into place in the container), I get this trace when I try >> to do anything related with the database: >> >> >> >> Improper use? Potential bug? Your feedback is welcome: >> >> Contact: [email protected] >> >> Version: BaseX 9.6 RC1 >> >> Java: IcedTea, 1.8.0_212 >> >> OS: Linux, amd64 >> >> Stack Trace: >> >> java.lang.NullPointerException >> >> at org.basex.data.DiskData.write(DiskData.java:146) >> >> at org.basex.data.DiskData.close(DiskData.java:160) >> >> at org.basex.core.Datas.unpin(Datas.java:52) >> >> at org.basex.core.cmd.Close.close(Close.java:45) >> >> at org.basex.query.QueryResources.close(QueryResources.java:92) >> >> at org.basex.query.QueryContext.close(QueryContext.java:515) >> >> at org.basex.query.QueryProcessor.close(QueryProcessor.java:251) >> >> at org.basex.core.cmd.AQuery.query(AQuery.java:132) >> >> at org.basex.core.cmd.XQuery.run(XQuery.java:22) >> >> at org.basex.core.Command.run(Command.java:257) >> >> at org.basex.core.Command.execute(Command.java:93) >> >> at >> org.basex.api.client.LocalSession.execute(LocalSession.java:132) >> >> at org.basex.api.client.Session.execute(Session.java:36) >> >> at org.basex.core.CLI.execute(CLI.java:92) >> >> at org.basex.core.CLI.execute(CLI.java:76) >> >> at org.basex.BaseX.console(BaseX.java:177) >> >> at org.basex.BaseX.<init>(BaseX.java:152) >> >> at org.basex.BaseX.main(BaseX.java:43) >> >> >> >> I hope this is useful. Right now I am blocked. >> >> >> >> Best Regards >> >> >> >> Peter Villadsen. >> >> >> >> >> >> *From:* Peter Villadsen >> *Sent:* Tuesday, March 4, 2025 12:43 PM >> *To:* Christian Grün <[email protected]> >> *Cc:* [email protected] >> *Subject:* RE: [EXTERNAL] Re: [basex-talk] HTTP server performance seems >> very slow... >> >> >> >> Christian, >> >> >> >> Yes, I have. Thank you for following up - I should have come back >> earlier. >> >> >> >> I have been experimenting with this for a while now, and the container >> image (the official one and the quodatum, newer 10.3 one) both crash when I >> try to use them, both through HTTP and TCP. I am still looking into it. If >> I do not manage to find out what the issue is, I will upload the stack >> traces. >> >> >> >> In both cases, I built my own container to include the database, so I can >> avoid the volumes and have the container be completely self-contained. The >> database is 19GB, so the container gets pretty big. >> >> >> >> Here is how I start the container: >> >> >> >> docker run -d -e BASEX_JVM=-Xmx19G -p 8080:8080 -p 1984:1984 -p 8984:8984 >> rainier05042023 >> >> >> >> In my humble opinion it is unfortunate that the official container image >> has not been updated for at least 3 years. It would be nice to have the >> newest bits there, supported by BaseX. >> >> >> >> Here is the dockerfile I use: >> >> >> >> # escape=` >> >> # Use the BaseX 10.3 image as the base image >> >> FROM basex/basexhttp >> >> >> >> # Copy the Windows database directory into the container so it is >> available >> >> # when the container starts, without providing a --volume parameter. >> >> # This is fine since the database is essentially read-only. >> >> >> >> WORKDIR /srv/basex/data >> >> COPY --chown=basex:basex Rainier05042023 "Rainier05042023/" >> >> >> >> # The older versions of BaseX just use admin/admin. >> >> # RUN echo "admin" | /srv/basex/bin/basex -cPASSWORD >> >> >> >> # Modify the CMD command so that the Rainier05042023 database is opened >> >> CMD /usr/local/bin/basexhttp -c "open Rainier05042023" >> >> >> >> LABEL description="Legacy BaseX with Rainier05042023 database" >> >> >> >> # Here is a build command that builds the container with the name >> Rainier05042023: >> >> # >> >> # cd to the directory containing this Dockerfile and run the command: >> >> # docker build -t rainier05042023 . >> >> # >> >> # When the docker container has been built it can be run with the name >> >> # provided in the build command i.e. rainier05042023. It can be saved >> >> # to a file with the command: >> >> # >> >> # docker save -o Rainier05042023.tar rainier05042023 >> >> # >> >> # and loaded with the command: >> >> # >> >> # docker load -i rainier05042023.tar >> >> # >> >> # The container can be run with the command: >> >> # docker run -d -e BASEX_JVM=-Xmx19G -p 8080:8080 -p 1984:1984 -p >> 8984:8984 rainier05042023 >> >> # The database can be accessed at http://localhost:8080/dba/ >> >> >> >> Best Regards >> >> >> >> Peter VIlladsen >> >> >> >> *From:* Christian Grün <[email protected]> >> *Sent:* Tuesday, March 4, 2025 6:05 AM >> *To:* Peter Villadsen <[email protected]> >> *Cc:* [email protected] >> *Subject:* [EXTERNAL] Re: [basex-talk] HTTP server performance seems >> very slow... >> >> >> >> Hi Peter, >> >> >> >> To be sure, could you confirm that you have received my mails? >> >> >> >> Best regards, >> >> Christian >> >> >> >> >> >> On Sat, Feb 22, 2025 at 12:41 PM Christian Grün < >> [email protected]> wrote: >> >> Hi Peter, >> >> >> >> > This leads me to believe that a lot of the time (>7 seconds) may be >> spent opening the database each time a POST is done? Is there a way to >> tweak the HTTP server to “remember” the connection with the current >> database for a little while? This may be against the REST principles, of >> course. The database is guaranteed to be read-only in my case. >> >> >> >> One option is to open the database with the initial basexhttp call. It >> will be kept open until the server is shut down: >> >> >> >> basexhttp -c"open name-of-db" >> >> >> >> Best, >> >> Christian >> >> >> >> >> >> On Sat, Feb 15, 2025 at 8:53 PM Peter Villadsen via BaseX-Talk < >> [email protected]> wrote: >> >> All, >> >> >> >> I have been using BaseX for a while, connecting to the TCP endpoint. I >> know the performance I typically get, and it is impressive! However, now I >> wanted to use the HTTP endpoint, and it seems the performance is at least 2 >> orders of magnitude worse! >> >> >> >> Here is the query that I am POSTing to >> http://localhost:8984/rest/RainFnd_6.0.10.0 >> >> >> >> <query xmlns=http://basex.org/rest> >> >> <text>/Class[@Package='ApplicationPlatform']/@Name</text> >> >> </query> >> >> >> >> This simple query will generate around 1500 results from the 13GB >> database (RainFnd_6.0.10.0 <http://localhost:8984/rest/RainFnd_6.0.10.0>). >> It takes just over 7 seconds to do this. If I do this in the BaseX GUI that >> is self contained, it takes around 20ms. >> >> >> >> However, it seems that the time spent executing the query against the >> database is negligible. Please consider this query: >> >> >> >> <query xmlns=http://basex.org/rest> >> >> <text>1 + 2</text> >> >> </query> >> >> >> >> In which there there is obviously no database access. It takes almost the >> same amount of time as the query that accesses the database. 7 seconds to >> calculate 1 + 2 is too long. >> >> >> >> If I post the 1 + 2 query to the endpoint without specifying the database >> on the URL: >> >> >> >> http://localhost:8984/rest >> >> >> >> it takes around 7 milliseconds, close to what I expected, certainly >> within expectations for the time spent sending the query over the wire and >> serializing etc. >> >> >> >> This leads me to believe that a lot of the time (>7 seconds) may be spent >> opening the database each time a POST is done? Is there a way to tweak the >> HTTP server to “remember” the connection with the current database for a >> little while? This may be against the REST principles, of course. The >> database is guaranteed to be read-only in my case. >> >> >> >> The problem is that this makes the HTTP server inappropriate for >> interactive applications. I can still use the TCP server, where I get the >> results I need, but using the HTTP would be simpler, and have less overhead >> in terms of code needed to communicate with the server. >> >> >> >> Please let me know if there is a way to accomplish acceptable performance >> with the HTTP server. >> >> >> >> >> >> Best Regards >> >> >> >> Peter Villadsen >> >> Principal Technical Program Manager >> >> Microsoft Business Applications Group >> >> >> >> >> >>
