Hi, The first round of tests looks good. We'll try it in production and see how it responds with multiple users.
The only issue (warning not error) I got was when trying to open a file if the server is busy with a process. The message is : [qtp1635546341-17] WARN com.bradmcevoy.http.LockInfo - resource is being locked with a null user. This won't really be locked at all... org.basex.query.QueryIOException: Cannot convert empty-sequence() to xs:string: $owner := (). at org.basex.http.webdav.WebDAVQuery.execute(WebDAVQuery.java:122) at org.basex.http.webdav.WebDAVLockService.lock(WebDAVLockService.java:74) at org.basex.http.webdav.WebDAVResource$4.get(WebDAVResource.java:138) at org.basex.http.webdav.WebDAVResource$4.get(WebDAVResource.java:1) at org.basex.http.webdav.WebDAVCode.eval(WebDAVCode.java:37) at org.basex.http.webdav.WebDAVCode.evalNoEx(WebDAVCode.java:53) at org.basex.http.webdav.WebDAVResource.lock(WebDAVResource.java:153) at com.bradmcevoy.http.webdav.LockHandler.processNewLock(LockHandler.java:225) at com.bradmcevoy.http.webdav.LockHandler.processExistingResource(LockHandler.java:110) at com.bradmcevoy.http.webdav.LockHandler.process(LockHandler.java:86) at com.bradmcevoy.http.StandardFilter.process(StandardFilter.java:52) at com.bradmcevoy.http.FilterChain.process(FilterChain.java:40) at com.bradmcevoy.http.HttpManager.process(HttpManager.java:228) at org.basex.http.webdav.WebDAVServlet.run(WebDAVServlet.java:34) at org.basex.http.BaseXServlet.service(BaseXServlet.java:59) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:856) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:535) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:531) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:319) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:175) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:133) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:754) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:672) at java.lang.Thread.run(Thread.java:745) Caused by: org.basex.query.QueryException: Cannot convert empty-sequence() to xs:string: $owner := (). at org.basex.query.QueryError.get(QueryError.java:1392) at org.basex.query.QueryError.typeError(QueryError.java:1575) at org.basex.query.value.type.SeqType.promote(SeqType.java:381) at org.basex.query.var.Var.checkType(Var.java:191) at org.basex.query.expr.gflwor.Let.optimize(Let.java:94) at org.basex.query.func.StaticFunc.inlineExpr(StaticFunc.java:286) at org.basex.query.func.StaticFuncCall.compile(StaticFuncCall.java:68) at org.basex.query.scope.MainModule.comp(MainModule.java:83) at org.basex.query.QueryCompiler.compile(QueryCompiler.java:114) at org.basex.query.QueryCompiler.compile(QueryCompiler.java:105) at org.basex.query.QueryContext.compile(QueryContext.java:302) at org.basex.query.QueryContext.iter(QueryContext.java:321) at org.basex.query.QueryProcessor.iter(QueryProcessor.java:90) at org.basex.query.QueryProcessor.value(QueryProcessor.java:99) at org.basex.http.webdav.WebDAVQuery.execute(WebDAVQuery.java:116) ... 44 more But no more duplicate and no more lock message. On Fri, May 18, 2018 at 2:15 PM, Christian Grün <[email protected]> wrote: > I believe we have fixed the reported issue. Furthermore, we didn’t > manage to trigger any of the exceptions (EOF, etc.) anymore. The new > snapshot is available. > > Cheers, > Christian > > > > On Fri, May 18, 2018 at 2:08 PM, France Baril > <[email protected]> wrote: > > Awesome, any progress is good. We've been dealing with that lock issue > for > > the longest time, but it took a while to figure out how to replicate it > > systematically. It was always so random. I'll see with the client when we > > can upgrade and will let you know how it goes. > > > > On Thu, May 17, 2018 at 9:05 PM, Christian Grün < > [email protected]> > > wrote: > >> > >> Hi France, > >> > >> Some updates: > >> > >> • I fixed the locking bug that caused a null pointer exception. > >> > >> • As you probably know, the WebDAV locks were organized in an > >> additional ~webdav database on disk. I decided to change this quite > >> fundamentally: From now on, the locks will be kept in main-memory. > >> Locks will get lost if BaseX is restarted (but I expect this to rarely > >> happen in productive environments). > >> > >> • The good news is that the oXygen WebDAV explorer will be much faster > >> now! I noticed that 50.000 internal log checks were performed with > >> oXygen. This didn’t happen with other WebDAV clients. > >> > >> I’d be pleased if you could check out the latest snapshot [1] and give > >> me an update if it works as expected. The actual problem you reported > >> has not been fixed yet, but I’m positive that things are clearing up. > >> > >> Best, > >> Christian > >> > >> [1] http://files.basex.org/releases/latest/ > >> > >> > >> > >> On Thu, May 17, 2018 at 5:42 PM, Christian Grün > >> <[email protected]> wrote: > >> > Hi France, > >> > > >> > The delay for retrieving the file list seems to be oXygen-specific: > >> > BaseX itself requires appr. 1 second to create a list of the 50.000 > >> > files, but it takes around 180 seconds until the resources are > >> > displayed in the oXygen WebDAV explorer. I tried another WebDAV > >> > implementation: With the WebDAV plugin of the windows application > >> > TotalCommander, the files are listed after 3 seconds. > >> > > >> > But back to your original question: My troubles started when I tried > >> > to open and close a file with oXygen (version 20): If I open a single > >> > resource, a NullPointerException is output by BaseX (on command line). > >> > If I close the file and try to reopen it, oXygen returns 500 (“Problem > >> > while trying to acquire lock”). > >> > > >> > Do you experience a similar behavior? Which versions of BaseX and > >> > oXygen are you currently working with? > >> > > >> > Unfortunately, the WebDAV protocol has been causing problems since the > >> > very beginning we implemented it. This is on the one hand due to the > >> > outdated library we use, on the other hand to the protocol itself > >> > (each WebDAV client seems to use it differently). Maybe you could have > >> > a look at Axxepta’s Argon Author plugin for oXygen: > >> > > >> > http://argon-author.com/ > >> > > >> > Best, > >> > Christian > >> > > >> > > >> > > >> > On Thu, May 17, 2018 at 10:55 AM, France Baril > >> > <[email protected]> wrote: > >> >> Perfect, happy we could finally find a way to get this issue > >> >> replicated!. > >> >> > >> >> We only have one DB with more than 1000 files, so display speed is > not > >> >> much > >> >> of an issue. We'd be happy to get speed anyway :-). The 50,000 files > >> >> were > >> >> just to make the bug easy to replicate. > >> >> > >> >> On Wed, May 16, 2018 at 4:04 PM, Christian Grün > >> >> <[email protected]> > >> >> wrote: > >> >>> > >> >>> Dear France, > >> >>> > >> >>> A first update: > >> >>> > >> >>> I noticed that the oXygen file access while updating the database > >> >>> causes various exceptions (which are written to the BaseX logs). As > a > >> >>> result, I also get duplicate files in the database. I will try to > find > >> >>> out if this is something we can resolve, or if it goes back to thev > >> >>> Milton WebDAV library we use. > >> >>> > >> >>> A minor info: You can speed up the duplicates lookup by using group > >> >>> by: > >> >>> > >> >>> let $duplicates := ( > >> >>> for $file-group in db:list('mydb') > >> >>> group by $path := string($file-group) > >> >>> let $count := count($file-group) > >> >>> where $count > 1 > >> >>> return <li>There are { $count } instances of { $path }.</li> > >> >>> ) > >> >>> return > >> >>> if ($duplicates) > >> >>> then <ul> { $duplicates }</ul> > >> >>> else <p>All is good. No duplicate found.</p> > >> >>> > >> >>> Apart from that, I noticed that it takes a very long time to list > the > >> >>> 50.000 files in oXygen. Yet another issues that may be due to the > >> >>> restrictions of WebDAV; but I’ll see if something can be done in > BaseX > >> >>> to get this accelerated. > >> >>> > >> >>> Best, > >> >>> Christian > >> >>> > >> >>> > >> >>> > >> >>> > >> >>> On Sun, May 13, 2018 at 3:36 PM, France Baril > >> >>> <[email protected]> wrote: > >> >>> > Hi, > >> >>> > > >> >>> > Just wondering if this slipped through the cracks. > >> >>> > > >> >>> > On Wed, May 2, 2018 at 1:11 PM, France Baril > >> >>> > <[email protected]> > >> >>> > wrote: > >> >>> >> > >> >>> >> Hi, > >> >>> >> > >> >>> >> We've been having this issue for a while and we think resolving > it > >> >>> >> may > >> >>> >> be > >> >>> >> the key to resolving an intermittent server 500 error that we've > >> >>> >> been > >> >>> >> having. > >> >>> >> > >> >>> >> When a user tries to save a file while a batch process is > running, > >> >>> >> BaseX > >> >>> >> saves duplicates of the file. > >> >>> >> > >> >>> >> How to reproduce: > >> >>> >> > >> >>> >> 1) Take a fresh BaseX 9.0.1 installation > >> >>> >> 2) Copy the attached .xqm in webapp > >> >>> >> 3) Create an empty DB called mydb > >> >>> >> 4) Access localhost:port-num/test/create-update-a-lot-of-files > to > >> >>> >> populate > >> >>> >> your db. > >> >>> >> 5) In OxygenXML, set a webdav connection to the db and open a > file, > >> >>> >> add > >> >>> >> a > >> >>> >> character in one of the elements, but don't save the file. > >> >>> >> 6) From the browser, access > >> >>> >> 'localhost:port-num/test/update-something' > >> >>> >> 7) While the process in the browser is still running, save the > file > >> >>> >> in > >> >>> >> Oxygen. You'll get a message saying that read timed out. Click ok > >> >>> >> and > >> >>> >> do not > >> >>> >> try saving the file again. > >> >>> >> 8) When the update-something process is done running, don't > resave > >> >>> >> the > >> >>> >> file in Oxygen, instead go to > >> >>> >> localhost:port-num/test/oups-duplicates. > >> >>> >> You'll get a message saying that some files are duplicated. If > >> >>> >> you > >> >>> >> don't try again from step #4 a few times. You'll only get > >> >>> >> duplicates if > >> >>> >> you > >> >>> >> get the time out message before the update-something process is > >> >>> >> still > >> >>> >> running. If you try to save the file many times, you'll get more > >> >>> >> duplicates, > >> >>> >> 4 or 6. > >> >>> >> > >> >>> >> We're not sure if it's a BaseX bug or if we are setting our user > >> >>> >> management and/or locking rules incorrectly. > >> >>> >> > >> >>> >> Do you have any suggestions? > >> >>> >> > >> >>> >> -- > >> >>> >> France Baril > >> >>> >> Architecte documentaire / Documentation architect > >> >>> >> [email protected] > >> >>> > > >> >>> > > >> >>> > > >> >>> > > >> >>> > -- > >> >>> > France Baril > >> >>> > Architecte documentaire / Documentation architect > >> >>> > [email protected] > >> >> > >> >> > >> >> > >> >> > >> >> -- > >> >> France Baril > >> >> Architecte documentaire / Documentation architect > >> >> [email protected] > > > > > > > > > > -- > > France Baril > > Architecte documentaire / Documentation architect > > [email protected] > -- France Baril Architecte documentaire / Documentation architect [email protected]
