Hi Christian I am coming back from you, replying to your StackOverflow reply.
I have implemented a system where on page load the ten first results are shown, and when a user clicks a button an Ajax request is sent, and the ten next results are looked up and appended to HTML. This works great, especially in cases where many hits are found. However, this is still quite slow when you are querying for rare cases. Therefore I am curious about your opinion on the following idea: What if you set the difference between $start and $end to 1, and immediately return each hit, and fire the next ajax request? This would mean A LOT of session open / session close to basex, but it would result in a user experience where the user can see the results 'streaming in' as they are found. Is this feasible, or do you strongly advise against this - if so, why? My assumption is that there are many server queries indeed, but they all happen one at a time, so disk usage should be relatively normal. CPU usage is something else I guess. What do you think? Thanks -----Oorspronkelijk bericht----- Van: Christian Grün [mailto:christian.gr...@gmail.com] Verzonden: donderdag 21 april 2016 15:22 Aan: Bram Vanroy | KU Leuven <bram.vanr...@student.kuleuven.be> CC: BaseX <basex-talk@mailman.uni-konstanz.de> Onderwerp: Re: [basex-talk] Querying Basex from a web interface PS: The current way to only return chunks of the result is to pass on the index to the first and last result in your result, and to do the filtering in XQuery: $results[position() = $start to $end] By returning one more result than requested, the client will know that there will be more results. This may be helpful, because computing the total result size is often much more expensive than returning only the first results. Does this help? Christian On Thu, Apr 21, 2016 at 3:18 PM, Christian Grün <christian.gr...@gmail.com> wrote: > Hi Bram, > > Thanks for your query. This is just a quick response, but you could > have a look at the new %rest:single annotation in BaseX [1]. Your > feedback is welcome. > > Christian > > [1] http://docs.basex.org/wiki/RESTXQ#Query_Execution > > > > On Thu, Apr 21, 2016 at 3:11 PM, Bram Vanroy | KU Leuven > <bram.vanr...@student.kuleuven.be> wrote: >> Hello all >> >> >> >> I have been reading through many emails on this list and I’ve learnt >> a lot about how Basex works and how others use it. A month or so back >> I have sent an email myself to this list concerning caching. Even >> though I have some more questions about that, I will leave that for >> another time. Today I am concerned about retrieving chunked input from Basex. >> >> >> >> (Question also found on StackOverflow, with a nice bounty! J >> http://stackoverflow.com/questions/36675388/efficient-and-user-friend >> ly-way-to-present-slow-loading-results) >> >> >> >> Case at hand: we use Basex to query a 50 million tokens corpus. We >> also make this available to other users through a website. The thing >> is that this is slow. For our own projects that’s no problem, we dive >> straight into the back-end and run a search command from terminal and >> let the query run for all the time it needs. However, for users it is >> paramount that they get a quick response. At the moment it is taking >> too long. I don’t blame BaseX. We love BaseX and are astounded by its >> efficiency and optimisations! However, we want to deliver the best >> user-experience to our users. >> >> >> >> We call a new session from PHP, wait to receive the results, do some >> post-processing and then load the result page. As said, this takes >> too much time. We’ve been looking into some solutions. The best one >> that I think should be possible, is returning chunks of the results. >> Do you know those websites that allow you to see results but only, >> like, 20 per page? I think something similar is appropriate. When a >> user has searched for a pattern, we only show the 20 or so first >> results just so they can get an idea of the results they’d find. >> Then, when they click a button, we should query for the twenty next >> results which are then appended to the list (JavaScript solution I >> guess), and so on. Until all results have been found. Additionally, I >> will also provide a button from which users can download all results >> in a text file. This is allowed to take a longer time. The main thing is >> that users should get early feedback and results on their query. >> >> >> >> Now the question is if something like this is possible in an >> efficient manner in BaseX. Can you form a query that only finds the >> 20 first results, and then the following 20 and so on – and is this >> faster than searching everything at once? In other words, when I am >> searching for the results >> 120-140 (after having pushed the button a couple of times), is BaseX >> smart enough to skip the search space it has already done to find the >> 120 previous hits? If so, that would be great. Could you help me on >> my way, with some PHP/XQuery code that is suitable? >> >> >> >> I also highly encourage you to participate on StackOverflow. As I >> said, I am offering a 200 bounty – for the people who are interested in >> Internet fame. >> J >> >> >> >> >> >> Thank you for your time >> >> >> >> Kind regards >> >> >> >> Bram Vanroy >> >> http://bramvanroy.be
