Re: IndexedDB >> Proposed API Change: cursor.advance BACKWARD when direction is "prev"
On Fri, May 23, 2014 at 6:24 PM, marc fawzi wrote: > Here is a jsfiddle showing how .advance behaves when the range is > restricted by .only > > Create new e.g. 7 items with names like "marc" and tags like "w1 w3 w5 w2" > (random selection of tags with some tags appearing across multiple records > (per the attached image) > > Enter "w2" or "w5" in the box next to 'get by tag' and click 'get by tag' > > You'll see the first 2 matching items, with primary keys 7 and 6 > > Click 'get by tag' again and you'll see the next 2 matching items, with > primary keys 4 and 2 > > Click 'get by tag' again and you'll see the next and last matching item, > with primary key 1 > > Notice the way I advance the cursor each time in order to re-continue the > search from where I left off in the previous invocation is by using the > number of items already found > > http://jsfiddle.net/marcfawzi/y5ELj/ > > Thanks for sharing this example. > It's the correct behavior but it would be easier imo if we have .find() > for what .continue() and .continue(key) does and use .continue() to mean > .advance(1) and .continue(n) to mean .advance(n) > > But I could be totally wrong. Just a harmless feedback at this point. > > It's a reasonable suggestion. Unfortunately, we already have multiple shipping implementations and code in the wild depending on the API as specified, and this would be a breaking change. It's useful feedback if we add new cursor/iteration APIs in the future, though - the current choice of "continue" and "advance" vs. e.g. "find" or "seek" is pretty arbitrary and can be a source of confusion. Thanks again for following up with examples to ensure we understood your feedback! > :) > > > On Fri, May 23, 2014 at 1:07 PM, marc fawzi wrote: > >> << >> Thanks for following up! At least two IDB implementers were worried that >> you'd found some browser bugs we couldn't reproduce. >> >> >> Yup. I had to figure this stuff out as the API is very low level (which >> is why it can also be used in very powerful ways and also potentially very >> confusing for the uninitiated) >> >> <> not restricted, if the cursor's key=7 and direction='prev' then I would >> expect after advance(2) that key=5. If you're seeing key=2 can you post a >> sample somewhere (e.g. jsfiddle.com?)>> >> >> In the case I have say 7 items [1,2,3,4,5,6,7] and the cursor's range is >> restricted by IDBKeyRange.only(val, "prev") ... so if the matching (or in >> range) items are at 7, 6, 4, 2, 1 then I can obtain them individually or in >> contiguous ranges by advancing the cursor on each consecutive invocation of >> my search routine, like so: on first invocation advance(1) from 7 to 6, on >> second invocation advance(2) from 7 to 4, on third invocation advance(3) >> from 7 to 2 and on fourth invocation advance(4) from 7 to 1. I could also >> use advance to advance by 1 within each invocation until no matching items >> are found but only up to 2 times an invocation (for a store with 700 or >> 7 items we can advance by 1 about 200 times per invocation, but that's >> arbitrary) >> >> I can definitely post a jsfiddle if you believe the above is not in >> accordance with the spec. >> >> As to continue(n) or continue(any string), i would make that >> .find(something) >> >> >> >> On Fri, May 23, 2014 at 10:41 AM, Joshua Bell wrote: >> >>> On Fri, May 23, 2014 at 9:40 AM, marc fawzi wrote: >>> I thought .continue/advance was similar to the 'continue' statement in a for loop in that everything below the statement will be ignored and the loop would start again from the next index. So my console logging was giving confusing results. I figured it out and it works fine now. >>> >>> Thanks for following up! At least two IDB implementers were worried that >>> you'd found some browser bugs we couldn't reproduce. >>> >>> For sanity's sake, I've resorted to adding a 'return' in my code in the .success callback after every .advance and .continue so the execution flow is easier to follow. It's very confusing, from execution flow perspective, for execution to continue past .continue/.advance while at once looping asynchronously. I understand it's two different instances of the .success callback but it was entirely not clear to me from reading the docs on MDN (for example) that .advance / .continue are async. >>> >>> Long term, we expect JS to evolve better ways of expressing async calls >>> and using async results. Promises are a first step, and hopefully the >>> language also grows some syntax for them. IDB should jump on that train >>> somehow. >>> >>> Also, the description of .advance in browser vendor's documentation, e.g. on MDN, says "Advance the cursor position forward by two places" for cursor.advance(2) but what they should really say is "advance the cursor position forward by two results." For example, let's say cursor first landed on an item with primary key = 7, and you is
Re: IndexedDB >> Proposed API Change: cursor.advance BACKWARD when direction is "prev"
<< Thanks for following up! At least two IDB implementers were worried that you'd found some browser bugs we couldn't reproduce. >> Yup. I had to figure this stuff out as the API is very low level (which is why it can also be used in very powerful ways and also potentially very confusing for the uninitiated) <> In the case I have say 7 items [1,2,3,4,5,6,7] and the cursor's range is restricted by IDBKeyRange.only(val, "prev") ... so if the matching (or in range) items are at 7, 6, 4, 2, 1 then I can obtain them individually or in contiguous ranges by advancing the cursor on each consecutive invocation of my search routine, like so: on first invocation advance(1) from 7 to 6, on second invocation advance(2) from 7 to 4, on third invocation advance(3) from 7 to 2 and on fourth invocation advance(4) from 7 to 1. I could also use advance to advance by 1 within each invocation until no matching items are found but only up to 2 times an invocation (for a store with 700 or 7 items we can advance by 1 about 200 times per invocation, but that's arbitrary) I can definitely post a jsfiddle if you believe the above is not in accordance with the spec. As to continue(n) or continue(any string), i would make that .find(something) On Fri, May 23, 2014 at 10:41 AM, Joshua Bell wrote: > On Fri, May 23, 2014 at 9:40 AM, marc fawzi wrote: > >> I thought .continue/advance was similar to the 'continue' statement in a >> for loop in that everything below the statement will be ignored and the >> loop would start again from the next index. So my console logging was >> giving confusing results. I figured it out and it works fine now. >> > > Thanks for following up! At least two IDB implementers were worried that > you'd found some browser bugs we couldn't reproduce. > > >> For sanity's sake, I've resorted to adding a 'return' in my code in the >> .success callback after every .advance and .continue so the execution flow >> is easier to follow. It's very confusing, from execution flow perspective, >> for execution to continue past .continue/.advance while at once looping >> asynchronously. I understand it's two different instances of the .success >> callback but it was entirely not clear to me from reading the docs on MDN >> (for example) that .advance / .continue are async. >> > > Long term, we expect JS to evolve better ways of expressing async calls > and using async results. Promises are a first step, and hopefully the > language also grows some syntax for them. IDB should jump on that train > somehow. > > >> Also, the description of .advance in browser vendor's documentation, e.g. >> on MDN, says "Advance the cursor position forward by two places" for >> cursor.advance(2) but what they should really say is "advance the cursor >> position forward by two results." For example, let's say cursor first >> landed on an item with primary key = 7, and you issue the statement >> cursor.advance(2), I would expect it to go to the item with primary key 5 >> (for cursor direction = "prev") but instead it goes to the item with >> primary key 2 because that's the 2nd match for the range argument from the >> cursor's current position >> > > What "range argument" are you referring to? > > Assuming the store has [1,2,3,4,5,6,7,8,9] and the cursor's range is not > restricted, if the cursor's key=7 and direction='prev' then I would expect > after advance(2) that key=5. If you're seeing key=2 can you post a sample > somewhere (e.g. jsfiddle.com?) > > >> , which means that .advance(n) would be far more clear semantically >> speaking if it was simply done as .continue(n) ... I guess if there is an >> understanding that the cursor is always at a matching item and that it >> could only continue/advance to the next/prev matching item, not literal >> 'positions' in the table (i.e. sequentially through the list of all items) >> then there would be no confusion but the very concept of a cursor is >> foreign to most front end developers, and that's where the confusion comes >> from for many. >> >> My inclination as a front end developer, so far removed from database >> terminology, would be >> >> 1) to deprecate .advance in favor of .continue(n) and >> > > continue(n) already has meaning - it jumps ahead to the key with value n > > >> >> 2) if it makes sense (you have to say why it may not) have >> .continue()/.continue(n) cause the return of the execution flow similar to >> 'continue' in a for loop. >> > > The API can't change the language - you return from functions via return > or throw. Further, there are reasons you may want to do further processing > after calling continue() - e.g. there may be multiple cursors (e.g. in a > join operation) or for better performance you can call continue() as early > as possible so that the database can do its work while you're processing > the previous result. > > > >> >> What do you think? >> >> >> >> On Wed, May 21, 2014 at 10:42 AM, Joshua Bell wrote: >> >>> >>> >>> >>> On Wed, May 21, 2014 at 7:
Re: IndexedDB >> Proposed API Change: cursor.advance BACKWARD when direction is "prev"
On Fri, May 23, 2014 at 9:40 AM, marc fawzi wrote: > I thought .continue/advance was similar to the 'continue' statement in a > for loop in that everything below the statement will be ignored and the > loop would start again from the next index. So my console logging was > giving confusing results. I figured it out and it works fine now. > Thanks for following up! At least two IDB implementers were worried that you'd found some browser bugs we couldn't reproduce. > For sanity's sake, I've resorted to adding a 'return' in my code in the > .success callback after every .advance and .continue so the execution flow > is easier to follow. It's very confusing, from execution flow perspective, > for execution to continue past .continue/.advance while at once looping > asynchronously. I understand it's two different instances of the .success > callback but it was entirely not clear to me from reading the docs on MDN > (for example) that .advance / .continue are async. > Long term, we expect JS to evolve better ways of expressing async calls and using async results. Promises are a first step, and hopefully the language also grows some syntax for them. IDB should jump on that train somehow. > Also, the description of .advance in browser vendor's documentation, e.g. > on MDN, says "Advance the cursor position forward by two places" for > cursor.advance(2) but what they should really say is "advance the cursor > position forward by two results." For example, let's say cursor first > landed on an item with primary key = 7, and you issue the statement > cursor.advance(2), I would expect it to go to the item with primary key 5 > (for cursor direction = "prev") but instead it goes to the item with > primary key 2 because that's the 2nd match for the range argument from the > cursor's current position > What "range argument" are you referring to? Assuming the store has [1,2,3,4,5,6,7,8,9] and the cursor's range is not restricted, if the cursor's key=7 and direction='prev' then I would expect after advance(2) that key=5. If you're seeing key=2 can you post a sample somewhere (e.g. jsfiddle.com?) > , which means that .advance(n) would be far more clear semantically > speaking if it was simply done as .continue(n) ... I guess if there is an > understanding that the cursor is always at a matching item and that it > could only continue/advance to the next/prev matching item, not literal > 'positions' in the table (i.e. sequentially through the list of all items) > then there would be no confusion but the very concept of a cursor is > foreign to most front end developers, and that's where the confusion comes > from for many. > > My inclination as a front end developer, so far removed from database > terminology, would be > > 1) to deprecate .advance in favor of .continue(n) and > continue(n) already has meaning - it jumps ahead to the key with value n > > 2) if it makes sense (you have to say why it may not) have > .continue()/.continue(n) cause the return of the execution flow similar to > 'continue' in a for loop. > The API can't change the language - you return from functions via return or throw. Further, there are reasons you may want to do further processing after calling continue() - e.g. there may be multiple cursors (e.g. in a join operation) or for better performance you can call continue() as early as possible so that the database can do its work while you're processing the previous result. > > What do you think? > > > > On Wed, May 21, 2014 at 10:42 AM, Joshua Bell wrote: > >> >> >> >> On Wed, May 21, 2014 at 7:32 AM, Arthur Barstow wrote: >> >>> [ Bcc www-tag ; Marc - please use public-webapps for IDB discussions ] >>> >>> On 5/20/14 7:46 PM, marc fawzi wrote: >>> Hi everyone, I've been using IndexedDB for a week or so and I've noticed that cursor.advance(n) will always move n items forward regardless of cursor direction. In other words, when the cursor direction is set to "prev" as in: range = IDBKeyRange.only(someValue, "prev") and primary key is auto-incremented, the cursor, upon cursor.advance(n), will actually advance n items in the opposite direction to the cursor.continue() operation. >>> >>> >> That runs contrary to the spec. Both continue() and advance() reference >> the "steps for iterating a cursor" which picks up the direction from the >> cursor object; neither entry point alters the steps to affect the direction. >> >> When you say "you've noticed", are you observing a particular browser's >> implementation or are you interpreting the spec? I did a quick test and >> Chrome, Firefox, and IE all appear to behave as I expected when intermixing >> continue() and advance() calls with direction 'prev' - the cursor always >> moves in the same direction regardless of which call is used. >> >> Can you share sample code that demonstrates the problem, and indicate >> which browser(s) you've tested? >> >> >> >> >>> This is not only an issue of "broken sy
Re: IndexedDB >> Proposed API Change: cursor.advance BACKWARD when direction is "prev"
I thought .continue/advance was similar to the 'continue' statement in a for loop in that everything below the statement will be ignored and the loop would start again from the next index. So my console logging was giving confusing results. I figured it out and it works fine now. For sanity's sake, I've resorted to adding a 'return' in my code in the .success callback after every .advance and .continue so the execution flow is easier to follow. It's very confusing, from execution flow perspective, for execution to continue past .continue/.advance while at once looping asynchronously. I understand it's two different instances of the .success callback but it was entirely not clear to me from reading the docs on MDN (for example) that .advance / .continue are async. Also, the description of .advance in browser vendor's documentation, e.g. on MDN, says "Advance the cursor position forward by two places" for cursor.advance(2) but what they should really say is "advance the cursor position forward by two results." For example, let's say cursor first landed on an item with primary key = 7, and you issue the statement cursor.advance(2), I would expect it to go to the item with primary key 5 (for cursor direction = "prev") but instead it goes to the item with primary key 2 because that's the 2nd match for the range argument from the cursor's current position, which means that .advance(n) would be far more clear semantically speaking if it was simply done as .continue(n) ... I guess if there is an understanding that the cursor is always at a matching item and that it could only continue/advance to the next/prev matching item, not literal 'positions' in the table (i.e. sequentially through the list of all items) then there would be no confusion but the very concept of a cursor is foreign to most front end developers, and that's where the confusion comes from for many. My inclination as a front end developer, so far removed from database terminology, would be 1) to deprecate .advance in favor of .continue(n) and 2) if it makes sense (you have to say why it may not) have .continue()/.continue(n) cause the return of the execution flow similar to 'continue' in a for loop. What do you think? On Wed, May 21, 2014 at 10:42 AM, Joshua Bell wrote: > > > > On Wed, May 21, 2014 at 7:32 AM, Arthur Barstow wrote: > >> [ Bcc www-tag ; Marc - please use public-webapps for IDB discussions ] >> >> On 5/20/14 7:46 PM, marc fawzi wrote: >> >>> Hi everyone, >>> >>> I've been using IndexedDB for a week or so and I've noticed that >>> cursor.advance(n) will always move n items forward regardless of cursor >>> direction. In other words, when the cursor direction is set to "prev" as >>> in: range = IDBKeyRange.only(someValue, "prev") and primary key is >>> auto-incremented, the cursor, upon cursor.advance(n), will actually advance >>> n items in the opposite direction to the cursor.continue() operation. >> >> > That runs contrary to the spec. Both continue() and advance() reference > the "steps for iterating a cursor" which picks up the direction from the > cursor object; neither entry point alters the steps to affect the direction. > > When you say "you've noticed", are you observing a particular browser's > implementation or are you interpreting the spec? I did a quick test and > Chrome, Firefox, and IE all appear to behave as I expected when intermixing > continue() and advance() calls with direction 'prev' - the cursor always > moves in the same direction regardless of which call is used. > > Can you share sample code that demonstrates the problem, and indicate > which browser(s) you've tested? > > > > >> This is not only an issue of "broken symmetry" but it presents an >>> obstacle to doing things like: keeping a record of the primaryKey of the >>> last found item (after calling cursor.continue for say 200 times) and, long >>> after the transaction has ended, call our search function again and, upon >>> finding the same item it found first last time, advance the cursor to the >>> previously recorded primary key and call cursor.continue 200 times, from >>> that offset, and repeat whenever you need to fetch the next 200 matching >>> items. Such algorithm works in the forward direction (from oldest to newest >>> item) because cursor.advance(n) can be used to position the cursor forward >>> at the previously recorded primary key (of last found item) but it does not >>> work in the backward direction (from newest to oldest item) because there >>> is no way to make the cursor advance backward. It only advances forward, >>> regardless of its own set direction. >>> >>> This example is very rough and arbitrary. But it appears to me that the >>> cursor.advance needs to obey the cursor's own direction setting. It's >>> almost like having a car that only moves forward (and can't u-turn) and in >>> order to move backward you have to reverse the road. That's bonkers. >>> >>> What's up with that? >>> >>> How naive or terribly misguided a
Re: IndexedDB >> Proposed API Change: cursor.advance BACKWARD when direction is "prev"
On Wed, May 21, 2014 at 7:32 AM, Arthur Barstow wrote: > [ Bcc www-tag ; Marc - please use public-webapps for IDB discussions ] > > On 5/20/14 7:46 PM, marc fawzi wrote: > >> Hi everyone, >> >> I've been using IndexedDB for a week or so and I've noticed that >> cursor.advance(n) will always move n items forward regardless of cursor >> direction. In other words, when the cursor direction is set to "prev" as >> in: range = IDBKeyRange.only(someValue, "prev") and primary key is >> auto-incremented, the cursor, upon cursor.advance(n), will actually advance >> n items in the opposite direction to the cursor.continue() operation. > > That runs contrary to the spec. Both continue() and advance() reference the "steps for iterating a cursor" which picks up the direction from the cursor object; neither entry point alters the steps to affect the direction. When you say "you've noticed", are you observing a particular browser's implementation or are you interpreting the spec? I did a quick test and Chrome, Firefox, and IE all appear to behave as I expected when intermixing continue() and advance() calls with direction 'prev' - the cursor always moves in the same direction regardless of which call is used. Can you share sample code that demonstrates the problem, and indicate which browser(s) you've tested? > This is not only an issue of "broken symmetry" but it presents an obstacle >> to doing things like: keeping a record of the primaryKey of the last found >> item (after calling cursor.continue for say 200 times) and, long after the >> transaction has ended, call our search function again and, upon finding the >> same item it found first last time, advance the cursor to the previously >> recorded primary key and call cursor.continue 200 times, from that offset, >> and repeat whenever you need to fetch the next 200 matching items. Such >> algorithm works in the forward direction (from oldest to newest item) >> because cursor.advance(n) can be used to position the cursor forward at the >> previously recorded primary key (of last found item) but it does not work >> in the backward direction (from newest to oldest item) because there is no >> way to make the cursor advance backward. It only advances forward, >> regardless of its own set direction. >> >> This example is very rough and arbitrary. But it appears to me that the >> cursor.advance needs to obey the cursor's own direction setting. It's >> almost like having a car that only moves forward (and can't u-turn) and in >> order to move backward you have to reverse the road. That's bonkers. >> >> What's up with that? >> >> How naive or terribly misguided am I being? >> >> Thanks in advance. >> >> Marc >> > > >
IndexedDB >> Proposed API Change: cursor.advance BACKWARD when direction is "prev"
[ Bcc www-tag ; Marc - please use public-webapps for IDB discussions ] On 5/20/14 7:46 PM, marc fawzi wrote: Hi everyone, I've been using IndexedDB for a week or so and I've noticed that cursor.advance(n) will always move n items forward regardless of cursor direction. In other words, when the cursor direction is set to "prev" as in: range = IDBKeyRange.only(someValue, "prev") and primary key is auto-incremented, the cursor, upon cursor.advance(n), will actually advance n items in the opposite direction to the cursor.continue() operation. This is not only an issue of "broken symmetry" but it presents an obstacle to doing things like: keeping a record of the primaryKey of the last found item (after calling cursor.continue for say 200 times) and, long after the transaction has ended, call our search function again and, upon finding the same item it found first last time, advance the cursor to the previously recorded primary key and call cursor.continue 200 times, from that offset, and repeat whenever you need to fetch the next 200 matching items. Such algorithm works in the forward direction (from oldest to newest item) because cursor.advance(n) can be used to position the cursor forward at the previously recorded primary key (of last found item) but it does not work in the backward direction (from newest to oldest item) because there is no way to make the cursor advance backward. It only advances forward, regardless of its own set direction. This example is very rough and arbitrary. But it appears to me that the cursor.advance needs to obey the cursor's own direction setting. It's almost like having a car that only moves forward (and can't u-turn) and in order to move backward you have to reverse the road. That's bonkers. What's up with that? How naive or terribly misguided am I being? Thanks in advance. Marc
