Still, a slightly modified version of the original request does not
seem unreasonable. He would have to formulate his URLs something like
this:
myblog.com/comments/?q=the&first=1234
or maybe:
myblog.com/comments/?q=the&after=1234
I could see this being really useful, since encrypting (or worse,
storing on the server) the cursor is pretty painful. Furthermore, it
seems highly probable that as things are, many people will obliviously
write public webapps that take a raw cursor as a parameter. This
could be the new SQL injection attack.
Jeff
2010/2/9 Alkis Evlogimenos ('Αλκης Ευλογημένος) <evlogime...@gmail.com>:
> If the cursor had to skip entries by using an offset, its performance would
> depend on the size of the offset. This is what the current Query.fetch() api
> is doing when you give it an offset. A cursor is a pointer to the entry from
> which the next query will start. It has no notion of offset.
> On Tue, Feb 9, 2010 at 4:07 PM, Nickolas Daskalou <n...@daskalou.com> wrote:
>>
>> Does the production cursor string contain information about the app id,
>> kind, any filter()s or order()s, and (more importantly) some sort of
>> numerical value that indicates how many records the next query should
>> "skip"? If so, and if we could extract this information (and then use it
>> again to the reconstruct the cursor), that would make for much cleaner,
>> safer and intuitive URLs than including the entire cursor string (or some
>> sort of encrypted/encoded cursor string replacement).
>>
>>
>> 2010/2/10 Nick Johnson (Google) <nick.john...@google.com>
>>>
>>> Hi Nickolas,
>>>
>>> 2010/2/9 Nickolas Daskalou <n...@daskalou.com>
>>>>
>>>> I'd want to do this so that I could include parts of the cursor (such as
>>>> the offset) into a URL without including other parts (eg. the model kind
>>>> and
>>>> filters). I could then reconstruct the cursor on the server side based on
>>>> what was passed into the URL.
>>>
>>> The offset argument you're talking about is specific to the
>>> dev_appserver's implementation of cursors. In production, offsets are not
>>> used, so this won't work.
>>> -Nick Johnson
>>>
>>>>
>>>> For example, if I was searching for blog comments that contained the
>>>> word "the" (with the default order being the creation time, descending),
>>>> the
>>>> URL might look like this:
>>>>
>>>> myblog.com/comments/?q=the
>>>>
>>>> With model:
>>>>
>>>> class Comment(db.Model):
>>>> ....
>>>> created_at = db.DateTimeProperty(auto_now_add=True)
>>>> words = db.StringListProperty() # A list of all the words in a comment
>>>> (forget about exploding indexes for now)
>>>> ...
>>>>
>>>> The query object for this URL might look something like:
>>>>
>>>> ....
>>>> q =
>>>> Comment.all().filter('words',self.request.get('q')).order('-created_at')
>>>> ....
>>>>
>>>> To get to the 1001st comment, it'd be good if the URL looked something
>>>> like this:
>>>>
>>>> myblog.com/comments/?q=the&skip=1000
>>>>
>>>> instead of:
>>>>
>>>> myblog.com/comments/?q=the&cursor=[something ugly]
>>>>
>>>> so that when the request comes in, I can do this:
>>>>
>>>> ....
>>>> q =
>>>> Comment.all().filter('words',self.request.get('q')).order('-created_at')
>>>> cursor_template = q.cursor_template()
>>>> cursor =
>>>> db.Cursor.from_template(cursor_template,offset=int(self.request.get('skip')))
>>>> ....
>>>> (or something along these lines)
>>>>
>>>> Does that make sense?
>>>>
>>>>
>>>> On 10 February 2010 01:03, Nick Johnson (Google)
>>>> <nick.john...@google.com> wrote:
>>>>>
>>>>> Hi Nickolas,
>>>>>
>>>>> 2010/2/9 Nickolas Daskalou <n...@daskalou.com>
>>>>>>
>>>>>> Will we be able to construct our own cursors much the same way that we
>>>>>> are able to construct our own Datastore keys (Key.from_path())?
>>>>>
>>>>> No, not practically speaking.
>>>>>
>>>>>>
>>>>>> Also along the same lines, will we be able to "deconstruct" a cursor
>>>>>> to get its components (offset, start_inclusive etc.), as we can now do
>>>>>> with
>>>>>> keys (key.name(), key.id(), key.kind() etc.)?
>>>>>
>>>>> While you could do this, there's no guarantees that it'll work (or
>>>>> continue to work), as you'd be digging into internal implementation
>>>>> details.
>>>>> Why do you want to do this?
>>>>> -Nick Johnson
>>>>>>
>>>>>>
>>>>>> 2010/2/9 Nick Johnson (Google) <nick.john...@google.com>
>>>>>>>
>>>>>>> 2010/2/9 Stephen <sdea...@gmail.com>
>>>>>>>>
>>>>>>>> I'm asking if it's wise to store it as a query parameter embedded in
>>>>>>>> a
>>>>>>>> web page.
>>>>>>>
>>>>>>> You're right that it's unwise. Depending on how you construct your
>>>>>>> query, a user could potentially modify the cursor they send to you to
>>>>>>> return
>>>>>>> results from any query your datastore is capable of performing, which
>>>>>>> could
>>>>>>> result in you revealing information to the user that they shouldn't
>>>>>>> know.
>>>>>>> You should either store the cursor on the server-side, or encrypt it
>>>>>>> before
>>>>>>> sending it to the client.
>>>>>>> I was going to mention something about this in my post, but it
>>>>>>> slipped my mind.
>>>>>>> -Nick Johnson
>>>>>>>>
>>>>>>>> On Feb 9, 12:26 am, "Ikai L (Google)" <ika...@google.com> wrote:
>>>>>>>> > A cursor serializes to a Base64 encoded String, so you can store
>>>>>>>> > it anywhere
>>>>>>>> > you want to store strings: Memcached, Datastore, etc. You can even
>>>>>>>> > pass it
>>>>>>>> > as an URL parameter to task queues.
>>>>>>>> >
>>>>>>>> > 2010/2/8 Stephen <sdea...@gmail.com>
>>>>>>>> >
>>>>>>>> >
>>>>>>>> >
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > > Ah right, Nick's blog does say start_key and not offset. My bad.
>>>>>>>> >
>>>>>>>> > > Maybe there will be warnings in the upcoming documentation, but
>>>>>>>> > > my
>>>>>>>> > > first instinct was to embed the serialised cursor in the HTML as
>>>>>>>> > > the
>>>>>>>> > > 'next' link. But that doesn't look like a good idea as Nick's
>>>>>>>> > > decoded
>>>>>>>> > > query shows what's embedded:
>>>>>>>> >
>>>>>>>> > > PrimaryScan {
>>>>>>>> > > start_key: "shell\000TestModel\000foo\000\232bar\000\200"
>>>>>>>> > > start_inclusive: true
>>>>>>>> > > }
>>>>>>>> > > keys_only: false
>>>>>>>> >
>>>>>>>> > > First, you may or may not want to leak this info. Second, could
>>>>>>>> > > this
>>>>>>>> > > be altered on the client to change the query in any way that's
>>>>>>>> > > undesirable?
>>>>>>>> >
>>>>>>>> > > Once you have a cursor, where do you store it so you can use it
>>>>>>>> > > again?
>>>>>>>> >
>>>>>>>> > > On Feb 8, 10:17 pm, "Ikai L (Google)" <ika...@google.com> wrote:
>>>>>>>> > > > I got beaten to this answer. No, there is no traversal to get
>>>>>>>> > > > to the
>>>>>>>> > > offset.
>>>>>>>> >
>>>>>>>> > > > BigTable has an underlying mechanism for range queries on
>>>>>>>> > > > keys. Indexes
>>>>>>>> > > are
>>>>>>>> > > > essentially a key comprised of a concatenation of application
>>>>>>>> > > > ID, entity
>>>>>>>> > > > type, column, value. When a filter operation is performed, the
>>>>>>>> > > > datastore
>>>>>>>> > > > looks for a range matching this criteria, returning the set of
>>>>>>>> > > > keys. A
>>>>>>>> > > > cursor also adds the datastore key of the entity so it is
>>>>>>>> > > > possible to
>>>>>>>> > > > serialize where to begin the query. This is actually a bit
>>>>>>>> > > > awkward to
>>>>>>>> > > > explain without visuals. You can watch Ryan Barrett's talk
>>>>>>>> > > > here:
>>>>>>>> >
>>>>>>>> > > >http://www.youtube.com/watch?v=tx5gdoNpcZM
>>>>>>>> >
>>>>>>>> > > > Hopefully, we'll be able to post an article at some point in
>>>>>>>> > > > the future
>>>>>>>> > > > explaining how cursors work.
>>>>>>>> >
>>>>>>>> > > > 2010/2/8 Alkis Evlogimenos ('Αλκης Ευλογημένος)
>>>>>>>> > > > <evlogime...@gmail.com>
>>>>>>>> >
>>>>>>>> > > > > There is no offset. The protocol buffer stores a start_key
>>>>>>>> > > > > and a
>>>>>>>> > > boolean
>>>>>>>> > > > > denoting if this start key is inclusive or not. The
>>>>>>>> > > > > performance of
>>>>>>>> > > > > continuing the fetch from a cursor should be the same as the
>>>>>>>> > > performance of
>>>>>>>> > > > > the first entities you got from a query.
>>>>>>>> >
>>>>>>>> > > > > On Mon, Feb 8, 2010 at 4:33 PM, Stephen <sdea...@gmail.com>
>>>>>>>> > > > > wrote:
>>>>>>>> >
>>>>>>>> > > > >> On Feb 8, 7:06 pm, "Ikai L (Google)" <ika...@google.com>
>>>>>>>> > > > >> wrote:
>>>>>>>> > > > >> > The official docs are pending, but here's Nick Johnson to
>>>>>>>> > > > >> > the
>>>>>>>> > > rescue:
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > > >http://blog.notdot.net/2010/02/New-features-in-1-3-1-prerelease-Cursors
>>>>>>>> >
>>>>>>>> > > > >> What are the performance characteristics of cursors?
>>>>>>>> >
>>>>>>>> > > > >> The serialised cursor shows that it stores an offset. Does
>>>>>>>> > > > >> that mean
>>>>>>>> > > > >> that if the offset is one million, one million rows will
>>>>>>>> > > > >> have to be
>>>>>>>> > > > >> skipped before the next 10 are returned? This will be
>>>>>>>> > > > >> faster than
>>>>>>>> > > > >> doing it in your app, but not as quick as the existing
>>>>>>>> > > > >> bookmark
>>>>>>>> > > > >> techniques which use the primary key index.
>>>>>>>> >
>>>>>>>> > > > >> Or is the server-side stateful, like a typical SQL
>>>>>>>> > > > >> implementation of
>>>>>>>> > > > >> cursors? In which case, are there any limits to the number
>>>>>>>> > > > >> of active
>>>>>>>> > > > >> cursors? Or what if a cursor is resumed some time in the
>>>>>>>> > > > >> future; will
>>>>>>>> > > > >> it work at all, or work slower?
>>>>>>>> >
>>>>>>>> > > > >> --
>>>>>>>> > > > >> You received this message because you are subscribed to the
>>>>>>>> > > > >> Google
>>>>>>>> > > Groups
>>>>>>>> > > > >> "Google App Engine" group.
>>>>>>>> > > > >> To post to this group, send email to
>>>>>>>> > > google-appeng...@googlegroups.com.
>>>>>>>> > > > >> To unsubscribe from this group, send email to
>>>>>>>> > > > >>
>>>>>>>> > > > >> google-appengine+unsubscr...@googlegroups.com<google-appengine%2bunsubscr...@googlegroups.com>
>>>>>>>> > >
>>>>>>>> > > <google-appengine%2bunsubscr...@googlegroups.com<google-appengine%252bunsubscr...@googlegroups.com>
>>>>>>>> >
>>>>>>>> > > > >> .
>>>>>>>> > > > >> For more options, visit this group at
>>>>>>>> > > > >>http://groups.google.com/group/google-appengine?hl=en.
>>>>>>>> >
>>>>>>>> > > > > --
>>>>>>>> >
>>>>>>>> > > > > Alkis
>>>>>>>> >
>>>>>>>> > > > > --
>>>>>>>> > > > > You received this message because you are subscribed to the
>>>>>>>> > > > > Google
>>>>>>>> > > Groups
>>>>>>>> > > > > "Google App Engine" group.
>>>>>>>> > > > > To post to this group, send email to
>>>>>>>> > > > > google-appengine@googlegroups.com
>>>>>>>> > > .
>>>>>>>> > > > > To unsubscribe from this group, send email to
>>>>>>>> > > > >
>>>>>>>> > > > > google-appengine+unsubscr...@googlegroups.com<google-appengine%2bunsubscr...@googlegroups.com>
>>>>>>>> > >
>>>>>>>> > > <google-appengine%2bunsubscr...@googlegroups.com<google-appengine%252bunsubscr...@googlegroups.com>
>>>>>>>> >
>>>>>>>> > > > > .
>>>>>>>> > > > > For more options, visit this group at
>>>>>>>> > > > >http://groups.google.com/group/google-appengine?hl=en.
>>>>>>>> >
>>>>>>>> > > > --
>>>>>>>> > > > Ikai Lan
>>>>>>>> > > > Developer Programs Engineer, Google App Enginehttp://
>>>>>>>> > > googleappengine.blogspot.com|http://twitter.com/app_engine
>>>>>>>> >
>>>>>>>> > > --
>>>>>>>> > > You received this message because you are subscribed to the
>>>>>>>> > > Google Groups
>>>>>>>> > > "Google App Engine" group.
>>>>>>>> > > To post to this group, send email to
>>>>>>>> > > google-appeng...@googlegroups.com.
>>>>>>>> > > To unsubscribe from this group, send email to
>>>>>>>> > >
>>>>>>>> > > google-appengine+unsubscr...@googlegroups.com<google-appengine%2bunsubscr...@googlegroups.com>
>>>>>>>> > > .
>>>>>>>> > > For more options, visit this group at
>>>>>>>> > >http://groups.google.com/group/google-appengine?hl=en.
>>>>>>>> >
>>>>>>>> > --
>>>>>>>> > Ikai Lan
>>>>>>>> > Developer Programs Engineer, Google App
>>>>>>>> > Enginehttp://googleappengine.blogspot.com|http://twitter.com/app_engine
>>>>>>>>
>>>>>>>> --
>>>>>>>> You received this message because you are subscribed to the Google
>>>>>>>> Groups "Google App Engine" group.
>>>>>>>> To post to this group, send email to
>>>>>>>> google-appeng...@googlegroups.com.
>>>>>>>> To unsubscribe from this group, send email to
>>>>>>>> google-appengine+unsubscr...@googlegroups.com.
>>>>>>>> For more options, visit this group at
>>>>>>>> http://groups.google.com/group/google-appengine?hl=en.
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Nick Johnson, Developer Programs Engineer, App Engine
>>>>>>> Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration
>>>>>>> Number: 368047
>>>>>>>
>>>>>>> --
>>>>>>> You received this message because you are subscribed to the Google
>>>>>>> Groups "Google App Engine" group.
>>>>>>> To post to this group, send email to
>>>>>>> google-appeng...@googlegroups.com.
>>>>>>> To unsubscribe from this group, send email to
>>>>>>> google-appengine+unsubscr...@googlegroups.com.
>>>>>>> For more options, visit this group at
>>>>>>> http://groups.google.com/group/google-appengine?hl=en.
>>>>>>
>>>>>> --
>>>>>> You received this message because you are subscribed to the Google
>>>>>> Groups "Google App Engine" group.
>>>>>> To post to this group, send email to
>>>>>> google-appeng...@googlegroups.com.
>>>>>> To unsubscribe from this group, send email to
>>>>>> google-appengine+unsubscr...@googlegroups.com.
>>>>>> For more options, visit this group at
>>>>>> http://groups.google.com/group/google-appengine?hl=en.
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Nick Johnson, Developer Programs Engineer, App Engine
>>>>> Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration
>>>>> Number: 368047
>>>>>
>>>>> --
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "Google App Engine" group.
>>>>> To post to this group, send email to google-appeng...@googlegroups.com.
>>>>> To unsubscribe from this group, send email to
>>>>> google-appengine+unsubscr...@googlegroups.com.
>>>>> For more options, visit this group at
>>>>> http://groups.google.com/group/google-appengine?hl=en.
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "Google App Engine" group.
>>>> To post to this group, send email to google-appeng...@googlegroups.com.
>>>> To unsubscribe from this group, send email to
>>>> google-appengine+unsubscr...@googlegroups.com.
>>>> For more options, visit this group at
>>>> http://groups.google.com/group/google-appengine?hl=en.
>>>
>>>
>>>
>>> --
>>> Nick Johnson, Developer Programs Engineer, App Engine
>>> Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration
>>> Number: 368047
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups
>>> "Google App Engine" group.
>>> To post to this group, send email to google-appeng...@googlegroups.com.
>>> To unsubscribe from this group, send email to
>>> google-appengine+unsubscr...@googlegroups.com.
>>> For more options, visit this group at
>>> http://groups.google.com/group/google-appengine?hl=en.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Google App Engine" group.
>> To post to this group, send email to google-appeng...@googlegroups.com.
>> To unsubscribe from this group, send email to
>> google-appengine+unsubscr...@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/google-appengine?hl=en.
>
>
>
> --
>
> Alkis
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appeng...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
