Re: [google-appengine] Re: 1.3.1 SDK Prerelease - help us verify
They are faster - use less system resources - but I do not see an effective
way to paginate using the current implementation of cursors which are
"forward only" unless YOU store the cursor . Given that cursors are
"opaque", there is no guarantee if that will work over time.
2010/2/16 Waldemar Kornewald
> Hi,
> what's the advantage of cursors compared to key-based pagination? The
> latter at least allows for paginating backwards from any point. Why
> don't cursors just build on the same principle?
>
> Bye,
> Waldemar Kornewald
>
> On Feb 16, 5:46 pm, "Nick Johnson (Google)"
> wrote:
> > Hi Andy,
> >
> > 2010/2/16 Andy Freeman
> >
> > > > Furthermore, it
> > > > seems highly probable that as things are, many people will
> obliviously
> > > > write public webapps that take a raw cursor as a parameter. This
> > > > could be the new SQL injection attack.
> >
> > > Can you comment a bit more on the security issues?
> >
> > > AFAIK, cursors can not be used to write anything. The cursor still
> > > has to match the query with its parameters, so I don't see how they
> > > can synthesize a cursor to see anything that they haven't already seen
> > > (replay) or that they'd see by requesting more and more pages (skip
> > > ahead).
> >
> > I was mistaken when I stated that they shouldn't be sent to the user in
> the
> > clear. As you point out, in order to use a cursor, you still have to
> > reconstruct the original query, so a user could not modify a cursor to
> cause
> > you to display records they should not have access to.
> >
> > > The cursor may, as part of its "is this the right query" content,
> > > reveal something about the query.
> >
> > > Hmm - the latter seems somewhat serious. It isn't data modification,
> > > but it is a data reveal.
> >
> > This is true, though I wouldn't personally consider it a serious issue.
> That
> > is up to you, naturally.
> >
> >
> >
> > > What information can someone extract from a production cursor? Does
> > > it contain the parameters (bad) or signatures (okay if someone can't
> > > derive one parameter given the other parameters).
> >
> > It contains the complete key of the next record to be returned, along
> with
> > some extra information about the query. Feel free to experiment and see
> for
> > yourself, of course. :)
> >
> > -Nick Johnson
> >
> >
> >
> > > -andy
> >
> > > On Feb 9, 9:02 am, Jeff Schnitzer wrote:
> > > > Still, a slightly modified version of the original request does not
> > > > seem unreasonable. He would have to formulate his URLs something
> like
> > > > this:
> >
> > > > myblog.com/comments/?q=the&first=1234
> >
> > > > or maybe:
> >
> > > > myblog.com/comments/?q=the&after=1234
> >
> > > > I could see this being really useful, since encrypting (or worse,
> > > > storing on the server) the cursor is pretty painful. Furthermore, it
> > > > seems highly probable that as things are, many people will
> obliviously
> > > > write public webapps that take a raw cursor as a parameter. This
> > > > could be the new SQL injection attack.
> >
> > > > Jeff
> >
> > > > 2010/2/9 Alkis Evlogimenos ('Αλκης Ευλογημένος) <
> evlogime...@gmail.com>:>
> > > If the cursor had to skip entries by using an offset, its performance
> would
> > > > > depend on the size of the offset. This is what the current
> > > Query.fetch() api
> > > > > is doing when you give it an offset. A cursor is a pointer to the
> entry
> > > from
> > > > > which the next query will start. It has no notion of offset.
> > > > > On Tue, Feb 9, 2010 at 4:07 PM, Nickolas Daskalou <
> n...@daskalou.com>
> > > wrote:
> >
> > > > >> Does the production cursor string contain information about the
> app
> > > id,
> > > > >> kind, any filter()s or order()s, and (more importantly) some sort
> of
> > > > >> numerical value that indicates how many records the next query
> should
> > > > >> "skip"? If so, and if we could extract this information (and then
> use
> > > it
> > > > >> again to the reconstruct the cursor), that would make for much
> > > cleaner,
> > > > >> safer and intuitive URLs than including the entire cursor string
> (or
> > > some
> > > > >> sort of encrypted/encoded cursor string replacement).
> >
> > > > >> 2010/2/10 Nick Johnson (Google)
> >
> > > > >>> Hi Nickolas,
> >
> > > > >>> 2010/2/9 Nickolas Daskalou
> >
> > > > I'd want to do this so that I could include parts of the cursor
> > > (such as
> > > > the offset) into a URL without including other parts (eg. the
> model
> > > kind and
> > > > filters). I could then reconstruct the cursor on the server side
> > > based on
> > > > what was passed into the URL.
> >
> > > > >>> The offset argument you're talking about is specific to the
> > > > >>> dev_appserver's implementation of cursors. In production, offsets
> are
> > > not
> > > > >>> used, so this won't work.
> > > > >>> -Nick Johnson
> >
> > > > For example, if I was searching for blog comments that contained
> the
> > > > word "the" (with
[google-appengine] Re: 1.3.1 SDK Prerelease - help us verify
On Feb 17, 8:23 am, Takashi Matsuo wrote: > In my opinion, the only problem is the task for sending mail is *not* > idempotent. In other words, what if the task is executed more than > once? Presumably recipients will receive more than one mail. Its > somewhat annoying, isn't it? true! good point. idempotence is an important, general concern, across most systems. on app engine, if it's important that you only do something once, doing it in a request handler directly isn't really any better than doing it in a task, since both could die at any point and/or run multiple times. idempotence is expensive and difficult to get right, and the specifics often change significantly from project to project, so we generally leave it to developers themselves or client-side libraries. if you need it, i actually think i've seen open source libraries for app engine that do much of the heavy lifting for you. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: 1.3.1 SDK Prerelease - help us verify
On Wed, Feb 17, 2010 at 12:28 PM, ryan wrote: > On Feb 7, 5:52 pm, Takashi Matsuo wrote: >> >> For the time being, you can use following strategy for a workaround. >> 1) prepare a handler for sending particular mail >> 2) put this handler into the task queue in a transactional manner > > exactly! we actually don't even consider it a workaround, per se. it's > the recommended way to attach any API call or chunk of code to a > datastore transaction so that it's guaranteed to happen if the > transaction commits. > > it would take a significant amount of effort to attach another > individual API (like mail) to datastore transactions in the datastore > backend. given that, we chose to do just the task queue because it can > run arbitrary code, which means you can use it to make any API call > transactional. Its nice to know the opinion from App Engine team! Agreed that using transactional task queue for other APIs is somewhat reasonable and reliable. Having said that, the strategy is still not perfect. In my opinion, the only problem is the task for sending mail is *not* idempotent. In other words, what if the task is executed more than once? Presumably recipients will receive more than one mail. Its somewhat annoying, isn't it? As far as I know, a background process in charge of managing and triggering tasks *can* mis-recognize that the task execution is failed while the execution is actually succeeded, right? If so, there is still a possibility for receiving redundant mails. Definitely things become better than before, but is still not perfect. I know its a very difficult problem, but could you please be aware of this room for improvement? Anyway, thanks for the great new release. Developers around me and I appreciate your efforts very much! > granted, enqueueing a task to run an API call does take a little extra > setup. if that's a concern, though, the deferred library mostly > addresses it: > > http://code.google.com/appengine/articles/deferred.html > > i think it might not compatible with transactional tasks quite yet: > > http://code.google.com/p/googleappengine/issues/detail?id=2721 > > but assuming that's true i expect we'll fix it soon. Its very good to know that you're aware of the issue I posted :) (because it was left as NEW state, so I was afraid it was ignored) Regards, -- Takashi Matsuo Kay's daddy > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: 1.3.1 SDK Prerelease - help us verify
On Feb 10, 3:10 am, "Nick Johnson (Google)" wrote: > The cursor format is internal, and it's not really amenable to being parsed > like this, since it will vary depending on the type of query you're > executing. +1. the cursor format is an implementation detail. you definitely can decode a cursor, muck with it, and re-encode it, and it will work. that's definitely unsupported, though, so we'd discourage it. On Feb 16, 8:46 am, "Nick Johnson (Google)" wrote: > As you point out, in order to use a cursor, you still have to > reconstruct the original query, so a user could not modify a cursor to cause > you to display records they should not have access to. +1. this is important. > It contains the complete key of the next record to be returned, along with > some extra information about the query. Feel free to experiment and see for > yourself, of course. :) specifically, the "extra information" is your app id and the kinds and some property values, and possibly also property names, of one or more entities that were query results. the specific properties involved are the properties in the query. On Feb 16, 3:02 pm, Waldemar Kornewald wrote: > what's the advantage of cursors compared to key-based pagination? The > latter at least allows for paginating backwards from any point. Why > don't cursors just build on the same principle? the main advantage is that cursors are a built in, turnkey solution. key-based pagination takes much more work on the developer's part: extracting all of the property values and key required to resume the query, packing them up together and serializing them, then later injecting them back into the query as filter values to resume. what's more, key-based pagination often requires an extra custom index. you're right, though, key-based pagination does support paging backward, even if it generally requires yet another custom index. cursors don't easily support paging backward right now, but they definitely could. we can think about adding that in the future. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: 1.3.1 SDK Prerelease - help us verify
hah. ryan, meet groups. groups, meet ryan. that "Newer" link at the bottom is really useful. it shows all these extra messages that you don't see at first, sometimes even ones that have already said what you're about to say. so useful! :P On Feb 16, 7:28 pm, ryan wrote: > On Feb 7, 5:52 pm, Takashi Matsuo wrote: > > > > > For the time being, you can use following strategy for a workaround. > > 1) prepare a handler for sending particular mail > > 2) put this handler into the task queue in a transactional manner > > exactly! we actually don't even consider it a workaround, per se. it's > the recommended way to attach any API call or chunk of code to a > datastore transaction so that it's guaranteed to happen if the > transaction commits. > > it would take a significant amount of effort to attach another > individual API (like mail) to datastore transactions in the datastore > backend. given that, we chose to do just the task queue because it can > run arbitrary code, which means you can use it to make any API call > transactional. > > granted, enqueueing a task to run an API call does take a little extra > setup. if that's a concern, though, the deferred library mostly > addresses it: > > http://code.google.com/appengine/articles/deferred.html > > i think it might not compatible with transactional tasks quite yet: > > http://code.google.com/p/googleappengine/issues/detail?id=2721 > > but assuming that's true i expect we'll fix it soon. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: 1.3.1 SDK Prerelease - help us verify
On Feb 8, 1:33 pm, Stephen wrote: > What are the performance characteristics of cursors? good question! we'll address this in the docs, but for now... > The serialised cursor shows that it stores an offset. Does that mean > that if the offset is one million, one million rows will have to be > skipped before the next 10 are returned? This will be faster than > doing it in your app, but not as quick as the existing bookmark > techniques which use the primary key index. that offset field is very rarely used, only e.g. if you provide an offset on the original query, start it but don't actually fetch any results, then ask for the cursor. cursors store direct pointer(s) to the index row(s) where the query will resume scanning. in that sense, they work the same way as the existing bookmark techniques, except they're (obviously) much easier to use, work with any query, and don't require any extra property(ies). > Or is the server-side stateful, like a typical SQL implementation of > cursors? In which case, are there any limits to the number of active > cursors? Or what if a cursor is resumed some time in the future; will > it work at all, or work slower? no, cursors are stateless. all necessary information is included in the cursor blob itself. among other things, this happily means that resuming a cursor years later is just as fast as resuming it seconds later. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: 1.3.1 SDK Prerelease - help us verify
On Feb 7, 5:52 pm, Takashi Matsuo wrote: > > For the time being, you can use following strategy for a workaround. > 1) prepare a handler for sending particular mail > 2) put this handler into the task queue in a transactional manner exactly! we actually don't even consider it a workaround, per se. it's the recommended way to attach any API call or chunk of code to a datastore transaction so that it's guaranteed to happen if the transaction commits. it would take a significant amount of effort to attach another individual API (like mail) to datastore transactions in the datastore backend. given that, we chose to do just the task queue because it can run arbitrary code, which means you can use it to make any API call transactional. granted, enqueueing a task to run an API call does take a little extra setup. if that's a concern, though, the deferred library mostly addresses it: http://code.google.com/appengine/articles/deferred.html i think it might not compatible with transactional tasks quite yet: http://code.google.com/p/googleappengine/issues/detail?id=2721 but assuming that's true i expect we'll fix it soon. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: 1.3.1 SDK Prerelease - help us verify
On Feb 4, 3:31 pm, Kyle Heu wrote: > Do the Datastore Cursors solve the 1000 query limit? actually, we removed the 1000 result limit independent of cursors. you can now fetch or iterate more than 1000 results, no cursor needed. On Feb 5, 11:04 am, alf wrote: > and 30 sec time window? definitely not. :P -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: 1.3.1 SDK Prerelease - help us verify
Hi,
what's the advantage of cursors compared to key-based pagination? The
latter at least allows for paginating backwards from any point. Why
don't cursors just build on the same principle?
Bye,
Waldemar Kornewald
On Feb 16, 5:46 pm, "Nick Johnson (Google)"
wrote:
> Hi Andy,
>
> 2010/2/16 Andy Freeman
>
> > > Furthermore, it
> > > seems highly probable that as things are, many people will obliviously
> > > write public webapps that take a raw cursor as a parameter. This
> > > could be the new SQL injection attack.
>
> > Can you comment a bit more on the security issues?
>
> > AFAIK, cursors can not be used to write anything. The cursor still
> > has to match the query with its parameters, so I don't see how they
> > can synthesize a cursor to see anything that they haven't already seen
> > (replay) or that they'd see by requesting more and more pages (skip
> > ahead).
>
> I was mistaken when I stated that they shouldn't be sent to the user in the
> clear. As you point out, in order to use a cursor, you still have to
> reconstruct the original query, so a user could not modify a cursor to cause
> you to display records they should not have access to.
>
> > The cursor may, as part of its "is this the right query" content,
> > reveal something about the query.
>
> > Hmm - the latter seems somewhat serious. It isn't data modification,
> > but it is a data reveal.
>
> This is true, though I wouldn't personally consider it a serious issue. That
> is up to you, naturally.
>
>
>
> > What information can someone extract from a production cursor? Does
> > it contain the parameters (bad) or signatures (okay if someone can't
> > derive one parameter given the other parameters).
>
> It contains the complete key of the next record to be returned, along with
> some extra information about the query. Feel free to experiment and see for
> yourself, of course. :)
>
> -Nick Johnson
>
>
>
> > -andy
>
> > On Feb 9, 9:02 am, Jeff Schnitzer wrote:
> > > Still, a slightly modified version of the original request does not
> > > seem unreasonable. He would have to formulate his URLs something like
> > > this:
>
> > > myblog.com/comments/?q=the&first=1234
>
> > > or maybe:
>
> > > myblog.com/comments/?q=the&after=1234
>
> > > I could see this being really useful, since encrypting (or worse,
> > > storing on the server) the cursor is pretty painful. Furthermore, it
> > > seems highly probable that as things are, many people will obliviously
> > > write public webapps that take a raw cursor as a parameter. This
> > > could be the new SQL injection attack.
>
> > > Jeff
>
> > > 2010/2/9 Alkis Evlogimenos ('Αλκης Ευλογημένος) :>
> > If the cursor had to skip entries by using an offset, its performance would
> > > > depend on the size of the offset. This is what the current
> > Query.fetch() api
> > > > is doing when you give it an offset. A cursor is a pointer to the entry
> > from
> > > > which the next query will start. It has no notion of offset.
> > > > On Tue, Feb 9, 2010 at 4:07 PM, Nickolas Daskalou
> > wrote:
>
> > > >> Does the production cursor string contain information about the app
> > id,
> > > >> kind, any filter()s or order()s, and (more importantly) some sort of
> > > >> numerical value that indicates how many records the next query should
> > > >> "skip"? If so, and if we could extract this information (and then use
> > it
> > > >> again to the reconstruct the cursor), that would make for much
> > cleaner,
> > > >> safer and intuitive URLs than including the entire cursor string (or
> > some
> > > >> sort of encrypted/encoded cursor string replacement).
>
> > > >> 2010/2/10 Nick Johnson (Google)
>
> > > >>> Hi Nickolas,
>
> > > >>> 2010/2/9 Nickolas Daskalou
>
> > > I'd want to do this so that I could include parts of the cursor
> > (such as
> > > the offset) into a URL without including other parts (eg. the model
> > kind and
> > > filters). I could then reconstruct the cursor on the server side
> > based on
> > > what was passed into the URL.
>
> > > >>> The offset argument you're talking about is specific to the
> > > >>> dev_appserver's implementation of cursors. In production, offsets are
> > not
> > > >>> used, so this won't work.
> > > >>> -Nick Johnson
>
> > > For example, if I was searching for blog comments that contained the
> > > word "the" (with the default order being the creation time,
> > descending), the
> > > URL might look like this:
>
> > > myblog.com/comments/?q=the
>
> > > With model:
>
> > > class Comment(db.Model):
> > >
> > > created_at = db.DateTimeProperty(auto_now_add=True)
> > > words = db.StringListProperty() # A list of all the words in a
> > comment
> > > (forget about exploding indexes for now)
> > > ...
>
> > > The query object for this URL might look something like:
>
> > >
> > > q =
>
> > Comment.all().filter('words',self.request.get('q')).order('-created_at')
> >
[google-appengine] Re: 1.3.1 SDK Prerelease - help us verify
I just noticed that the maven repository was updated and all jars that I need are there. many thanks. On Feb 14, 6:27 pm, Christian Schuhegger wrote: > Hello, > > I understand that this is a prerelease, but is there a maven > repository where I can reference this new SDK? I did not find it > at:http://maven-gae-plugin.googlecode.com/svn/repository/com/google/appe... -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: 1.3.1 SDK Prerelease - help us verify
> Furthermore, it
> seems highly probable that as things are, many people will obliviously
> write public webapps that take a raw cursor as a parameter. This
> could be the new SQL injection attack.
Can you comment a bit more on the security issues?
AFAIK, cursors can not be used to write anything. The cursor still
has to match the query with its parameters, so I don't see how they
can synthesize a cursor to see anything that they haven't already seen
(replay) or that they'd see by requesting more and more pages (skip
ahead).
The cursor may, as part of its "is this the right query" content,
reveal something about the query.
Hmm - the latter seems somewhat serious. It isn't data modification,
but it is a data reveal.
What information can someone extract from a production cursor? Does
it contain the parameters (bad) or signatures (okay if someone can't
derive one parameter given the other parameters).
-andy
On Feb 9, 9:02 am, Jeff Schnitzer wrote:
> Still, a slightly modified version of the original request does not
> seem unreasonable. He would have to formulate his URLs something like
> this:
>
> myblog.com/comments/?q=the&first=1234
>
> or maybe:
>
> myblog.com/comments/?q=the&after=1234
>
> I could see this being really useful, since encrypting (or worse,
> storing on the server) the cursor is pretty painful. Furthermore, it
> seems highly probable that as things are, many people will obliviously
> write public webapps that take a raw cursor as a parameter. This
> could be the new SQL injection attack.
>
> Jeff
>
> 2010/2/9 Alkis Evlogimenos ('Αλκης Ευλογημένος) :> If
> the cursor had to skip entries by using an offset, its performance would
> > depend on the size of the offset. This is what the current Query.fetch() api
> > is doing when you give it an offset. A cursor is a pointer to the entry from
> > which the next query will start. It has no notion of offset.
> > On Tue, Feb 9, 2010 at 4:07 PM, Nickolas Daskalou wrote:
>
> >> Does the production cursor string contain information about the app id,
> >> kind, any filter()s or order()s, and (more importantly) some sort of
> >> numerical value that indicates how many records the next query should
> >> "skip"? If so, and if we could extract this information (and then use it
> >> again to the reconstruct the cursor), that would make for much cleaner,
> >> safer and intuitive URLs than including the entire cursor string (or some
> >> sort of encrypted/encoded cursor string replacement).
>
> >> 2010/2/10 Nick Johnson (Google)
>
> >>> Hi Nickolas,
>
> >>> 2010/2/9 Nickolas Daskalou
>
> I'd want to do this so that I could include parts of the cursor (such as
> the offset) into a URL without including other parts (eg. the model kind
> and
> filters). I could then reconstruct the cursor on the server side based on
> what was passed into the URL.
>
> >>> The offset argument you're talking about is specific to the
> >>> dev_appserver's implementation of cursors. In production, offsets are not
> >>> used, so this won't work.
> >>> -Nick Johnson
>
> For example, if I was searching for blog comments that contained the
> word "the" (with the default order being the creation time, descending),
> the
> URL might look like this:
>
> myblog.com/comments/?q=the
>
> With model:
>
> class Comment(db.Model):
>
> created_at = db.DateTimeProperty(auto_now_add=True)
> words = db.StringListProperty() # A list of all the words in a comment
> (forget about exploding indexes for now)
> ...
>
> The query object for this URL might look something like:
>
>
> q =
> Comment.all().filter('words',self.request.get('q')).order('-created_at')
>
>
> To get to the 1001st comment, it'd be good if the URL looked something
> like this:
>
> myblog.com/comments/?q=the&skip=1000
>
> instead of:
>
> myblog.com/comments/?q=the&cursor=[something ugly]
>
> so that when the request comes in, I can do this:
>
>
> q =
> Comment.all().filter('words',self.request.get('q')).order('-created_at')
> cursor_template = q.cursor_template()
> cursor =
> db.Cursor.from_template(cursor_template,offset=int(self.request.get('skip')))
>
> (or something along these lines)
>
> Does that make sense?
>
> On 10 February 2010 01:03, Nick Johnson (Google)
> wrote:
>
> > Hi Nickolas,
>
> > 2010/2/9 Nickolas Daskalou
>
> >> Will we be able to construct our own cursors much the same way that we
> >> are able to construct our own Datastore keys (Key.from_path())?
>
> > No, not practically speaking.
>
> >> Also along the same lines, will we be able to "deconstruct" a cursor
> >> to get its components (offset, start_inclusive etc.), as we can now do
> >> with
> >> keys (key.name(), key.id(), key.kind() etc.)?
>
> > While you could do this, there
[google-appengine] Re: 1.3.1 SDK Prerelease - help us verify
Hi Folks,
great discussion - I'm still unable to understand if cursors can be
used to implement a clear cut pagination solution (assuming we encrypt
the cursor key) and send it to the html page. As of now, the offset
and the total number of results seems like the only way to go.
By pagination, I mean (next and previous) and that being available at
any point:
a' la
http://www.google.com/search?hl=en&q=test+internet+speed&start=10&sa=N
On Feb 10, 3:10 am, "Nick Johnson (Google)"
wrote:
> Hi Nickolas,
>
> 2010/2/10 Nickolas Daskalou
>
>
>
>
>
> > What Jeff suggests would also be great. So one page 1 we could do something
> > like:
>
> >
> > next_page_url = 'http://myblog.com/comments/?q=the&first=%d'%
> > cursor.start_key().id()
> >
>
> > and then reconstruct the cursor on page 2:
>
> >
> > cursor =
> > db.Cursor(start_key=db.Key('Comment',int(self.request.get('first'))), )
> >
>
> > How about it Google? Will we be able to do this?
>
> The cursor format is internal, and it's not really amenable to being parsed
> like this, since it will vary depending on the type of query you're
> executing.
>
> -Nick Johnson
>
>
>
>
>
> > 2010/2/10 Jeff Schnitzer
>
> > In case it wasn't completely clear - 1234 in this example is the
> >> object's id, not an offset.
>
> >> Jeff
>
> >> On Tue, Feb 9, 2010 at 9:02 AM, Jeff Schnitzer
> >> wrote:
> >> > Still, a slightly modified version of the original request does not
> >> > seem unreasonable. He would have to formulate his URLs something like
> >> > this:
>
> >> > myblog.com/comments/?q=the&first=1234
>
> >> > or maybe:
>
> >> > myblog.com/comments/?q=the&after=1234
>
> >> > I could see this being really useful, since encrypting (or worse,
> >> > storing on the server) the cursor is pretty painful. Furthermore, it
> >> > seems highly probable that as things are, many people will obliviously
> >> > write public webapps that take a raw cursor as a parameter. This
> >> > could be the new SQL injection attack.
>
> >> > Jeff
>
> >> > 2010/2/9 Alkis Evlogimenos ('Αλκης Ευλογημένος) >> >:
> >> >> If the cursor had to skip entries by using an offset, its performance
> >> would
> >> >> depend on the size of the offset. This is what the current
> >> Query.fetch() api
> >> >> is doing when you give it an offset. A cursor is a pointer to the entry
> >> from
> >> >> which the next query will start. It has no notion of offset.
> >> >> On Tue, Feb 9, 2010 at 4:07 PM, Nickolas Daskalou
> >> wrote:
>
> >> >>> Does the production cursor string contain information about the app
> >> id,
> >> >>> kind, any filter()s or order()s, and (more importantly) some sort of
> >> >>> numerical value that indicates how many records the next query should
> >> >>> "skip"? If so, and if we could extract this information (and then use
> >> it
> >> >>> again to the reconstruct the cursor), that would make for much
> >> cleaner,
> >> >>> safer and intuitive URLs than including the entire cursor string (or
> >> some
> >> >>> sort of encrypted/encoded cursor string replacement).
>
> >> >>> 2010/2/10 Nick Johnson (Google)
>
> >> Hi Nickolas,
>
> >> 2010/2/9 Nickolas Daskalou
>
> >> > I'd want to do this so that I could include parts of the cursor
> >> (such as
> >> > the offset) into a URL without including other parts (eg. the model
> >> kind and
> >> > filters). I could then reconstruct the cursor on the server side
> >> based on
> >> > what was passed into the URL.
>
> >> The offset argument you're talking about is specific to the
> >> dev_appserver's implementation of cursors. In production, offsets are
> >> not
> >> used, so this won't work.
> >> -Nick Johnson
>
> >> > For example, if I was searching for blog comments that contained the
> >> > word "the" (with the default order being the creation time,
> >> descending), the
> >> > URL might look like this:
>
> >> > myblog.com/comments/?q=the
>
> >> > With model:
>
> >> > class Comment(db.Model):
> >> >
> >> > created_at = db.DateTimeProperty(auto_now_add=True)
> >> > words = db.StringListProperty() # A list of all the words in a
> >> comment
> >> > (forget about exploding indexes for now)
> >> > ...
>
> >> > The query object for this URL might look something like:
>
> >> >
> >> > q =
>
> >> Comment.all().filter('words',self.request.get('q')).order('-created_at')
> >> >
>
> >> > To get to the 1001st comment, it'd be good if the URL looked
> >> something
> >> > like this:
>
> >> > myblog.com/comments/?q=the&skip=1000
>
> >> > instead of:
>
> >> > myblog.com/comments/?q=the&cursor=[something ugly]
>
> >> > so that when the request comes in, I can do this:
>
> >> >
> >> > q =
>
> >> Comment.all().filter('words',self.request.get('q')).order('-created_at')
> >> > cursor_template = q.cursor_template()
> >> > cursor =
>
> >> db.Cursor.from_template(cursor_templ
[google-appengine] Re: 1.3.1 SDK Prerelease - help us verify
Hello, I understand that this is a prerelease, but is there a maven repository where I can reference this new SDK? I did not find it at: http://maven-gae-plugin.googlecode.com/svn/repository/com/google/appengine/ -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: 1.3.1 SDK Prerelease - help us verify
In case it wasn't completely clear - 1234 in this example is the
object's id, not an offset.
Jeff
On Tue, Feb 9, 2010 at 9:02 AM, Jeff Schnitzer wrote:
> Still, a slightly modified version of the original request does not
> seem unreasonable. He would have to formulate his URLs something like
> this:
>
> myblog.com/comments/?q=the&first=1234
>
> or maybe:
>
> myblog.com/comments/?q=the&after=1234
>
> I could see this being really useful, since encrypting (or worse,
> storing on the server) the cursor is pretty painful. Furthermore, it
> seems highly probable that as things are, many people will obliviously
> write public webapps that take a raw cursor as a parameter. This
> could be the new SQL injection attack.
>
> Jeff
>
> 2010/2/9 Alkis Evlogimenos ('Αλκης Ευλογημένος) :
>> If the cursor had to skip entries by using an offset, its performance would
>> depend on the size of the offset. This is what the current Query.fetch() api
>> is doing when you give it an offset. A cursor is a pointer to the entry from
>> which the next query will start. It has no notion of offset.
>> On Tue, Feb 9, 2010 at 4:07 PM, Nickolas Daskalou wrote:
>>>
>>> Does the production cursor string contain information about the app id,
>>> kind, any filter()s or order()s, and (more importantly) some sort of
>>> numerical value that indicates how many records the next query should
>>> "skip"? If so, and if we could extract this information (and then use it
>>> again to the reconstruct the cursor), that would make for much cleaner,
>>> safer and intuitive URLs than including the entire cursor string (or some
>>> sort of encrypted/encoded cursor string replacement).
>>>
>>>
>>> 2010/2/10 Nick Johnson (Google)
Hi Nickolas,
2010/2/9 Nickolas Daskalou
>
> I'd want to do this so that I could include parts of the cursor (such as
> the offset) into a URL without including other parts (eg. the model kind
> and
> filters). I could then reconstruct the cursor on the server side based on
> what was passed into the URL.
The offset argument you're talking about is specific to the
dev_appserver's implementation of cursors. In production, offsets are not
used, so this won't work.
-Nick Johnson
>
> For example, if I was searching for blog comments that contained the
> word "the" (with the default order being the creation time, descending),
> the
> URL might look like this:
>
> myblog.com/comments/?q=the
>
> With model:
>
> class Comment(db.Model):
>
> created_at = db.DateTimeProperty(auto_now_add=True)
> words = db.StringListProperty() # A list of all the words in a comment
> (forget about exploding indexes for now)
> ...
>
> The query object for this URL might look something like:
>
>
> q =
> Comment.all().filter('words',self.request.get('q')).order('-created_at')
>
>
> To get to the 1001st comment, it'd be good if the URL looked something
> like this:
>
> myblog.com/comments/?q=the&skip=1000
>
> instead of:
>
> myblog.com/comments/?q=the&cursor=[something ugly]
>
> so that when the request comes in, I can do this:
>
>
> q =
> Comment.all().filter('words',self.request.get('q')).order('-created_at')
> cursor_template = q.cursor_template()
> cursor =
> db.Cursor.from_template(cursor_template,offset=int(self.request.get('skip')))
>
> (or something along these lines)
>
> Does that make sense?
>
>
> On 10 February 2010 01:03, Nick Johnson (Google)
> wrote:
>>
>> Hi Nickolas,
>>
>> 2010/2/9 Nickolas Daskalou
>>>
>>> Will we be able to construct our own cursors much the same way that we
>>> are able to construct our own Datastore keys (Key.from_path())?
>>
>> No, not practically speaking.
>>
>>>
>>> Also along the same lines, will we be able to "deconstruct" a cursor
>>> to get its components (offset, start_inclusive etc.), as we can now do
>>> with
>>> keys (key.name(), key.id(), key.kind() etc.)?
>>
>> While you could do this, there's no guarantees that it'll work (or
>> continue to work), as you'd be digging into internal implementation
>> details.
>> Why do you want to do this?
>> -Nick Johnson
>>>
>>>
>>> 2010/2/9 Nick Johnson (Google)
2010/2/9 Stephen
>
> I'm asking if it's wise to store it as a query parameter embedded in
> a
> web page.
You're right that it's unwise. Depending on how you construct your
query, a user could potentially modify the cursor they send to you to
return
results from any query your datastore is capable of performing, which
could
result in you revealing information to the user that they shouldn't
Re: [google-appengine] Re: 1.3.1 SDK Prerelease - help us verify
Still, a slightly modified version of the original request does not
seem unreasonable. He would have to formulate his URLs something like
this:
myblog.com/comments/?q=the&first=1234
or maybe:
myblog.com/comments/?q=the&after=1234
I could see this being really useful, since encrypting (or worse,
storing on the server) the cursor is pretty painful. Furthermore, it
seems highly probable that as things are, many people will obliviously
write public webapps that take a raw cursor as a parameter. This
could be the new SQL injection attack.
Jeff
2010/2/9 Alkis Evlogimenos ('Αλκης Ευλογημένος) :
> If the cursor had to skip entries by using an offset, its performance would
> depend on the size of the offset. This is what the current Query.fetch() api
> is doing when you give it an offset. A cursor is a pointer to the entry from
> which the next query will start. It has no notion of offset.
> On Tue, Feb 9, 2010 at 4:07 PM, Nickolas Daskalou wrote:
>>
>> Does the production cursor string contain information about the app id,
>> kind, any filter()s or order()s, and (more importantly) some sort of
>> numerical value that indicates how many records the next query should
>> "skip"? If so, and if we could extract this information (and then use it
>> again to the reconstruct the cursor), that would make for much cleaner,
>> safer and intuitive URLs than including the entire cursor string (or some
>> sort of encrypted/encoded cursor string replacement).
>>
>>
>> 2010/2/10 Nick Johnson (Google)
>>>
>>> Hi Nickolas,
>>>
>>> 2010/2/9 Nickolas Daskalou
I'd want to do this so that I could include parts of the cursor (such as
the offset) into a URL without including other parts (eg. the model kind
and
filters). I could then reconstruct the cursor on the server side based on
what was passed into the URL.
>>>
>>> The offset argument you're talking about is specific to the
>>> dev_appserver's implementation of cursors. In production, offsets are not
>>> used, so this won't work.
>>> -Nick Johnson
>>>
For example, if I was searching for blog comments that contained the
word "the" (with the default order being the creation time, descending),
the
URL might look like this:
myblog.com/comments/?q=the
With model:
class Comment(db.Model):
created_at = db.DateTimeProperty(auto_now_add=True)
words = db.StringListProperty() # A list of all the words in a comment
(forget about exploding indexes for now)
...
The query object for this URL might look something like:
q =
Comment.all().filter('words',self.request.get('q')).order('-created_at')
To get to the 1001st comment, it'd be good if the URL looked something
like this:
myblog.com/comments/?q=the&skip=1000
instead of:
myblog.com/comments/?q=the&cursor=[something ugly]
so that when the request comes in, I can do this:
q =
Comment.all().filter('words',self.request.get('q')).order('-created_at')
cursor_template = q.cursor_template()
cursor =
db.Cursor.from_template(cursor_template,offset=int(self.request.get('skip')))
(or something along these lines)
Does that make sense?
On 10 February 2010 01:03, Nick Johnson (Google)
wrote:
>
> Hi Nickolas,
>
> 2010/2/9 Nickolas Daskalou
>>
>> Will we be able to construct our own cursors much the same way that we
>> are able to construct our own Datastore keys (Key.from_path())?
>
> No, not practically speaking.
>
>>
>> Also along the same lines, will we be able to "deconstruct" a cursor
>> to get its components (offset, start_inclusive etc.), as we can now do
>> with
>> keys (key.name(), key.id(), key.kind() etc.)?
>
> While you could do this, there's no guarantees that it'll work (or
> continue to work), as you'd be digging into internal implementation
> details.
> Why do you want to do this?
> -Nick Johnson
>>
>>
>> 2010/2/9 Nick Johnson (Google)
>>>
>>> 2010/2/9 Stephen
I'm asking if it's wise to store it as a query parameter embedded in
a
web page.
>>>
>>> You're right that it's unwise. Depending on how you construct your
>>> query, a user could potentially modify the cursor they send to you to
>>> return
>>> results from any query your datastore is capable of performing, which
>>> could
>>> result in you revealing information to the user that they shouldn't
>>> know.
>>> You should either store the cursor on the server-side, or encrypt it
>>> before
>>> sending it to the client.
>>> I was going to mention something about this in my post, but it
>>> slipped my mind.
>>> -Nick Johnson
On Feb 9, 12:26 am, "Ikai L (Google)
Re: [google-appengine] Re: 1.3.1 SDK Prerelease - help us verify
If the cursor had to skip entries by using an offset, its performance would
depend on the size of the offset. This is what the current Query.fetch() api
is doing when you give it an offset. A cursor is a pointer to the entry from
which the next query will start. It has no notion of offset.
On Tue, Feb 9, 2010 at 4:07 PM, Nickolas Daskalou wrote:
> Does the production cursor string contain information about the app id,
> kind, any filter()s or order()s, and (more importantly) some sort of
> numerical value that indicates how many records the next query should
> "skip"? If so, and if we could extract this information (and then use it
> again to the reconstruct the cursor), that would make for much cleaner,
> safer and intuitive URLs than including the entire cursor string (or some
> sort of encrypted/encoded cursor string replacement).
>
>
> 2010/2/10 Nick Johnson (Google)
>
> Hi Nickolas,
>>
>> 2010/2/9 Nickolas Daskalou
>>
>>> I'd want to do this so that I could include parts of the cursor (such as
>>> the offset) into a URL without including other parts (eg. the model kind and
>>> filters). I could then reconstruct the cursor on the server side based on
>>> what was passed into the URL.
>>>
>>
>> The offset argument you're talking about is specific to the
>> dev_appserver's implementation of cursors. In production, offsets are not
>> used, so this won't work.
>>
>> -Nick Johnson
>>
>>
>>>
>>> For example, if I was searching for blog comments that contained the word
>>> "the" (with the default order being the creation time, descending), the URL
>>> might look like this:
>>>
>>> myblog.com/comments/?q=the
>>>
>>> With model:
>>>
>>> class Comment(db.Model):
>>>
>>> created_at = db.DateTimeProperty(auto_now_add=True)
>>> words = db.StringListProperty() # A list of all the words in a comment
>>> (forget about exploding indexes for now)
>>> ...
>>>
>>> The query object for this URL might look something like:
>>>
>>>
>>> q =
>>> Comment.all().filter('words',self.request.get('q')).order('-created_at')
>>>
>>>
>>> To get to the 1001st comment, it'd be good if the URL looked something
>>> like this:
>>>
>>> myblog.com/comments/?q=the&skip=1000
>>>
>>> instead of:
>>>
>>> myblog.com/comments/?q=the&cursor=[something ugly]
>>>
>>> so that when the request comes in, I can do this:
>>>
>>>
>>> q =
>>> Comment.all().filter('words',self.request.get('q')).order('-created_at')
>>> cursor_template = q.cursor_template()
>>> cursor =
>>> db.Cursor.from_template(cursor_template,offset=int(self.request.get('skip')))
>>>
>>> (or something along these lines)
>>>
>>> Does that make sense?
>>>
>>>
>>>
>>> On 10 February 2010 01:03, Nick Johnson (Google) <
>>> nick.john...@google.com> wrote:
>>>
Hi Nickolas,
2010/2/9 Nickolas Daskalou
Will we be able to construct our own cursors much the same way that we
> are able to construct our own Datastore keys (Key.from_path())?
>
No, not practically speaking.
>
> Also along the same lines, will we be able to "deconstruct" a cursor to
> get its components (offset, start_inclusive etc.), as we can now do with
> keys (key.name(), key.id(), key.kind() etc.)?
>
While you could do this, there's no guarantees that it'll work (or
continue to work), as you'd be digging into internal implementation
details.
Why do you want to do this?
-Nick Johnson
>
> 2010/2/9 Nick Johnson (Google)
>
>> 2010/2/9 Stephen
>>
>>
>>> I'm asking if it's wise to store it as a query parameter embedded in
>>> a
>>> web page.
>>>
>>
>> You're right that it's unwise. Depending on how you construct your
>> query, a user could potentially modify the cursor they send to you to
>> return
>> results from any query your datastore is capable of performing, which
>> could
>> result in you revealing information to the user that they shouldn't know.
>> You should either store the cursor on the server-side, or encrypt it
>> before
>> sending it to the client.
>>
>> I was going to mention something about this in my post, but it slipped
>> my mind.
>>
>> -Nick Johnson
>>
>>>
>>>
>>> On Feb 9, 12:26 am, "Ikai L (Google)" wrote:
>>> > A cursor serializes to a Base64 encoded String, so you can store it
>>> anywhere
>>> > you want to store strings: Memcached, Datastore, etc. You can even
>>> pass it
>>> > as an URL parameter to task queues.
>>> >
>>> > 2010/2/8 Stephen
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > > Ah right, Nick's blog does say start_key and not offset. My bad.
>>> >
>>> > > Maybe there will be warnings in the upcoming documentation, but
>>> my
>>> > > first instinct was to embed the serialised cursor in the HTML as
>>> the
>>> > > 'next' link. But that doesn't look like a good ide
Re: [google-appengine] Re: 1.3.1 SDK Prerelease - help us verify
Does the production cursor string contain information about the app id,
kind, any filter()s or order()s, and (more importantly) some sort of
numerical value that indicates how many records the next query should
"skip"? If so, and if we could extract this information (and then use it
again to the reconstruct the cursor), that would make for much cleaner,
safer and intuitive URLs than including the entire cursor string (or some
sort of encrypted/encoded cursor string replacement).
2010/2/10 Nick Johnson (Google)
> Hi Nickolas,
>
> 2010/2/9 Nickolas Daskalou
>
>> I'd want to do this so that I could include parts of the cursor (such as
>> the offset) into a URL without including other parts (eg. the model kind and
>> filters). I could then reconstruct the cursor on the server side based on
>> what was passed into the URL.
>>
>
> The offset argument you're talking about is specific to the dev_appserver's
> implementation of cursors. In production, offsets are not used, so this
> won't work.
>
> -Nick Johnson
>
>
>>
>> For example, if I was searching for blog comments that contained the word
>> "the" (with the default order being the creation time, descending), the URL
>> might look like this:
>>
>> myblog.com/comments/?q=the
>>
>> With model:
>>
>> class Comment(db.Model):
>>
>> created_at = db.DateTimeProperty(auto_now_add=True)
>> words = db.StringListProperty() # A list of all the words in a comment
>> (forget about exploding indexes for now)
>> ...
>>
>> The query object for this URL might look something like:
>>
>>
>> q =
>> Comment.all().filter('words',self.request.get('q')).order('-created_at')
>>
>>
>> To get to the 1001st comment, it'd be good if the URL looked something
>> like this:
>>
>> myblog.com/comments/?q=the&skip=1000
>>
>> instead of:
>>
>> myblog.com/comments/?q=the&cursor=[something ugly]
>>
>> so that when the request comes in, I can do this:
>>
>>
>> q =
>> Comment.all().filter('words',self.request.get('q')).order('-created_at')
>> cursor_template = q.cursor_template()
>> cursor =
>> db.Cursor.from_template(cursor_template,offset=int(self.request.get('skip')))
>>
>> (or something along these lines)
>>
>> Does that make sense?
>>
>>
>>
>> On 10 February 2010 01:03, Nick Johnson (Google) > > wrote:
>>
>>> Hi Nickolas,
>>>
>>> 2010/2/9 Nickolas Daskalou
>>>
>>> Will we be able to construct our own cursors much the same way that we
are able to construct our own Datastore keys (Key.from_path())?
>>>
>>> No, not practically speaking.
>>>
>>>
Also along the same lines, will we be able to "deconstruct" a cursor to
get its components (offset, start_inclusive etc.), as we can now do with
keys (key.name(), key.id(), key.kind() etc.)?
>>>
>>> While you could do this, there's no guarantees that it'll work (or
>>> continue to work), as you'd be digging into internal implementation details.
>>> Why do you want to do this?
>>>
>>> -Nick Johnson
>>>
>>>
2010/2/9 Nick Johnson (Google)
> 2010/2/9 Stephen
>
>
>> I'm asking if it's wise to store it as a query parameter embedded in a
>> web page.
>>
>
> You're right that it's unwise. Depending on how you construct your
> query, a user could potentially modify the cursor they send to you to
> return
> results from any query your datastore is capable of performing, which
> could
> result in you revealing information to the user that they shouldn't know.
> You should either store the cursor on the server-side, or encrypt it
> before
> sending it to the client.
>
> I was going to mention something about this in my post, but it slipped
> my mind.
>
> -Nick Johnson
>
>>
>>
>> On Feb 9, 12:26 am, "Ikai L (Google)" wrote:
>> > A cursor serializes to a Base64 encoded String, so you can store it
>> anywhere
>> > you want to store strings: Memcached, Datastore, etc. You can even
>> pass it
>> > as an URL parameter to task queues.
>> >
>> > 2010/2/8 Stephen
>> >
>> >
>> >
>> >
>> >
>> > > Ah right, Nick's blog does say start_key and not offset. My bad.
>> >
>> > > Maybe there will be warnings in the upcoming documentation, but my
>> > > first instinct was to embed the serialised cursor in the HTML as
>> the
>> > > 'next' link. But that doesn't look like a good idea as Nick's
>> decoded
>> > > query shows what's embedded:
>> >
>> > > PrimaryScan {
>> > > start_key: "shell\000TestModel\000foo\000\232bar\000\200"
>> > > start_inclusive: true
>> > > }
>> > > keys_only: false
>> >
>> > > First, you may or may not want to leak this info. Second, could
>> this
>> > > be altered on the client to change the query in any way that's
>> > > undesirable?
>> >
>> > > Once you have a cursor, where do you store it so you can use it
>> again?
>> >
>>
Re: [google-appengine] Re: 1.3.1 SDK Prerelease - help us verify
Hi Nickolas,
2010/2/9 Nickolas Daskalou
> I'd want to do this so that I could include parts of the cursor (such as
> the offset) into a URL without including other parts (eg. the model kind and
> filters). I could then reconstruct the cursor on the server side based on
> what was passed into the URL.
>
The offset argument you're talking about is specific to the dev_appserver's
implementation of cursors. In production, offsets are not used, so this
won't work.
-Nick Johnson
>
> For example, if I was searching for blog comments that contained the word
> "the" (with the default order being the creation time, descending), the URL
> might look like this:
>
> myblog.com/comments/?q=the
>
> With model:
>
> class Comment(db.Model):
>
> created_at = db.DateTimeProperty(auto_now_add=True)
> words = db.StringListProperty() # A list of all the words in a comment
> (forget about exploding indexes for now)
> ...
>
> The query object for this URL might look something like:
>
>
> q =
> Comment.all().filter('words',self.request.get('q')).order('-created_at')
>
>
> To get to the 1001st comment, it'd be good if the URL looked something like
> this:
>
> myblog.com/comments/?q=the&skip=1000
>
> instead of:
>
> myblog.com/comments/?q=the&cursor=[something ugly]
>
> so that when the request comes in, I can do this:
>
>
> q =
> Comment.all().filter('words',self.request.get('q')).order('-created_at')
> cursor_template = q.cursor_template()
> cursor =
> db.Cursor.from_template(cursor_template,offset=int(self.request.get('skip')))
>
> (or something along these lines)
>
> Does that make sense?
>
>
>
> On 10 February 2010 01:03, Nick Johnson (Google)
> wrote:
>
>> Hi Nickolas,
>>
>> 2010/2/9 Nickolas Daskalou
>>
>> Will we be able to construct our own cursors much the same way that we are
>>> able to construct our own Datastore keys (Key.from_path())?
>>>
>>
>> No, not practically speaking.
>>
>>
>>>
>>> Also along the same lines, will we be able to "deconstruct" a cursor to
>>> get its components (offset, start_inclusive etc.), as we can now do with
>>> keys (key.name(), key.id(), key.kind() etc.)?
>>>
>>
>> While you could do this, there's no guarantees that it'll work (or
>> continue to work), as you'd be digging into internal implementation details.
>> Why do you want to do this?
>>
>> -Nick Johnson
>>
>>
>>>
>>> 2010/2/9 Nick Johnson (Google)
>>>
2010/2/9 Stephen
> I'm asking if it's wise to store it as a query parameter embedded in a
> web page.
>
You're right that it's unwise. Depending on how you construct your
query, a user could potentially modify the cursor they send to you to
return
results from any query your datastore is capable of performing, which could
result in you revealing information to the user that they shouldn't know.
You should either store the cursor on the server-side, or encrypt it before
sending it to the client.
I was going to mention something about this in my post, but it slipped
my mind.
-Nick Johnson
>
>
> On Feb 9, 12:26 am, "Ikai L (Google)" wrote:
> > A cursor serializes to a Base64 encoded String, so you can store it
> anywhere
> > you want to store strings: Memcached, Datastore, etc. You can even
> pass it
> > as an URL parameter to task queues.
> >
> > 2010/2/8 Stephen
> >
> >
> >
> >
> >
> > > Ah right, Nick's blog does say start_key and not offset. My bad.
> >
> > > Maybe there will be warnings in the upcoming documentation, but my
> > > first instinct was to embed the serialised cursor in the HTML as
> the
> > > 'next' link. But that doesn't look like a good idea as Nick's
> decoded
> > > query shows what's embedded:
> >
> > > PrimaryScan {
> > > start_key: "shell\000TestModel\000foo\000\232bar\000\200"
> > > start_inclusive: true
> > > }
> > > keys_only: false
> >
> > > First, you may or may not want to leak this info. Second, could
> this
> > > be altered on the client to change the query in any way that's
> > > undesirable?
> >
> > > Once you have a cursor, where do you store it so you can use it
> again?
> >
> > > On Feb 8, 10:17 pm, "Ikai L (Google)" wrote:
> > > > I got beaten to this answer. No, there is no traversal to get to
> the
> > > offset.
> >
> > > > BigTable has an underlying mechanism for range queries on keys.
> Indexes
> > > are
> > > > essentially a key comprised of a concatenation of application ID,
> entity
> > > > type, column, value. When a filter operation is performed, the
> datastore
> > > > looks for a range matching this criteria, returning the set of
> keys. A
> > > > cursor also adds the datastore key of the entity so it is
> possible to
> > > > serialize where to begin the query. This is actually a b
Re: [google-appengine] Re: 1.3.1 SDK Prerelease - help us verify
I'd want to do this so that I could include parts of the cursor (such as the
offset) into a URL without including other parts (eg. the model kind and
filters). I could then reconstruct the cursor on the server side based on
what was passed into the URL.
For example, if I was searching for blog comments that contained the word
"the" (with the default order being the creation time, descending), the URL
might look like this:
myblog.com/comments/?q=the
With model:
class Comment(db.Model):
created_at = db.DateTimeProperty(auto_now_add=True)
words = db.StringListProperty() # A list of all the words in a comment
(forget about exploding indexes for now)
...
The query object for this URL might look something like:
q = Comment.all().filter('words',self.request.get('q')).order('-created_at')
To get to the 1001st comment, it'd be good if the URL looked something like
this:
myblog.com/comments/?q=the&skip=1000
instead of:
myblog.com/comments/?q=the&cursor=[something ugly]
so that when the request comes in, I can do this:
q = Comment.all().filter('words',self.request.get('q')).order('-created_at')
cursor_template = q.cursor_template()
cursor =
db.Cursor.from_template(cursor_template,offset=int(self.request.get('skip')))
(or something along these lines)
Does that make sense?
On 10 February 2010 01:03, Nick Johnson (Google) wrote:
> Hi Nickolas,
>
> 2010/2/9 Nickolas Daskalou
>
> Will we be able to construct our own cursors much the same way that we are
>> able to construct our own Datastore keys (Key.from_path())?
>>
>
> No, not practically speaking.
>
>
>>
>> Also along the same lines, will we be able to "deconstruct" a cursor to
>> get its components (offset, start_inclusive etc.), as we can now do with
>> keys (key.name(), key.id(), key.kind() etc.)?
>>
>
> While you could do this, there's no guarantees that it'll work (or continue
> to work), as you'd be digging into internal implementation details. Why do
> you want to do this?
>
> -Nick Johnson
>
>
>>
>> 2010/2/9 Nick Johnson (Google)
>>
>>> 2010/2/9 Stephen
>>>
>>>
I'm asking if it's wise to store it as a query parameter embedded in a
web page.
>>>
>>> You're right that it's unwise. Depending on how you construct your query,
>>> a user could potentially modify the cursor they send to you to return
>>> results from any query your datastore is capable of performing, which could
>>> result in you revealing information to the user that they shouldn't know.
>>> You should either store the cursor on the server-side, or encrypt it before
>>> sending it to the client.
>>>
>>> I was going to mention something about this in my post, but it slipped my
>>> mind.
>>>
>>> -Nick Johnson
>>>
On Feb 9, 12:26 am, "Ikai L (Google)" wrote:
> A cursor serializes to a Base64 encoded String, so you can store it
anywhere
> you want to store strings: Memcached, Datastore, etc. You can even
pass it
> as an URL parameter to task queues.
>
> 2010/2/8 Stephen
>
>
>
>
>
> > Ah right, Nick's blog does say start_key and not offset. My bad.
>
> > Maybe there will be warnings in the upcoming documentation, but my
> > first instinct was to embed the serialised cursor in the HTML as the
> > 'next' link. But that doesn't look like a good idea as Nick's
decoded
> > query shows what's embedded:
>
> > PrimaryScan {
> > start_key: "shell\000TestModel\000foo\000\232bar\000\200"
> > start_inclusive: true
> > }
> > keys_only: false
>
> > First, you may or may not want to leak this info. Second, could this
> > be altered on the client to change the query in any way that's
> > undesirable?
>
> > Once you have a cursor, where do you store it so you can use it
again?
>
> > On Feb 8, 10:17 pm, "Ikai L (Google)" wrote:
> > > I got beaten to this answer. No, there is no traversal to get to
the
> > offset.
>
> > > BigTable has an underlying mechanism for range queries on keys.
Indexes
> > are
> > > essentially a key comprised of a concatenation of application ID,
entity
> > > type, column, value. When a filter operation is performed, the
datastore
> > > looks for a range matching this criteria, returning the set of
keys. A
> > > cursor also adds the datastore key of the entity so it is possible
to
> > > serialize where to begin the query. This is actually a bit awkward
to
> > > explain without visuals. You can watch Ryan Barrett's talk here:
>
> > >http://www.youtube.com/watch?v=tx5gdoNpcZM
>
> > > Hopefully, we'll be able to post an article at some point in the
future
> > > explaining how cursors work.
>
> > > 2010/2/8 Alkis Evlogimenos ('Αλκης Ευλογημένος) <
evlogime...@gmail.com>
>
> > > > There is no offset. The pro
Re: [google-appengine] Re: 1.3.1 SDK Prerelease - help us verify
Hi Nickolas,
2010/2/9 Nickolas Daskalou
> Will we be able to construct our own cursors much the same way that we are
> able to construct our own Datastore keys (Key.from_path())?
>
No, not practically speaking.
>
> Also along the same lines, will we be able to "deconstruct" a cursor to get
> its components (offset, start_inclusive etc.), as we can now do with keys (
> key.name(), key.id(), key.kind() etc.)?
>
While you could do this, there's no guarantees that it'll work (or continue
to work), as you'd be digging into internal implementation details. Why do
you want to do this?
-Nick Johnson
>
> 2010/2/9 Nick Johnson (Google)
>
>> 2010/2/9 Stephen
>>
>>
>>> I'm asking if it's wise to store it as a query parameter embedded in a
>>> web page.
>>>
>>
>> You're right that it's unwise. Depending on how you construct your query,
>> a user could potentially modify the cursor they send to you to return
>> results from any query your datastore is capable of performing, which could
>> result in you revealing information to the user that they shouldn't know.
>> You should either store the cursor on the server-side, or encrypt it before
>> sending it to the client.
>>
>> I was going to mention something about this in my post, but it slipped my
>> mind.
>>
>> -Nick Johnson
>>
>>>
>>>
>>> On Feb 9, 12:26 am, "Ikai L (Google)" wrote:
>>> > A cursor serializes to a Base64 encoded String, so you can store it
>>> anywhere
>>> > you want to store strings: Memcached, Datastore, etc. You can even pass
>>> it
>>> > as an URL parameter to task queues.
>>> >
>>> > 2010/2/8 Stephen
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > > Ah right, Nick's blog does say start_key and not offset. My bad.
>>> >
>>> > > Maybe there will be warnings in the upcoming documentation, but my
>>> > > first instinct was to embed the serialised cursor in the HTML as the
>>> > > 'next' link. But that doesn't look like a good idea as Nick's decoded
>>> > > query shows what's embedded:
>>> >
>>> > > PrimaryScan {
>>> > > start_key: "shell\000TestModel\000foo\000\232bar\000\200"
>>> > > start_inclusive: true
>>> > > }
>>> > > keys_only: false
>>> >
>>> > > First, you may or may not want to leak this info. Second, could this
>>> > > be altered on the client to change the query in any way that's
>>> > > undesirable?
>>> >
>>> > > Once you have a cursor, where do you store it so you can use it
>>> again?
>>> >
>>> > > On Feb 8, 10:17 pm, "Ikai L (Google)" wrote:
>>> > > > I got beaten to this answer. No, there is no traversal to get to
>>> the
>>> > > offset.
>>> >
>>> > > > BigTable has an underlying mechanism for range queries on keys.
>>> Indexes
>>> > > are
>>> > > > essentially a key comprised of a concatenation of application ID,
>>> entity
>>> > > > type, column, value. When a filter operation is performed, the
>>> datastore
>>> > > > looks for a range matching this criteria, returning the set of
>>> keys. A
>>> > > > cursor also adds the datastore key of the entity so it is possible
>>> to
>>> > > > serialize where to begin the query. This is actually a bit awkward
>>> to
>>> > > > explain without visuals. You can watch Ryan Barrett's talk here:
>>> >
>>> > > >http://www.youtube.com/watch?v=tx5gdoNpcZM
>>> >
>>> > > > Hopefully, we'll be able to post an article at some point in the
>>> future
>>> > > > explaining how cursors work.
>>> >
>>> > > > 2010/2/8 Alkis Evlogimenos ('Αλκης Ευλογημένος) <
>>> evlogime...@gmail.com>
>>> >
>>> > > > > There is no offset. The protocol buffer stores a start_key and a
>>> > > boolean
>>> > > > > denoting if this start key is inclusive or not. The performance
>>> of
>>> > > > > continuing the fetch from a cursor should be the same as the
>>> > > performance of
>>> > > > > the first entities you got from a query.
>>> >
>>> > > > > On Mon, Feb 8, 2010 at 4:33 PM, Stephen
>>> wrote:
>>> >
>>> > > > >> On Feb 8, 7:06 pm, "Ikai L (Google)" wrote:
>>> > > > >> > The official docs are pending, but here's Nick Johnson to the
>>> > > rescue:
>>> >
>>> > >
>>> http://blog.notdot.net/2010/02/New-features-in-1-3-1-prerelease-Cursors
>>> >
>>> > > > >> What are the performance characteristics of cursors?
>>> >
>>> > > > >> The serialised cursor shows that it stores an offset. Does that
>>> mean
>>> > > > >> that if the offset is one million, one million rows will have to
>>> be
>>> > > > >> skipped before the next 10 are returned? This will be faster
>>> than
>>> > > > >> doing it in your app, but not as quick as the existing bookmark
>>> > > > >> techniques which use the primary key index.
>>> >
>>> > > > >> Or is the server-side stateful, like a typical SQL
>>> implementation of
>>> > > > >> cursors? In which case, are there any limits to the number of
>>> active
>>> > > > >> cursors? Or what if a cursor is resumed some time in the future;
>>> will
>>> > > > >> it work at all, or work slower?
>>> >
>>> > > > >> --
>>> > > > >> You received this message because you are subscribed to the
>>> Google
>>> > > Groups
>>> > >
Re: [google-appengine] Re: 1.3.1 SDK Prerelease - help us verify
Will we be able to construct our own cursors much the same way that we are
able to construct our own Datastore keys (Key.from_path())?
Also along the same lines, will we be able to "deconstruct" a cursor to get
its components (offset, start_inclusive etc.), as we can now do with keys (
key.name(), key.id(), key.kind() etc.)?
2010/2/9 Nick Johnson (Google)
> 2010/2/9 Stephen
>
>
>> I'm asking if it's wise to store it as a query parameter embedded in a
>> web page.
>>
>
> You're right that it's unwise. Depending on how you construct your query, a
> user could potentially modify the cursor they send to you to return results
> from any query your datastore is capable of performing, which could result
> in you revealing information to the user that they shouldn't know. You
> should either store the cursor on the server-side, or encrypt it before
> sending it to the client.
>
> I was going to mention something about this in my post, but it slipped my
> mind.
>
> -Nick Johnson
>
>>
>>
>> On Feb 9, 12:26 am, "Ikai L (Google)" wrote:
>> > A cursor serializes to a Base64 encoded String, so you can store it
>> anywhere
>> > you want to store strings: Memcached, Datastore, etc. You can even pass
>> it
>> > as an URL parameter to task queues.
>> >
>> > 2010/2/8 Stephen
>> >
>> >
>> >
>> >
>> >
>> > > Ah right, Nick's blog does say start_key and not offset. My bad.
>> >
>> > > Maybe there will be warnings in the upcoming documentation, but my
>> > > first instinct was to embed the serialised cursor in the HTML as the
>> > > 'next' link. But that doesn't look like a good idea as Nick's decoded
>> > > query shows what's embedded:
>> >
>> > > PrimaryScan {
>> > > start_key: "shell\000TestModel\000foo\000\232bar\000\200"
>> > > start_inclusive: true
>> > > }
>> > > keys_only: false
>> >
>> > > First, you may or may not want to leak this info. Second, could this
>> > > be altered on the client to change the query in any way that's
>> > > undesirable?
>> >
>> > > Once you have a cursor, where do you store it so you can use it again?
>> >
>> > > On Feb 8, 10:17 pm, "Ikai L (Google)" wrote:
>> > > > I got beaten to this answer. No, there is no traversal to get to the
>> > > offset.
>> >
>> > > > BigTable has an underlying mechanism for range queries on keys.
>> Indexes
>> > > are
>> > > > essentially a key comprised of a concatenation of application ID,
>> entity
>> > > > type, column, value. When a filter operation is performed, the
>> datastore
>> > > > looks for a range matching this criteria, returning the set of keys.
>> A
>> > > > cursor also adds the datastore key of the entity so it is possible
>> to
>> > > > serialize where to begin the query. This is actually a bit awkward
>> to
>> > > > explain without visuals. You can watch Ryan Barrett's talk here:
>> >
>> > > >http://www.youtube.com/watch?v=tx5gdoNpcZM
>> >
>> > > > Hopefully, we'll be able to post an article at some point in the
>> future
>> > > > explaining how cursors work.
>> >
>> > > > 2010/2/8 Alkis Evlogimenos ('Αλκης Ευλογημένος) <
>> evlogime...@gmail.com>
>> >
>> > > > > There is no offset. The protocol buffer stores a start_key and a
>> > > boolean
>> > > > > denoting if this start key is inclusive or not. The performance of
>> > > > > continuing the fetch from a cursor should be the same as the
>> > > performance of
>> > > > > the first entities you got from a query.
>> >
>> > > > > On Mon, Feb 8, 2010 at 4:33 PM, Stephen
>> wrote:
>> >
>> > > > >> On Feb 8, 7:06 pm, "Ikai L (Google)" wrote:
>> > > > >> > The official docs are pending, but here's Nick Johnson to the
>> > > rescue:
>> >
>> > >
>> http://blog.notdot.net/2010/02/New-features-in-1-3-1-prerelease-Cursors
>> >
>> > > > >> What are the performance characteristics of cursors?
>> >
>> > > > >> The serialised cursor shows that it stores an offset. Does that
>> mean
>> > > > >> that if the offset is one million, one million rows will have to
>> be
>> > > > >> skipped before the next 10 are returned? This will be faster than
>> > > > >> doing it in your app, but not as quick as the existing bookmark
>> > > > >> techniques which use the primary key index.
>> >
>> > > > >> Or is the server-side stateful, like a typical SQL implementation
>> of
>> > > > >> cursors? In which case, are there any limits to the number of
>> active
>> > > > >> cursors? Or what if a cursor is resumed some time in the future;
>> will
>> > > > >> it work at all, or work slower?
>> >
>> > > > >> --
>> > > > >> You received this message because you are subscribed to the
>> Google
>> > > Groups
>> > > > >> "Google App Engine" group.
>> > > > >> To post to this group, send email to
>> > > google-appeng...@googlegroups.com.
>> > > > >> To unsubscribe from this group, send email to
>> > > > >> google-appengine+unsubscr...@googlegroups.com
>>
>> >
>> > >
>>
>> >
>> >
>> > > > >> .
>> > > > >> For more options, visit this group at
>> > > > >>http://groups.google.com/group/google-appengine?hl=en.
>> >
>> > > > > --
>>
Re: [google-appengine] Re: 1.3.1 SDK Prerelease - help us verify
2010/2/9 Stephen
>
> I'm asking if it's wise to store it as a query parameter embedded in a
> web page.
>
You're right that it's unwise. Depending on how you construct your query, a
user could potentially modify the cursor they send to you to return results
from any query your datastore is capable of performing, which could result
in you revealing information to the user that they shouldn't know. You
should either store the cursor on the server-side, or encrypt it before
sending it to the client.
I was going to mention something about this in my post, but it slipped my
mind.
-Nick Johnson
>
>
> On Feb 9, 12:26 am, "Ikai L (Google)" wrote:
> > A cursor serializes to a Base64 encoded String, so you can store it
> anywhere
> > you want to store strings: Memcached, Datastore, etc. You can even pass
> it
> > as an URL parameter to task queues.
> >
> > 2010/2/8 Stephen
> >
> >
> >
> >
> >
> > > Ah right, Nick's blog does say start_key and not offset. My bad.
> >
> > > Maybe there will be warnings in the upcoming documentation, but my
> > > first instinct was to embed the serialised cursor in the HTML as the
> > > 'next' link. But that doesn't look like a good idea as Nick's decoded
> > > query shows what's embedded:
> >
> > > PrimaryScan {
> > > start_key: "shell\000TestModel\000foo\000\232bar\000\200"
> > > start_inclusive: true
> > > }
> > > keys_only: false
> >
> > > First, you may or may not want to leak this info. Second, could this
> > > be altered on the client to change the query in any way that's
> > > undesirable?
> >
> > > Once you have a cursor, where do you store it so you can use it again?
> >
> > > On Feb 8, 10:17 pm, "Ikai L (Google)" wrote:
> > > > I got beaten to this answer. No, there is no traversal to get to the
> > > offset.
> >
> > > > BigTable has an underlying mechanism for range queries on keys.
> Indexes
> > > are
> > > > essentially a key comprised of a concatenation of application ID,
> entity
> > > > type, column, value. When a filter operation is performed, the
> datastore
> > > > looks for a range matching this criteria, returning the set of keys.
> A
> > > > cursor also adds the datastore key of the entity so it is possible to
> > > > serialize where to begin the query. This is actually a bit awkward to
> > > > explain without visuals. You can watch Ryan Barrett's talk here:
> >
> > > >http://www.youtube.com/watch?v=tx5gdoNpcZM
> >
> > > > Hopefully, we'll be able to post an article at some point in the
> future
> > > > explaining how cursors work.
> >
> > > > 2010/2/8 Alkis Evlogimenos ('Αλκης Ευλογημένος) <
> evlogime...@gmail.com>
> >
> > > > > There is no offset. The protocol buffer stores a start_key and a
> > > boolean
> > > > > denoting if this start key is inclusive or not. The performance of
> > > > > continuing the fetch from a cursor should be the same as the
> > > performance of
> > > > > the first entities you got from a query.
> >
> > > > > On Mon, Feb 8, 2010 at 4:33 PM, Stephen wrote:
> >
> > > > >> On Feb 8, 7:06 pm, "Ikai L (Google)" wrote:
> > > > >> > The official docs are pending, but here's Nick Johnson to the
> > > rescue:
> >
> > >http://blog.notdot.net/2010/02/New-features-in-1-3-1-prerelease-Cursors
> >
> > > > >> What are the performance characteristics of cursors?
> >
> > > > >> The serialised cursor shows that it stores an offset. Does that
> mean
> > > > >> that if the offset is one million, one million rows will have to
> be
> > > > >> skipped before the next 10 are returned? This will be faster than
> > > > >> doing it in your app, but not as quick as the existing bookmark
> > > > >> techniques which use the primary key index.
> >
> > > > >> Or is the server-side stateful, like a typical SQL implementation
> of
> > > > >> cursors? In which case, are there any limits to the number of
> active
> > > > >> cursors? Or what if a cursor is resumed some time in the future;
> will
> > > > >> it work at all, or work slower?
> >
> > > > >> --
> > > > >> You received this message because you are subscribed to the Google
> > > Groups
> > > > >> "Google App Engine" group.
> > > > >> To post to this group, send email to
> > > google-appeng...@googlegroups.com.
> > > > >> To unsubscribe from this group, send email to
> > > > >> google-appengine+unsubscr...@googlegroups.com
>
> >
> > >
>
> >
> >
> > > > >> .
> > > > >> For more options, visit this group at
> > > > >>http://groups.google.com/group/google-appengine?hl=en.
> >
> > > > > --
> >
> > > > > Alkis
> >
> > > > > --
> > > > > You received this message because you are subscribed to the Google
> > > Groups
> > > > > "Google App Engine" group.
> > > > > To post to this group, send email to
> google-appengine@googlegroups.com
> > > .
> > > > > To unsubscribe from this group, send email to
> > > > > google-appengine+unsubscr...@googlegroups.com
>
> >
> > >
>
> >
> >
> > > > > .
> > > > > For more options, visit this group at
> > > > >http://groups.google.com/group/google-appengine?hl=en.
> >
>
[google-appengine] Re: 1.3.1 SDK Prerelease - help us verify
I'm asking if it's wise to store it as a query parameter embedded in a
web page.
On Feb 9, 12:26 am, "Ikai L (Google)" wrote:
> A cursor serializes to a Base64 encoded String, so you can store it anywhere
> you want to store strings: Memcached, Datastore, etc. You can even pass it
> as an URL parameter to task queues.
>
> 2010/2/8 Stephen
>
>
>
>
>
> > Ah right, Nick's blog does say start_key and not offset. My bad.
>
> > Maybe there will be warnings in the upcoming documentation, but my
> > first instinct was to embed the serialised cursor in the HTML as the
> > 'next' link. But that doesn't look like a good idea as Nick's decoded
> > query shows what's embedded:
>
> > PrimaryScan {
> > start_key: "shell\000TestModel\000foo\000\232bar\000\200"
> > start_inclusive: true
> > }
> > keys_only: false
>
> > First, you may or may not want to leak this info. Second, could this
> > be altered on the client to change the query in any way that's
> > undesirable?
>
> > Once you have a cursor, where do you store it so you can use it again?
>
> > On Feb 8, 10:17 pm, "Ikai L (Google)" wrote:
> > > I got beaten to this answer. No, there is no traversal to get to the
> > offset.
>
> > > BigTable has an underlying mechanism for range queries on keys. Indexes
> > are
> > > essentially a key comprised of a concatenation of application ID, entity
> > > type, column, value. When a filter operation is performed, the datastore
> > > looks for a range matching this criteria, returning the set of keys. A
> > > cursor also adds the datastore key of the entity so it is possible to
> > > serialize where to begin the query. This is actually a bit awkward to
> > > explain without visuals. You can watch Ryan Barrett's talk here:
>
> > >http://www.youtube.com/watch?v=tx5gdoNpcZM
>
> > > Hopefully, we'll be able to post an article at some point in the future
> > > explaining how cursors work.
>
> > > 2010/2/8 Alkis Evlogimenos ('Αλκης Ευλογημένος)
>
> > > > There is no offset. The protocol buffer stores a start_key and a
> > boolean
> > > > denoting if this start key is inclusive or not. The performance of
> > > > continuing the fetch from a cursor should be the same as the
> > performance of
> > > > the first entities you got from a query.
>
> > > > On Mon, Feb 8, 2010 at 4:33 PM, Stephen wrote:
>
> > > >> On Feb 8, 7:06 pm, "Ikai L (Google)" wrote:
> > > >> > The official docs are pending, but here's Nick Johnson to the
> > rescue:
>
> >http://blog.notdot.net/2010/02/New-features-in-1-3-1-prerelease-Cursors
>
> > > >> What are the performance characteristics of cursors?
>
> > > >> The serialised cursor shows that it stores an offset. Does that mean
> > > >> that if the offset is one million, one million rows will have to be
> > > >> skipped before the next 10 are returned? This will be faster than
> > > >> doing it in your app, but not as quick as the existing bookmark
> > > >> techniques which use the primary key index.
>
> > > >> Or is the server-side stateful, like a typical SQL implementation of
> > > >> cursors? In which case, are there any limits to the number of active
> > > >> cursors? Or what if a cursor is resumed some time in the future; will
> > > >> it work at all, or work slower?
>
> > > >> --
> > > >> You received this message because you are subscribed to the Google
> > Groups
> > > >> "Google App Engine" group.
> > > >> To post to this group, send email to
> > google-appeng...@googlegroups.com.
> > > >> To unsubscribe from this group, send email to
> > > >> google-appengine+unsubscr...@googlegroups.com
> >
>
> > > >> .
> > > >> For more options, visit this group at
> > > >>http://groups.google.com/group/google-appengine?hl=en.
>
> > > > --
>
> > > > Alkis
>
> > > > --
> > > > You received this message because you are subscribed to the Google
> > Groups
> > > > "Google App Engine" group.
> > > > To post to this group, send email to google-appengine@googlegroups.com
> > .
> > > > To unsubscribe from this group, send email to
> > > > google-appengine+unsubscr...@googlegroups.com
> >
>
> > > > .
> > > > For more options, visit this group at
> > > >http://groups.google.com/group/google-appengine?hl=en.
>
> > > --
> > > Ikai Lan
> > > Developer Programs Engineer, Google App Enginehttp://
> > googleappengine.blogspot.com|http://twitter.com/app_engine
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Google App Engine" group.
> > To post to this group, send email to google-appeng...@googlegroups.com.
> > To unsubscribe from this group, send email to
> > google-appengine+unsubscr...@googlegroups.com
> > .
> > For more options, visit this group at
> >http://groups.google.com/group/google-appengine?hl=en.
>
> --
> Ikai Lan
> Developer Programs Engineer, Google App
> Enginehttp://googleappengine.blogspot.com|http://twitter.com/app_engine
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email
Re: [google-appengine] Re: 1.3.1 SDK Prerelease - help us verify
A cursor serializes to a Base64 encoded String, so you can store it anywhere
you want to store strings: Memcached, Datastore, etc. You can even pass it
as an URL parameter to task queues.
2010/2/8 Stephen
>
> Ah right, Nick's blog does say start_key and not offset. My bad.
>
> Maybe there will be warnings in the upcoming documentation, but my
> first instinct was to embed the serialised cursor in the HTML as the
> 'next' link. But that doesn't look like a good idea as Nick's decoded
> query shows what's embedded:
>
> PrimaryScan {
> start_key: "shell\000TestModel\000foo\000\232bar\000\200"
> start_inclusive: true
> }
> keys_only: false
>
> First, you may or may not want to leak this info. Second, could this
> be altered on the client to change the query in any way that's
> undesirable?
>
> Once you have a cursor, where do you store it so you can use it again?
>
>
> On Feb 8, 10:17 pm, "Ikai L (Google)" wrote:
> > I got beaten to this answer. No, there is no traversal to get to the
> offset.
> >
> > BigTable has an underlying mechanism for range queries on keys. Indexes
> are
> > essentially a key comprised of a concatenation of application ID, entity
> > type, column, value. When a filter operation is performed, the datastore
> > looks for a range matching this criteria, returning the set of keys. A
> > cursor also adds the datastore key of the entity so it is possible to
> > serialize where to begin the query. This is actually a bit awkward to
> > explain without visuals. You can watch Ryan Barrett's talk here:
> >
> > http://www.youtube.com/watch?v=tx5gdoNpcZM
> >
> > Hopefully, we'll be able to post an article at some point in the future
> > explaining how cursors work.
> >
> > 2010/2/8 Alkis Evlogimenos ('Αλκης Ευλογημένος)
> >
> >
> >
> > > There is no offset. The protocol buffer stores a start_key and a
> boolean
> > > denoting if this start key is inclusive or not. The performance of
> > > continuing the fetch from a cursor should be the same as the
> performance of
> > > the first entities you got from a query.
> >
> > > On Mon, Feb 8, 2010 at 4:33 PM, Stephen wrote:
> >
> > >> On Feb 8, 7:06 pm, "Ikai L (Google)" wrote:
> > >> > The official docs are pending, but here's Nick Johnson to the
> rescue:
> >
> > >> >
> http://blog.notdot.net/2010/02/New-features-in-1-3-1-prerelease-Cursors
> >
> > >> What are the performance characteristics of cursors?
> >
> > >> The serialised cursor shows that it stores an offset. Does that mean
> > >> that if the offset is one million, one million rows will have to be
> > >> skipped before the next 10 are returned? This will be faster than
> > >> doing it in your app, but not as quick as the existing bookmark
> > >> techniques which use the primary key index.
> >
> > >> Or is the server-side stateful, like a typical SQL implementation of
> > >> cursors? In which case, are there any limits to the number of active
> > >> cursors? Or what if a cursor is resumed some time in the future; will
> > >> it work at all, or work slower?
> >
> > >> --
> > >> You received this message because you are subscribed to the Google
> Groups
> > >> "Google App Engine" group.
> > >> To post to this group, send email to
> google-appeng...@googlegroups.com.
> > >> To unsubscribe from this group, send email to
> > >> google-appengine+unsubscr...@googlegroups.com
>
> >
> > >> .
> > >> For more options, visit this group at
> > >>http://groups.google.com/group/google-appengine?hl=en.
> >
> > > --
> >
> > > Alkis
> >
> > > --
> > > You received this message because you are subscribed to the Google
> Groups
> > > "Google App Engine" group.
> > > To post to this group, send email to google-appengine@googlegroups.com
> .
> > > To unsubscribe from this group, send email to
> > > google-appengine+unsubscr...@googlegroups.com
>
> >
> > > .
> > > For more options, visit this group at
> > >http://groups.google.com/group/google-appengine?hl=en.
> >
> > --
> > Ikai Lan
> > Developer Programs Engineer, Google App Enginehttp://
> googleappengine.blogspot.com|http://twitter.com/app_engine
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appeng...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>
--
Ikai Lan
Developer Programs Engineer, Google App Engine
http://googleappengine.blogspot.com | http://twitter.com/app_engine
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: 1.3.1 SDK Prerelease - help us verify
Ah right, Nick's blog does say start_key and not offset. My bad.
Maybe there will be warnings in the upcoming documentation, but my
first instinct was to embed the serialised cursor in the HTML as the
'next' link. But that doesn't look like a good idea as Nick's decoded
query shows what's embedded:
PrimaryScan {
start_key: "shell\000TestModel\000foo\000\232bar\000\200"
start_inclusive: true
}
keys_only: false
First, you may or may not want to leak this info. Second, could this
be altered on the client to change the query in any way that's
undesirable?
Once you have a cursor, where do you store it so you can use it again?
On Feb 8, 10:17 pm, "Ikai L (Google)" wrote:
> I got beaten to this answer. No, there is no traversal to get to the offset.
>
> BigTable has an underlying mechanism for range queries on keys. Indexes are
> essentially a key comprised of a concatenation of application ID, entity
> type, column, value. When a filter operation is performed, the datastore
> looks for a range matching this criteria, returning the set of keys. A
> cursor also adds the datastore key of the entity so it is possible to
> serialize where to begin the query. This is actually a bit awkward to
> explain without visuals. You can watch Ryan Barrett's talk here:
>
> http://www.youtube.com/watch?v=tx5gdoNpcZM
>
> Hopefully, we'll be able to post an article at some point in the future
> explaining how cursors work.
>
> 2010/2/8 Alkis Evlogimenos ('Αλκης Ευλογημένος)
>
>
>
> > There is no offset. The protocol buffer stores a start_key and a boolean
> > denoting if this start key is inclusive or not. The performance of
> > continuing the fetch from a cursor should be the same as the performance of
> > the first entities you got from a query.
>
> > On Mon, Feb 8, 2010 at 4:33 PM, Stephen wrote:
>
> >> On Feb 8, 7:06 pm, "Ikai L (Google)" wrote:
> >> > The official docs are pending, but here's Nick Johnson to the rescue:
>
> >> >http://blog.notdot.net/2010/02/New-features-in-1-3-1-prerelease-Cursors
>
> >> What are the performance characteristics of cursors?
>
> >> The serialised cursor shows that it stores an offset. Does that mean
> >> that if the offset is one million, one million rows will have to be
> >> skipped before the next 10 are returned? This will be faster than
> >> doing it in your app, but not as quick as the existing bookmark
> >> techniques which use the primary key index.
>
> >> Or is the server-side stateful, like a typical SQL implementation of
> >> cursors? In which case, are there any limits to the number of active
> >> cursors? Or what if a cursor is resumed some time in the future; will
> >> it work at all, or work slower?
>
> >> --
> >> You received this message because you are subscribed to the Google Groups
> >> "Google App Engine" group.
> >> To post to this group, send email to google-appeng...@googlegroups.com.
> >> To unsubscribe from this group, send email to
> >> google-appengine+unsubscr...@googlegroups.com
> >> .
> >> For more options, visit this group at
> >>http://groups.google.com/group/google-appengine?hl=en.
>
> > --
>
> > Alkis
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Google App Engine" group.
> > To post to this group, send email to google-appeng...@googlegroups.com.
> > To unsubscribe from this group, send email to
> > google-appengine+unsubscr...@googlegroups.com
> > .
> > For more options, visit this group at
> >http://groups.google.com/group/google-appengine?hl=en.
>
> --
> Ikai Lan
> Developer Programs Engineer, Google App
> Enginehttp://googleappengine.blogspot.com|http://twitter.com/app_engine
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: 1.3.1 SDK Prerelease - help us verify
I got beaten to this answer. No, there is no traversal to get to the offset.
BigTable has an underlying mechanism for range queries on keys. Indexes are
essentially a key comprised of a concatenation of application ID, entity
type, column, value. When a filter operation is performed, the datastore
looks for a range matching this criteria, returning the set of keys. A
cursor also adds the datastore key of the entity so it is possible to
serialize where to begin the query. This is actually a bit awkward to
explain without visuals. You can watch Ryan Barrett's talk here:
http://www.youtube.com/watch?v=tx5gdoNpcZM
Hopefully, we'll be able to post an article at some point in the future
explaining how cursors work.
2010/2/8 Alkis Evlogimenos ('Αλκης Ευλογημένος)
> There is no offset. The protocol buffer stores a start_key and a boolean
> denoting if this start key is inclusive or not. The performance of
> continuing the fetch from a cursor should be the same as the performance of
> the first entities you got from a query.
>
> On Mon, Feb 8, 2010 at 4:33 PM, Stephen wrote:
>
>>
>>
>> On Feb 8, 7:06 pm, "Ikai L (Google)" wrote:
>> > The official docs are pending, but here's Nick Johnson to the rescue:
>> >
>> > http://blog.notdot.net/2010/02/New-features-in-1-3-1-prerelease-Cursors
>>
>>
>> What are the performance characteristics of cursors?
>>
>> The serialised cursor shows that it stores an offset. Does that mean
>> that if the offset is one million, one million rows will have to be
>> skipped before the next 10 are returned? This will be faster than
>> doing it in your app, but not as quick as the existing bookmark
>> techniques which use the primary key index.
>>
>> Or is the server-side stateful, like a typical SQL implementation of
>> cursors? In which case, are there any limits to the number of active
>> cursors? Or what if a cursor is resumed some time in the future; will
>> it work at all, or work slower?
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Google App Engine" group.
>> To post to this group, send email to google-appeng...@googlegroups.com.
>> To unsubscribe from this group, send email to
>> google-appengine+unsubscr...@googlegroups.com
>> .
>> For more options, visit this group at
>> http://groups.google.com/group/google-appengine?hl=en.
>>
>>
>
>
> --
>
> Alkis
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appeng...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
--
Ikai Lan
Developer Programs Engineer, Google App Engine
http://googleappengine.blogspot.com | http://twitter.com/app_engine
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: 1.3.1 SDK Prerelease - help us verify
There is no offset. The protocol buffer stores a start_key and a boolean denoting if this start key is inclusive or not. The performance of continuing the fetch from a cursor should be the same as the performance of the first entities you got from a query. On Mon, Feb 8, 2010 at 4:33 PM, Stephen wrote: > > > On Feb 8, 7:06 pm, "Ikai L (Google)" wrote: > > The official docs are pending, but here's Nick Johnson to the rescue: > > > > http://blog.notdot.net/2010/02/New-features-in-1-3-1-prerelease-Cursors > > > What are the performance characteristics of cursors? > > The serialised cursor shows that it stores an offset. Does that mean > that if the offset is one million, one million rows will have to be > skipped before the next 10 are returned? This will be faster than > doing it in your app, but not as quick as the existing bookmark > techniques which use the primary key index. > > Or is the server-side stateful, like a typical SQL implementation of > cursors? In which case, are there any limits to the number of active > cursors? Or what if a cursor is resumed some time in the future; will > it work at all, or work slower? > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- Alkis -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: 1.3.1 SDK Prerelease - help us verify
On Feb 8, 7:06 pm, "Ikai L (Google)" wrote: > The official docs are pending, but here's Nick Johnson to the rescue: > > http://blog.notdot.net/2010/02/New-features-in-1-3-1-prerelease-Cursors What are the performance characteristics of cursors? The serialised cursor shows that it stores an offset. Does that mean that if the offset is one million, one million rows will have to be skipped before the next 10 are returned? This will be faster than doing it in your app, but not as quick as the existing bookmark techniques which use the primary key index. Or is the server-side stateful, like a typical SQL implementation of cursors? In which case, are there any limits to the number of active cursors? Or what if a cursor is resumed some time in the future; will it work at all, or work slower? -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: 1.3.1 SDK Prerelease - help us verify
The official docs are pending, but here's Nick Johnson to the rescue: http://blog.notdot.net/2010/02/New-features-in-1-3-1-prerelease-Cursors -- Ikai Lan Developer Programs Engineer, Google App Engine http://googleappengine.blogspot.com | http://twitter.com/app_engine -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: 1.3.1 SDK Prerelease - help us verify
Didn't find doc about the "Support for Custom Admin Console pages" ? Could that means that we can add custom data to Admin Console webapp ? PS.: Ikai, sorry about the direct message -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: 1.3.1 SDK Prerelease - help us verify
Didn't find doc about "Support for Custom Admin Console pages" ? Is this a way to add our data in the Admin Console Web app ? -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: 1.3.1 SDK Prerelease - help us verify
Nick, et al.: Could we get a pointer or pointers to in-line documentation as related to query cursors? If this is what it sounds like, it is far-and-away the biggest and most useful new feature in the SDK, but we need some hints about how it is used. I've browsed the source code and found plenty of referneces to cursors, but there's not much actualyl useful info about how they are used. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: 1.3.1 SDK Prerelease - help us verify
and 30 sec time window? thx. On Feb 5, 12:31 am, Kyle Heu wrote: > Do the Datastore Cursors solve the 1000 query limit? > > On Feb 3, 5:03 pm, Ikai Lan wrote: > > > > > Hello App Engine Developers, > > > As part of our ongoing efforts to improve release quality and > > transparency, we will start prereleasing SDKs for early testing. We > > hope this gives developers a chance to participate in our release > > process by trying out new changes and sending feedback. As of this > > morning, the prerelease SDK for our next release, 1.3.1, is available > > in the familiar download location (note that the filename ends in > > 'prerelease.zip'): > > >http://code.google.com/p/googleappengine/downloads/list > > > If you're interested, please download and give it a try locally with > > your favorite App Engine code. Please note that, as a prerelease, this > > SDK is not yet supported and still subject to change. Thus, please > > don't take critical dependencies or make substantial changes to > > production apps based on this SDK. > > > Importantly, this prerelease is purely for the SDK and is intended for > > local testing and development in dev_appserver. The server-side of App > > Engine (our production environment) is not at 1.3.1, so deploying with > > this SDK is not yet supported. In the future, we might enable a > > complete SDK and server test environment for prereleases. > > > Please try 1.3.1 for local development and send us your feedback! > > > Thanks, > > > App Engine Team > > > Python > > = > > - New support for Datastore Query Cursors > > - New support for Transactional Task Creation > > - Additional file extensions permitted when sending mail including .doc and > > .xls > > http://code.google.com/p/googleappengine/issues/detail?id=494 > > - New Grab Tail added to Memcache API > > - Support for Custom Admin Console pages > > - New "month" and "synchronized" syntax for Cron configuration > > - Application Stats library now included in with SDK > > - Bulk Loader supports bulk downloading all kinds simultaneously > > - appcfg.py validates SSL certificates for HTTPS connections > > - Support for ETags, If-matches, If-not-matches HTTP Headers, as well as 304 > > error codes now available on static files (not available on the > > dev_appserver or Blobstore > > blobs)http://code.google.com/p/googleappengine/issues/detail?id=575 > > > Java > > = > > - Datastore Query Cursors > > - Transactional Tasks > > - Additional file extensions permitted when sending mail including .doc and > > .xsl > > http://code.google.com/p/googleappengine/issues/detail?id=494 > > - Grab Tail added to Memcache API > > - Support for Custom Admin Console pages > > - Java Precompilation is now on by default. > > - Developers can opt-out of precompilation by setting the flag in > > appengine-web.xml > > false > > - New built-in support for unit testing (see appengine-testing.jar) > > http://code.google.com/p/googleappengine/issues/detail?id=326 > > - net.sf.jsr107 package included as an alternative to the low-level Memcache > > API > > - javax.annotation.Resource/Resources added to the package whitelist > > - New "month" and "synchronized" syntax for Cron configuration > > - URLFetch supports asynchronous requests > > -http://code.google.com/p/googleappengine/issues/detail?id=1899 > > - appcfg.sh uses HTTPS for application deployment > > - appcfg.sh adds request_logs --append > > - Changes to the order queries without a specified sort order are returned. > > Only queries that use IN will see different results. > > - Added support for multiple != filters on the same property > > - Improved support for keys-only queries when using IN and != filters > > - Support for ETags, If-matches, If-not-matches HTTP Headers, as well as 304 > > error codes now available on static files (not available on the > > dev_appserver or Blobstore blobs) > > http://code.google.com/p/googleappengine/issues/detail?id=575 > > - Fixed issue where the maximum transform count was enforced for composite > > operations > > http://code.google.com/p/googleappengine/issues/detail?id=1656 > > - Fixed issue with whitespace on the end of strings in web.xml > > http://code.google.com/p/googleappengine/issues/detail?id=2242 > > - Fixed "Not Found" issue when defining in web.xml > > http://code.google.com/p/googleappengine/issues/detail?id=1477 > > - Fixed issue when defining in web.xml > > http://code.google.com/p/googleappengine/issues/detail?id=1249 > > - Fixed issue where cancelling a deployment in progress would > > unintentionally delete packages > > http://code.google.com/p/googleappengine/issues/detail?id=2255 > > - Fixed issue with QuotaService.getCpuTimeInMegaCycles() returning 0 > > http://code.google.com/p/googleappengine/issues/detail?id=2639 > > - Fixed issue where JSP exceptions will be incorrectly cast causing a > > ClassCastException > > http://code.google.com/p/googleappengine/issues/detail?id=1438 > > > -- > > Ikai Lan > > De
[google-appengine] Re: 1.3.1 SDK Prerelease - help us verify
Do the Datastore Cursors solve the 1000 query limit? On Feb 3, 5:03 pm, Ikai Lan wrote: > Hello App Engine Developers, > > As part of our ongoing efforts to improve release quality and > transparency, we will start prereleasing SDKs for early testing. We > hope this gives developers a chance to participate in our release > process by trying out new changes and sending feedback. As of this > morning, the prerelease SDK for our next release, 1.3.1, is available > in the familiar download location (note that the filename ends in > 'prerelease.zip'): > > http://code.google.com/p/googleappengine/downloads/list > > If you're interested, please download and give it a try locally with > your favorite App Engine code. Please note that, as a prerelease, this > SDK is not yet supported and still subject to change. Thus, please > don't take critical dependencies or make substantial changes to > production apps based on this SDK. > > Importantly, this prerelease is purely for the SDK and is intended for > local testing and development in dev_appserver. The server-side of App > Engine (our production environment) is not at 1.3.1, so deploying with > this SDK is not yet supported. In the future, we might enable a > complete SDK and server test environment for prereleases. > > Please try 1.3.1 for local development and send us your feedback! > > Thanks, > > App Engine Team > > Python > = > - New support for Datastore Query Cursors > - New support for Transactional Task Creation > - Additional file extensions permitted when sending mail including .doc and > .xls > http://code.google.com/p/googleappengine/issues/detail?id=494 > - New Grab Tail added to Memcache API > - Support for Custom Admin Console pages > - New "month" and "synchronized" syntax for Cron configuration > - Application Stats library now included in with SDK > - Bulk Loader supports bulk downloading all kinds simultaneously > - appcfg.py validates SSL certificates for HTTPS connections > - Support for ETags, If-matches, If-not-matches HTTP Headers, as well as 304 > error codes now available on static files (not available on the > dev_appserver or Blobstore > blobs)http://code.google.com/p/googleappengine/issues/detail?id=575 > > Java > = > - Datastore Query Cursors > - Transactional Tasks > - Additional file extensions permitted when sending mail including .doc and > .xsl > http://code.google.com/p/googleappengine/issues/detail?id=494 > - Grab Tail added to Memcache API > - Support for Custom Admin Console pages > - Java Precompilation is now on by default. > - Developers can opt-out of precompilation by setting the flag in > appengine-web.xml > false > - New built-in support for unit testing (see appengine-testing.jar) > http://code.google.com/p/googleappengine/issues/detail?id=326 > - net.sf.jsr107 package included as an alternative to the low-level Memcache > API > - javax.annotation.Resource/Resources added to the package whitelist > - New "month" and "synchronized" syntax for Cron configuration > - URLFetch supports asynchronous requests > -http://code.google.com/p/googleappengine/issues/detail?id=1899 > - appcfg.sh uses HTTPS for application deployment > - appcfg.sh adds request_logs --append > - Changes to the order queries without a specified sort order are returned. > Only queries that use IN will see different results. > - Added support for multiple != filters on the same property > - Improved support for keys-only queries when using IN and != filters > - Support for ETags, If-matches, If-not-matches HTTP Headers, as well as 304 > error codes now available on static files (not available on the > dev_appserver or Blobstore blobs) > http://code.google.com/p/googleappengine/issues/detail?id=575 > - Fixed issue where the maximum transform count was enforced for composite > operations > http://code.google.com/p/googleappengine/issues/detail?id=1656 > - Fixed issue with whitespace on the end of strings in web.xml > http://code.google.com/p/googleappengine/issues/detail?id=2242 > - Fixed "Not Found" issue when defining in web.xml > http://code.google.com/p/googleappengine/issues/detail?id=1477 > - Fixed issue when defining in web.xml > http://code.google.com/p/googleappengine/issues/detail?id=1249 > - Fixed issue where cancelling a deployment in progress would > unintentionally delete packages > http://code.google.com/p/googleappengine/issues/detail?id=2255 > - Fixed issue with QuotaService.getCpuTimeInMegaCycles() returning 0 > http://code.google.com/p/googleappengine/issues/detail?id=2639 > - Fixed issue where JSP exceptions will be incorrectly cast causing a > ClassCastException > http://code.google.com/p/googleappengine/issues/detail?id=1438 > > -- > Ikai Lan > Developer Programs Engineer, Google App > Enginehttp://googleappengine.blogspot.com|http://twitter.com/app_engine -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to g
[google-appengine] Re: 1.3.1 SDK Prerelease - help us verify
As I see, this code was added in r93 and it's SDK 1.2.8. Does it mean you just didn't mention this feature in 1.2.8? On Feb 4, 3:18 pm, "Nick Johnson (Google)" wrote: > On Thu, Feb 4, 2010 at 10:01 AM, Koen Bok wrote: > > Seems like an exciting update! > > > - New Grab Tail added to Memcache API > > > What does this mean? > > See the inline docs, > here:http://code.google.com/p/googleappengine/source/browse/trunk/python/g... > > -Nick Johnson > > > > > > > > > On Feb 4, 3:07 am, Jason C wrote: > > > I think it's mean to refer to 304 as an "error code" - it's the best > > > HTTP response code of all, and think of what a wonderful place the > > > Internet would be if everyone knew how to use it properly! ;) > > > > j > > > > On Feb 3, 4:03 pm, Ikai Lan wrote: > > > > > Hello App Engine Developers, > > > > > As part of our ongoing efforts to improve release quality and > > > > transparency, we will start prereleasing SDKs for early testing. We > > > > hope this gives developers a chance to participate in our release > > > > process by trying out new changes and sending feedback. As of this > > > > morning, the prerelease SDK for our next release, 1.3.1, is available > > > > in the familiar download location (note that the filename ends in > > > > 'prerelease.zip'): > > > > >http://code.google.com/p/googleappengine/downloads/list > > > > > If you're interested, please download and give it a try locally with > > > > your favorite App Engine code. Please note that, as a prerelease, this > > > > SDK is not yet supported and still subject to change. Thus, please > > > > don't take critical dependencies or make substantial changes to > > > > production apps based on this SDK. > > > > > Importantly, this prerelease is purely for the SDK and is intended for > > > > local testing and development in dev_appserver. The server-side of App > > > > Engine (our production environment) is not at 1.3.1, so deploying with > > > > this SDK is not yet supported. In the future, we might enable a > > > > complete SDK and server test environment for prereleases. > > > > > Please try 1.3.1 for local development and send us your feedback! > > > > > Thanks, > > > > > App Engine Team > > > > > Python > > > > = > > > > - New support for Datastore Query Cursors > > > > - New support for Transactional Task Creation > > > > - Additional file extensions permitted when sending mail including .doc > > and > > > > .xls > > > > http://code.google.com/p/googleappengine/issues/detail?id=494 > > > > - New Grab Tail added to Memcache API > > > > - Support for Custom Admin Console pages > > > > - New "month" and "synchronized" syntax for Cron configuration > > > > - Application Stats library now included in with SDK > > > > - Bulk Loader supports bulk downloading all kinds simultaneously > > > > - appcfg.py validates SSL certificates for HTTPS connections > > > > - Support for ETags, If-matches, If-not-matches HTTP Headers, as well > > as 304 > > > > error codes now available on static files (not available on the > > > > dev_appserver or Blobstore blobs) > >http://code.google.com/p/googleappengine/issues/detail?id=575 > > > > > Java > > > > = > > > > - Datastore Query Cursors > > > > - Transactional Tasks > > > > - Additional file extensions permitted when sending mail including .doc > > and > > > > .xsl > > > > http://code.google.com/p/googleappengine/issues/detail?id=494 > > > > - Grab Tail added to Memcache API > > > > - Support for Custom Admin Console pages > > > > - Java Precompilation is now on by default. > > > > - Developers can opt-out of precompilation by setting the flag in > > > > appengine-web.xml > > > > false > > > > - New built-in support for unit testing (see appengine-testing.jar) > > > > http://code.google.com/p/googleappengine/issues/detail?id=326 > > > > - net.sf.jsr107 package included as an alternative to the low-level > > Memcache > > > > API > > > > - javax.annotation.Resource/Resources added to the package whitelist > > > > - New "month" and "synchronized" syntax for Cron configuration > > > > - URLFetch supports asynchronous requests > > > > -http://code.google.com/p/googleappengine/issues/detail?id=1899 > > > > - appcfg.sh uses HTTPS for application deployment > > > > - appcfg.sh adds request_logs --append > > > > - Changes to the order queries without a specified sort order are > > returned. > > > > Only queries that use IN will see different results. > > > > - Added support for multiple != filters on the same property > > > > - Improved support for keys-only queries when using IN and != filters > > > > - Support for ETags, If-matches, If-not-matches HTTP Headers, as well > > as 304 > > > > error codes now available on static files (not available on the > > > > dev_appserver or Blobstore blobs) > > > > http://code.google.com/p/googleappengine/issues/detail?id=575 > > > > - Fixed issue where the maximum transform count was enforced for > > composite > > > > operations > > > > http://code.goog
[google-appengine] Re: 1.3.1 SDK Prerelease - help us verify
Very interesting it couldn't some language .mo dialectindependent adds for instance arabic months some noted should. Easy output enables arabic months we can display العملية 04 فبراير أهلا، دخول -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: 1.3.1 SDK Prerelease - help us verify
Maybe itsn't best place for this question, but... Have you any plans to finish an XMPP API with and support (http://code.google.com/p/googleappengine/issues/detail? id=2071) On Feb 3, 5:03 pm, Ikai Lan wrote: > Hello App Engine Developers, > > As part of our ongoing efforts to improve release quality and > transparency, we will start prereleasing SDKs for early testing. We > hope this gives developers a chance to participate in our release > process by trying out new changes and sending feedback. As of this > morning, the prerelease SDK for our next release, 1.3.1, is available > in the familiar download location (note that the filename ends in > 'prerelease.zip'): > > http://code.google.com/p/googleappengine/downloads/list > > If you're interested, please download and give it a try locally with > your favorite App Engine code. Please note that, as a prerelease, this > SDK is not yet supported and still subject to change. Thus, please > don't take critical dependencies or make substantial changes to > production apps based on this SDK. > > Importantly, this prerelease is purely for the SDK and is intended for > local testing and development in dev_appserver. The server-side of App > Engine (our production environment) is not at 1.3.1, so deploying with > this SDK is not yet supported. In the future, we might enable a > complete SDK and server test environment for prereleases. > > Please try 1.3.1 for local development and send us your feedback! > > Thanks, > > App Engine Team > > Python > = > - New support for Datastore Query Cursors > - New support for Transactional Task Creation > - Additional file extensions permitted when sending mail including .doc and > .xls > http://code.google.com/p/googleappengine/issues/detail?id=494 > - New Grab Tail added to Memcache API > - Support for Custom Admin Console pages > - New "month" and "synchronized" syntax for Cron configuration > - Application Stats library now included in with SDK > - Bulk Loader supports bulk downloading all kinds simultaneously > - appcfg.py validates SSL certificates for HTTPS connections > - Support for ETags, If-matches, If-not-matches HTTP Headers, as well as 304 > error codes now available on static files (not available on the > dev_appserver or Blobstore > blobs)http://code.google.com/p/googleappengine/issues/detail?id=575 > > Java > = > - Datastore Query Cursors > - Transactional Tasks > - Additional file extensions permitted when sending mail including .doc and > .xsl > http://code.google.com/p/googleappengine/issues/detail?id=494 > - Grab Tail added to Memcache API > - Support for Custom Admin Console pages > - Java Precompilation is now on by default. > - Developers can opt-out of precompilation by setting the flag in > appengine-web.xml > false > - New built-in support for unit testing (see appengine-testing.jar) > http://code.google.com/p/googleappengine/issues/detail?id=326 > - net.sf.jsr107 package included as an alternative to the low-level Memcache > API > - javax.annotation.Resource/Resources added to the package whitelist > - New "month" and "synchronized" syntax for Cron configuration > - URLFetch supports asynchronous requests > -http://code.google.com/p/googleappengine/issues/detail?id=1899 > - appcfg.sh uses HTTPS for application deployment > - appcfg.sh adds request_logs --append > - Changes to the order queries without a specified sort order are returned. > Only queries that use IN will see different results. > - Added support for multiple != filters on the same property > - Improved support for keys-only queries when using IN and != filters > - Support for ETags, If-matches, If-not-matches HTTP Headers, as well as 304 > error codes now available on static files (not available on the > dev_appserver or Blobstore blobs) > http://code.google.com/p/googleappengine/issues/detail?id=575 > - Fixed issue where the maximum transform count was enforced for composite > operations > http://code.google.com/p/googleappengine/issues/detail?id=1656 > - Fixed issue with whitespace on the end of strings in web.xml > http://code.google.com/p/googleappengine/issues/detail?id=2242 > - Fixed "Not Found" issue when defining in web.xml > http://code.google.com/p/googleappengine/issues/detail?id=1477 > - Fixed issue when defining in web.xml > http://code.google.com/p/googleappengine/issues/detail?id=1249 > - Fixed issue where cancelling a deployment in progress would > unintentionally delete packages > http://code.google.com/p/googleappengine/issues/detail?id=2255 > - Fixed issue with QuotaService.getCpuTimeInMegaCycles() returning 0 > http://code.google.com/p/googleappengine/issues/detail?id=2639 > - Fixed issue where JSP exceptions will be incorrectly cast causing a > ClassCastException > http://code.google.com/p/googleappengine/issues/detail?id=1438 > > -- > Ikai Lan > Developer Programs Engineer, Google App > Enginehttp://googleappengine.blogspot.com|http://twitter.com/app_engine -- You received this
[google-appengine] Re: 1.3.1 SDK Prerelease - help us verify
Datastore Query Cursors seem very interesting. Where can we find documentation (inline or otherwise) on them? On Feb 4, 7:18 am, "Nick Johnson (Google)" wrote: > On Thu, Feb 4, 2010 at 10:01 AM, Koen Bok wrote: > > Seems like an exciting update! > > > - New Grab Tail added to Memcache API > > > What does this mean? > > See the inline docs, > here:http://code.google.com/p/googleappengine/source/browse/trunk/python/g... > > -Nick Johnson > > > > > > > > > On Feb 4, 3:07 am, Jason C wrote: > > > I think it's mean to refer to 304 as an "error code" - it's the best > > > HTTP response code of all, and think of what a wonderful place the > > > Internet would be if everyone knew how to use it properly! ;) > > > > j > > > > On Feb 3, 4:03 pm, Ikai Lan wrote: > > > > > Hello App Engine Developers, > > > > > As part of our ongoing efforts to improve release quality and > > > > transparency, we will start prereleasing SDKs for early testing. We > > > > hope this gives developers a chance to participate in our release > > > > process by trying out new changes and sending feedback. As of this > > > > morning, the prerelease SDK for our next release, 1.3.1, is available > > > > in the familiar download location (note that the filename ends in > > > > 'prerelease.zip'): > > > > >http://code.google.com/p/googleappengine/downloads/list > > > > > If you're interested, please download and give it a try locally with > > > > your favorite App Engine code. Please note that, as a prerelease, this > > > > SDK is not yet supported and still subject to change. Thus, please > > > > don't take critical dependencies or make substantial changes to > > > > production apps based on this SDK. > > > > > Importantly, this prerelease is purely for the SDK and is intended for > > > > local testing and development in dev_appserver. The server-side of App > > > > Engine (our production environment) is not at 1.3.1, so deploying with > > > > this SDK is not yet supported. In the future, we might enable a > > > > complete SDK and server test environment for prereleases. > > > > > Please try 1.3.1 for local development and send us your feedback! > > > > > Thanks, > > > > > App Engine Team > > > > > Python > > > > = > > > > - New support for Datastore Query Cursors > > > > - New support for Transactional Task Creation > > > > - Additional file extensions permitted when sending mail including .doc > > and > > > > .xls > > > > http://code.google.com/p/googleappengine/issues/detail?id=494 > > > > - New Grab Tail added to Memcache API > > > > - Support for Custom Admin Console pages > > > > - New "month" and "synchronized" syntax for Cron configuration > > > > - Application Stats library now included in with SDK > > > > - Bulk Loader supports bulk downloading all kinds simultaneously > > > > - appcfg.py validates SSL certificates for HTTPS connections > > > > - Support for ETags, If-matches, If-not-matches HTTP Headers, as well > > as 304 > > > > error codes now available on static files (not available on the > > > > dev_appserver or Blobstore blobs) > >http://code.google.com/p/googleappengine/issues/detail?id=575 > > > > > Java > > > > = > > > > - Datastore Query Cursors > > > > - Transactional Tasks > > > > - Additional file extensions permitted when sending mail including .doc > > and > > > > .xsl > > > > http://code.google.com/p/googleappengine/issues/detail?id=494 > > > > - Grab Tail added to Memcache API > > > > - Support for Custom Admin Console pages > > > > - Java Precompilation is now on by default. > > > > - Developers can opt-out of precompilation by setting the flag in > > > > appengine-web.xml > > > > false > > > > - New built-in support for unit testing (see appengine-testing.jar) > > > > http://code.google.com/p/googleappengine/issues/detail?id=326 > > > > - net.sf.jsr107 package included as an alternative to the low-level > > Memcache > > > > API > > > > - javax.annotation.Resource/Resources added to the package whitelist > > > > - New "month" and "synchronized" syntax for Cron configuration > > > > - URLFetch supports asynchronous requests > > > > -http://code.google.com/p/googleappengine/issues/detail?id=1899 > > > > - appcfg.sh uses HTTPS for application deployment > > > > - appcfg.sh adds request_logs --append > > > > - Changes to the order queries without a specified sort order are > > returned. > > > > Only queries that use IN will see different results. > > > > - Added support for multiple != filters on the same property > > > > - Improved support for keys-only queries when using IN and != filters > > > > - Support for ETags, If-matches, If-not-matches HTTP Headers, as well > > as 304 > > > > error codes now available on static files (not available on the > > > > dev_appserver or Blobstore blobs) > > > > http://code.google.com/p/googleappengine/issues/detail?id=575 > > > > - Fixed issue where the maximum transform count was enforced for > > composite > > > > operations > > > > http://code.google.com/
[google-appengine] Re: 1.3.1 SDK Prerelease - help us verify
Finally, Datastore Query Cursors. Thank you. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: 1.3.1 SDK Prerelease - help us verify
I believe that the CRON changes are to address this issue: http://code.google.com/p/googleappengine/issues/detail?id=1261&can=5&colspec=ID%20Type%20Status%20Priority%20Stars%20Owner%20Summary%20Log%20Component Can you please explain the new syntax? Thanks! -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: 1.3.1 SDK Prerelease - help us verify
On Thu, Feb 4, 2010 at 10:01 AM, Koen Bok wrote: > Seems like an exciting update! > > - New Grab Tail added to Memcache API > > What does this mean? > See the inline docs, here: http://code.google.com/p/googleappengine/source/browse/trunk/python/google/appengine/api/memcache/__init__.py#934 -Nick Johnson > > On Feb 4, 3:07 am, Jason C wrote: > > I think it's mean to refer to 304 as an "error code" - it's the best > > HTTP response code of all, and think of what a wonderful place the > > Internet would be if everyone knew how to use it properly! ;) > > > > j > > > > On Feb 3, 4:03 pm, Ikai Lan wrote: > > > > > > > > > Hello App Engine Developers, > > > > > As part of our ongoing efforts to improve release quality and > > > transparency, we will start prereleasing SDKs for early testing. We > > > hope this gives developers a chance to participate in our release > > > process by trying out new changes and sending feedback. As of this > > > morning, the prerelease SDK for our next release, 1.3.1, is available > > > in the familiar download location (note that the filename ends in > > > 'prerelease.zip'): > > > > >http://code.google.com/p/googleappengine/downloads/list > > > > > If you're interested, please download and give it a try locally with > > > your favorite App Engine code. Please note that, as a prerelease, this > > > SDK is not yet supported and still subject to change. Thus, please > > > don't take critical dependencies or make substantial changes to > > > production apps based on this SDK. > > > > > Importantly, this prerelease is purely for the SDK and is intended for > > > local testing and development in dev_appserver. The server-side of App > > > Engine (our production environment) is not at 1.3.1, so deploying with > > > this SDK is not yet supported. In the future, we might enable a > > > complete SDK and server test environment for prereleases. > > > > > Please try 1.3.1 for local development and send us your feedback! > > > > > Thanks, > > > > > App Engine Team > > > > > Python > > > = > > > - New support for Datastore Query Cursors > > > - New support for Transactional Task Creation > > > - Additional file extensions permitted when sending mail including .doc > and > > > .xls > > > http://code.google.com/p/googleappengine/issues/detail?id=494 > > > - New Grab Tail added to Memcache API > > > - Support for Custom Admin Console pages > > > - New "month" and "synchronized" syntax for Cron configuration > > > - Application Stats library now included in with SDK > > > - Bulk Loader supports bulk downloading all kinds simultaneously > > > - appcfg.py validates SSL certificates for HTTPS connections > > > - Support for ETags, If-matches, If-not-matches HTTP Headers, as well > as 304 > > > error codes now available on static files (not available on the > > > dev_appserver or Blobstore blobs) > http://code.google.com/p/googleappengine/issues/detail?id=575 > > > > > Java > > > = > > > - Datastore Query Cursors > > > - Transactional Tasks > > > - Additional file extensions permitted when sending mail including .doc > and > > > .xsl > > > http://code.google.com/p/googleappengine/issues/detail?id=494 > > > - Grab Tail added to Memcache API > > > - Support for Custom Admin Console pages > > > - Java Precompilation is now on by default. > > > - Developers can opt-out of precompilation by setting the flag in > > > appengine-web.xml > > > false > > > - New built-in support for unit testing (see appengine-testing.jar) > > > http://code.google.com/p/googleappengine/issues/detail?id=326 > > > - net.sf.jsr107 package included as an alternative to the low-level > Memcache > > > API > > > - javax.annotation.Resource/Resources added to the package whitelist > > > - New "month" and "synchronized" syntax for Cron configuration > > > - URLFetch supports asynchronous requests > > > -http://code.google.com/p/googleappengine/issues/detail?id=1899 > > > - appcfg.sh uses HTTPS for application deployment > > > - appcfg.sh adds request_logs --append > > > - Changes to the order queries without a specified sort order are > returned. > > > Only queries that use IN will see different results. > > > - Added support for multiple != filters on the same property > > > - Improved support for keys-only queries when using IN and != filters > > > - Support for ETags, If-matches, If-not-matches HTTP Headers, as well > as 304 > > > error codes now available on static files (not available on the > > > dev_appserver or Blobstore blobs) > > > http://code.google.com/p/googleappengine/issues/detail?id=575 > > > - Fixed issue where the maximum transform count was enforced for > composite > > > operations > > > http://code.google.com/p/googleappengine/issues/detail?id=1656 > > > - Fixed issue with whitespace on the end of strings in web.xml > > > http://code.google.com/p/googleappengine/issues/detail?id=2242 > > > - Fixed "Not Found" issue when defining in web.xml > > > http://code.google.com/p/googleappengin
[google-appengine] Re: 1.3.1 SDK Prerelease - help us verify
Seems like an exciting update! - New Grab Tail added to Memcache API What does this mean? On Feb 4, 3:07 am, Jason C wrote: > I think it's mean to refer to 304 as an "error code" - it's the best > HTTP response code of all, and think of what a wonderful place the > Internet would be if everyone knew how to use it properly! ;) > > j > > On Feb 3, 4:03 pm, Ikai Lan wrote: > > > > > Hello App Engine Developers, > > > As part of our ongoing efforts to improve release quality and > > transparency, we will start prereleasing SDKs for early testing. We > > hope this gives developers a chance to participate in our release > > process by trying out new changes and sending feedback. As of this > > morning, the prerelease SDK for our next release, 1.3.1, is available > > in the familiar download location (note that the filename ends in > > 'prerelease.zip'): > > >http://code.google.com/p/googleappengine/downloads/list > > > If you're interested, please download and give it a try locally with > > your favorite App Engine code. Please note that, as a prerelease, this > > SDK is not yet supported and still subject to change. Thus, please > > don't take critical dependencies or make substantial changes to > > production apps based on this SDK. > > > Importantly, this prerelease is purely for the SDK and is intended for > > local testing and development in dev_appserver. The server-side of App > > Engine (our production environment) is not at 1.3.1, so deploying with > > this SDK is not yet supported. In the future, we might enable a > > complete SDK and server test environment for prereleases. > > > Please try 1.3.1 for local development and send us your feedback! > > > Thanks, > > > App Engine Team > > > Python > > = > > - New support for Datastore Query Cursors > > - New support for Transactional Task Creation > > - Additional file extensions permitted when sending mail including .doc and > > .xls > > http://code.google.com/p/googleappengine/issues/detail?id=494 > > - New Grab Tail added to Memcache API > > - Support for Custom Admin Console pages > > - New "month" and "synchronized" syntax for Cron configuration > > - Application Stats library now included in with SDK > > - Bulk Loader supports bulk downloading all kinds simultaneously > > - appcfg.py validates SSL certificates for HTTPS connections > > - Support for ETags, If-matches, If-not-matches HTTP Headers, as well as 304 > > error codes now available on static files (not available on the > > dev_appserver or Blobstore > > blobs)http://code.google.com/p/googleappengine/issues/detail?id=575 > > > Java > > = > > - Datastore Query Cursors > > - Transactional Tasks > > - Additional file extensions permitted when sending mail including .doc and > > .xsl > > http://code.google.com/p/googleappengine/issues/detail?id=494 > > - Grab Tail added to Memcache API > > - Support for Custom Admin Console pages > > - Java Precompilation is now on by default. > > - Developers can opt-out of precompilation by setting the flag in > > appengine-web.xml > > false > > - New built-in support for unit testing (see appengine-testing.jar) > > http://code.google.com/p/googleappengine/issues/detail?id=326 > > - net.sf.jsr107 package included as an alternative to the low-level Memcache > > API > > - javax.annotation.Resource/Resources added to the package whitelist > > - New "month" and "synchronized" syntax for Cron configuration > > - URLFetch supports asynchronous requests > > -http://code.google.com/p/googleappengine/issues/detail?id=1899 > > - appcfg.sh uses HTTPS for application deployment > > - appcfg.sh adds request_logs --append > > - Changes to the order queries without a specified sort order are returned. > > Only queries that use IN will see different results. > > - Added support for multiple != filters on the same property > > - Improved support for keys-only queries when using IN and != filters > > - Support for ETags, If-matches, If-not-matches HTTP Headers, as well as 304 > > error codes now available on static files (not available on the > > dev_appserver or Blobstore blobs) > > http://code.google.com/p/googleappengine/issues/detail?id=575 > > - Fixed issue where the maximum transform count was enforced for composite > > operations > > http://code.google.com/p/googleappengine/issues/detail?id=1656 > > - Fixed issue with whitespace on the end of strings in web.xml > > http://code.google.com/p/googleappengine/issues/detail?id=2242 > > - Fixed "Not Found" issue when defining in web.xml > > http://code.google.com/p/googleappengine/issues/detail?id=1477 > > - Fixed issue when defining in web.xml > > http://code.google.com/p/googleappengine/issues/detail?id=1249 > > - Fixed issue where cancelling a deployment in progress would > > unintentionally delete packages > > http://code.google.com/p/googleappengine/issues/detail?id=2255 > > - Fixed issue with QuotaService.getCpuTimeInMegaCycles() returning 0 > > http://code.google.com/p/googleappengine/
[google-appengine] Re: 1.3.1 SDK Prerelease - help us verify
I think it's mean to refer to 304 as an "error code" - it's the best HTTP response code of all, and think of what a wonderful place the Internet would be if everyone knew how to use it properly! ;) j On Feb 3, 4:03 pm, Ikai Lan wrote: > Hello App Engine Developers, > > As part of our ongoing efforts to improve release quality and > transparency, we will start prereleasing SDKs for early testing. We > hope this gives developers a chance to participate in our release > process by trying out new changes and sending feedback. As of this > morning, the prerelease SDK for our next release, 1.3.1, is available > in the familiar download location (note that the filename ends in > 'prerelease.zip'): > > http://code.google.com/p/googleappengine/downloads/list > > If you're interested, please download and give it a try locally with > your favorite App Engine code. Please note that, as a prerelease, this > SDK is not yet supported and still subject to change. Thus, please > don't take critical dependencies or make substantial changes to > production apps based on this SDK. > > Importantly, this prerelease is purely for the SDK and is intended for > local testing and development in dev_appserver. The server-side of App > Engine (our production environment) is not at 1.3.1, so deploying with > this SDK is not yet supported. In the future, we might enable a > complete SDK and server test environment for prereleases. > > Please try 1.3.1 for local development and send us your feedback! > > Thanks, > > App Engine Team > > Python > = > - New support for Datastore Query Cursors > - New support for Transactional Task Creation > - Additional file extensions permitted when sending mail including .doc and > .xls > http://code.google.com/p/googleappengine/issues/detail?id=494 > - New Grab Tail added to Memcache API > - Support for Custom Admin Console pages > - New "month" and "synchronized" syntax for Cron configuration > - Application Stats library now included in with SDK > - Bulk Loader supports bulk downloading all kinds simultaneously > - appcfg.py validates SSL certificates for HTTPS connections > - Support for ETags, If-matches, If-not-matches HTTP Headers, as well as 304 > error codes now available on static files (not available on the > dev_appserver or Blobstore > blobs)http://code.google.com/p/googleappengine/issues/detail?id=575 > > Java > = > - Datastore Query Cursors > - Transactional Tasks > - Additional file extensions permitted when sending mail including .doc and > .xsl > http://code.google.com/p/googleappengine/issues/detail?id=494 > - Grab Tail added to Memcache API > - Support for Custom Admin Console pages > - Java Precompilation is now on by default. > - Developers can opt-out of precompilation by setting the flag in > appengine-web.xml > false > - New built-in support for unit testing (see appengine-testing.jar) > http://code.google.com/p/googleappengine/issues/detail?id=326 > - net.sf.jsr107 package included as an alternative to the low-level Memcache > API > - javax.annotation.Resource/Resources added to the package whitelist > - New "month" and "synchronized" syntax for Cron configuration > - URLFetch supports asynchronous requests > -http://code.google.com/p/googleappengine/issues/detail?id=1899 > - appcfg.sh uses HTTPS for application deployment > - appcfg.sh adds request_logs --append > - Changes to the order queries without a specified sort order are returned. > Only queries that use IN will see different results. > - Added support for multiple != filters on the same property > - Improved support for keys-only queries when using IN and != filters > - Support for ETags, If-matches, If-not-matches HTTP Headers, as well as 304 > error codes now available on static files (not available on the > dev_appserver or Blobstore blobs) > http://code.google.com/p/googleappengine/issues/detail?id=575 > - Fixed issue where the maximum transform count was enforced for composite > operations > http://code.google.com/p/googleappengine/issues/detail?id=1656 > - Fixed issue with whitespace on the end of strings in web.xml > http://code.google.com/p/googleappengine/issues/detail?id=2242 > - Fixed "Not Found" issue when defining in web.xml > http://code.google.com/p/googleappengine/issues/detail?id=1477 > - Fixed issue when defining in web.xml > http://code.google.com/p/googleappengine/issues/detail?id=1249 > - Fixed issue where cancelling a deployment in progress would > unintentionally delete packages > http://code.google.com/p/googleappengine/issues/detail?id=2255 > - Fixed issue with QuotaService.getCpuTimeInMegaCycles() returning 0 > http://code.google.com/p/googleappengine/issues/detail?id=2639 > - Fixed issue where JSP exceptions will be incorrectly cast causing a > ClassCastException > http://code.google.com/p/googleappengine/issues/detail?id=1438 > > -- > Ikai Lan > Developer Programs Engineer, Google App > Enginehttp://googleappengine.blogspot.com|http://twitter.com/app_engi
[google-appengine] Re: 1.3.1 SDK Prerelease - help us verify
Is there any documentation on the new functionality released for python, specifically datastore query cursors? On Feb 3, 4:03 pm, Ikai Lan wrote: > Hello App Engine Developers, > > As part of our ongoing efforts to improve release quality and > transparency, we will start prereleasing SDKs for early testing. We > hope this gives developers a chance to participate in our release > process by trying out new changes and sending feedback. As of this > morning, the prerelease SDK for our next release, 1.3.1, is available > in the familiar download location (note that the filename ends in > 'prerelease.zip'): > > http://code.google.com/p/googleappengine/downloads/list > > If you're interested, please download and give it a try locally with > your favorite App Engine code. Please note that, as a prerelease, this > SDK is not yet supported and still subject to change. Thus, please > don't take critical dependencies or make substantial changes to > production apps based on this SDK. > > Importantly, this prerelease is purely for the SDK and is intended for > local testing and development in dev_appserver. The server-side of App > Engine (our production environment) is not at 1.3.1, so deploying with > this SDK is not yet supported. In the future, we might enable a > complete SDK and server test environment for prereleases. > > Please try 1.3.1 for local development and send us your feedback! > > Thanks, > > App Engine Team > > Python > = > - New support for Datastore Query Cursors > - New support for Transactional Task Creation > - Additional file extensions permitted when sending mail including .doc and > .xls > http://code.google.com/p/googleappengine/issues/detail?id=494 > - New Grab Tail added to Memcache API > - Support for Custom Admin Console pages > - New "month" and "synchronized" syntax for Cron configuration > - Application Stats library now included in with SDK > - Bulk Loader supports bulk downloading all kinds simultaneously > - appcfg.py validates SSL certificates for HTTPS connections > - Support for ETags, If-matches, If-not-matches HTTP Headers, as well as 304 > error codes now available on static files (not available on the > dev_appserver or Blobstore > blobs)http://code.google.com/p/googleappengine/issues/detail?id=575 > > Java > = > - Datastore Query Cursors > - Transactional Tasks > - Additional file extensions permitted when sending mail including .doc and > .xsl > http://code.google.com/p/googleappengine/issues/detail?id=494 > - Grab Tail added to Memcache API > - Support for Custom Admin Console pages > - Java Precompilation is now on by default. > - Developers can opt-out of precompilation by setting the flag in > appengine-web.xml > false > - New built-in support for unit testing (see appengine-testing.jar) > http://code.google.com/p/googleappengine/issues/detail?id=326 > - net.sf.jsr107 package included as an alternative to the low-level Memcache > API > - javax.annotation.Resource/Resources added to the package whitelist > - New "month" and "synchronized" syntax for Cron configuration > - URLFetch supports asynchronous requests > -http://code.google.com/p/googleappengine/issues/detail?id=1899 > - appcfg.sh uses HTTPS for application deployment > - appcfg.sh adds request_logs --append > - Changes to the order queries without a specified sort order are returned. > Only queries that use IN will see different results. > - Added support for multiple != filters on the same property > - Improved support for keys-only queries when using IN and != filters > - Support for ETags, If-matches, If-not-matches HTTP Headers, as well as 304 > error codes now available on static files (not available on the > dev_appserver or Blobstore blobs) > http://code.google.com/p/googleappengine/issues/detail?id=575 > - Fixed issue where the maximum transform count was enforced for composite > operations > http://code.google.com/p/googleappengine/issues/detail?id=1656 > - Fixed issue with whitespace on the end of strings in web.xml > http://code.google.com/p/googleappengine/issues/detail?id=2242 > - Fixed "Not Found" issue when defining in web.xml > http://code.google.com/p/googleappengine/issues/detail?id=1477 > - Fixed issue when defining in web.xml > http://code.google.com/p/googleappengine/issues/detail?id=1249 > - Fixed issue where cancelling a deployment in progress would > unintentionally delete packages > http://code.google.com/p/googleappengine/issues/detail?id=2255 > - Fixed issue with QuotaService.getCpuTimeInMegaCycles() returning 0 > http://code.google.com/p/googleappengine/issues/detail?id=2639 > - Fixed issue where JSP exceptions will be incorrectly cast causing a > ClassCastException > http://code.google.com/p/googleappengine/issues/detail?id=1438 > > -- > Ikai Lan > Developer Programs Engineer, Google App > Enginehttp://googleappengine.blogspot.com|http://twitter.com/app_engine -- You received this message because you are subscribed to the Google Groups "Google
