[google-appengine] Re: app-engine-data-pipelines session video
Brett, any plans to make an article of this talk?
This feels like such a key strategy to getting stuff done on
datastore, it should be part of the sdk.
On Jun 7, 11:44 pm, Brett Slatkin wrote:
> I was using an integer hash to reduce the key size. You don't need to hash
> the whole thing. Bigtable will split tablets based on a string prefix, so
> all that matters is the data distribution beyond that prefix. So
> "foo-" is just as effective as "", or even
> better since it's shorter.
>
> 2010/6/7 Jaroslav Záruba
>
>
>
> > Thank you, Brett.
>
> > Would it be wrong to hash whole work_index instead of only hashing its
> > second half? sum_name, knuth_hash(index)
> > By md5-ing only the sequence number I get work_index of 'mySumName' + 32B.
> > If I hashed mySumName together with the seq.number the key would be only
> > 32B. (Still quite huge though.)
> > Given how frequent a vote entity is I would like to have the keys as short
> > as possible.
>
> > Regards
> > J. Záruba
>
> > On Mon, Jun 7, 2010 at 11:21 PM, Brett Slatkin > > wrote:
>
> >> Hey all,
>
> >> The int(time.time()/30) part of the task name is to prevent queue stalls.
> >> When memcache gets evicted the work index counter will be reset to zero.
> >> That means new fork-join work items may insert tasks that are named the
> >> same
> >> as tasks that were already inserted. By including a time window of ~30
> >> seconds in the task name, we ensure that this problem can only last for
> >> about thirty seconds. This is also why you should raise an exception when
> >> you see a TombstonedTaskError exception.
>
> >> Worst-case scenario if the clocks are wonky is that two tasks are run to
> >> do the fan-in work instead of just one, which is an acceptable trade-off in
> >> many cases and a fundamental possibility when using the task queue API.
> >> This
> >> can be mitigated using pigeon-hole acknowledgment entities, like I use in
> >> my
> >> materialized view example.
>
> >> Hope that helps,
>
> >> -Brett
>
> >> On Mon, Jun 7, 2010 at 2:14 PM, Tristan wrote:
>
> >>> not a python guy but, the purpose of int (now / 30) will be to come up
> >>> with the same name for a span of time (30 milliseconds?).
>
> >>> notice that int(1/30) = 0 int (3/30) = 0 int (29/30) = 0 and
> >>> int(32/30) = 1. this is a way to come up with that task name
> >>> uniquely.
>
> >>> although now i'm confused because doesn't he say later on that time is
> >>> a bad thing to use for synchronization and sequence numbers should be
> >>> used instead?
>
> >>> On Jun 7, 2:40 am, Jaroslav Záruba wrote:
> >>> > Also if someone knew what is the purpose of "now / 30" in the task
> >>> name,
> >>> > please:http://www.youtube.com/watch?v=zSDC_TU7rtc#t=41m35
>
> >>> > Regards
> >>> > J. Záruba
>
> >>> > 2010/6/7 Jaroslav Záruba
>
> >>> > > Hello
>
> >>> > > I'm reading through the PDF that Brett Slatkin has published for
> >>> %subj
> >>> > > %.
> >>> > >http://tinyurl.com/3523mej
>
> >>> > > In the video (the Fan-in part) Brett says that the work_index has to
> >>> > > be a hash, so that 'you distribute the load across the BigTable'
> >>> > >http://www.youtube.com/watch?v=zSDC_TU7rtc#t=48m44
>
> >>> > > And this is how work_index is created:
> >>> > > work_index = '%s-%d' % (sum_name, knuth_hash(index))
> >>> > > ...which I guess creates something like
> >>> 'votesMovieXYZ-54657651321987'
>
> >>> > > My question is why only one half of work_index is hashed? Is it
> >>> > > important?
> >>> > > Would it be bad to do md5('%s-%d' % (sum_name, index)) so that the
> >>> > > hash would be like '6gw8hq6'?
>
> >>> > > Regards
> >>> > > J. Záruba
>
> >>> --
> >>> You received this message because you are subscribed to the Google Groups
> >>> "Google App Engine" group.
> >>> To post to this group, send email to google-appeng...@googlegroups.com.
> >>> To unsubscribe from this group, send email to
> >>> google-appengine+unsubscr...@googlegroups.com >>> e...@googlegroups.com>
> >>> .
> >>> For more options, visit this group at
> >>>http://groups.google.com/group/google-appengine?hl=en.
>
> >> --
> >> You received this message because you are subscribed to the Google Groups
> >> "Google App Engine" group.
> >> To post to this group, send email to google-appeng...@googlegroups.com.
> >> To unsubscribe from this group, send email to
> >> google-appengine+unsubscr...@googlegroups.com >> e...@googlegroups.com>
> >> .
> >> For more options, visit this group at
> >>http://groups.google.com/group/google-appengine?hl=en.
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Google App Engine" group.
> > To post to this group, send email to google-appeng...@googlegroups.com.
> > To unsubscribe from this group, send email to
> > google-appengine+unsubscr...@googlegroups.com > e...@googlegroups.com>
> > .
> > For more options, visit this group at
> >http://groups.google.com/group/google-appengine?hl=en.
--
You received this message because you a
Re: [google-appengine] Deferred Library Not Working Intermittently
I'd like to update this thread with a new set of observations. Most
importantly, I have divorced myself from the notion that this has anythign
to do with the deferred library.
When I took the datastore puts out of the deferred function, they still
failed to complete. When I put the puts inside a transaction, it helps a
little bit, but the puts still fail about a third of the time. My question
is why this would be happening on the SDK. There is nothing else running in
the background when these puts are getting issued.
Your thoughts are much appreciated!
On Sun, Jun 13, 2010 at 2:58 PM, Alon Carmel wrote:
> having the same issue exactly. deferred lib runs two tasks then halts
> without uppering the counter. i thought i had some issue.
> -
> Cheers,
> public static function AlonCarmel() {
> //Contact me
> var email = 'a...@aloncarmel.me';
> var twitter = '@aloncarmel';
> var web = 'http://aloncarmel.me';
> var phone = '+972-54-4860380';
> }
>
> * If you received an unsolicited email from by mistake that wasn't of your
> matter please delete immediately. All E-mail sent from Alon Carmel is
> copyrighted to Alon Carmel 2008. Any details revealed in e-mails sent by
> Alon Carmel are owned by the Author only. Any attempt to duplicate or
> imitate any of the Content is prohibited under copyright law 2008.
>
>
>
> On Sun, Jun 13, 2010 at 11:00 PM, Aaron Cheung
> wrote:
>
>> Hi Nick,
>>
>> Unfortunately, putting the update inside a transaction didn't solve the
>> problem. This problem is currently occuring on the SDK, where there
>> shouldn't be any concurrency issues. I looked into AppStats again, and I
>> can clearly seeing the datastore.v3Commit taking place. And, I am pretty
>> sure the values are being passed in and set properly. It just fails a large
>> portion of the time. Any other ideas?
>>
>> Very best and thanks for your help,
>> Aaron
>>
>>
>> On Sun, Jun 13, 2010 at 4:47 AM, Nick Johnson (Google) <
>> nick.john...@google.com> wrote:
>>
>>> Hi Aaron,
>>>
>>> Since you're not doing the update inside a transaction, it's likely that
>>> what's happening is that multiple tasks are updating the entity at the same
>>> time, which results in a concurrency issue: Both tasks fetch the entity and
>>> modify it, and the one that does the put last is the one whose results are
>>> visible.
>>>
>>> Doing the update in a transaction should fix this.
>>>
>>> -Nick Johnson
>>>
>>> On Sun, Jun 13, 2010 at 8:50 AM, Aaron Cheung
>>> wrote:
>>>
Hi guys,
I'm still having trouble debugging this error. Would appreciate any
thoughts!
Thanks,
Aaron
On Sat, Jun 12, 2010 at 3:55 AM, Aaron wrote:
> Hi!
>
> I am currently implementing the deferred library in my code, and I am
> running into a very very strange problem. The function that I am
> deferring gets passed an entity key. It does two tasks: 1) Gets the
> entity, retrieves a reference property, alters it, and then puts it
> back 2)Sends out an email
>
> The email is getting sent out every time the deferred function is
> called. BUT, the first task only completes 2/3 of the time. I have
> been replicating this error over and over and have yet to find any
> pattern in the failures. Here is the deferred function
>
> def complete_insert(self, data, answer_key, userkey):
>a = db.get(answer_key)
>q = a.question
>q.answercount += 1
>q.answerers += [str(userkey)]
>q.answerers = list(set(q.answerers))
>q.answered = True
>q.lastanswer = str(answer_key)
>q.lastanswerdate = datetime.now()
>q.put()
>
>mail = Mail()
>followers = db.get(q.indexes.followerlist)
>emails = []
>for follower in followers:
>mail.send(emails, "question-followers-email", params =
> {"question": common.strip_html(q.question),
> "details":common.strip_html(q.details), "slug":q.slug,
> "answer":common.strip_html(data["answer"])} )
>
> A couple observations that make this even more boggling. I can tell
> when the function fails because the counter does not increment and the
> other properties do not change. However, when it fails, I can see the
> put() being done in AppStats under /ah/deferred. FURthermore, I have
> checked that the values being passed into the function are the ones
> that I am expecting. (I did this by adding another put that wrote the
> arguments to a test model)
>
> I have absolutely no idea what could be going on here. Is this a bug?
>
> Very best and thanks for your time,
> Aaron
>
> --
> You received this message because you are subscribed to the Google
> Groups "Google App Engine" group.
> To post to this group, send email to google-appengine@googlegroups.com
> .
> To unsubscribe from this group,
[google-appengine] Re: Critical bug report: index or ReferenceProperty bug
Thanks for your reply. Unfortunately my application has 7M UserSong objects. From a user submitted bug report I've managed to find 10 entites that seem to not be in the index, but potetnitally this is probably thousands if not more. Due to the size of the datastore it would be really expensive and time consuming to go over all of them and put them again. And even if I do, I would like to know that I wouldn't have to do this every now and then. I have only been using GAE the last two months and I hope I shouldn't have to be assuming that indexes don't work whenever I get an empty result. Cheers, Gergely On Jun 14, 7:51 am, djidjadji wrote: > Have you tried to loop over the UserSong objects and put() them again. > It will update the indexes. You can try it with one object you know > that fails to see if it solves the problem. > Sometimes an index is not updated correctly when you put() an object, > this was a problem in the early months of GAE. > > 2010/6/14 Gergely Orosz : > > > > > Hello! > > > I am reporting a bug related to either indexes or ReferenceProperties. > > > I have an application with a Song entity type and a UserSong entity > > type that has a userId string field and a song ReferenceProperty field > > that links to Songs. I have an combined index for song and userId > > fields within UserSongs entity that is serving without reported > > errors. > > > The problem is that when I query UserSongs with "SELECT * FROM > > UserSong WHERE userId=XXX AND song=key(XXX)" sometimes I get an empty > > result when the entity should be found; e.g. it appears both in the > > "SELECT * FROM UserSong WHERE userId=XXX" list and in the "SELECT * > > FROM UserSong WHERE song=key(XXX)" query as well. I would therefore > > expect it to appear in the combined "SELECT * FROM UserSong WHERE > > userId=XXX AND song=key(XXX)" query as well as I have the necessary > > indexes serving. This behaviour seems absolutely random; it happens > > really rarely, but should actually never happen. > > > I am able to reproduce the issue with specific values. My question > > where can I send this bug on to as this is a major issue for my > > application (e.g. my users are unable to fetch some of their existing > > entites). > > > As this is a critical issue for my application, it would appreciate to > > get some official help. > > > Thank you. > > > Gergely > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to google-appeng...@googlegroups.com. > > To unsubscribe from this group, send email to > > google-appengine+unsubscr...@googlegroups.com. > > For more options, visit this group > > athttp://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Deferred Library Not Working Intermittently
for me its just not running the deferred the third time. i see two tasks
maybe three running then everything halts. its like somewhere along the way
the deadtimeexceeded is not firing or the deferred doesnt have enough time
to fire up upon error.
i resolved by going task wise and just spreading the load across many tasks
(300) of them instead of doing it deferred.
-
Cheers,
public static function AlonCarmel() {
//Contact me
var email = 'a...@aloncarmel.me';
var twitter = '@aloncarmel';
var web = 'http://aloncarmel.me';
var phone = '+972-54-4860380';
}
* If you received an unsolicited email from by mistake that wasn't of your
matter please delete immediately. All E-mail sent from Alon Carmel is
copyrighted to Alon Carmel 2008. Any details revealed in e-mails sent by
Alon Carmel are owned by the Author only. Any attempt to duplicate or
imitate any of the Content is prohibited under copyright law 2008.
On Mon, Jun 14, 2010 at 11:07 AM, Aaron Cheung wrote:
> I'd like to update this thread with a new set of observations. Most
> importantly, I have divorced myself from the notion that this has anythign
> to do with the deferred library.
>
> When I took the datastore puts out of the deferred function, they still
> failed to complete. When I put the puts inside a transaction, it helps a
> little bit, but the puts still fail about a third of the time. My question
> is why this would be happening on the SDK. There is nothing else running in
> the background when these puts are getting issued.
>
> Your thoughts are much appreciated!
>
> On Sun, Jun 13, 2010 at 2:58 PM, Alon Carmel wrote:
>
>> having the same issue exactly. deferred lib runs two tasks then halts
>> without uppering the counter. i thought i had some issue.
>> -
>> Cheers,
>> public static function AlonCarmel() {
>> //Contact me
>> var email = 'a...@aloncarmel.me';
>> var twitter = '@aloncarmel';
>> var web = 'http://aloncarmel.me';
>> var phone = '+972-54-4860380';
>> }
>>
>> * If you received an unsolicited email from by mistake that wasn't of your
>> matter please delete immediately. All E-mail sent from Alon Carmel is
>> copyrighted to Alon Carmel 2008. Any details revealed in e-mails sent by
>> Alon Carmel are owned by the Author only. Any attempt to duplicate or
>> imitate any of the Content is prohibited under copyright law 2008.
>>
>>
>>
>> On Sun, Jun 13, 2010 at 11:00 PM, Aaron Cheung
>> wrote:
>>
>>> Hi Nick,
>>>
>>> Unfortunately, putting the update inside a transaction didn't solve the
>>> problem. This problem is currently occuring on the SDK, where there
>>> shouldn't be any concurrency issues. I looked into AppStats again, and I
>>> can clearly seeing the datastore.v3Commit taking place. And, I am pretty
>>> sure the values are being passed in and set properly. It just fails a large
>>> portion of the time. Any other ideas?
>>>
>>> Very best and thanks for your help,
>>> Aaron
>>>
>>>
>>> On Sun, Jun 13, 2010 at 4:47 AM, Nick Johnson (Google) <
>>> nick.john...@google.com> wrote:
>>>
Hi Aaron,
Since you're not doing the update inside a transaction, it's likely that
what's happening is that multiple tasks are updating the entity at the same
time, which results in a concurrency issue: Both tasks fetch the entity and
modify it, and the one that does the put last is the one whose results are
visible.
Doing the update in a transaction should fix this.
-Nick Johnson
On Sun, Jun 13, 2010 at 8:50 AM, Aaron Cheung >>> > wrote:
> Hi guys,
>
> I'm still having trouble debugging this error. Would appreciate any
> thoughts!
>
> Thanks,
> Aaron
>
>
> On Sat, Jun 12, 2010 at 3:55 AM, Aaron wrote:
>
>> Hi!
>>
>> I am currently implementing the deferred library in my code, and I am
>> running into a very very strange problem. The function that I am
>> deferring gets passed an entity key. It does two tasks: 1) Gets the
>> entity, retrieves a reference property, alters it, and then puts it
>> back 2)Sends out an email
>>
>> The email is getting sent out every time the deferred function is
>> called. BUT, the first task only completes 2/3 of the time. I have
>> been replicating this error over and over and have yet to find any
>> pattern in the failures. Here is the deferred function
>>
>> def complete_insert(self, data, answer_key, userkey):
>>a = db.get(answer_key)
>>q = a.question
>>q.answercount += 1
>>q.answerers += [str(userkey)]
>>q.answerers = list(set(q.answerers))
>>q.answered = True
>>q.lastanswer = str(answer_key)
>>q.lastanswerdate = datetime.now()
>>q.put()
>>
>>mail = Mail()
>>followers = db.get(q.indexes.followerlist)
>>emails = []
>>for follower in followers:
>>mail.send(emails,
Re: [google-appengine] Datastore
i also suggest always putting in mind that selects unlike inserts are more
problematic in datastore unlike sql. so youll want to save all the data you
may think you want in advance. remember datastore row model structure can be
altered as you go by adding more entities to the model.
i always save all the data ill need and also referencekey to stuff just for
keeping things in order.
-
Cheers,
public static function AlonCarmel() {
//Contact me
var email = 'a...@aloncarmel.me';
var twitter = '@aloncarmel';
var web = 'http://aloncarmel.me';
var phone = '+972-54-4860380';
}
* If you received an unsolicited email from by mistake that wasn't of your
matter please delete immediately. All E-mail sent from Alon Carmel is
copyrighted to Alon Carmel 2008. Any details revealed in e-mails sent by
Alon Carmel are owned by the Author only. Any attempt to duplicate or
imitate any of the Content is prohibited under copyright law 2008.
2010/6/14 Ian Lewis
> Ben,
>
> With non-relational databases like appengine's datastore you need to
> create data based on the way you will query it rather than the way you
> want to store it.
>
> In your case it looks like you would need to create a model to store
> the data you want to query (essentially an index that you manage). A
> model something like the following might work.
>
> PeopleOwnItemsIndex {
>person (reference to the person owning the items/people)
>owner (reference to the person owning the item)
>item (reference to the item owned)
> }
>
> You'll be querying on the person field so the query will need to go in
> your datastore-indexes.xml or index.yaml
>
> When you change owners or add items you'll need to update the data for
> this model. If you want to consolidate models you could also get only
> your own items by creating an index on person+owner and query it with
> person = me, owner = me. That would be instead of using PersonOwnItem,
> but the difference would be you would have a record for each person
> that can see the item rather than only one for the person owning the
> item.
>
> Ian
>
> On Sun, Jun 13, 2010 at 4:10 AM, Ben Woodhead
> wrote:
> > Hello Everybody,
> >
> > I am very comfortable with SQL and relational databases and I am
> > trying to wrap my head around GAE. Below is an example of how I would
> > tackle my problem in SQL and it would be really great if someone could
> > give me some hints on how to accomplish something similar in GAE. Its
> > basically a hierarchy of ownership.
> >
> > The 2 base tables:
> >
> > table People {
> > int personId;
> > string name;
> > }
> >
> > table items {
> >int itemId;
> >string itemName;
> > }
> >
> > The 2 relational tables:
> > table PeopleOwnPeople {
> >int ownersId;
> >int personId;
> > }
> >
> > table PeopleOwnItems {
> >int personId;
> >int itemId;
> > }
> >
> > So now that we have a basic structure I can start to explain the idea
> > or problem. I want to be able to search for all items owned by me and
> > the people i own:
> >
> > select items.* from Items, PeopleOwnPeople, PeopleOwnItems where
> > (PeopleOwnPeople.ownersId=234 and
> > PeopleOwnPeople.personId=PeopleownItems.personId) or
> > PeopleOwnItems.personId = 234 ;
> >
> > I am looking to use GWT as my frontend to this project so I am likely
> > going to be using Java for the backend but that isn't set in stone
> > yet.
> >
> > Thanks any and all help.
> > Ben
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> > To post to this group, send email to google-appeng...@googlegroups.com.
> > To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com
> .
> > For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
> >
> >
>
>
>
> --
> ===
> 株式会社ビープラウド イアン・ルイス
> 〒150-0021
> 東京都渋谷区恵比寿西2-3-2 NSビル6階
> email: ianmle...@beproud.jp
> TEL:03-6416-9836
> FAX:03-6416-9837
> http://www.beproud.jp/
> ===
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appeng...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] SLA Chain from "Google" to "App Developer" to "End User"
Hi All, This is a roadmap question hoping for a response from a member of the Google App Engine Team. When GAE finally moves out of Beta (is January 2011 too optimistic?), there is a general expectation that developers will be able to run mission-critical web apps with best-of-breed scaling, redundancy and downtime. Please correct me if I'm wrong in this assumption (and if I am wrong then that raises other obvious questions). When that nirvana moment arrives, I expect that Google will have fine- tuned its SLA to reflect its confidence in the robustness of its GAE platform and associated infrastructure. But what about GAE developers whose apps provide "critical" services to end-users - a CRM application would be a good example. In this case, the developer would be expected to provide an SLA to his customers. Obviously the two main technical considerations in framing this SLA will be the quality of the App Software (the developer's responsibility) and the reliability of the platform/infra-structure (Google's responsibility). Looking purely at the platform/infra-structure side of things, a conservative end-user SLA might water down Google uptime guarantees just to give a little wriggle room. But the guarantees are not the real issue here. The difficulty lies in what recourse the end-user has to an SLA violation. If Google fails to meet it's uptime guarantees, e.g. due to a major outage taking a couple of days to stabilise, then what recourse can the developer realistically expect from Goggle if he has 10,000 angry business customers? Of course this question is pertinent to all "hosting" companies. The critical difference here is the level of control which the developer has in a GAE scenario compared to, say, a dedicated server cluster. For example, in a dedicated server scenario, you could have a worst- case disaster-recovery plan in place to get your web app back on-line within say 8 hours. But with GAE, that level of control is completely relinquished to Google. And it is worsened by the fact that your app is mixed up with zillions of other apps in the GAE soup. I'm not expecting miracle answers here. I'm thinking more in terms of how GAE developers can approach end-user SLAs, bearing in mind that a watered-down SLA could be a deal breaker for end-users. I'm wondering also if Google is thinking of offering enhanced GAE services for developers of critical apps, offering an extra level of redundancy using a special "backup" cloud on independent infra-structure. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: SLA Chain from "Google" to "App Developer" to "End User"
Hello, Google announced a "business" version of AppEngine some time ago http://code.google.com/appengine/business/ It's got an SLA which is published in draft at http://code.google.com/appengine/business/sla.html Prices for public apps are still not listed (and I guess it would take quite a long time to get them published). I don't know if regular AppEngine will benefit from a similar SLA, but in the Business FAQ they state developers can "get these features by migrating your app to Google App Engine for Business", so I guess no SLA will be published for regular GAE. I hope terms stated in the SLA draft can help you figure out what you can offer to customers. As a developer I'd better leave SLAs and customer refunds questions to lawyers, since they can be more accurate in evaluating their implications (sad but true) :-(. Regards Lorenzo On Jun 14, 1:23 pm, rvjcallanan wrote: > Hi All, > > This is a roadmap question hoping for a response from a member of the > Google App Engine Team. > > When GAE finally moves out of Beta (is January 2011 too optimistic?), > there is a general expectation that developers will be able to run > mission-critical web apps with best-of-breed scaling, redundancy and > downtime. Please correct me if I'm wrong in this assumption (and if I > am wrong then that raises other obvious questions). > > When that nirvana moment arrives, I expect that Google will have fine- > tuned its SLA to reflect its confidence in the robustness of its GAE > platform and associated infrastructure. > > But what about GAE developers whose apps provide "critical" services > to end-users - a CRM application would be a good example. In this > case, the developer would be expected to provide an SLA to his > customers. Obviously the two main technical considerations in framing > this SLA will be the quality of the App Software (the developer's > responsibility) and the reliability of the platform/infra-structure > (Google's responsibility). > > Looking purely at the platform/infra-structure side of things, a > conservative end-user SLA might water down Google uptime guarantees > just to give a little wriggle room. But the guarantees are not the > real issue here. The difficulty lies in what recourse the end-user has > to an SLA violation. If Google fails to meet it's uptime guarantees, > e.g. due to a major outage taking a couple of days to stabilise, then > what recourse can the developer realistically expect from Goggle if he > has 10,000 angry business customers? > > Of course this question is pertinent to all "hosting" companies. The > critical difference here is the level of control which the developer > has in a GAE scenario compared to, say, a dedicated server cluster. > For example, in a dedicated server scenario, you could have a worst- > case disaster-recovery plan in place to get your web app back on-line > within say 8 hours. But with GAE, that level of control is completely > relinquished to Google. And it is worsened by the fact that your app > is mixed up with zillions of other apps in the GAE soup. > > I'm not expecting miracle answers here. I'm thinking more in terms of > how GAE developers can approach end-user SLAs, bearing in mind that a > watered-down SLA could be a deal breaker for end-users. I'm wondering > also if Google is thinking of offering enhanced GAE services for > developers of critical apps, offering an extra level of redundancy > using a special "backup" cloud on independent infra-structure. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] HTTPS on my own domain
> > Is it feasible to somehow make it appear that this is https on my > domain? Not currently entirely with Google. It but is appently on the roadmap for AppEngine for Business. The only way currently would be to setup a HTTPS capable proxy. Your proxy receives HTTPS requests from the user, and forwards them in the background to https://app-id.appspot.com/. This would maintain encryption security as both sites of the proxy would be encrypted channels. I beleive you can setup a subdomain to point to your own server, eg use secure.domain.com for secure which go via proxy. www.domain.com goes via normal Apps proxy to AppEngine. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Datastore
Hey, Thanks a lot. The project is similar to what you would expect from a social application. I will use sharing photos as an example. Search 1: Display all my photos and my friends photos ordered by date Search 2: Display groups of photos limited by tags from my photos and my friends photos ordered by date Search 3: Display the top rated photos from everybody. ( categories for this too ). Search 4: Scores are applied to all members in the tree. So I need to be able to walk the owner tree to find the original poster. Normally how I approach something is that I figure out the searches I need to do and the build the most flexible SQL datastore to reflect that and hopefully further growth. When I apply this to datastore I hit the no relationships and panic. I am sure theirs a solution and once I get the ball rolling I will be fine. Thanks again, Ben -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: NeedIndexException - in production server
It's not a problem with indexing, it's a problem with your data. If you don't add composite indexes, the system will attempt to serve queries that contain only equality filters using a merge-join strategy, which is less efficient than a simple index lookup (with the benefit of not needing to add the indexes, which use space and write- time CPU). If your data becomes such that the query can not be done efficiently with the merge-join anymore, you'll start to see that exception and you'll need to add a composite index. On Jun 12, 2:00 am, Ravi wrote: > Hi Google Team, > > All these days our application was working fine and we have not > changed any datastore index or codebase of our application. > Everyday we upload new set of data to our application. Today also we > have uploaded new set of data but today after uploading the data we > are getting very strange exception as below: > > com.google.appengine.api.datastore.DatastoreNeedIndexException: The > built-in indices are not efficient enough for this query and your > data. Please add a composite index for this query.. An index is > missing but we are unable to tell you which one due to a bug in the > App Engine SDK. If your query only contains equality filters you most > likely need a composite index on all the properties referenced in > those filters. > > We are worried after seeing the above exception, we dont have any > complex queries, all the queries contain equals to filter (==). > We tested the same set of data and query in our development > environment, it is working fine. > For testing purpose, we uploaded the same set of data to our other > testing appengine applcation, it is working fine there also. > > Only in our production system, we are getting > DatastoreNeedIndexException. > > Could you please confirm is there any problem with the datastore > indexing. > Data stored in our production datastore is more than 0.5 GB, will > thee be any problem in indexing if there is high amount of data. > > Application ID: bigpaisa2.appspot.com > Latest application version: http://061102.latest.bigpaisa2.appspot.com > - In this version DatastoreNeedIndexException is occuring. > > Please help us as soon as possible. > > Thanks, > Ravi -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: NeedIndexException - in production server
Hi Geoffrey Spear, Thanks for your clarification. We suspected the same reason. Last few days back our data was less around 20,000 rows there was no problem. Only when more data is added we are facing this problem. Our table is too large which is around 50 fields/columns and around 50,000 rows. Query filter is also dynamically generated (user selects few fields and submits). Our servlet will query database depending on the user selected fields. In this scenario we dont know which fields will be selected by user and certainly I cant have composite index for 50 combinations. Is there any work around for this problem?? Thanks, Ravi On Jun 14, 5:59 pm, Geoffrey Spear wrote: > It's not a problem with indexing, it's a problem with your data. If > you don't add composite indexes, the system will attempt to serve > queries that contain only equality filters using a merge-join > strategy, which is less efficient than a simple index lookup (with the > benefit of not needing to add the indexes, which use space and write- > time CPU). If your data becomes such that the query can not be done > efficiently with the merge-join anymore, you'll start to see that > exception and you'll need to add a composite index. > > On Jun 12, 2:00 am, Ravi wrote: > > > > > Hi Google Team, > > > All these days our application was working fine and we have not > > changed any datastore index or codebase of our application. > > Everyday we upload new set of data to our application. Today also we > > have uploaded new set of data but today after uploading the data we > > are getting very strange exception as below: > > > com.google.appengine.api.datastore.DatastoreNeedIndexException: The > > built-in indices are not efficient enough for this query and your > > data. Please add a composite index for this query.. An index is > > missing but we are unable to tell you which one due to a bug in the > > App Engine SDK. If your query only contains equality filters you most > > likely need a composite index on all the properties referenced in > > those filters. > > > We are worried after seeing the above exception, we dont have any > > complex queries, all the queries contain equals to filter (==). > > We tested the same set of data and query in our development > > environment, it is working fine. > > For testing purpose, we uploaded the same set of data to our other > > testing appengine applcation, it is working fine there also. > > > Only in our production system, we are getting > > DatastoreNeedIndexException. > > > Could you please confirm is there any problem with the datastore > > indexing. > > Data stored in our production datastore is more than 0.5 GB, will > > thee be any problem in indexing if there is high amount of data. > > > Application ID: bigpaisa2.appspot.com > > Latest application version: http://061102.latest.bigpaisa2.appspot.com > > - In this version DatastoreNeedIndexException is occuring. > > > Please help us as soon as possible. > > > Thanks, > > Ravi -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Awesome Latency
+1 Nice job! On Sun, Jun 13, 2010 at 11:49 PM, nischalshetty wrote: > +1 and want to profusely thank the entire Google App Engine team. The > red errors in my log file related to datastore seem to have all > disappeared! Hip Hip Hurray :D > > -Nischal > > On Jun 14, 4:27 am, nickmilon wrote: >> Great job App team ! >> Sorry for your week-end ;-( >> data store latency is dramatically reduced >> !http://gaengine.blogspot.com/2010/06/datastore-latency-ii.html >> >> Happy coding :-) > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: 403 error - automated queries
The "we are sorry" dialog is happening all the time with purely human interaction now, unfortunately. Please read the post http://technorati.com/blogging/article/dear-google-i-am-human; it includes a bizarre explanation from a Google developer. On Jun 1, 3:52 pm, "Ikai L (Google)" wrote: > Are the gadgets making requests from App Engine? Or are they calling > makeRequest to App Engine which is in turn using the GData API? > > > > > > On Sun, May 23, 2010 at 5:56 PM, mike wrote: > > Hi, I have 3 gadgets that retrieve data from Google base. the gadgets > > are hosted on App Enginer. About a week ago, the gadgets started > > getting 403 errors and the gadgets are > > basically broken. I have not had a problem over the last year and > > longer. The error reads: > > > "We're sorry..but your computer or network may be sending automated > > queries. To protect our users, we can't process your request right > > now. > >http://www.google.com/support/bin/answer.py?answer=86640"; > > > I evaluated the traffic and can not identify a source of automated > > requests. There is also no evidence > > of a virus. > > > How can I get the account reset? At this point I have lost a lot of > > users. Here is a link to the sites having the problem (which feed > > gadgets). For some reason, sometimes the first query shows results but > > after that the 403 error shows up. > > >http://appdiner.appspot.com > >http://basestuff.appspot.com > >http://searchhomesales.appspot.com/ > > > Thanks, > > Mike J. > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to google-appeng...@googlegroups.com. > > To unsubscribe from this group, send email to > > google-appengine+unsubscr...@googlegroups.com > e...@googlegroups.com> > > . > > For more options, visit this group at > >http://groups.google.com/group/google-appengine?hl=en. > > -- > Ikai Lan > Developer Programs Engineer, Google App Engine > Blog:http://googleappengine.blogspot.com > Twitter:http://twitter.com/app_engine > Reddit:http://www.reddit.com/r/appengine -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] NeedIndexException - in production server
This means that our query planner could not run the given query efficiently (using zigzag merge join) with the indexes available. Currently this can happen if you pull a lot of results or if the data being queried is especially inefficient for the given query. Given that this error is data dependent it can be hard to reproduce or know when it will happen. We are working to remove this restriction, but in the mean time you should add a composite index that contains the properties you are querying on (or if you're clever you can use a subset of those properties, I would avoid more than one multivalued property). For more information about why this happens you can watch some IO talks: http://www.youtube.com/watch?v=tx5gdoNpcZM http://www.youtube.com/watch?v=ofhEyDBpngM On Fri, Jun 11, 2010 at 11:00 PM, Ravi wrote: > Hi Google Team, > > All these days our application was working fine and we have not > changed any datastore index or codebase of our application. > Everyday we upload new set of data to our application. Today also we > have uploaded new set of data but today after uploading the data we > are getting very strange exception as below: > > com.google.appengine.api.datastore.DatastoreNeedIndexException: The > built-in indices are not efficient enough for this query and your > data. Please add a composite index for this query.. An index is > missing but we are unable to tell you which one due to a bug in the > App Engine SDK. If your query only contains equality filters you most > likely need a composite index on all the properties referenced in > those filters. > > We are worried after seeing the above exception, we dont have any > complex queries, all the queries contain equals to filter (==). > We tested the same set of data and query in our development > environment, it is working fine. > For testing purpose, we uploaded the same set of data to our other > testing appengine applcation, it is working fine there also. > > Only in our production system, we are getting > DatastoreNeedIndexException. > > Could you please confirm is there any problem with the datastore > indexing. > Data stored in our production datastore is more than 0.5 GB, will > thee be any problem in indexing if there is high amount of data. > > Application ID: bigpaisa2.appspot.com > Latest application version: http://061102.latest.bigpaisa2.appspot.com > - In this version DatastoreNeedIndexException is occuring. > > Please help us as soon as possible. > > Thanks, > Ravi > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Problems Enabling Billing
I wanted to enable billing for my first app so I could use the Blob Store API, but every time I configure it and go through Google Checkout, I get 2 emails. The first is a thank you letter with my order number. The second is a message telling me my order has been canceled. The reason cited is: Comments from Google Services:"This inactive subscription has been canceled" Has anyone run into this issue before? Am I doing something wrong? Thanks -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] help!! 'Invalid / used nonce' problem about twitter api on google app engine
I create a jsp project about twitter api.When I run it on my own local
jsp server,tomcat,it works correctly.So I upload it to google app
engine.On the app engine,If I visit the unnecessary authorized
api,such as public_timeline,it works well.But when it need
authorize,such as home_timeline,it always response the 'Invalid / used
nonce' error.Why?
here is the location detail:
my location:China
google app engine's location:may be US,absolutely not in China
When program get the access_token,it will send request to
home_timeline.
There are two http head from twitter api via two different web server:
google app engine:
OAuth oauth_consumer_key="9WXY7kD9XiznbN4zRMyNuA",
oauth_nonce="92fa4a6aa648672cf26dbb05a9b4a744",
oauth_signature="iq9N97qB1x9Ae251cxv%2Bvvmyjn0%3D",
oauth_signature_method="HMAC-SHA1", oauth_timestamp="1276437638",
oauth_token="154805754-WwhuUbBdwmfOcajq0jxfDg4Ers8St4N6lHe3FmrU",
oauth_version="1.0"
my local tomcat:
OAuth oauth_consumer_key="9WXY7kD9XiznbN4zRMyNuA",
oauth_nonce="4e3de3e506b8cf961d3d02d2aca1c8ed",
oauth_signature="860zZiqdK9DuXOvUDBLZMMhgm2M%3D",
oauth_signature_method="HMAC-SHA1", oauth_timestamp="1276437655",
oauth_token="154805754-WwhuUbBdwmfOcajq0jxfDg4Ers8St4N6lHe3FmrU",
oauth_version="1.0"
part of code:
tk = sb.getAccessToken(tk, pin);
req = new Request(Request.Verb.GET,"http://api.twitter.com/statuses/
home_timeline.xml");
sb.signRequest(req, tk);
resp = req.send();
out.println(resp.getBody());
res=req.getHeaders().get("Authorization");
System.out.println(res);
ps:
pin is the user authorized code,
tk is the access_token,
signRequest means add the tk to request stream,
the result will be printed to the web browser,
res is the http header which like I mentioned above.
Need any more infomation?
Thanks for tolerating my ravings
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Datastore
http://www.youtube.com/watch?v=AgaL6NGpkB8 is a good resource and I believe
explicitly covers this scenario.
On Sun, Jun 13, 2010 at 10:41 PM, Ian Lewis wrote:
> Ben,
>
> With non-relational databases like appengine's datastore you need to
> create data based on the way you will query it rather than the way you
> want to store it.
>
> In your case it looks like you would need to create a model to store
> the data you want to query (essentially an index that you manage). A
> model something like the following might work.
>
> PeopleOwnItemsIndex {
>person (reference to the person owning the items/people)
>owner (reference to the person owning the item)
>item (reference to the item owned)
> }
>
> You'll be querying on the person field so the query will need to go in
> your datastore-indexes.xml or index.yaml
>
> When you change owners or add items you'll need to update the data for
> this model. If you want to consolidate models you could also get only
> your own items by creating an index on person+owner and query it with
> person = me, owner = me. That would be instead of using PersonOwnItem,
> but the difference would be you would have a record for each person
> that can see the item rather than only one for the person owning the
> item.
>
> Ian
>
> On Sun, Jun 13, 2010 at 4:10 AM, Ben Woodhead
> wrote:
> > Hello Everybody,
> >
> > I am very comfortable with SQL and relational databases and I am
> > trying to wrap my head around GAE. Below is an example of how I would
> > tackle my problem in SQL and it would be really great if someone could
> > give me some hints on how to accomplish something similar in GAE. Its
> > basically a hierarchy of ownership.
> >
> > The 2 base tables:
> >
> > table People {
> > int personId;
> > string name;
> > }
> >
> > table items {
> >int itemId;
> >string itemName;
> > }
> >
> > The 2 relational tables:
> > table PeopleOwnPeople {
> >int ownersId;
> >int personId;
> > }
> >
> > table PeopleOwnItems {
> >int personId;
> >int itemId;
> > }
> >
> > So now that we have a basic structure I can start to explain the idea
> > or problem. I want to be able to search for all items owned by me and
> > the people i own:
> >
> > select items.* from Items, PeopleOwnPeople, PeopleOwnItems where
> > (PeopleOwnPeople.ownersId=234 and
> > PeopleOwnPeople.personId=PeopleownItems.personId) or
> > PeopleOwnItems.personId = 234 ;
> >
> > I am looking to use GWT as my frontend to this project so I am likely
> > going to be using Java for the backend but that isn't set in stone
> > yet.
> >
> > Thanks any and all help.
> > Ben
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> > To post to this group, send email to google-appeng...@googlegroups.com.
> > To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com
> .
> > For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
> >
> >
>
>
>
> --
> ===
> 株式会社ビープラウド イアン・ルイス
> 〒150-0021
> 東京都渋谷区恵比寿西2-3-2 NSビル6階
> email: ianmle...@beproud.jp
> TEL:03-6416-9836
> FAX:03-6416-9837
> http://www.beproud.jp/
> ===
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appeng...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Billing settings not changing
I have been trying to change the billing settings of an application, everything works exactly right, except that the settings don't change. I have tried both increasing the budget and just readjusting it. Same thing happens, tells me it is changing, after awhile that message goes away, but it is still the same. I have tried this multiple times, waiting between 20min and 3hours after each attempt, no change. Trying this on a different app doesn't work either. It's not a matter of my payment being rejected either, since I get the emails providing a receipt and such, and making changes that don't increase the weekly cost (and don't require re-google-checkouting) don't work either. The last event on the billing history log was on the 12th, and that was a usage report. Is anyone else having this problem? (Can you adjust your quotas?) -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Awesome Latency
Thanks for the feedback, guys. We're not quite out of the woods yet. We'll be monitoring this list as well as our instrumentation for latency issues. There's still a good amount of work we need to do. We'll send notifications if we believe the work will impact production availability. Be sure that you're subscribed to the downtime-notify list if you want to get these updates: https://groups.google.com/group/google-appengine-downtime-notify On Mon, Jun 14, 2010 at 9:11 AM, Rafael Sierra wrote: > +1 Nice job! > > On Sun, Jun 13, 2010 at 11:49 PM, nischalshetty > wrote: > > +1 and want to profusely thank the entire Google App Engine team. The > > red errors in my log file related to datastore seem to have all > > disappeared! Hip Hip Hurray :D > > > > -Nischal > > > > On Jun 14, 4:27 am, nickmilon wrote: > >> Great job App team ! > >> Sorry for your week-end ;-( > >> data store latency is dramatically reduced ! > http://gaengine.blogspot.com/2010/06/datastore-latency-ii.html > >> > >> Happy coding :-) > > > > -- > > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > > To post to this group, send email to google-appeng...@googlegroups.com. > > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com > . > > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > > > > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- Ikai Lan Developer Programs Engineer, Google App Engine Blog: http://googleappengine.blogspot.com Twitter: http://twitter.com/app_engine Reddit: http://www.reddit.com/r/appengine -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: SLA Chain from "Google" to "App Developer" to "End User"
Thanks Lorenzo, I have looked at both of your links. I realise that nowadays SLAs are more smokescreen than meaningful. For example, a Google service charge refund is of little consolation if a developer has 30,000 angry customers. When it comes to this kind of thing, prevention is better than cure. And if prevention is not always possible then some level of control is desirable in the recovery process. That is why "premium developer support", "guaranteed turnaround times" and "access to trained support engineers" are the noises I like to hear from the "GAE for Business" people. These are the only things that really matter at the end of the day. But I would leave the following proviso: For now, these encouraging noises are only words. In practice, GAE service problems are not going to be spread out nicely over time and space as might be the case with, say, a hardware manufacturer. A serious GAE outage will probably affect zillions of app deployments *simultaneously*. And there are only so many expert support staff which Google can have on standby. So the above noises need to be backed up by serious measures in the future, one of which might involve the special redundant backup cloud I suggested for customers willing to pay for added control and peace- of-mind. At a flick of a switch, or should I say: a click of a mouse, I would be able to activate a standby deployment until Google sorts out a major outage. But this is not trivial to implement - there are all sorts of issues re: DNS, data consistency, etc. Anyway, I'm sure Google is already very aware of where this is all heading. But it is easy to forget about these things when a major project like this is picking up pace and everyone is excited and energised. And tacking on solutions at the end can be difficult as many software developers have found to their cost with 24x7x365 systems. Things like backup and recovery are the "boring" bits which those poor old greying enterprise vendors (who employ people like me) do very well. Perhaps the vibrant and youthful culture which has brought so much success to Google could do with a few grand-daddies about the place. And no, this is not a plug for anti-ageism :-) We live in interesting times. On Jun 14, 1:21 pm, "l.denardo" wrote: > Hello, > Google announced a "business" version of AppEngine some time ago > > http://code.google.com/appengine/business/ > > It's got an SLA which is published in draft > athttp://code.google.com/appengine/business/sla.html > > Prices for public apps are still not listed (and I guess it would take > quite a long time to get them published). I don't know if regular > AppEngine will benefit from a similar SLA, but in the Business FAQ > they state developers can "get these features by migrating your app to > Google App Engine for Business", so I guess no SLA will be published > for regular GAE. > > I hope terms stated in the SLA draft can help you figure out what you > can offer to customers. As a developer I'd better leave SLAs and > customer refunds questions to lawyers, since they can be more accurate > in evaluating their implications (sad but true) :-(. > > Regards > Lorenzo > > On Jun 14, 1:23 pm, rvjcallanan wrote: > > > > > Hi All, > > > This is a roadmap question hoping for a response from a member of the > > Google App Engine Team. > > > When GAE finally moves out of Beta (is January 2011 too optimistic?), > > there is a general expectation that developers will be able to run > > mission-critical web apps with best-of-breed scaling, redundancy and > > downtime. Please correct me if I'm wrong in this assumption (and if I > > am wrong then that raises other obvious questions). > > > When that nirvana moment arrives, I expect that Google will have fine- > > tuned its SLA to reflect its confidence in the robustness of its GAE > > platform and associated infrastructure. > > > But what about GAE developers whose apps provide "critical" services > > to end-users - a CRM application would be a good example. In this > > case, the developer would be expected to provide an SLA to his > > customers. Obviously the two main technical considerations in framing > > this SLA will be the quality of the App Software (the developer's > > responsibility) and the reliability of the platform/infra-structure > > (Google's responsibility). > > > Looking purely at the platform/infra-structure side of things, a > > conservative end-user SLA might water down Google uptime guarantees > > just to give a little wriggle room. But the guarantees are not the > > real issue here. The difficulty lies in what recourse the end-user has > > to an SLA violation. If Google fails to meet it's uptime guarantees, > > e.g. due to a major outage taking a couple of days to stabilise, then > > what recourse can the developer realistically expect from Goggle if he > > has 10,000 angry business customers? > > > Of course this question is pertinent to all "hosting" companies. The > > critical difference here is the
Re: [google-appengine] Problems Enabling Billing
I got the same issue before few minutes when I tried to change my billing settings. It seems that it's a common issue. Regards, Miroslav On 06/14/2010 05:25 PM, cwaldbieser wrote: I wanted to enable billing for my first app so I could use the Blob Store API, but every time I configure it and go through Google Checkout, I get 2 emails. The first is a thank you letter with my order number. The second is a message telling me my order has been canceled. The reason cited is: Comments from Google Services:"This inactive subscription has been canceled" Has anyone run into this issue before? Am I doing something wrong? Thanks -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Channel API shipping delay?
It's shipping soon to trusted testers, not the general public. Stay tuned. On Fri, Jun 11, 2010 at 7:33 PM, GoSharp Lite wrote: > Moishe Lettvin told us he is shipping Channel API very soon during > Google I/O last month. Could PM in charge of this project tell us > when will be the planned shipping date? > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- Ikai Lan Developer Programs Engineer, Google App Engine Blog: http://googleappengine.blogspot.com Twitter: http://twitter.com/app_engine Reddit: http://www.reddit.com/r/appengine -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Issues raising billing quotas
Okay, almost a month later, but I'm having this same issue today. I've increased my quotas 4 times to no avail. It seems that I've been hasty by not waiting the three hours, but who likes to wait three hours while their paid application is down? The first change I made about 2-1/2 hours ago. I hope this means that the changes will come through in the next half hour. On May 26, 3:08 pm, Chris Killpack wrote: > The updated quota was applied about 3 hours after I made the changes, and > received the Checkout email receipt. After a change, the messaging about > preventing further changes for 30 minutes, lead me to believe that is the > upper limit on how long before the changes took affect. > > Chris > > > > On Tue, May 25, 2010 at 6:32 PM, Chris Killpack wrote: > > I received Google Checkout invoices for the correct amount, but those > > changes are not reflected in the dashboard - it still shows the unchanged > > daily maximum and storage quotas. It's been about two hours since I tried to > > increase my quota. > > > Chris > > > On Tue, May 25, 2010 at 5:08 PM, Chris Killpack wrote: > > >> I have exhausted my storage quota, and twice today I have tried and failed > >> to increase my storage quota. I go to the app's Billings Settings, increase > >> the Stored Data quota by $0.02 (which increases the maximum daily limit). > >> Then I go through the Google Checkout process, received the emails, seen > >> the > >> dashboard messages that it will take 30 minutes to process and then... > >> nothing. Still at my previous levels, still at 100% usage. > > >> Anyone else seen this problem? > > >> Chris -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Mail API saves mail in users folders even if mail is sent by GAE
Yes, this is intended behavior. The sender is BCC'd on any email sent on their behalf. On Fri, Jun 11, 2010 at 5:47 AM, l.denardo wrote: > Hello, > I have an application deployed in GAE (Java) using Mail API. > The app is restricted in access to users from my Google Apps domain. > To receive feedback from users, I send from client (GWT + gwt > dispatch) an action containing a subject and message. > On server side I prepare an email message, set the logged in user as > the sender and send the message to a mail group. > > The point is that users can find that email in their "sent mail" > folder in Google Mail, even if they didn't actively send the message > by themselves. > > I would like to know if this is an attended behaviour, and if it can > be turned off in some way: I think it's a bit confusing for users to > find mails they did not send in their own folders. > > Java mail documentation in > http://code.google.com/appengine/docs/java/mail/ > did not help. > > Thanks for clarification > Lorenzo > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- Ikai Lan Developer Programs Engineer, Google App Engine Blog: http://googleappengine.blogspot.com Twitter: http://twitter.com/app_engine Reddit: http://www.reddit.com/r/appengine -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Cannot Raise Billing Quota
My application has been over quota for nearly two hours now. I've tried four times to raise the quota. I began trying about an hour and a half before it went over quota, but the billing system will not reflect any of the changes that I have made. I have tried emailing google services via the google payments system to no avail. I've even tried to call that terrible automated system that google calls a help line. This is just one in a long line of errors and problems that app engine has caused me, and I'm finally beginning to feel like I've made a bad choice in choosing this service. I made it through the atrocious datastore timeouts, the long bouts of server outages, the ridiculous hoops to map my url through google apps, weird datastore issues (http://stackoverflow.com/questions/3007478/appengine-datastore-record- cannot-be-deleted-or-changed) created by your May 25th outage, and months of coding. I'm finally reaching the breaking point and all I really require is a small peep from someone who works at google to state "oh yes we are having an issue and we will help you fix this problem." But every email I send goes unanswered, everywhere I look for some technical support for google services or google app engine just refers me to documentation which assumes that google's software and services actually work. Now I'm left here without the ability to work on my app because it's over quota, and plenty of free time and frustration to write this verbose complaint which will probably fall on deaf ears. Anyways, someone from google please help me fix my app, and fix your system so that it doesn't take hours to update billing settings on an application. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] How many exploding indexes is too much in general?
You're limited to 200 indexes for a billing enabled app and 100 for a non-billing enabled application. The number of indexes does not introduce a scalability cap, it just makes every write much, much more expensive. The question here is whether or not your budget can support this. On Sat, Jun 12, 2010 at 11:23 AM, Harshal wrote: > Hi, > > Until Google releases the version which they showcased in I/O where we > won't have to deal with exploding indexes, I guess our choices are very > limited and we would have to live with it. > > I would like to go ahead and ask, what is acceptable number indexes people > have in general? I have an entity which has something like 42 indexes > (because I have to filter in various ways on many properties many times). Is > it scalable ? ( I am seeing high CPU spikes but I am fine with that, well at > least as of now). > > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- Ikai Lan Developer Programs Engineer, Google App Engine Blog: http://googleappengine.blogspot.com Twitter: http://twitter.com/app_engine Reddit: http://www.reddit.com/r/appengine -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Index building seems to be stuck
These look to be serving to me. It's possible index building was delayed (even more so) due to the maintenance on Saturday. I'll check to see if background jobs are still backed up. On Sun, Jun 13, 2010 at 10:08 AM, Amir wrote: > My index page has been saying "BUILDING" for almost 24 hours now... I > think it's stuck. Can someone please take a look? appid is: mambo- > pics > Thanks, > Amir > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- Ikai Lan Developer Programs Engineer, Google App Engine Blog: http://googleappengine.blogspot.com Twitter: http://twitter.com/app_engine Reddit: http://www.reddit.com/r/appengine -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] cannot delete old version of my application
Are you uploading versions or new applications? You should have a limit of 100 versions, not 10. On Sun, Jun 13, 2010 at 4:22 AM, Ohad R wrote: > hi there > i upload several versions of my app to GAE. after i get to 10 > versions, i was notified that i must delete old versions. but some > versions cannot be deleted from some unknown reason. when i try to > delete it , i get an error: > Server Error > A server error has occurred. > Return to Applications screen » > > anyone has an idea? > (my app name is blpatch...@appspot.com) > > thanks a lot, > Ohad > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- Ikai Lan Developer Programs Engineer, Google App Engine Blog: http://googleappengine.blogspot.com Twitter: http://twitter.com/app_engine Reddit: http://www.reddit.com/r/appengine -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] index building takes so long
Can you provide us with your application ID? On Sat, Jun 12, 2010 at 10:28 PM, james_027 wrote: > Hi, > > On this page > http://code.google.com/appengine/kb/general.html#stuck_indexes > , it says that I can ask help in the group, if my index has been in > building status for so long. > > Thanks, > James > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- Ikai Lan Developer Programs Engineer, Google App Engine Blog: http://googleappengine.blogspot.com Twitter: http://twitter.com/app_engine Reddit: http://www.reddit.com/r/appengine -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] help!! 'Invalid / used nonce' problem about twitter api on google app engine
Are you sending the same nonce to Twitter? A nonce is a number that must be
unique per request (nonce = number once).
On Sun, Jun 13, 2010 at 11:45 PM, ggcc11 wrote:
> I create a jsp project about twitter api.When I run it on my own local
> jsp server,tomcat,it works correctly.So I upload it to google app
> engine.On the app engine,If I visit the unnecessary authorized
> api,such as public_timeline,it works well.But when it need
> authorize,such as home_timeline,it always response the 'Invalid / used
> nonce' error.Why?
>
> here is the location detail:
> my location:China
> google app engine's location:may be US,absolutely not in China
>
> When program get the access_token,it will send request to
> home_timeline.
> There are two http head from twitter api via two different web server:
>
> google app engine:
> OAuth oauth_consumer_key="9WXY7kD9XiznbN4zRMyNuA",
> oauth_nonce="92fa4a6aa648672cf26dbb05a9b4a744",
> oauth_signature="iq9N97qB1x9Ae251cxv%2Bvvmyjn0%3D",
> oauth_signature_method="HMAC-SHA1", oauth_timestamp="1276437638",
> oauth_token="154805754-WwhuUbBdwmfOcajq0jxfDg4Ers8St4N6lHe3FmrU",
> oauth_version="1.0"
>
> my local tomcat:
> OAuth oauth_consumer_key="9WXY7kD9XiznbN4zRMyNuA",
> oauth_nonce="4e3de3e506b8cf961d3d02d2aca1c8ed",
> oauth_signature="860zZiqdK9DuXOvUDBLZMMhgm2M%3D",
> oauth_signature_method="HMAC-SHA1", oauth_timestamp="1276437655",
> oauth_token="154805754-WwhuUbBdwmfOcajq0jxfDg4Ers8St4N6lHe3FmrU",
> oauth_version="1.0"
>
> part of code:
>
> tk = sb.getAccessToken(tk, pin);
> req = new Request(Request.Verb.GET,"http://api.twitter.com/statuses/
> home_timeline.xml");
> sb.signRequest(req, tk);
> resp = req.send();
> out.println(resp.getBody());
> res=req.getHeaders().get("Authorization");
> System.out.println(res);
>
> ps:
> pin is the user authorized code,
> tk is the access_token,
> signRequest means add the tk to request stream,
> the result will be printed to the web browser,
> res is the http header which like I mentioned above.
>
> Need any more infomation?
>
> Thanks for tolerating my ravings
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appeng...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>
--
Ikai Lan
Developer Programs Engineer, Google App Engine
Blog: http://googleappengine.blogspot.com
Twitter: http://twitter.com/app_engine
Reddit: http://www.reddit.com/r/appengine
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: 403 error - automated queries
I think that post is related to search, not GData. On Sun, Jun 13, 2010 at 10:08 PM, Braingain wrote: > The "we are sorry" dialog is happening all the time with purely human > interaction now, unfortunately. Please read the post > http://technorati.com/blogging/article/dear-google-i-am-human; it > includes a bizarre explanation from a Google developer. > > On Jun 1, 3:52 pm, "Ikai L (Google)" wrote: > > Are the gadgets making requests from App Engine? Or are they calling > > makeRequest to App Engine which is in turn using the GData API? > > > > > > > > > > > > On Sun, May 23, 2010 at 5:56 PM, mike wrote: > > > Hi, I have 3 gadgets that retrieve data from Google base. the gadgets > > > are hosted on App Enginer. About a week ago, the gadgets started > > > getting 403 errors and the gadgets are > > > basically broken. I have not had a problem over the last year and > > > longer. The error reads: > > > > > "We're sorry..but your computer or network may be sending automated > > > queries. To protect our users, we can't process your request right > > > now. > > >http://www.google.com/support/bin/answer.py?answer=86640"; > > > > > I evaluated the traffic and can not identify a source of automated > > > requests. There is also no evidence > > > of a virus. > > > > > How can I get the account reset? At this point I have lost a lot of > > > users. Here is a link to the sites having the problem (which feed > > > gadgets). For some reason, sometimes the first query shows results but > > > after that the 403 error shows up. > > > > >http://appdiner.appspot.com > > >http://basestuff.appspot.com > > >http://searchhomesales.appspot.com/ > > > > > Thanks, > > > Mike J. > > > > > -- > > > You received this message because you are subscribed to the Google > Groups > > > "Google App Engine" group. > > > To post to this group, send email to google-appengine@googlegroups.com > . > > > To unsubscribe from this group, send email to > > > google-appengine+unsubscr...@googlegroups.com e...@googlegroups.com> > > > . > > > For more options, visit this group at > > >http://groups.google.com/group/google-appengine?hl=en. > > > > -- > > Ikai Lan > > Developer Programs Engineer, Google App Engine > > Blog:http://googleappengine.blogspot.com > > Twitter:http://twitter.com/app_engine > > Reddit:http://www.reddit.com/r/appengine > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- Ikai Lan Developer Programs Engineer, Google App Engine Blog: http://googleappengine.blogspot.com Twitter: http://twitter.com/app_engine Reddit: http://www.reddit.com/r/appengine -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Cannot Raise Billing Quota
Have you filled out the billing issues form? http://code.google.com/support/bin/request.py?contact_type=AppEngineBillingSupport We're working through reported issues. On Mon, Jun 14, 2010 at 11:38 AM, Daniel wrote: > My application has been over quota for nearly two hours now. I've > tried four times to raise the quota. I began trying about an hour and > a half before it went over quota, but the billing system will not > reflect any of the changes that I have made. I have tried emailing > google services via the google payments system to no avail. I've even > tried to call that terrible automated system that google calls a help > line. > > This is just one in a long line of errors and problems that app engine > has caused me, and I'm finally beginning to feel like I've made a bad > choice in choosing this service. I made it through the atrocious > datastore timeouts, the long bouts of server outages, the ridiculous > hoops to map my url through google apps, weird datastore issues > (http://stackoverflow.com/questions/3007478/appengine-datastore-record- > cannot-be-deleted-or-changed) created by your May 25th outage, and > months of coding. I'm finally reaching the breaking point and all I > really require is a small peep from someone who works at google to > state "oh yes we are having an issue and we will help you fix this > problem." But every email I send goes unanswered, everywhere I look > for some technical support for google services or google app engine > just refers me to documentation which assumes that google's software > and services actually work. > > Now I'm left here without the ability to work on my app because it's > over quota, and plenty of free time and frustration to write this > verbose complaint which will probably fall on deaf ears. > > Anyways, someone from google please help me fix my app, and fix your > system so that it doesn't take hours to update billing settings on an > application. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- Ikai Lan Developer Programs Engineer, Google App Engine Blog: http://googleappengine.blogspot.com Twitter: http://twitter.com/app_engine Reddit: http://www.reddit.com/r/appengine -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Cannot Raise Billing Quota
No I hadn't tried that. Surprisingly I couldn't find a link for that on any of the billing settings pages. I am just a bit more frustrated by the fact that my app has been down for 3 hours and I'm just now able to file a support ticket. On Jun 14, 3:23 pm, "Ikai L (Google)" wrote: > Have you filled out the billing issues form? > > http://code.google.com/support/bin/request.py?contact_type=AppEngineB... > > We're working through reported issues. > > > > > > On Mon, Jun 14, 2010 at 11:38 AM, Daniel wrote: > > My application has been over quota for nearly two hours now. I've > > tried four times to raise the quota. I began trying about an hour and > > a half before it went over quota, but the billing system will not > > reflect any of the changes that I have made. I have tried emailing > > google services via the google payments system to no avail. I've even > > tried to call that terrible automated system that google calls a help > > line. > > > This is just one in a long line of errors and problems that app engine > > has caused me, and I'm finally beginning to feel like I've made a bad > > choice in choosing this service. I made it through the atrocious > > datastore timeouts, the long bouts of server outages, the ridiculous > > hoops to map my url through google apps, weird datastore issues > > (http://stackoverflow.com/questions/3007478/appengine-datastore-record- > > cannot-be-deleted-or-changed) created by your May 25th outage, and > > months of coding. I'm finally reaching the breaking point and all I > > really require is a small peep from someone who works at google to > > state "oh yes we are having an issue and we will help you fix this > > problem." But every email I send goes unanswered, everywhere I look > > for some technical support for google services or google app engine > > just refers me to documentation which assumes that google's software > > and services actually work. > > > Now I'm left here without the ability to work on my app because it's > > over quota, and plenty of free time and frustration to write this > > verbose complaint which will probably fall on deaf ears. > > > Anyways, someone from google please help me fix my app, and fix your > > system so that it doesn't take hours to update billing settings on an > > application. > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to google-appeng...@googlegroups.com. > > To unsubscribe from this group, send email to > > google-appengine+unsubscr...@googlegroups.com > e...@googlegroups.com> > > . > > For more options, visit this group at > >http://groups.google.com/group/google-appengine?hl=en. > > -- > Ikai Lan > Developer Programs Engineer, Google App Engine > Blog:http://googleappengine.blogspot.com > Twitter:http://twitter.com/app_engine > Reddit:http://www.reddit.com/r/appengine -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: How many exploding indexes is too much in general?
Actually, this is an interesting question. Is it generally better to sort using the datastore, or sort using python code. For example, let's say you have a list of people, and you need to get various qualities about the people (let's say the list is a couple hundred people or less). One approach can be to use multiple indexes: Order people by name; and age > 50. Order reverse name and height > 5 feet. A different approach is to memcache the entire list, and filter it using list comprehensions. Besides the usual "profile your code" response - is there any rule-of- thumbs to use to select an approach? johnP On Jun 14, 11:48 am, "Ikai L (Google)" wrote: > You're limited to 200 indexes for a billing enabled app and 100 for a > non-billing enabled application. > > The number of indexes does not introduce a scalability cap, it just makes > every write much, much more expensive. The question here is whether or not > your budget can support this. > > > > > > On Sat, Jun 12, 2010 at 11:23 AM, Harshal wrote: > > Hi, > > > Until Google releases the version which they showcased in I/O where we > > won't have to deal with exploding indexes, I guess our choices are very > > limited and we would have to live with it. > > > I would like to go ahead and ask, what is acceptable number indexes people > > have in general? I have an entity which has something like 42 indexes > > (because I have to filter in various ways on many properties many times). Is > > it scalable ? ( I am seeing high CPU spikes but I am fine with that, well at > > least as of now). > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to google-appeng...@googlegroups.com. > > To unsubscribe from this group, send email to > > google-appengine+unsubscr...@googlegroups.com > e...@googlegroups.com> > > . > > For more options, visit this group at > >http://groups.google.com/group/google-appengine?hl=en. > > -- > Ikai Lan > Developer Programs Engineer, Google App Engine > Blog:http://googleappengine.blogspot.com > Twitter:http://twitter.com/app_engine > Reddit:http://www.reddit.com/r/appengine -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Listing entities with TextProperty efficiently
Hey, If I have a model with a TextProperty and I want to list it, that means a query such as Entry.all().fetch(10) could potentially fetch several megabytes of data (not in my case, but just a worst case scenario). In a list I would only show some of the smaller properties, such as a title and whatnot. Would you say I should clone every Entry into EntryShort which would have only the properties necessary for the list and a reference to the original Entry-object? Or will it do fine to just fetch this potentially huge amount of data every now and then and memcache the list (excluding the large properties)? /Andreas Blixt -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] How many exploding indexes is too much in general?
Thanks Ika for that reply, appreciate it. Hope you guys are in good shape to release that magical 'no exploding indexes' solution that was showcased in I/O. Harshal On Tue, Jun 15, 2010 at 12:18 AM, Ikai L (Google) wrote: > You're limited to 200 indexes for a billing enabled app and 100 for a > non-billing enabled application. > > The number of indexes does not introduce a scalability cap, it just makes > every write much, much more expensive. The question here is whether or not > your budget can support this. > > On Sat, Jun 12, 2010 at 11:23 AM, Harshal wrote: > >> Hi, >> >> Until Google releases the version which they showcased in I/O where we >> won't have to deal with exploding indexes, I guess our choices are very >> limited and we would have to live with it. >> >> I would like to go ahead and ask, what is acceptable number indexes people >> have in general? I have an entity which has something like 42 indexes >> (because I have to filter in various ways on many properties many times). Is >> it scalable ? ( I am seeing high CPU spikes but I am fine with that, well at >> least as of now). >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To post to this group, send email to google-appeng...@googlegroups.com. >> To unsubscribe from this group, send email to >> google-appengine+unsubscr...@googlegroups.com >> . >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> > > > > -- > Ikai Lan > Developer Programs Engineer, Google App Engine > Blog: http://googleappengine.blogspot.com > Twitter: http://twitter.com/app_engine > Reddit: http://www.reddit.com/r/appengine > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Billing settings not changing
I am having this issue and have posted about it as well. I was told to file a complaint here: http://code.google.com/support/bin/request.py?contact_type=AppEngineBillingSupport Good luck, my app has been down for hours. On Jun 14, 6:04 am, mcilrain wrote: > I have been trying to change the billing settings of an application, > everything works exactly right, except that the settings don't change. > > I have tried both increasing the budget and just readjusting it. Same > thing happens, tells me it is changing, after awhile that message goes > away, but it is still the same. I have tried this multiple times, > waiting between 20min and 3hours after each attempt, no change. > > Trying this on a different app doesn't work either. > > It's not a matter of my payment being rejected either, since I get the > emails providing a receipt and such, and making changes that don't > increase the weekly cost (and don't require re-google-checkouting) > don't work either. > > The last event on the billing history log was on the 12th, and that > was a usage report. > > Is anyone else having this problem? (Can you adjust your quotas?) -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Problems Enabling Billing
+1. I've reported the problem to google through a form that a google apps rep sent me. Please get on this! Thanks. On Jun 14, 2010, at 1:07 PM, Miroslav Genov wrote: > I got the same issue before few minutes when I tried to change my billing > settings. It seems that it's a common issue. > > Regards, > Miroslav > > On 06/14/2010 05:25 PM, cwaldbieser wrote: >> I wanted to enable billing for my first app so I could use the Blob >> Store API, but every time I configure it and go through Google >> Checkout, I get 2 emails. The first is a thank you letter with my >> order number. The second is a message telling me my order has been >> canceled. The reason cited is: >> >> Comments from Google Services:"This inactive subscription has been >> canceled" >> >> Has anyone run into this issue before? Am I doing something wrong? >> >> Thanks >> >> > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Datastore
you may find this thread useful:
http://groups.google.com/group/google-appengine-java/browse_thread/thread/4298b7f54f1b58db/ddfabdb64110effc?q=
On Jun 14, 12:57 am, Alfred Fuller
wrote:
> http://www.youtube.com/watch?v=AgaL6NGpkB8is a good resource and I believe
> explicitly covers this scenario.
>
>
>
> On Sun, Jun 13, 2010 at 10:41 PM, Ian Lewis wrote:
> > Ben,
>
> > With non-relational databases like appengine's datastore you need to
> > create data based on the way you will query it rather than the way you
> > want to store it.
>
> > In your case it looks like you would need to create a model to store
> > the data you want to query (essentially an index that you manage). A
> > model something like the following might work.
>
> > PeopleOwnItemsIndex {
> >person (reference to the person owning the items/people)
> >owner (reference to the person owning the item)
> >item (reference to the item owned)
> > }
>
> > You'll be querying on the person field so the query will need to go in
> > your datastore-indexes.xml or index.yaml
>
> > When you change owners or add items you'll need to update the data for
> > this model. If you want to consolidate models you could also get only
> > your own items by creating an index on person+owner and query it with
> > person = me, owner = me. That would be instead of using PersonOwnItem,
> > but the difference would be you would have a record for each person
> > that can see the item rather than only one for the person owning the
> > item.
>
> > Ian
>
> > On Sun, Jun 13, 2010 at 4:10 AM, Ben Woodhead
> > wrote:
> > > Hello Everybody,
>
> > > I am very comfortable with SQL and relational databases and I am
> > > trying to wrap my head around GAE. Below is an example of how I would
> > > tackle my problem in SQL and it would be really great if someone could
> > > give me some hints on how to accomplish something similar in GAE. Its
> > > basically a hierarchy of ownership.
>
> > > The 2 base tables:
>
> > > table People {
> > > int personId;
> > > string name;
> > > }
>
> > > table items {
> > >int itemId;
> > >string itemName;
> > > }
>
> > > The 2 relational tables:
> > > table PeopleOwnPeople {
> > >int ownersId;
> > >int personId;
> > > }
>
> > > table PeopleOwnItems {
> > >int personId;
> > >int itemId;
> > > }
>
> > > So now that we have a basic structure I can start to explain the idea
> > > or problem. I want to be able to search for all items owned by me and
> > > the people i own:
>
> > > select items.* from Items, PeopleOwnPeople, PeopleOwnItems where
> > > (PeopleOwnPeople.ownersId=234 and
> > > PeopleOwnPeople.personId=PeopleownItems.personId) or
> > > PeopleOwnItems.personId = 234 ;
>
> > > I am looking to use GWT as my frontend to this project so I am likely
> > > going to be using Java for the backend but that isn't set in stone
> > > yet.
>
> > > Thanks any and all help.
> > > Ben
>
> > > --
> > > You received this message because you are subscribed to the Google Groups
> > "Google App Engine" group.
> > > To post to this group, send email to google-appeng...@googlegroups.com.
> > > To unsubscribe from this group, send email to
> > google-appengine+unsubscr...@googlegroups.com > e...@googlegroups.com>
> > .
> > > For more options, visit this group at
> >http://groups.google.com/group/google-appengine?hl=en.
>
> > --
> > ===
> > 株式会社ビープラウド イアン・ルイス
> > 〒150-0021
> > 東京都渋谷区恵比寿西2-3-2 NSビル6階
> > email: ianmle...@beproud.jp
> > TEL:03-6416-9836
> > FAX:03-6416-9837
> >http://www.beproud.jp/
> > ===
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Google App Engine" group.
> > To post to this group, send email to google-appeng...@googlegroups.com.
> > To unsubscribe from this group, send email to
> > google-appengine+unsubscr...@googlegroups.com > e...@googlegroups.com>
> > .
> > For more options, visit this group at
> >http://groups.google.com/group/google-appengine?hl=en.
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Cannot enable billing
I need to enable billing so I can use the BlobStore API (bit of a bait- n-switch there not warning me it was only paid-for, but let that be). I pressed the "Enable Billing" button, went through the Google Checkout, I got a receipt in the mail, my status on the Billing Settings page when to "Activating Billing" for a half-hour and then went back to "Free". I tried the usual things (different credit-cards and so on) but no love. I've opened an issue with support for this. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: How many exploding indexes is too much in general?
No, no rules of thumb. If you're expecting lots of small datasets with a complex sort, you're almost always going to want to do it locally instead of in the datastore. Just be aware that if the memory usage on your application bloats too much that it could be restarted. On Mon, Jun 14, 2010 at 12:36 PM, johnP wrote: > Actually, this is an interesting question. Is it generally better to > sort using the datastore, or sort using python code. For example, > let's say you have a list of people, and you need to get various > qualities about the people (let's say the list is a couple hundred > people or less). One approach can be to use multiple indexes: Order > people by name; and age > 50. Order reverse name and height > 5 > feet. > > A different approach is to memcache the entire list, and filter it > using list comprehensions. > > Besides the usual "profile your code" response - is there any rule-of- > thumbs to use to select an approach? > > johnP > > > > > > > On Jun 14, 11:48 am, "Ikai L (Google)" wrote: > > You're limited to 200 indexes for a billing enabled app and 100 for a > > non-billing enabled application. > > > > The number of indexes does not introduce a scalability cap, it just makes > > every write much, much more expensive. The question here is whether or > not > > your budget can support this. > > > > > > > > > > > > On Sat, Jun 12, 2010 at 11:23 AM, Harshal wrote: > > > Hi, > > > > > Until Google releases the version which they showcased in I/O where we > > > won't have to deal with exploding indexes, I guess our choices are very > > > limited and we would have to live with it. > > > > > I would like to go ahead and ask, what is acceptable number indexes > people > > > have in general? I have an entity which has something like 42 indexes > > > (because I have to filter in various ways on many properties many > times). Is > > > it scalable ? ( I am seeing high CPU spikes but I am fine with that, > well at > > > least as of now). > > > > > -- > > > You received this message because you are subscribed to the Google > Groups > > > "Google App Engine" group. > > > To post to this group, send email to google-appengine@googlegroups.com > . > > > To unsubscribe from this group, send email to > > > google-appengine+unsubscr...@googlegroups.com e...@googlegroups.com> > > > . > > > For more options, visit this group at > > >http://groups.google.com/group/google-appengine?hl=en. > > > > -- > > Ikai Lan > > Developer Programs Engineer, Google App Engine > > Blog:http://googleappengine.blogspot.com > > Twitter:http://twitter.com/app_engine > > Reddit:http://www.reddit.com/r/appengine > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- Ikai Lan Developer Programs Engineer, Google App Engine Blog: http://googleappengine.blogspot.com Twitter: http://twitter.com/app_engine Reddit: http://www.reddit.com/r/appengine -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: How many exploding indexes is too much in general?
Thanks for the answer. The key is to know that there *is* a continuum, where at one end, it is better to sort locally and at the other end, sort with indexes. johnP On Jun 14, 2:09 pm, "Ikai L (Google)" wrote: > No, no rules of thumb. If you're expecting lots of small datasets with a > complex sort, you're almost always going to want to do it locally instead of > in the datastore. Just be aware that if the memory usage on your application > bloats too much that it could be restarted. > > > > > > On Mon, Jun 14, 2010 at 12:36 PM, johnP wrote: > > Actually, this is an interesting question. Is it generally better to > > sort using the datastore, or sort using python code. For example, > > let's say you have a list of people, and you need to get various > > qualities about the people (let's say the list is a couple hundred > > people or less). One approach can be to use multiple indexes: Order > > people by name; and age > 50. Order reverse name and height > 5 > > feet. > > > A different approach is to memcache the entire list, and filter it > > using list comprehensions. > > > Besides the usual "profile your code" response - is there any rule-of- > > thumbs to use to select an approach? > > > johnP > > > On Jun 14, 11:48 am, "Ikai L (Google)" wrote: > > > You're limited to 200 indexes for a billing enabled app and 100 for a > > > non-billing enabled application. > > > > The number of indexes does not introduce a scalability cap, it just makes > > > every write much, much more expensive. The question here is whether or > > not > > > your budget can support this. > > > > On Sat, Jun 12, 2010 at 11:23 AM, Harshal wrote: > > > > Hi, > > > > > Until Google releases the version which they showcased in I/O where we > > > > won't have to deal with exploding indexes, I guess our choices are very > > > > limited and we would have to live with it. > > > > > I would like to go ahead and ask, what is acceptable number indexes > > people > > > > have in general? I have an entity which has something like 42 indexes > > > > (because I have to filter in various ways on many properties many > > times). Is > > > > it scalable ? ( I am seeing high CPU spikes but I am fine with that, > > well at > > > > least as of now). > > > > > -- > > > > You received this message because you are subscribed to the Google > > Groups > > > > "Google App Engine" group. > > > > To post to this group, send email to google-appengine@googlegroups.com > > . > > > > To unsubscribe from this group, send email to > > > > google-appengine+unsubscr...@googlegroups.com > > > e...@googlegroups.com> > e...@googlegroups.com> > > > > . > > > > For more options, visit this group at > > > >http://groups.google.com/group/google-appengine?hl=en. > > > > -- > > > Ikai Lan > > > Developer Programs Engineer, Google App Engine > > > Blog:http://googleappengine.blogspot.com > > > Twitter:http://twitter.com/app_engine > > > Reddit:http://www.reddit.com/r/appengine > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to google-appeng...@googlegroups.com. > > To unsubscribe from this group, send email to > > google-appengine+unsubscr...@googlegroups.com > e...@googlegroups.com> > > . > > For more options, visit this group at > >http://groups.google.com/group/google-appengine?hl=en. > > -- > Ikai Lan > Developer Programs Engineer, Google App Engine > Blog:http://googleappengine.blogspot.com > Twitter:http://twitter.com/app_engine > Reddit:http://www.reddit.com/r/appengine -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Listing entities with TextProperty efficiently
IMHO splitting up entities may be regarded as premature optimization unless your calculations say daily bandwidth usage to datastore is going to be a problem. Regards, Brandon Thomson On Mon, Jun 14, 2010 at 3:44 PM, Blixt wrote: > Hey, > > If I have a model with a TextProperty and I want to list it, that > means a query such as Entry.all().fetch(10) could potentially fetch > several megabytes of data (not in my case, but just a worst case > scenario). In a list I would only show some of the smaller properties, > such as a title and whatnot. > > Would you say I should clone every Entry into EntryShort which would > have only the properties necessary for the list and a reference to the > original Entry-object? Or will it do fine to just fetch this > potentially huge amount of data every now and then and memcache the > list (excluding the large properties)? > > /Andreas Blixt > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Seeing a lot of ApplicationError 2 errors since maintenance?
Looking at the logs, ApplicationError 2 *appears* be thrown by the application itself and not by the service. Could you publish the stack trace? On Jun 13, 3:55 am, Jan Z wrote: > Hi - as per request, app id is access-manager. > > Happy to provide IP's and URL's through email? We tested against an > EC2 instance, Rackspace and an internal server, all had the same > problem. Google.com didn't. > > Thanks! > > Jan > > On Jun 13, 11:13 am, Jan Z wrote: > > > Is anyone else experiencing this? > > > J -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: NeedIndexException - in production server
Yes, you can create composite indexes that contain columns that are often queried together. The merge join algorithm will use these indexes to make queries that include those specific columns more efficient. Or if you see this error more often on some queries than others, create composite indexes for those queries (or the common columns between those queries) http://www.youtube.com/watch?v=ofhEyDBpngM at time 30:30 gives an example of this. On Mon, Jun 14, 2010 at 7:58 AM, Ravi wrote: > Hi Geoffrey Spear, > Thanks for your clarification. > We suspected the same reason. Last few days back our data was less > around 20,000 rows there was no problem. > Only when more data is added we are facing this problem. > > Our table is too large which is around 50 fields/columns and around > 50,000 rows. Query filter is also dynamically generated (user selects > few fields and submits). > Our servlet will query database depending on the user selected fields. > In this scenario we dont know which fields will be selected by user > and certainly I cant have composite index for 50 combinations. > > Is there any work around for this problem?? > > Thanks, > Ravi > > > > > > On Jun 14, 5:59 pm, Geoffrey Spear wrote: > > It's not a problem with indexing, it's a problem with your data. If > > you don't add composite indexes, the system will attempt to serve > > queries that contain only equality filters using a merge-join > > strategy, which is less efficient than a simple index lookup (with the > > benefit of not needing to add the indexes, which use space and write- > > time CPU). If your data becomes such that the query can not be done > > efficiently with the merge-join anymore, you'll start to see that > > exception and you'll need to add a composite index. > > > > On Jun 12, 2:00 am, Ravi wrote: > > > > > > > > > Hi Google Team, > > > > > All these days our application was working fine and we have not > > > changed any datastore index or codebase of our application. > > > Everyday we upload new set of data to our application. Today also we > > > have uploaded new set of data but today after uploading the data we > > > are getting very strange exception as below: > > > > > com.google.appengine.api.datastore.DatastoreNeedIndexException: The > > > built-in indices are not efficient enough for this query and your > > > data. Please add a composite index for this query.. An index is > > > missing but we are unable to tell you which one due to a bug in the > > > App Engine SDK. If your query only contains equality filters you most > > > likely need a composite index on all the properties referenced in > > > those filters. > > > > > We are worried after seeing the above exception, we dont have any > > > complex queries, all the queries contain equals to filter (==). > > > We tested the same set of data and query in our development > > > environment, it is working fine. > > > For testing purpose, we uploaded the same set of data to our other > > > testing appengine applcation, it is working fine there also. > > > > > Only in our production system, we are getting > > > DatastoreNeedIndexException. > > > > > Could you please confirm is there any problem with the datastore > > > indexing. > > > Data stored in our production datastore is more than 0.5 GB, will > > > thee be any problem in indexing if there is high amount of data. > > > > > Application ID: bigpaisa2.appspot.com > > > Latest application version: > http://061102.latest.bigpaisa2.appspot.com > > > - In this version DatastoreNeedIndexException is occuring. > > > > > Please help us as soon as possible. > > > > > Thanks, > > > Ravi > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Billing settings not changing
It seems the problem eventually got resolved, as my billing settings finally got updated. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Cannot enable billing
Well, the blobstore isn't really paid-for, as it has a free quota, but you have to enable billing in order to get it, it is a little strange. If you want things to be as free as possible, set the daily budget as low as possible (I think it's $1), and put that all towards a quota that you never use up. On Jun 15, 5:55 am, Michael Lorton wrote: > I need to enable billing so I can use the BlobStore API (bit of a bait- > n-switch there not warning me it was only paid-for, but let that be). > > I pressed the "Enable Billing" button, went through the Google > Checkout, I got a receipt in the mail, my status on the Billing > Settings page when to "Activating Billing" for a half-hour and then > went back to "Free". > > I tried the usual things (different credit-cards and so on) but no > love. > > I've opened an issue with support for this. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Entity group question
I have a set of entities all of the same kind, AnalyticsCounter, in my
datastore. For correctness I would like to update many of these entities in
a single transaction, which requires they are all in the same entity group.
In my case I don't have a hierarchy, instead I just have a set of entities.
My understanding is that I need to parent all of them to another
AnalyticsCounter entity to achieve this. However when I tried this, I
received the fateful "Cannot operate on different entity groups in a
transaction error.".
I created the parent entity in my constructor, and then applied the parent
when I created an instance of AnalyticsCounter, but the transaction fails on
the put method. Interestingly the "different entity groups" lists both the
newly created instance and the parent instance. I tried creating the parent
two different ways, first just creating a key:
self._parent_key = db.Key.from_path('AnalyticsCounter', 'virtual_root')
and then, creating an actual instance.
self._parent_instance = AnalyticsCounter.get_or_insert(key_name='root')
but met the same failure on both cases.
Briefly, my transaction looks like this:
counters = [list of key_names]
def txn():
counters = AnalyticsCounter.get_by_key_name(counters)
for counter in counters:
if counter is None:
counter = AnalyticsCounter(parent=self._parent_instance,
key_name='blah')
counter.foo = 'bar'
counter.put() # <- dies here
db.run_in_transaction(txn)
Can someone shed some light on what I am doing wrong? Thanks,
Chris
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
