Re: [google-appengine] Architecture approaches for puts
Why not use allocate_ids to generate the ids? That might simplify the process a bit. http://code.google.com/appengine/docs/python/datastore/functions.html#allocate_ids I've been using a similar process for batch updates for quite some time. Works well for my case, but in my case there is not a user involved. It is an auto-mated sync process to another system's database, so I have a unique id to use for lookups to avoid duplicates. What happens if the client does not get the response in step 4. Also, I assume if you get a failure, and resend the entity you'll use the previous id? Robert On Tue, Nov 9, 2010 at 15:07, stevep wrote: > I would like some feedback about pluses / minuses for handling new records. > Currently I need to optimize how the client request handler processes new > entity put()s. Several custom indices for the model are used, so puts run > too close to the 1,000 ms limit (were running over the limit prior to Nov. > 6th maintenance – thanks Google). > The entities are written with unique integer key values. Integers are > generated using Google’s recommended sharded process. Client currently POSTs > a new record to the GAE handler. If handler does not send back a successful > response, client will retry POST “n” times (at least twice, but possibly > more). Continued failures past “n” will prompt user that record could not be > created, saves data locally, and asks user to try later. > Planned new process will use Task Queue. > 1) Client POSTs new entity data to the handler. At this point, user sees a > dialog box saying record is being written. > 2) Handler will use the shards to generate the next integer value for the > key. > 3) Handler sets up a task queue with the new key value and record data, and > responds back to the client with they key value. > 4) Client receives key value back from handler, and changes to inform user > that record write is being confirmed on the server (or as before retries > entire POST if response is an error code). > 5) Client waits a second or two (for task queue to finish), then issues a > GET to the handler to read the new record using the key value. > 6) Handler does a simple key value read of the new record. Responds back to > client either with found or not found status. > 7) If client gets found response, then we are done. If not found, or error > response client will wait a few seconds, and issue another GET. > 7) If after “n” tries, no GET yields a successful read, then client informs > user that record could not be written, and “please try again in a few > minutes” (saving new record data locally). > I know this is not ideal, but believe it is a valid, given GAE’s > limitations, as an approach to minimize lost writes. Would very much > appreciate feedback. I should note that the imposition of a few seconds > delay while writing the record should not be an issue given it is a single > transaction at the end of a previous creative process which has engaged user > for several minutes. Also, we do not use logic that cannot handle gaps > (missing) integer values in the model's key values. > TIA, > stevep > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Only one property per query may have inequality filters?
Hey Ajaxer,
Yes, GQL has an IN operator. You can use it via GqlQuery or the Query object.
http://code.google.com/appengine/docs/python/datastore/gqlreference.html
http://code.google.com/appengine/docs/python/datastore/queryclass.html
>From the docs:
Note: The IN and != operators use multiple queries behind the scenes.
For example, the IN operator executes a separate underlying datastore
query for every item in the list. The entities returned are a result
of the cross-product of all the underlying datastore queries and are
de-duplicated. A maximum of 30 datastore queries are allowed for any
single GQL query.
Robert
On Tue, Nov 9, 2010 at 23:42, ajaxer wrote:
> thanks.
>
> there is no 'or' in gql?
>
> records = QMessage.gql('where time > :1 and room = :2 and fromUser
> = :3 or toUser = :3 order by time limit', time, room, user)
>
>
> BadQueryError: Parse Error: Expected no additional symbols at symbol
> or
>
> On Nov 10, 1:01 am, Robert Kluin wrote:
>> http://code.google.com/appengine/docs/python/datastore/queriesandinde...
>>
>> I am just guessing about your application, but I would drop the user
>> filter from the query and handle it in your app code. Or you can run
>> two queries and merge the results yourself.
>>
>> Robert
>>
>>
>>
>>
>>
>>
>>
>> On Tue, Nov 9, 2010 at 11:39, ajaxer wrote:
>> > BadFilterError: BadFilterError: invalid filter: Only one property per
>> > query may have inequality filters (<=, >=, <, >)..
>>
>> > my code:
>>
>> > records = QMessage.all().filter('room = ', room).filter('isPrivate
>> > =',
>> > False).filter('fromUser !=', user).filter('time > ', time)
>>
>> > how can i solve this problem?
>>
>> > --
>> > You received this message because you are subscribed to the Google Groups
>> > "Google App Engine" group.
>> > To post to this group, send email to google-appeng...@googlegroups.com.
>> > To unsubscribe from this group, send email to
>> > google-appengine+unsubscr...@googlegroups.com.
>> > For more options, visit this group
>> > athttp://groups.google.com/group/google-appengine?hl=en.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appeng...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Only one property per query may have inequality filters?
thanks.
there is no 'or' in gql?
records = QMessage.gql('where time > :1 and room = :2 and fromUser
= :3 or toUser = :3 order by time limit', time, room, user)
BadQueryError: Parse Error: Expected no additional symbols at symbol
or
On Nov 10, 1:01 am, Robert Kluin wrote:
> http://code.google.com/appengine/docs/python/datastore/queriesandinde...
>
> I am just guessing about your application, but I would drop the user
> filter from the query and handle it in your app code. Or you can run
> two queries and merge the results yourself.
>
> Robert
>
>
>
>
>
>
>
> On Tue, Nov 9, 2010 at 11:39, ajaxer wrote:
> > BadFilterError: BadFilterError: invalid filter: Only one property per
> > query may have inequality filters (<=, >=, <, >)..
>
> > my code:
>
> > records = QMessage.all().filter('room = ', room).filter('isPrivate
> > =',
> > False).filter('fromUser !=', user).filter('time > ', time)
>
> > how can i solve this problem?
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Google App Engine" group.
> > To post to this group, send email to google-appeng...@googlegroups.com.
> > To unsubscribe from this group, send email to
> > google-appengine+unsubscr...@googlegroups.com.
> > For more options, visit this group
> > athttp://groups.google.com/group/google-appengine?hl=en.
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Unknown error 503 Occurs on appengine Python
And it could help to format the code properly. If I am not mistaken,
white-space is important in Python; so no leading spaces makes it hard
to spot many types of errors.
Robert
On Tue, Nov 9, 2010 at 18:33, Tim Hoffman wrote:
> Hi
>
> You should include a stacktrace from your control panel so we can have
> some idea of what is going on.
>
> Rgds
>
> T
>
> On Nov 9, 4:08 pm, theprateeksharma
> wrote:
>> I am a complete newbie to appengine , python and these api so please
>> pardon me for errors
>>
>> I have used google spreadsheet api and google doc list api on
>> appengine using python
>>
>> part of code causing error is specified below.If further details are
>> required i m ready to share it
>>
>> for i in range(int(total_number_of_boxes)):
>> boxname='strText'+str(i+1) classname=self.request.get(boxname)
>> attendancesheet =
>> self.gd_client.Copy(self._Retrieve_Document(self.standard_attendance_regist
>> er),
>> classname) self._Move_Documents(attendancesheet,schoolfolder) t_key =
>> self._Retrieve_Key_Spreadsheet(attendancesheet) s_key =
>> t_key.partition('A') feed = self.gs_client.GetWorksheetsFeed(s_key[2])
>> w_id = self._Retrieve_Key_Firstworksheet(feed.entry[0])
>> self._CellsUpdateAction(row, col, classname)
>> self._CellsUpdateAction(row, col+1, s_key[2])
>> self._CellsUpdateAction(row, col+2, w_id) self._CellsUpdateAction(row,
>> col+3, admin_name) self._CellsUpdateAction(row, col+4, admin_email)
>> T_name='t'+str(counter)
>> e=Directaccess(teacherusername=T_name,teacherpassword=T_password,teacherspr
>> eadsdheetid=s_key,teacherworksheetid=w_id)
>> self._Addnewacl_Permission(attendancesheet,admin_email,'user','writer')
>> row=row+1 column=1 counter=counter+1
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appeng...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Fan-in with materialized views: A sketch
Dmitry,
Glad to hear the bucket size helped!
Please let me know how it goes. If you have good results, maybe we
can find a clean way to facilitate directly doing the work done by
create work.
Robert
On Tue, Nov 9, 2010 at 18:11, Dmitry wrote:
> Robert, thanks a lot for your sugestions!
> Increasing bucket size made a huge difference. Need to study
> theoretical part... and find the optimal bucket size for 50/sec.
>
> yep, I use creatework directly without fanout. I will try to insert
> 'work' models within my original data transaction and compare the
> performance.
>
> On Nov 9, 3:14 am, Robert Kluin wrote:
>> Hey Dmitry,
>> I am working on getting some decent documentation about when you
>> might want to use fanout versus directly using creatwork. And, about
>> usage in general. If I am dealing with one or two aggregations I
>> usually use creatework directly. You can only insert five
>> transactional tasks in one database transaction, so with four you
>> could directly use creatework eliminating a fanout task.
>>
>> As far as rates go, I have been using a rate of 35/s and bucket size
>> of 40. However, I also get periodic queue backups. I think the max
>> rate / sec is currently 50, but I thought there was an announcement it
>> was getting increased (maybe I am just remembering the increase to
>> 50/sec announcement though). You might want to bump your rate up to
>> 50/sec. I always use a dedicated queue for creatework and aggregation
>> tasks. In one of my apps I use multiple queues to get a bit higher
>> throughput.
>>
>> I generally prefer to use creatework tasks; they cleanly handle any
>> failures that occur and keeps my primary processing running as fast as
>> possible. However, when I first started using this type of
>> aggregation technique I created the 'work' models and attempted to
>> insert the aggregator task (non-transactionaly!) within my primary
>> transaction. If your primary processing is within tasks, and your
>> tasks are fast enough, give it a shot. Converting CreateWorkHandler
>> to something you can use directly should not be a big deal.
>>
>> Robert
>>
>> On Mon, Nov 8, 2010 at 18:14, Dmitry wrote:
>> > Hi Robert,
>>
>> > What queue configuration do you use for your system?
>> > I came to another problem. I usually process several feeds in parallel
>> > and can insert up to 20-30 new items to the database. With 4
>> > aggregators it's >80 create_work tasks in one moment. So after a
>> > minute I can have up to 1000 tasks in queue... so I have up to 5
>> > minutes delay in processing.
>>
>> > It seems that for initial aggregation I should insert create work
>> > models not in tasks.
>> > I messed up again:)
>>
>> > On Nov 5, 6:46 am, Robert Kluin wrote:
>> >> Dmitry,
>> >> I finally got the time to make these changes. Let me know if that
>> >> works for your use-case.
>>
>> >> I really appreciate all of your suggestions and help with this.
>>
>> >> Robert
>>
>> >> 2010/11/3 Dmitry :
>>
>> >> > oops I read expression in wrong direction. This will definitely work!
>>
>> >> > On Nov 3, 7:43 pm, Robert Kluin wrote:
>> >> >> Dmitry,
>> >> >> š Right, I know those will cause problems. So what about my suggested
>> >> >> solution of using:
>>
>> >> >> šif not re.match("^[a-zA-Z0-9-]+$", task_name):
>> >> >> š š š task_name = šsha1_hash(task_name)
>>
>> >> >> That should correctly handle your use cases, since the full name will
>> >> >> be hashed.
>>
>> >> >> Are there issues with that solution I am not seeing?
>>
>> >> >> Robert
>>
>> >> >> On Nov 3, 2010, at 3:52, Dmitry wrote:
>>
>> >> >> > Robert,
>>
>> >> >> > You will get into the trouble with these aggregations:
>>
>> >> >> > urls:
>> >> >> > http://ÐÒÁ×ÉÔÅÌØÓÔ×Ï.ÒÆ/search/?phrase=ÎÁÌÏǧion=gov_events ->
>> >> >> > httpsearchphrase
>> >> >> > http://ÐÒÁ×ÉÔÅÌØÓÔ×Ï.ÒÆ/search/?phrase=ÐÒÅÚÉÄÅÎÔ§ion=gov_events
>> >> >> > ->
>> >> >> > httpsearchphrase
>>
>> >> >> > or usernames:
>> >> >> > ÍÓÔÉÔÅÌØ2000 -> 2000
>> >> >> > ÔÅÓÔ2000 -> 2000
>>
>> >> >> > but anyway in most cases your approach will work well:) You can leave
>> >> >> > it up to the user (add some kind of flag "use_hash").
>>
>> >> >> > or we can try to url encode strings:
>> >> >> > urllib.quote(task_name.encode('utf-8'))
>> >> >> > http3AD0BFD180D0B0D0B2D0B8D182D0B5D0BBD18CD181D182D0B2D0BED180D184search3Fphrase3DD0BDD0B0D0BBD0BED0B3
>> >> >> > http3AD0BFD180D0B0D0B2D0B8D182D0B5D0BBD18CD181D182D0B2D0BED180D184search3Fphrase3DD0BFD180D0B5D0B7D0B8D0B4D0B5D0BDD182
>>
>> >> >> > but this is not better that hash :-D
>>
>> >> >> > thanks
>>
>> >> >> > On Nov 3, 7:13 am, Robert Kluin wrote:
>> >> >> >> Hey Dmitry,
>> >> >> >> š I am sure the "fix" in that commit is _not_ a good idea.
>> >> >> >> šOriginally
>> >> >> >> I stuck it in because I use entity keys as the task-name, sometimes
>> >> >> >> they contains characters not allowed in task-names. šI actually
>> >> >> >> debated for several days about pushing that update out; šfinally
Re: [google-appengine] Keys only queries do not support IN filters?
Hi djidjadji,
I'm implement so, but why Google doesn't support it directly?
--
keakon
On Wed, Nov 10, 2010 at 9:24 AM, djidjadji wrote:
> A query with 'IN' is translated behind the scene as separate queries.
> You can perform them yourself
>
> result = []
> for kw in keywords:
> result.extend(Article.all(keys_only=True).filter('keywords =',
> kw).fetch(10))
> if len(result)>=10: break
> result = result[:10]
>
> 2010/11/9 风笑雪 :
>> When I run this query:
>> Article.all(keys_only=True).filter('keywords IN', keywords).fetch(10)
>>
>> It raise a BadQueryError: Keys only queries do not support IN filters.
>> http://code.google.com/p/googleappengine/source/browse/trunk/python/google/appengine/ext/db/__init__.py#2197
>>
>> But I can't find this restriction in document:
>> http://code.google.com/intl/en/appengine/docs/python/datastore/queriesandindexes.html#Restrictions_on_Queries
>>
>> --
>> keakon
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Google App Engine" group.
>> To post to this group, send email to google-appeng...@googlegroups.com.
>> To unsubscribe from this group, send email to
>> google-appengine+unsubscr...@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/google-appengine?hl=en.
>>
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appeng...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Problems pointing my GoDaddy domain to App Engine (no other posts have been able to help!)
The domain was not purchased through Google Apps, nor is it currently managed by Google Apps. I bought it and configure it at GoDaddy. Do I need to transfer management of this domain to Google Apps? I'd rather not. I set the CNAME entry at GoDaddy, but do I need to set the Nameservers to be something google specific? I only set up CNAME entries and MX entries. But it's still not working and it's been > 24 hours. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Keys only queries do not support IN filters?
A query with 'IN' is translated behind the scene as separate queries.
You can perform them yourself
result = []
for kw in keywords:
result.extend(Article.all(keys_only=True).filter('keywords =',
kw).fetch(10))
if len(result)>=10: break
result = result[:10]
2010/11/9 风笑雪 :
> When I run this query:
> Article.all(keys_only=True).filter('keywords IN', keywords).fetch(10)
>
> It raise a BadQueryError: Keys only queries do not support IN filters.
> http://code.google.com/p/googleappengine/source/browse/trunk/python/google/appengine/ext/db/__init__.py#2197
>
> But I can't find this restriction in document:
> http://code.google.com/intl/en/appengine/docs/python/datastore/queriesandindexes.html#Restrictions_on_Queries
>
> --
> keakon
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appeng...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Mystery data usage
No, it won't auto clear indexes. You'll need to use something like the Mapper API to read all the entities, modifies them and re-saves them. -- Ikai Lan Developer Programs Engineer, Google App Engine Blogger: http://googleappengine.blogspot.com Reddit: http://www.reddit.com/r/appengine Twitter: http://twitter.com/app_engine On Tue, Nov 9, 2010 at 2:44 PM, Erik wrote: > > Thanks for the information and tips Robert, somehow I missed that > AppEngine automatically indexes properties. > Simply adding the unindexed extension to the JDO class should > eventually clear the auto-generated indexes? > > Cheers & thanks again, > -Erik > > On Nov 9, 10:52 am, Robert Kluin wrote: > > Hi Erik, > > As far as I know the persistence manager would not create any extra > entities. > > > > It sounds like you might want to explicitly disable indexing on any > > fields you will not be querying on. See the 'Properties that Aren't > > Indexed' section on the 'Queries and Indexes' page. > http://code.google.com/appengine/docs/java/datastore/queriesandindexe... > > > > You might also want to use shorter kind and property names. Kind > > and property names are stored with every entity, so it can add up > > pretty fast. > > > > Robert > > > > On Tue, Nov 9, 2010 at 08:05, Erik wrote: > > > Hi Robert, > > > > > Thanks for the response, I did wait several days before sending that > > > message, but shortly afterwards the quota cleared to zero. I am using > > > a persistence manager with jdo, would these create temporary > > > entities? > > > > > I need to explicitly index my keys in descending order for mapreduce, > > > and thought I needed to index my collections but as I am not > > > performing any queries don't think that is necessary. My persistent > > > objects consist of a single property with type of long collection, > > > entities are fetched by id. > > > > > My latest 30MB dataset has already grown to consume 710MB, that is an > > > enormous increase. Datastore statistics say only 131MB is being used > > > with 71% of that as metadata. > > > > > Any other ideas? > > > > > Thanks, > > > -Erik > > > > > On Nov 8, 1:03 pm, Robert Kluin wrote: > > >> Hi Erik, > > >> Several common sources of datastore stats / quota number funkiness: > > >> 1) The numbers are not updated in real time. Sometimes it can > > >> take a day for the numbers to get updated. > > >> 2) Because of 1, if you are using a session library (or something > > >> similar) that create lots of temporary entities the numbers can > > >> 'appear' to be out-of-sync. > > > > >> I assume when you say switched to explicit indexes, you mean you > > >> explicitly disabled indexing of any properties that do not need > > >> indexes? > > > > >> Also, you might want to star issue 2740. > > >> http://code.google.com/p/googleappengine/issues/detail?id=2740 > > > > >> Robert > > > > >> On Sat, Nov 6, 2010 at 19:29, Erik wrote: > > > > >> > Hello all, > > > > >> > I have been uploading a dataset which is composed of 100MB of CSV > > >> > values. During the process of uploading with bulkloader, which > never > > >> > completed, the datastore expanded to consume 2GB of usage. I > decided > > >> > to explicitly index my data and clear the datastore before a > re-import > > >> > using the blobstore/mapreduce method. However, after clearing the > > >> > datastore there is still a persistent .37GB of data usage remaining, > > >> > while datastore statistics say 53KB is being consumed by 174 > objects, > > >> > and nothing shows in datastore viewer. > > > > >> > Any ideas on how I can recover the .37GB of quota which is > > >> > mysteriously being used in my empty datastore? > > > > >> > Many thanks, > > >> > -Erik > > > > >> > -- > > >> > You received this message because you are subscribed to the Google > Groups "Google App Engine" group. > > >> > To post to this group, send email to > google-appeng...@googlegroups.com. > > >> > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com > . > > >> > For more options, visit this group athttp:// > groups.google.com/group/google-appengine?hl=en. > > > > > -- > > > You received this message because you are subscribed to the Google > Groups "Google App Engine" group. > > > To post to this group, send email to google-appengine@googlegroups.com > . > > > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com > . > > > For more options, visit this group athttp:// > groups.google.com/group/google-appengine?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message beca
Re: [google-appengine] app engine articles
People got busy =). We're always looking for technical content from our partners, however, so if you've got something interesting please let us know. Wesley has an article coming out soon about frameworks. -- Ikai Lan Developer Programs Engineer, Google App Engine Blogger: http://googleappengine.blogspot.com Reddit: http://www.reddit.com/r/appengine Twitter: http://twitter.com/app_engine On Tue, Nov 9, 2010 at 3:22 PM, Jamie H wrote: > I'm just curious what happened to the articles at > http://code.google.com/appengine/articles/ > > Nothing new has shown up in a while. They were very informative... > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: AppEngine fail?
Yes, its working fine now. I dont know what happend, it wasnt working for me for about 15min. No response from Google so far. On 9 November 2010 23:39, Rafael Sierra wrote: > Is Appengine normal again? What happened? > > On Tue, Nov 9, 2010 at 9:15 PM, Andrius wrote: > > Yes, I can confirm that as well! > > > > On Nov 9, 11:12 pm, Rafael Sierra wrote: > >> Every request is failing, dashboard, app, etc > >> > >> -- > >> Rafael Sierrahttp://blog.rafaelsdm.com > > > > -- > > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > > To post to this group, send email to google-appeng...@googlegroups.com. > > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com > . > > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > > > > > > > -- > Rafael Sierra > http://blog.rafaelsdm.com > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: AppEngine fail?
Is Appengine normal again? What happened? On Tue, Nov 9, 2010 at 9:15 PM, Andrius wrote: > Yes, I can confirm that as well! > > On Nov 9, 11:12 pm, Rafael Sierra wrote: >> Every request is failing, dashboard, app, etc >> >> -- >> Rafael Sierrahttp://blog.rafaelsdm.com > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- Rafael Sierra http://blog.rafaelsdm.com -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Unknown error 503 Occurs on appengine Python
Hi
You should include a stacktrace from your control panel so we can have
some idea of what is going on.
Rgds
T
On Nov 9, 4:08 pm, theprateeksharma
wrote:
> I am a complete newbie to appengine , python and these api so please
> pardon me for errors
>
> I have used google spreadsheet api and google doc list api on
> appengine using python
>
> part of code causing error is specified below.If further details are
> required i m ready to share it
>
> for i in range(int(total_number_of_boxes)):
> boxname='strText'+str(i+1) classname=self.request.get(boxname)
> attendancesheet =
> self.gd_client.Copy(self._Retrieve_Document(self.standard_attendance_regist
> er),
> classname) self._Move_Documents(attendancesheet,schoolfolder) t_key =
> self._Retrieve_Key_Spreadsheet(attendancesheet) s_key =
> t_key.partition('A') feed = self.gs_client.GetWorksheetsFeed(s_key[2])
> w_id = self._Retrieve_Key_Firstworksheet(feed.entry[0])
> self._CellsUpdateAction(row, col, classname)
> self._CellsUpdateAction(row, col+1, s_key[2])
> self._CellsUpdateAction(row, col+2, w_id) self._CellsUpdateAction(row,
> col+3, admin_name) self._CellsUpdateAction(row, col+4, admin_email)
> T_name='t'+str(counter)
> e=Directaccess(teacherusername=T_name,teacherpassword=T_password,teacherspr
> eadsdheetid=s_key,teacherworksheetid=w_id)
> self._Addnewacl_Permission(attendancesheet,admin_email,'user','writer')
> row=row+1 column=1 counter=counter+1
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] 500 Server Error in all apps
It's up again! On 9 November 2010 23:14, Andrius wrote: > Hi, > > none of my apps load and even GAE admin console is not loading. > Getting error: > Error: Server Error > The server encountered an error and could not complete your request. > If the problem persists, please report your problem and mention this > error message and the query that caused it. > > Anyone having the same issues? App engine status says nothing about > it.. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] app engine articles
I'm just curious what happened to the articles at http://code.google.com/appengine/articles/ Nothing new has shown up in a while. They were very informative... -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: AppEngine fail?
Yes, I can confirm that as well! On Nov 9, 11:12 pm, Rafael Sierra wrote: > Every request is failing, dashboard, app, etc > > -- > Rafael Sierrahttp://blog.rafaelsdm.com -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] 500 Server Error in all apps
Hi, none of my apps load and even GAE admin console is not loading. Getting error: Error: Server Error The server encountered an error and could not complete your request. If the problem persists, please report your problem and mention this error message and the query that caused it. Anyone having the same issues? App engine status says nothing about it.. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] AppEngine fail?
Every request is failing, dashboard, app, etc -- Rafael Sierra http://blog.rafaelsdm.com -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Fan-in with materialized views: A sketch
Robert, thanks a lot for your sugestions!
Increasing bucket size made a huge difference. Need to study
theoretical part... and find the optimal bucket size for 50/sec.
yep, I use creatework directly without fanout. I will try to insert
'work' models within my original data transaction and compare the
performance.
On Nov 9, 3:14 am, Robert Kluin wrote:
> Hey Dmitry,
> I am working on getting some decent documentation about when you
> might want to use fanout versus directly using creatwork. And, about
> usage in general. If I am dealing with one or two aggregations I
> usually use creatework directly. You can only insert five
> transactional tasks in one database transaction, so with four you
> could directly use creatework eliminating a fanout task.
>
> As far as rates go, I have been using a rate of 35/s and bucket size
> of 40. However, I also get periodic queue backups. I think the max
> rate / sec is currently 50, but I thought there was an announcement it
> was getting increased (maybe I am just remembering the increase to
> 50/sec announcement though). You might want to bump your rate up to
> 50/sec. I always use a dedicated queue for creatework and aggregation
> tasks. In one of my apps I use multiple queues to get a bit higher
> throughput.
>
> I generally prefer to use creatework tasks; they cleanly handle any
> failures that occur and keeps my primary processing running as fast as
> possible. However, when I first started using this type of
> aggregation technique I created the 'work' models and attempted to
> insert the aggregator task (non-transactionaly!) within my primary
> transaction. If your primary processing is within tasks, and your
> tasks are fast enough, give it a shot. Converting CreateWorkHandler
> to something you can use directly should not be a big deal.
>
> Robert
>
> On Mon, Nov 8, 2010 at 18:14, Dmitry wrote:
> > Hi Robert,
>
> > What queue configuration do you use for your system?
> > I came to another problem. I usually process several feeds in parallel
> > and can insert up to 20-30 new items to the database. With 4
> > aggregators it's >80 create_work tasks in one moment. So after a
> > minute I can have up to 1000 tasks in queue... so I have up to 5
> > minutes delay in processing.
>
> > It seems that for initial aggregation I should insert create work
> > models not in tasks.
> > I messed up again:)
>
> > On Nov 5, 6:46 am, Robert Kluin wrote:
> >> Dmitry,
> >> I finally got the time to make these changes. Let me know if that
> >> works for your use-case.
>
> >> I really appreciate all of your suggestions and help with this.
>
> >> Robert
>
> >> 2010/11/3 Dmitry :
>
> >> > oops I read expression in wrong direction. This will definitely work!
>
> >> > On Nov 3, 7:43 pm, Robert Kluin wrote:
> >> >> Dmitry,
> >> >> š Right, I know those will cause problems. So what about my suggested
> >> >> solution of using:
>
> >> >> šif not re.match("^[a-zA-Z0-9-]+$", task_name):
> >> >> š š š task_name = šsha1_hash(task_name)
>
> >> >> That should correctly handle your use cases, since the full name will
> >> >> be hashed.
>
> >> >> Are there issues with that solution I am not seeing?
>
> >> >> Robert
>
> >> >> On Nov 3, 2010, at 3:52, Dmitry wrote:
>
> >> >> > Robert,
>
> >> >> > You will get into the trouble with these aggregations:
>
> >> >> > urls:
> >> >> > http://ÐÒÁ×ÉÔÅÌØÓÔ×Ï.ÒÆ/search/?phrase=ÎÁÌÏǧion=gov_events ->
> >> >> > httpsearchphrase
> >> >> > http://ÐÒÁ×ÉÔÅÌØÓÔ×Ï.ÒÆ/search/?phrase=ÐÒÅÚÉÄÅÎÔ§ion=gov_events ->
> >> >> > httpsearchphrase
>
> >> >> > or usernames:
> >> >> > ÍÓÔÉÔÅÌØ2000 -> 2000
> >> >> > ÔÅÓÔ2000 -> 2000
>
> >> >> > but anyway in most cases your approach will work well:) You can leave
> >> >> > it up to the user (add some kind of flag "use_hash").
>
> >> >> > or we can try to url encode strings:
> >> >> > urllib.quote(task_name.encode('utf-8'))
> >> >> > http3AD0BFD180D0B0D0B2D0B8D182D0B5D0BBD18CD181D182D0B2D0BED180D184search3Fphrase3DD0BDD0B0D0BBD0BED0B3
> >> >> > http3AD0BFD180D0B0D0B2D0B8D182D0B5D0BBD18CD181D182D0B2D0BED180D184search3Fphrase3DD0BFD180D0B5D0B7D0B8D0B4D0B5D0BDD182
>
> >> >> > but this is not better that hash :-D
>
> >> >> > thanks
>
> >> >> > On Nov 3, 7:13 am, Robert Kluin wrote:
> >> >> >> Hey Dmitry,
> >> >> >> š I am sure the "fix" in that commit is _not_ a good idea.
> >> >> >> šOriginally
> >> >> >> I stuck it in because I use entity keys as the task-name, sometimes
> >> >> >> they contains characters not allowed in task-names. šI actually
> >> >> >> debated for several days about pushing that update out; šfinally I
> >> >> >> decide to push and hope someone would notice and offer their
> >> >> >> thoughts.
>
> >> >> >> š I like your idea a lot. šBut, for many aggregations I like to use
> >> >> >> entity keys, it makes it possible for me to visually see what a task
> >> >> >> is doing. šWhat do you think about something like the following
> >> >> >> approach:
>
> >> >> >> š if not re.match("^[a
[google-appengine] Re: Mystery data usage
Thanks for the information and tips Robert, somehow I missed that AppEngine automatically indexes properties. Simply adding the unindexed extension to the JDO class should eventually clear the auto-generated indexes? Cheers & thanks again, -Erik On Nov 9, 10:52 am, Robert Kluin wrote: > Hi Erik, > As far as I know the persistence manager would not create any extra > entities. > > It sounds like you might want to explicitly disable indexing on any > fields you will not be querying on. See the 'Properties that Aren't > Indexed' section on the 'Queries and Indexes' > page.http://code.google.com/appengine/docs/java/datastore/queriesandindexe... > > You might also want to use shorter kind and property names. Kind > and property names are stored with every entity, so it can add up > pretty fast. > > Robert > > On Tue, Nov 9, 2010 at 08:05, Erik wrote: > > Hi Robert, > > > Thanks for the response, I did wait several days before sending that > > message, but shortly afterwards the quota cleared to zero. I am using > > a persistence manager with jdo, would these create temporary > > entities? > > > I need to explicitly index my keys in descending order for mapreduce, > > and thought I needed to index my collections but as I am not > > performing any queries don't think that is necessary. My persistent > > objects consist of a single property with type of long collection, > > entities are fetched by id. > > > My latest 30MB dataset has already grown to consume 710MB, that is an > > enormous increase. Datastore statistics say only 131MB is being used > > with 71% of that as metadata. > > > Any other ideas? > > > Thanks, > > -Erik > > > On Nov 8, 1:03 pm, Robert Kluin wrote: > >> Hi Erik, > >> Several common sources of datastore stats / quota number funkiness: > >> 1) The numbers are not updated in real time. Sometimes it can > >> take a day for the numbers to get updated. > >> 2) Because of 1, if you are using a session library (or something > >> similar) that create lots of temporary entities the numbers can > >> 'appear' to be out-of-sync. > > >> I assume when you say switched to explicit indexes, you mean you > >> explicitly disabled indexing of any properties that do not need > >> indexes? > > >> Also, you might want to star issue 2740. > >> http://code.google.com/p/googleappengine/issues/detail?id=2740 > > >> Robert > > >> On Sat, Nov 6, 2010 at 19:29, Erik wrote: > > >> > Hello all, > > >> > I have been uploading a dataset which is composed of 100MB of CSV > >> > values. During the process of uploading with bulkloader, which never > >> > completed, the datastore expanded to consume 2GB of usage. I decided > >> > to explicitly index my data and clear the datastore before a re-import > >> > using the blobstore/mapreduce method. However, after clearing the > >> > datastore there is still a persistent .37GB of data usage remaining, > >> > while datastore statistics say 53KB is being consumed by 174 objects, > >> > and nothing shows in datastore viewer. > > >> > Any ideas on how I can recover the .37GB of quota which is > >> > mysteriously being used in my empty datastore? > > >> > Many thanks, > >> > -Erik > > >> > -- > >> > You received this message because you are subscribed to the Google > >> > Groups "Google App Engine" group. > >> > To post to this group, send email to google-appeng...@googlegroups.com. > >> > To unsubscribe from this group, send email to > >> > google-appengine+unsubscr...@googlegroups.com. > >> > For more options, visit this group > >> > athttp://groups.google.com/group/google-appengine?hl=en. > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to google-appeng...@googlegroups.com. > > To unsubscribe from this group, send email to > > google-appengine+unsubscr...@googlegroups.com. > > For more options, visit this group > > athttp://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Storage Quota issue w/ Total Stored Data
That looks strange to me. It looks like the number of indexed properties you have is reasonable. Could you be reading and writing entities at a high rate? Did you have more entities before, and the updater is just running slowly? In the meantime, I'd enable billing to get around this limit. -- Ikai Lan Developer Programs Engineer, Google App Engine Blogger: http://googleappengine.blogspot.com Reddit: http://www.reddit.com/r/appengine Twitter: http://twitter.com/app_engine On Tue, Nov 9, 2010 at 7:51 AM, dflorey wrote: > Hi, > can you check this app-id: zracontacts > It has 6MB of stored data, but 100% quota on the datastore :-( > > Daniel > > On Sep 21, 4:23 pm, "Ikai Lan (Google)" > > > > wrote: > > No, the storage quota is not tied to Gmail or Docs. It could be updating > > slowly. Datastore statistics are updated asynchronously, but the > dashboard > > should be updating. What's your application ID? > > > > -- > > Ikai Lan > > Developer Programs Engineer, Google App Engine > > Blogger:http://googleappengine.blogspot.com > > Reddit:http://www.reddit.com/r/appengine > > Twitter:http://twitter.com/app_engine > > > > > > > > > > > > > > > > On Mon, Sep 20, 2010 at 5:41 PM, brianl wrote: > > > Yesterday I nearly hit the daily Storage Quota for Total Stored Data, > > > was at .93 GB. Using the Datastore Viewer I deleted all of my > > > entities yesterday. Today I was expecting the Total Stored Data quota > > > to be at 0, but it's still at .93 GB. > > > > > Wondering what's is going on? The GAE storage quota isn't tied to > > > storage amounts w/ Gmail and Google Docs? > > > > > -- > > > You received this message because you are subscribed to the Google > Groups > > > "Google App Engine" group. > > > To post to this group, send email to google-appengine@googlegroups.com > . > > > To unsubscribe from this group, send email to > > > google-appengine+unsubscr...@googlegroups.com e...@googlegroups.com> > > > . > > > For more options, visit this group at > > >http://groups.google.com/group/google-appengine?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Index problem with GAE for Java
Using Java and JDO. The package path is needed... fails without that. As I said earlier, it will work if I remove the where clause (or use a different property) so I am able to connect to the datastore and pull data. I can even look at the data from the Admin console so it is there and accessible. I just can't seem to get some query related indexes to work. I've been all OVER the Google docs researching this... it's a bit convoluted and you have to piece information together from several places. I originally didn't want to use either JDO or JPA, but am using the open source geomodel software to do a proximity search on geo- coordinates in the data and that code requires JDO. Frustrating that something so simple can be so difficult. Thanks for your ideas. On Nov 9, 1:59 pm, Jeff Schwartz wrote: > Should have asked what language you are using (Java or Python) as well as > if you are using jdo/jpa etc. > > The entity type you are querying - myPackage.HospitalData - is the > 'myPackage' part a package name? If so, why? The datastore doesn't know > about packages, it only knows about Entity types which are named. > > When you go into either production or dev console can you see the entities > you are querying for? The name they are listed under is their entity type > and that is the name you should use in your queries. > > Google's datastore docs are very good & provide a lot of information about > querying & indexes. > > If you are using JPA/JDO (that's assuming you are using Java) along with > datanucleus may I suggest you don't because the datastore is not SQL which > these 2 technologies are really geared towards and only serve IMO at least > to confuse the issue. Look into replacing it with a datastore centric ORM. I > personally use Objectify & I swear by it. > > Jeff > > > > On Tue, Nov 9, 2010 at 2:30 PM, Rmac wrote: > > Nope, that throws a different error: Identifier expected at character > > 1 in "*" > > > On Nov 9, 10:43 am, Jeff Schwartz wrote: > > > Try: > > > > select * from HospitalData where Version = '1' order by Hospital_Name asc > > > > Jeff > > > > On Tue, Nov 9, 2010 at 11:16 AM, Rmac wrote: > > > > Thanks Jeff... here you go... > > > > > *The string query supplied to the Query: * > > > > > select from myPackage.HospitalData where Version == '1' order by > > Hospital_Name asc > > > > > *The GAE response:* > > > > no matching index found.. > ancestor="false" source="manual"> > > > > > > > > > > > > > > > > > *Those entries are in the datastore-indexes.xml file and show up on the > > admin console as below:* > > > > HospitalData > > > > Version ▲,Hospital_Name▲,City▲,State▲,ZIP_Code▲,Hospital_Type▲ > > > > > If I leave off the "where Version == '1'" the query works (or if I use > > Hospital_Name in the where instead). Currently, all entities have a > > property value of 1 for Version. This shouldn't be so difficult to figure > > out for a simple example like this! > > > > > Thanks for any feedback. > > > > > -- > > > > You received this message because you are subscribed to the Google > > Groups > > > > "Google App Engine" group. > > > > To post to this group, send email to google-appengine@googlegroups.com > > . > > > > To unsubscribe from this group, send email to > > > > google-appengine+unsubscr...@googlegroups.com > > > > > > > . > > > > For more options, visit this group at > > > >http://groups.google.com/group/google-appengine?hl=en. > > > > -- > > > Jeff > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to google-appeng...@googlegroups.com. > > To unsubscribe from this group, send email to > > google-appengine+unsubscr...@googlegroups.com > > . > > For more options, visit this group at > >http://groups.google.com/group/google-appengine?hl=en. > > -- > Jeff -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Architecture approaches for puts
I would like some feedback about pluses / minuses for handling new records. Currently I need to optimize how the client request handler processes new entity put()s. Several custom indices for the model are used, so puts run too close to the 1,000 ms limit (were running over the limit prior to Nov. 6th maintenance – thanks Google). The entities are written with unique integer key values. Integers are generated using Google’s recommended sharded process. Client currently POSTs a new record to the GAE handler. If handler does not send back a successful response, client will retry POST “n” times (at least twice, but possibly more). Continued failures past “n” will prompt user that record could not be created, saves data locally, and asks user to try later. Planned new process will use Task Queue. 1) Client POSTs new entity data to the handler. At this point, user sees a dialog box saying record is being written. 2) Handler will use the shards to generate the next integer value for the key. 3) Handler sets up a task queue with the new key value and record data, and responds back to the client with they key value. 4) Client receives key value back from handler, and changes to inform user that record write is being confirmed on the server (or as before retries entire POST if response is an error code). 5) Client waits a second or two (for task queue to finish), then issues a GET to the handler to read the new record using the key value. 6) Handler does a simple key value read of the new record. Responds back to client either with found or not found status. 7) If client gets found response, then we are done. If not found, or error response client will wait a few seconds, and issue another GET. 7) If after “n” tries, no GET yields a successful read, then client informs user that record could not be written, and “please try again in a few minutes” (saving new record data locally). I know this is not ideal, but believe it is a valid, given GAE’s limitations, as an approach to minimize lost writes. Would very much appreciate feedback. I should note that the imposition of a few seconds delay while writing the record should not be an issue given it is a single transaction at the end of a previous creative process which has engaged user for several minutes. Also, we do not use logic that cannot handle gaps (missing) integer values in the model's key values. TIA, stevep -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Index problem with GAE for Java
Should have asked what language you are using (Java or Python) as well as if you are using jdo/jpa etc. The entity type you are querying - myPackage.HospitalData - is the 'myPackage' part a package name? If so, why? The datastore doesn't know about packages, it only knows about Entity types which are named. When you go into either production or dev console can you see the entities you are querying for? The name they are listed under is their entity type and that is the name you should use in your queries. Google's datastore docs are very good & provide a lot of information about querying & indexes. If you are using JPA/JDO (that's assuming you are using Java) along with datanucleus may I suggest you don't because the datastore is not SQL which these 2 technologies are really geared towards and only serve IMO at least to confuse the issue. Look into replacing it with a datastore centric ORM. I personally use Objectify & I swear by it. Jeff On Tue, Nov 9, 2010 at 2:30 PM, Rmac wrote: > Nope, that throws a different error: Identifier expected at character > 1 in "*" > > On Nov 9, 10:43 am, Jeff Schwartz wrote: > > Try: > > > > select * from HospitalData where Version = '1' order by Hospital_Name asc > > > > Jeff > > > > > > > > On Tue, Nov 9, 2010 at 11:16 AM, Rmac wrote: > > > Thanks Jeff... here you go... > > > > > *The string query supplied to the Query: * > > > > > select from myPackage.HospitalData where Version == '1' order by > Hospital_Name asc > > > > > *The GAE response:* > > > no matching index found.. ancestor="false" source="manual"> > > > > > > > > > > > > > > *Those entries are in the datastore-indexes.xml file and show up on the > admin console as below:* > > > HospitalData > > > Version ▲,Hospital_Name▲,City▲,State▲,ZIP_Code▲,Hospital_Type▲ > > > > > If I leave off the "where Version == '1'" the query works (or if I use > Hospital_Name in the where instead). Currently, all entities have a > property value of 1 for Version. This shouldn't be so difficult to figure > out for a simple example like this! > > > > > Thanks for any feedback. > > > > > -- > > > You received this message because you are subscribed to the Google > Groups > > > "Google App Engine" group. > > > To post to this group, send email to google-appengine@googlegroups.com > . > > > To unsubscribe from this group, send email to > > > google-appengine+unsubscr...@googlegroups.com > > > > > > . > > > For more options, visit this group at > > >http://groups.google.com/group/google-appengine?hl=en. > > > > -- > > Jeff > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- Jeff -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Mapper problem
Looking back at my MapReduce status page, things started acting up about 1.5 hours ago, but were running fine for at least the 4-5 hours before that. On Nov 10, 4:14 am, songs wrote: > Hi, > > I'm running into a weird issue with my mappers. If I kick them off > manually, they run fine. When they get kicked off from cron, I get > the errors below in my logs and they're stuck retrying in the task > queue. > > === > Property mapreduce_spec must be convertible to a 'google.appengine.ext.mapreduce.model.MapreduceSpec'> instance > () > === > > I thought this had been running fine for weeks and I haven't touched > anything related to the mapper creation/definition/kick-off stuff. > > Any ideas? > > Thanks, > Steve -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Index problem with GAE for Java
Nope, that throws a different error: Identifier expected at character 1 in "*" On Nov 9, 10:43 am, Jeff Schwartz wrote: > Try: > > select * from HospitalData where Version = '1' order by Hospital_Name asc > > Jeff > > > > On Tue, Nov 9, 2010 at 11:16 AM, Rmac wrote: > > Thanks Jeff... here you go... > > > *The string query supplied to the Query: * > > > select from myPackage.HospitalData where Version == '1' order by > > Hospital_Name asc > > > *The GAE response:* > > no matching index found.. > ancestor="false" source="manual"> > > > > > > > > > *Those entries are in the datastore-indexes.xml file and show up on the > > admin console as below:* > > HospitalData > > Version ▲,Hospital_Name▲,City▲,State▲,ZIP_Code▲,Hospital_Type▲ > > > If I leave off the "where Version == '1'" the query works (or if I use > > Hospital_Name in the where instead). Currently, all entities have a > > property value of 1 for Version. This shouldn't be so difficult to figure > > out for a simple example like this! > > > Thanks for any feedback. > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to google-appeng...@googlegroups.com. > > To unsubscribe from this group, send email to > > google-appengine+unsubscr...@googlegroups.com > > . > > For more options, visit this group at > >http://groups.google.com/group/google-appengine?hl=en. > > -- > Jeff -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Problems pointing my GoDaddy domain to App Engine (no other posts have been able to help!)
You only need follow these steps: 1. Go to Google Apps to register your domain and verify it. 2. Go to Google App Engine dashboard - Application Settings, click "Add domain" to bind your domain (without www). 3. You'll be taken to Google Apps, add a url such as www.yourdomain.com, and set its CNAME to ghs.google.com. 4. Wait for DNS updating. During this time, you can edit your hosts file to point www.yourdomain.com to ghs.google.com, and check if it works. -- keakon On Wed, Nov 10, 2010 at 12:00 AM, Kevin M wrote: > So I have a domain hosted on GoDaddy.com and I just set up Google App > Engine and Google Apps. I registered my new domain with Google Apps so > that I can see it in my dashboard and I believe I did what I needed to > do to verify I'm the owner of the domain. I added the CNAME entry in > GoDaddy for www to point to ghs.google.com and removed all others. I > set up the MX records I saw in another post so at least my email is > going where I expect it to. But, my new URL is not going to my google > app engine application for some reason. Do I need to change the Name > Servers from GoDaddy name servers to Google name servers? Do I need > A(Host) entries? I'm confused. Please help! > > Sorry if this is double posted... > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Mapper problem
Hi, I'm running into a weird issue with my mappers. If I kick them off manually, they run fine. When they get kicked off from cron, I get the errors below in my logs and they're stuck retrying in the task queue. === Property mapreduce_spec must be convertible to a instance () === I thought this had been running fine for weeks and I haven't touched anything related to the mapper creation/definition/kick-off stuff. Any ideas? Thanks, Steve -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Datastore and File Upload Size
Which URL do you use for uploading your file ?
I'm supossing that by "Anything beyond 1MB doesn't reach the servlet",
you mean : An error occur like an "RequestTooLargeException" is
throwed.
To upload your file into the Blobstore service, you have to generate a
temporary upload URL using
blobstoreService.createUploadUrl("/upload") (this method will generate
a url like this : http//yourapp/_ah/upload/1A4343HAB3242C )
When the upload is complete, you'll be notified on the /upload (url
used in createUploadUrl method).
cf
http://code.google.com/intl/fr/appengine/docs/java/blobstore/overview.html#Uploading_a_Blob
Regards
On 16 oct, 04:46, MasterGaurav wrote:
> Hi,
>
> I trying to upload file onto Blob Store. I am able to upload files
> only upto 1MB. Anything beyond 1MB doesn't reach the servlet.
>
> Is it a hard limit or am I doing something wrong?
>
> I'm using simple code...
>
> ---
> BlobstoreService blobSvc =
> BlobstoreServiceFactory.getBlobstoreService();
> Map blobs = blobSvc.getUploadedBlobs(req);
> ---
>
> -Gauravwww.mastergaurav.com
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Unknown error 503 Occurs on appengine Python
I am a complete newbie to appengine , python and these api so please
pardon me for errors
I have used google spreadsheet api and google doc list api on
appengine using python
part of code causing error is specified below.If further details are
required i m ready to share it
for i in range(int(total_number_of_boxes)):
boxname='strText'+str(i+1) classname=self.request.get(boxname)
attendancesheet =
self.gd_client.Copy(self._Retrieve_Document(self.standard_attendance_register),
classname) self._Move_Documents(attendancesheet,schoolfolder) t_key =
self._Retrieve_Key_Spreadsheet(attendancesheet) s_key =
t_key.partition('A') feed = self.gs_client.GetWorksheetsFeed(s_key[2])
w_id = self._Retrieve_Key_Firstworksheet(feed.entry[0])
self._CellsUpdateAction(row, col, classname)
self._CellsUpdateAction(row, col+1, s_key[2])
self._CellsUpdateAction(row, col+2, w_id) self._CellsUpdateAction(row,
col+3, admin_name) self._CellsUpdateAction(row, col+4, admin_email)
T_name='t'+str(counter)
e=Directaccess(teacherusername=T_name,teacherpassword=T_password,teacherspreadsdheetid=s_key,teacherworksheetid=w_id)
self._Addnewacl_Permission(attendancesheet,admin_email,'user','writer')
row=row+1 column=1 counter=counter+1
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Unknown Error
I am using google spreadsheet api version 1 and google document list api on appengine using python Please suggest what is causing this error Status: 500 Internal Server Error Content-Type: text/html; charset=utf-8 Cache-Control: no-cache Expires: Fri, 01 Jan 1990 00:00:00 GMT Content-Length: 1227 Traceback (most recent call last): File "/base/python_runtime/python_lib/versions/1/google/appengine/ ext/webapp/__init__.py", line 513, in __call__ handler.post(*groups) File "/base/data/home/apps/testcomputerspectra/1.346090215332055849/ intialise.py", line 152, in post attendancesheet = self.gd_client.Copy(self._Retrieve_Document(self.standard_attendance_register), classname) File "/base/data/home/apps/testcomputerspectra/1.346090215332055849/ gdata/docs/client.py", line 322, in copy return self.post(entry, DOCLIST_FEED_URI, auth_token=auth_token, **kwargs) File "/base/data/home/apps/testcomputerspectra/1.346090215332055849/ gdata/client.py", line 685, in post desired_class=desired_class, **kwargs) File "/base/data/home/apps/testcomputerspectra/1.346090215332055849/ gdata/client.py", line 320, in request RequestError) RequestError: Server responded with: 503, GData ServiceUnavailableException An unknown error has occurred. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Problems pointing my GoDaddy domain to App Engine (no other posts have been able to help!)
So I have a domain hosted on GoDaddy.com and I just set up Google App Engine and Google Apps. I registered my new domain with Google Apps so that I can see it in my dashboard and I believe I did what I needed to do to verify I'm the owner of the domain. I added the CNAME entry in GoDaddy for www to point to ghs.google.com and removed all others. I set up the MX records I saw in another post so at least my email is going where I expect it to. But, my new URL is not going to my google app engine application for some reason. Do I need to change the Name Servers from GoDaddy name servers to Google name servers? Do I need A(Host) entries? I'm confused. Please help! Sorry if this is double posted... -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Keys only queries do not support IN filters?
When I run this query:
Article.all(keys_only=True).filter('keywords IN', keywords).fetch(10)
It raise a BadQueryError: Keys only queries do not support IN filters.
http://code.google.com/p/googleappengine/source/browse/trunk/python/google/appengine/ext/db/__init__.py#2197
But I can't find this restriction in document:
http://code.google.com/intl/en/appengine/docs/python/datastore/queriesandindexes.html#Restrictions_on_Queries
--
keakon
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Index problem with GAE for Java
Thanks Jeff... here you go... The string query supplied to the Query: select from myPackage.HospitalData where Version == '1' order by Hospital_Name asc The GAE response: no matching index found.. Those entries are in the datastore-indexes.xml file and show up on the admin console as below: HospitalData Version ▲,Hospital_Name▲,City▲,State▲,ZIP_Code▲,Hospital_Type▲ If I leave off the "where Version == '1'" the query works (or if I use Hospital_Name in the where instead). Currently, all entities have a property value of 1 for Version. This shouldn't be so difficult to figure out for a simple example like this! Thanks for any feedback. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Only one property per query may have inequality filters?
http://code.google.com/appengine/docs/python/datastore/queriesandindexes.html#Restrictions_on_Queries
I am just guessing about your application, but I would drop the user
filter from the query and handle it in your app code. Or you can run
two queries and merge the results yourself.
Robert
On Tue, Nov 9, 2010 at 11:39, ajaxer wrote:
> BadFilterError: BadFilterError: invalid filter: Only one property per
> query may have inequality filters (<=, >=, <, >)..
>
> my code:
>
> records = QMessage.all().filter('room = ', room).filter('isPrivate
> =',
> False).filter('fromUser !=', user).filter('time > ', time)
>
> how can i solve this problem?
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appeng...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Index problem with GAE for Java
Try: select * from HospitalData where Version = '1' order by Hospital_Name asc Jeff On Tue, Nov 9, 2010 at 11:16 AM, Rmac wrote: > Thanks Jeff... here you go... > > *The string query supplied to the Query: * > > select from myPackage.HospitalData where Version == '1' order by > Hospital_Name asc > > *The GAE response:* > no matching index found.. ancestor="false" source="manual"> > > > > > *Those entries are in the datastore-indexes.xml file and show up on the admin > console as below:* > HospitalData > Version ▲,Hospital_Name▲,City▲,State▲,ZIP_Code▲,Hospital_Type▲ > > > > If I leave off the "where Version == '1'" the query works (or if I use > Hospital_Name in the where instead). Currently, all entities have a property > value of 1 for Version. This shouldn't be so difficult to figure out for a > simple example like this! > > Thanks for any feedback. > > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- Jeff -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Only one property per query may have inequality filters?
BadFilterError: BadFilterError: invalid filter: Only one property per
query may have inequality filters (<=, >=, <, >)..
my code:
records = QMessage.all().filter('room = ', room).filter('isPrivate
=',
False).filter('fromUser !=', user).filter('time > ', time)
how can i solve this problem?
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Mystery data usage
Hi Erik, As far as I know the persistence manager would not create any extra entities. It sounds like you might want to explicitly disable indexing on any fields you will not be querying on. See the 'Properties that Aren't Indexed' section on the 'Queries and Indexes' page. http://code.google.com/appengine/docs/java/datastore/queriesandindexes.html You might also want to use shorter kind and property names. Kind and property names are stored with every entity, so it can add up pretty fast. Robert On Tue, Nov 9, 2010 at 08:05, Erik wrote: > Hi Robert, > > Thanks for the response, I did wait several days before sending that > message, but shortly afterwards the quota cleared to zero. I am using > a persistence manager with jdo, would these create temporary > entities? > > I need to explicitly index my keys in descending order for mapreduce, > and thought I needed to index my collections but as I am not > performing any queries don't think that is necessary. My persistent > objects consist of a single property with type of long collection, > entities are fetched by id. > > My latest 30MB dataset has already grown to consume 710MB, that is an > enormous increase. Datastore statistics say only 131MB is being used > with 71% of that as metadata. > > Any other ideas? > > Thanks, > -Erik > > On Nov 8, 1:03 pm, Robert Kluin wrote: >> Hi Erik, >> Several common sources of datastore stats / quota number funkiness: >> 1) The numbers are not updated in real time. Sometimes it can >> take a day for the numbers to get updated. >> 2) Because of 1, if you are using a session library (or something >> similar) that create lots of temporary entities the numbers can >> 'appear' to be out-of-sync. >> >> I assume when you say switched to explicit indexes, you mean you >> explicitly disabled indexing of any properties that do not need >> indexes? >> >> Also, you might want to star issue 2740. >> http://code.google.com/p/googleappengine/issues/detail?id=2740 >> >> Robert >> >> On Sat, Nov 6, 2010 at 19:29, Erik wrote: >> >> > Hello all, >> >> > I have been uploading a dataset which is composed of 100MB of CSV >> > values. During the process of uploading with bulkloader, which never >> > completed, the datastore expanded to consume 2GB of usage. I decided >> > to explicitly index my data and clear the datastore before a re-import >> > using the blobstore/mapreduce method. However, after clearing the >> > datastore there is still a persistent .37GB of data usage remaining, >> > while datastore statistics say 53KB is being consumed by 174 objects, >> > and nothing shows in datastore viewer. >> >> > Any ideas on how I can recover the .37GB of quota which is >> > mysteriously being used in my empty datastore? >> >> > Many thanks, >> > -Erik >> >> > -- >> > You received this message because you are subscribed to the Google Groups >> > "Google App Engine" group. >> > To post to this group, send email to google-appeng...@googlegroups.com. >> > To unsubscribe from this group, send email to >> > google-appengine+unsubscr...@googlegroups.com. >> > For more options, visit this group >> > athttp://groups.google.com/group/google-appengine?hl=en. >> >> > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Storage Quota issue w/ Total Stored Data
Hi, can you check this app-id: zracontacts It has 6MB of stored data, but 100% quota on the datastore :-( Daniel On Sep 21, 4:23 pm, "Ikai Lan (Google)" wrote: > No, the storage quota is not tied to Gmail or Docs. It could be updating > slowly. Datastore statistics are updated asynchronously, but the dashboard > should be updating. What's your application ID? > > -- > Ikai Lan > Developer Programs Engineer, Google App Engine > Blogger:http://googleappengine.blogspot.com > Reddit:http://www.reddit.com/r/appengine > Twitter:http://twitter.com/app_engine > > > > > > > > On Mon, Sep 20, 2010 at 5:41 PM, brianl wrote: > > Yesterday I nearly hit the daily Storage Quota for Total Stored Data, > > was at .93 GB. Using the Datastore Viewer I deleted all of my > > entities yesterday. Today I was expecting the Total Stored Data quota > > to be at 0, but it's still at .93 GB. > > > Wondering what's is going on? The GAE storage quota isn't tied to > > storage amounts w/ Gmail and Google Docs? > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to google-appeng...@googlegroups.com. > > To unsubscribe from this group, send email to > > google-appengine+unsubscr...@googlegroups.com > e...@googlegroups.com> > > . > > For more options, visit this group at > >http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Early Christmas Present from Google?
Are you able to share any details on what exactly happened behind the scenes? On 8 Nov., 20:15, "Ikai Lan (Google)" wrote: > Hey everyone, > > Thanks for the great feedback! Going forward, we're cautiously optimistic > about the datastore improvements. You should continue to code using best > practices - that is, don't assume datastore get() will always run in 5ms and > place make 200 get calls serially. We'll be watching the performance closely > in the meantime. > > -- > Ikai Lan > Developer Programs Engineer, Google App Engine > Blogger:http://googleappengine.blogspot.com > Reddit:http://www.reddit.com/r/appengine > Twitter:http://twitter.com/app_engine > > > > > > > > On Sun, Nov 7, 2010 at 4:23 PM, Eugene D wrote: > > Blazin'! Great work! > > > On Nov 8, 1:14 am, Tom Wu wrote: > > > +1 > > > > Great datastore improve. > > > > 2010/11/8 Marc Provost > > > > > +1 > > > > > Thanks googlers! This is awesome. > > > > > On Nov 7, 4:58 pm, nickmilon wrote: > > > > > + 1 > > > > > Impressive performance gains - congratulation to Google and App > > Engine > > > > > team. > > > > > Lets hope current performance will be a benchmark for the future. > > > > > > On Nov 7, 12:17 am, Greg wrote: > > > > > > > Check out the datastore stats after today's maintenance... > > >http://code.google.com/status/appengine/detail/datastore/2010/11/06#a. > > > > .. > > > > > -- > > > > You received this message because you are subscribed to the Google > > Groups > > > > "Google App Engine" group. > > > > To post to this group, send email to google-appengine@googlegroups.com > > . > > > > To unsubscribe from this group, send email to > > > > google-appengine+unsubscr...@googlegroups.com > > > e...@googlegroups.com> > e...@googlegroups.com> > > > > . > > > > For more options, visit this group at > > > >http://groups.google.com/group/google-appengine?hl=en. > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to google-appeng...@googlegroups.com. > > To unsubscribe from this group, send email to > > google-appengine+unsubscr...@googlegroups.com > e...@googlegroups.com> > > . > > For more options, visit this group at > >http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: App Gallery no longer available
Thanks for the info. In terms of checking the Reddit page for updates, I don't see any article there with App Gallery in the title. Is there some announcement about dropping it there? If so, would someone please post a link to it? Dropping the App Gallery isn't big deal to me, as I hadn't looked at it in weeks or probably months. If I knew it was going away, however, I'm sure I would have taken one last look at my app pages. It is a little disconcerting to find out from other developers that Google changed something. A short note to this newsgroup would have been nice. On Nov 8, 2:04 pm, "Ikai Lan (Google)" wrote: > Hey guys, > > Apologies for the delayed responses on this thread. The App Gallery was > something that was really awesome in App Engine's infancy, but App Engine > has since matured a bit and it's really not the best way to showcase App > Engine's capabilities. We've decommissioned it. If you're interested in App > Engine happenings, you'll definitely want to regularly check out our Reddit > page: > > http://www.reddit.com/r/appengine > > We should have better communicated our intents with this page. We're > unlikely to bring the App Gallery back. I'd personally love to see an App > Engine application that serves the function the App Gallery served, maybe > one that even uses the Prediction API (http://code.google.com/apis/predict/) > to sort out the spam and do rankings. If this is a project community members > are interested in taking on, let me know and I'll get you access to the > prediction API. > > On a related note, due to the changes coming in Google Groups, we will be > porting the Open Source projects page as well as the "Will it Play" Java > frameworks page to our Project Hosting site: > > http://code.google.com/p/googleappengine/w/list > > -- > Ikai Lan > Developer Programs Engineer, Google App Engine > Blogger:http://googleappengine.blogspot.com > Reddit:http://www.reddit.com/r/appengine > Twitter:http://twitter.com/app_engine > > > > > > > > On Mon, Nov 8, 2010 at 1:03 AM, Tonny <12br...@gmail.com> wrote: > > I you wan't to emphasize the importance of this thread, please mark is > > a favorite - I tink Google Engineers are monitoring those for > > importance. > > > On Nov 8, 8:53 am, Julian Namaro wrote: > > > I think a specific "App Engine" tag in a general Web App Gallery would > > > make more sense than what we had before. > > > After all, what you can do with App Engine is pretty much the same > > > than what you can do with other technology stacks, and much of what > > > the user experiences is the common client-side part anyway. > > > > On Nov 7, 6:21 am, nickmilon wrote: > > > > > Also the gallery was a kind of museum where you could see how GAE > > > > appls have been evolved in time from the early days circa April 2008 > > > > till now, always advancing using new features of the platform or as > > > > the features were understood by the developers. > > > > Now if they even do substitute this with a new site I doubt early > > > > developers will go and resubmit their work. > > > > So I only hope Google reconsiders this, fixes the problems related to > > > > spam and classification of appls and restore the site. > > > > > On Nov 6, 5:17 am, Julian Namaro wrote: > > > > > > An App Gallery is a great idea but Google's one looked like a weekend > > > > > project thrown out in the wild and was pretty useless. > > > > > There was a lot of spam, unhelpful reviews, and occasionally you > > could > > > > > see Desktop software (running only on Windows, not even cross- > > > > > plateform) with a whole bunch of 5-stars ratings in this "App Engine" > > > > > gallery. > > > > > > My guess is that they're deprecating the gallery in favor of the > > > > > Chrome Web Store. Hope this will be a more serious attempt :] > > > > > > On Nov 5, 6:14 am, MLTrim wrote: > > > > > > > Do you know why App Gallery is no longer available? > > > > > > >http://appgallery.appspot.com > > > > > > > thanks > > > > > > Michele > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to google-appeng...@googlegroups.com. > > To unsubscribe from this group, send email to > > google-appengine+unsubscr...@googlegroups.com > e...@googlegroups.com> > > . > > For more options, visit this group at > >http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: deploy time out...
Ive been having lots of problems deploying recently, rather annoyingly it keeps telling me that my web.xml is invalid, even though ive not touched it since last time, its not until I look deeper in the logs that i found it was actually a timeout issue. This has been happening a lot recently. Usually if I just keep trying and trying eventually it goes through (as it has while ive been writing this!) Just thought Id raise it again, as Im not sure if its being picked up anywhere On Sep 28, 8:23 pm, "Ikai Lan (Google)" wrote: > We're aware of the issues around deploy. We're working on resolving them. > > -- > Ikai Lan > Developer Programs Engineer, Google App Engine > Blogger:http://googleappengine.blogspot.com > Reddit:http://www.reddit.com/r/appengine > Twitter:http://twitter.com/app_engine > > On Tue, Sep 28, 2010 at 10:35 AM, techboy wrote: > > Still slow today. I've tried to deploy 4 or 5 times. No success yet. > > > On Sep 27, 2:26 pm, Francois Masurel wrote: > > > Yep, it's quite slow today for me too... > > > > On 27 sep, 22:22, oth wrote: > > > > > Unfortunately yes, waiting is the only option. > > > > > Thanks > > > > > On Sep 26, 3:16 pm, Scott wrote: > > > > > > I'm seeing the same problem today, since about 3 pm EDT. > > > > > > Is our only option to wait? > > > > > > ~ Scott > > > > > > On Sep 24, 11:01 pm, Jason King wrote: > > > > > > > I've been having problems deploying my app. Of course, as I write > > > > > > this, it finally deployed. My two previous attempts today resulted > > in > > > > > > an apparent exception after about 10 mins of failing to deploy. > > Has > > > > > > there been an issue today or is this just happening to me... > > > > > > > Cloned 700 files. > > > > > > Deploying new version. > > > > > > Checking if new version is ready to serve. > > > > > > Will check again in 1 seconds. > > > > > > Checking if new version is ready to serve. > > > > > > Will check again in 2 seconds. > > > > > > Checking if new version is ready to serve. > > > > > > Will check again in 4 seconds. > > > > > > Checking if new version is ready to serve. > > > > > > Will check again in 8 seconds. > > > > > > Checking if new version is ready to serve. > > > > > > Will check again in 16 seconds. > > > > > > Checking if new version is ready to serve. > > > > > > Will check again in 32 seconds. > > > > > > Checking if new version is ready to serve. > > > > > > Will check again in 60 seconds. > > > > > > Checking if new version is ready to serve. > > > > > > Will check again in 60 seconds. > > > > > > Checking if new version is ready to serve. > > > > > > Will check again in 60 seconds. > > > > > > Checking if new version is ready to serve. > > > > > > Will check again in 60 seconds. > > > > > > Checking if new version is ready to serve. > > > > > > Will check again in 60 seconds. > > > > > > Checking if new version is ready to serve. > > > > > > Will check again in 60 seconds. > > > > > > Checking if new version is ready to serve. > > > > > > Will check again in 60 seconds. > > > > > > Checking if new version is ready to serve. > > > > > > Will check again in 60 seconds. > > > > > > Checking if new version is ready to serve. > > > > > > Closing update: new version is ready to start serving. > > > > > > Uploading index definitions. > > > > > > 2010-09-24 19:53:16 (Process exited with code 0) > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to google-appeng...@googlegroups.com. > > To unsubscribe from this group, send email to > > google-appengine+unsubscr...@googlegroups.com > > . > > For more options, visit this group at > >http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: deploy failures
I also receive this error : SEVERE: Received IOException parsing the input stream for my_path/war/WEB_INF//web.xml) I am using java sdk and Eclipse Helios. After I restart the eclipse it's working ok. On Nov 9, 2:47 pm, Erik wrote: > I have also been getting constant deployment errors with Java > deployment for the past couple days, seems like something is funny on > the google side, don't think my connection is flaky: > > Nov 9, 2010 7:18:59 AM > com.google.apphosting.utils.config.AbstractConfigXmlReader > getTopLevelNode > SEVERE: Received IOException parsing the input stream for /home/erik/ > workspace/wikihop/war/WEB-INF/web.xml > java.net.ConnectException: Connection timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) > ... > SEVERE: Received exception processing /home/erik/workspace/wikihop/war/ > WEB-INF/web.xml > com.google.apphosting.utils.config.AppEngineConfigException: Received > IOException parsing the input stream for /home/erik/workspace/wikihop/ > war/WEB-INF/web.xml > at > com.google.apphosting.utils.config.AbstractConfigXmlReader.getTopLevelNode( > AbstractConfigXmlReader.java: > 210) > ... > Bad configuration: Received IOException parsing the input stream for / > home/erik/workspace/wikihop/war/WEB-INF/web.xml > Caused by: Connection timed out -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Can't send mail to Google Group.
Hi everyone: I have an application that sending mail to Google Group. But since today, it can't. Is there anything changed, or I miss some mail policy? Thanks. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Can't send mail to Google Group.
Recently, I received the mail send from GAE to Google Groups. But delay about 7 hours. Daylight saving time？ On Nov 9, 9:32 pm, Toomore wrote: > Hi everyone: > > I have an application that sending mail to Google Group. But since > today, it can't. > Is there anything changed, or I miss some mail policy? > > Thanks. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Mystery data usage
Hi Robert, Thanks for the response, I did wait several days before sending that message, but shortly afterwards the quota cleared to zero. I am using a persistence manager with jdo, would these create temporary entities? I need to explicitly index my keys in descending order for mapreduce, and thought I needed to index my collections but as I am not performing any queries don't think that is necessary. My persistent objects consist of a single property with type of long collection, entities are fetched by id. My latest 30MB dataset has already grown to consume 710MB, that is an enormous increase. Datastore statistics say only 131MB is being used with 71% of that as metadata. Any other ideas? Thanks, -Erik On Nov 8, 1:03 pm, Robert Kluin wrote: > Hi Erik, > Several common sources of datastore stats / quota number funkiness: > 1) The numbers are not updated in real time. Sometimes it can > take a day for the numbers to get updated. > 2) Because of 1, if you are using a session library (or something > similar) that create lots of temporary entities the numbers can > 'appear' to be out-of-sync. > > I assume when you say switched to explicit indexes, you mean you > explicitly disabled indexing of any properties that do not need > indexes? > > Also, you might want to star issue 2740. > http://code.google.com/p/googleappengine/issues/detail?id=2740 > > Robert > > On Sat, Nov 6, 2010 at 19:29, Erik wrote: > > > Hello all, > > > I have been uploading a dataset which is composed of 100MB of CSV > > values. During the process of uploading with bulkloader, which never > > completed, the datastore expanded to consume 2GB of usage. I decided > > to explicitly index my data and clear the datastore before a re-import > > using the blobstore/mapreduce method. However, after clearing the > > datastore there is still a persistent .37GB of data usage remaining, > > while datastore statistics say 53KB is being consumed by 174 objects, > > and nothing shows in datastore viewer. > > > Any ideas on how I can recover the .37GB of quota which is > > mysteriously being used in my empty datastore? > > > Many thanks, > > -Erik > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to google-appeng...@googlegroups.com. > > To unsubscribe from this group, send email to > > google-appengine+unsubscr...@googlegroups.com. > > For more options, visit this group > > athttp://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: deploy failures
I have also been getting constant deployment errors with Java deployment for the past couple days, seems like something is funny on the google side, don't think my connection is flaky: Nov 9, 2010 7:18:59 AM com.google.apphosting.utils.config.AbstractConfigXmlReader getTopLevelNode SEVERE: Received IOException parsing the input stream for /home/erik/ workspace/wikihop/war/WEB-INF/web.xml java.net.ConnectException: Connection timed out at java.net.PlainSocketImpl.socketConnect(Native Method) ... SEVERE: Received exception processing /home/erik/workspace/wikihop/war/ WEB-INF/web.xml com.google.apphosting.utils.config.AppEngineConfigException: Received IOException parsing the input stream for /home/erik/workspace/wikihop/ war/WEB-INF/web.xml at com.google.apphosting.utils.config.AbstractConfigXmlReader.getTopLevelNode(AbstractConfigXmlReader.java: 210) ... Bad configuration: Received IOException parsing the input stream for / home/erik/workspace/wikihop/war/WEB-INF/web.xml Caused by: Connection timed out -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Index problem with GAE for Java
Code? On Mon, Nov 8, 2010 at 11:46 PM, Rmac wrote: > Hi, > > I am struggling with getting a query to work that has a simple > equality condition on one property in the where clause and one sort by > on another property. GAE responds with a "no matching index found" > error along with the index entries to put in the datastore-indexes.xml > file. I put those supplied entries in the file, redeploy, and then > later also look at the GAE administration console to see those indexes > are actually in "serving" status. But, rerunning the query results in > the same error. Does anyone have an idea how to get indexes to work > in Java GAE? > > Thanks in advance. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- Jeff -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Mail receive services during scheduled downtime
During a scheduled datastore downtime, what happens to incoming emails? If the mail receipt mechanism is disabled, that should be fine, since most senders will re-try automatically. But if the messages still get posted to /_ah/mail/ as a successful delivery, then the application has nowhere to save them. Is the only option to re-send these messages elsewhere? -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
