date:20100322

Re: [appengine-java] GWT+JDO+GAE enhancement nightmare

2010-03-22 Thread Shawn Brown

 This seems pretty basic. Can somebody wake me from this nightmare?

Sorry, been there...

If I was starting over, I'd seriously think about:

http://code.google.com/p/objectify-appengine/

for two reasons:

1) Objectify entities can be used in GWT without the need for Data
Transfer Objects.
2) it seems cold start up time is better when you take the jdo and
datanucleus stuff out. (judging by mail on this list)

There are much more knowledgeable people on this list but ... yeah I
know it's a nightmare to get it figured out in the beginning and don't
wonder if objectify isn't simpler.

Shawn

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] GWT+JDO+GAE enhancement nightmare

2010-03-22 Thread John Patterson



On 22 Mar 2010, at 13:20, Shawn Brown wrote:


1) Objectify entities can be used in GWT without the need for Data
Transfer Objects.


Twig data models also have this property but with the advantage that  
they have no dependencies on the low-level datastore - just pure  
unrestricted pojos.



--
You received this message because you are subscribed to the Google Groups Google 
App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Re: GAE 1.3.1 java.io.IOException: Truncated quoted printable data when processing a Hotmail email

2010-03-22 Thread Peter Ondruska

Something has gotten broken with recent release because my code used
to work fine with Hotmail/Yahoo sent messages. Looks like I have to go
through release notes for 1.3.1 and fix processing incoming emails.. :-
(

On Mar 21, 11:48 pm, Lucian Baciu lucianba...@gmail.com wrote:
 The mail processing API doesn't seem to work when processing a Hotmail
 email message. I get:

 java.io.IOException: Truncated quoted printable data
         at
 org.apache.geronimo.mail.util.QuotedPrintableEncoder.decodeNonspaceChar(Quo 
 tedPrintableEncoder.java:
 597)
         at
 org.apache.geronimo.mail.util.QuotedPrintableEncoder.decode(QuotedPrintable 
 Encoder.java:
 584)
         at
 org.apache.geronimo.mail.util.QuotedPrintableDecoderStream.read(QuotedPrint 
 ableDecoderStream.java:
 80)
         at
 org.apache.geronimo.mail.handlers.TextHandler.getContent(TextHandler.java:
 107)
         at javax.activation.DataSourceDataContentHandler.getContent(Unknown
 Source)
         at javax.activation.DataHandler.getContent(Unknown Source)
         at javax.mail.internet.MimeBodyPart.getContent(MimeBodyPart.java:392)

 I am using the code that should work from 
 here:http://groups.google.com/group/google-appengine-java/browse_thread/th...

 The code fails on this line: Object bp = p.getContent();

 Thanks

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Re: a question about transactional tasks

2010-03-22 Thread AJ



On Mar 21, 6:45 pm, Max Ross (Google) maxr+appeng...@google.com
wrote:
 Hi AJ,

 Your understanding is correct and I don't quite know what we meant when we
 said same level of isolation and consistency.  I'll get this sorted out.

thanks, I appreciate the reply!

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] Tweak static file without re-deploying

2010-03-22 Thread Ikai L (Google)

There's no way to just update static assets. I find that I always have
to update my HTML files whenever I update my static assets anyway. As
a safeguard against caches, it's a common practice to reference your
CSS and JavaScript files like this

script src=/something.js?${LAST-MODIFIED-TIMESTAMP}/script

Most browsers and proxies should respect cache policies, but this is
an almost foolproof way of doing it.

On Mon, Mar 22, 2010 at 1:38 PM, Steve Pritchard steve...@gmail.com wrote:
 Hi all,

 Is there any way to tweak or add a static file (such as a .css file)
 using without redeploying the whole application.  I do not mean from
 inside the servlet engine - I understand why this is not allowed.  I
 mean, does the deployer have a 'light-weight' mode that will allow it
 to 'zap'  the present deployment static file.

 It seems overkill to redeploy the whole application just to modify one
 or 2 characters in a .css file.

 Thanks,
 Steve

 --
 You received this message because you are subscribed to the Google Groups 
 Google App Engine for Java group.
 To post to this group, send email to google-appengine-j...@googlegroups.com.
 To unsubscribe from this group, send email to 
 google-appengine-java+unsubscr...@googlegroups.com.
 For more options, visit this group at 
 http://groups.google.com/group/google-appengine-java?hl=en.





-- 
Ikai Lan
Developer Programs Engineer, Google App Engine
http://googleappengine.blogspot.com | http://twitter.com/app_engine

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[google-appengine] How do you handle with Error: ReferenceProperty failed to be resolved

2010-03-22 Thread eng. Ilian Iliev

Hi all,

I`m sure lot you have faced this error and I saw some post for it but
couldn`t find some more
general way for handling it.

Iterating over the whole datastore after my app is broken doesn`t
seems to me the fastest way.
Knowing the connections between models and cascade delete/update if
some item is deleted looks like one of the ways.
The other maybe is just suppressing and logging this erros, but I
don`t know how to do it.

I`ll be glad if you share your ideas and experience with this problem.

Greetings,
Ilian Iliev

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: HELP! 7 GB of ghost data - billable but not in datastore, blobs, or versions.

2010-03-22 Thread homunq

Thanks for responding. I've starred the bug - it seems basic. More
answers below.

On Mar 21, 7:06 pm, Eli Jones eli.jo...@gmail.com wrote:
Depending on how your Models are defined (how many properties and what the
property types are there) and depending on custom indexes, 10 to 1 is
reasonable.

Even if I have essentially no custom indexes? (Only one ever, on a few
dozen entities with two integers each which handle cron housekeeping).

For my app, I aggressively set most properties to Indexed=false and I have
no custom indexes. I'm still at 2 to 1 for Datastore usage versus Size of
All Entities.

When you say datastore usage, you mean total data, right? Or is
there some other place this number is reported which I've missed?

To fully explore what is going on, you'd need to post what your Model
definitions are.. and then post what custom indexes are defined on any of
the properties.

I have no custom indexes. I have only two entities. 99.9% of the data
is in the Expando which has about 45 stringproperties (initally all
indexed, now around 30 are; many frequently empty string); 4 integer
properties; 4 dates; 4 lists (about 4-8 members typically); from 3-18
expando string lists (of 20 types total; an average of 6 lists with 2
members each); and 2 expando strings (mostly but not always present).

If you've been creating and then later on deleting custom indexes or
changing your Models around.. running vacuum indexes might help (but that's
just a shot in the dark).

I have done nothing with custom indexes, except the one miniscule
housekeeping one on a few entities. For the main entity kind, is there
any way to delete a default index without re-touching all the entities
one-by-one?

On Sun, Mar 21, 2010 at 7:19 PM, homunq jameson.qu...@gmail.com wrote:
My app is showing only 739 MB of datastore data, only 1 version
(150MB), no blobs, a tiny amount of index and memcache use (2K in
each case) - and yet 7.7GB of billable data! What the heck is going
on? How can I fix it?

Last updated Total number of entities Size of all entities
3:27:42 ago 279,689 739 MBytes

Total Stored Data $0.005/GByte-day 86% 7.70 of
9.00 GBytes
$0.04 / $0.04

This situation has persisted for well over 24 hours now, it's not just
a figment of the update period. Also, there's negligible non-default
indexes, and probably a total of around 2 GB if you count all my
uploaded code versions ever (but since they all had the same version
string, they should have overwritten). I probably have a fair amount
of data in the logs - I have a number of cron jobs, collectively they
run just under twice a minute - but supposedly logs are not billable
data.

This is an app I'm developing for a client. Until this issue is fixed,
I'm certainly not going to bill for my work, which is otherwise done.
So I'm anxious to fix this ASAP. If there's somewhere else I should/
could be taking this question (aside from IRC, where I've brought it
up twice), I'd be happy to learn it. If there's any further info which
could help resolve this, I'd be happy to share it, too. (My app id is
vulahealth)

Thanks,
Jameson

(Second post, I think the first post was moderated into oblivion - it
was my first post on this list.)

--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com
.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.

--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: How to call a URL of your own application?

2010-03-22 Thread dhruvbird

On Mar 22, 6:30 am, Eli Jones eli.jo...@gmail.com wrote:
Depends on how you want the code to work..

If you just want to send of work to get processed by those other handlers,
then firing off a Task (either with Task Queue or Deferred Task) would be
perfect for that.

Actually, this is a task that will create a new task in the task
queue, so I didn't want to create 2 tasks when 1 is what is required.
The thing is that that task has a Name, and the current caller doesn't
know what that name is. I need to be able to call some other function
to determine it's name and then create that task.

Regards,
-Dhruv.

http://code.google.com/appengine/docs/python/taskqueue/overview.htmlhttp://code.google.com/appengine/articles/deferred.html

On Sun, Mar 21, 2010 at 1:32 PM, dhruvbird dhruvb...@gmail.com wrote:
Hello,
I would like to call a URL of my own application. How can I go about
doing that w/o using URL fetch? One option would be to manually
construct a request object and call the function depending upon the
URL mapping table. However, is there a way to do this w/o that mess
and w/o using URL fetch (which will cost me a request)?

for eg: If I have these URLs:
/mail/send/
/chat/send/
/home/

If I want to call them from within the handler of /home/, how can I do
that?

Regards,
-Dhruv.

--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] ~7 GB of ghost data???

2010-03-22 Thread Brett Shelley

Are you storing anything in the Blob Store? If so, well, deleting Blobs
from the AppSpot console does not work. Perhaps the problem is systemic.
But, if it helps revenue, then why fix it?

-Brett

On Mon, Mar 22, 2010 at 5:26 AM, Robert Kluin robert.kl...@gmail.comwrote:

I do not think they charge separately for backups and replicas. I am
pretty sure they have stated before that the cost of those services is
already included in the storage charge.

I can not find the post that referenced this though.

On Mon, Mar 22, 2010 at 12:21 PM, Tom Wu service.g2...@gmail.com wrote:
GAE is cluster which include master slaver, backup system... etc..
So the quota is much bigger than your local file.

2010/3/22 杨浩 skzr@gmail.com

I have the same about thaTt!
my entities:167MB but total's 1G,It's over quota!
the other 833MB it's meta datas!
It's very confused!

2010/3/21 homunq jameson.qu...@gmail.com

Something is wrong. My app is showing with 7.42GB of total stored
data, but only 615 MB of datastore. There is only one version string
uploaded, which is almost 150MB, and nothing in the blobstore. This
discrepancy has been getting worse - several hours ago (longer than
the period since datastore statistics were updated, if you're
wondering), there were the same 615 MB in the datastore, and only
3.09GB of total stored data. (at that time, my theory was that it
was old uploads of tweaks to the same version - but the numbers have
gone far, far beyond that explanation now.) It's not some exploding
index; the only non-default index I have is on an entity type with
just 33 entities.

Here's the line from my dashboard:
Total Stored Data$0.005/GByte-day82% 7.42
of
9.00 GBytes
$0.04 / $0.04

And here is the word from my datastore statistics:
Last updatedTotal number of entitiesSize of all entities
1:32:13 ago 232,867 615 MBytes
(metadata 11%, if that matters)

Please, can someone help me figure out this issue? I'd be happy to
share any info or code which would help track this down. My app id is
vulahealth.

--
You received this message because you are subscribed to the Google
Groups
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com
.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.

--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] ~7 GB of ghost data???

2010-03-22 Thread Nick Johnson (Google)

Hi,

The discrepancy between datastore stats volume and stored data is generally
due to indexing overhead, which is not included in the datastore stats. This
can be very high for entities with many properties, or with long entity and
property names or entity keys. Do you have reason to suppose that's not the
case in your situation?

-Nick Johnson

On Sun, Mar 21, 2010 at 3:39 AM, homunq jameson.qu...@gmail.com wrote:

Here's the line from my dashboard:
Total Stored Data$0.005/GByte-day82% 7.42 of
9.00 GBytes
$0.04 / $0.04

And here is the word from my datastore statistics:
Last updatedTotal number of entitiesSize of all entities
1:32:13 ago 232,867 615 MBytes
(metadata 11%, if that matters)

Please, can someone help me figure out this issue? I'd be happy to
share any info or code which would help track this down. My app id is
vulahealth.

--
Nick Johnson, Developer Programs Engineer, App Engine
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number:
368047

--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: HELP! 7 GB of ghost data - billable but not in datastore, blobs, or versions.

2010-03-22 Thread Wooble



On Mar 22, 4:20 am, homunq jameson.qu...@gmail.com wrote:
 I have no custom indexes. I have only two entities. 99.9% of the data
 is in the Expando which has about 45 stringproperties (initally all
 indexed, now around 30 are; many frequently empty string); 4 integer
 properties; 4 dates; 4 lists (about 4-8 members typically); from 3-18
 expando string lists (of 20 types total; an average of 6 lists with 2
 members each); and 2 expando strings (mostly but not always present).

For each indexed property, there will be 2 index entries (one in the
forward index and one in the reverse index), each of which contains
the actual data, the name of the property, the name of your
application, and if there's a parent entity information about the
parent will be in the key as well.  This quickly adds up.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Will GAE ever be open source?

2010-03-22 Thread Wooble



On Mar 22, 12:36 am, Iap iap...@gmail.com wrote:
 My concern is that the open-source GAE would also expose the weakness of
 GAE.
 That makes the GAE more vulnerable under attacking.

I hope you're not coding in a programming language that's open
source.  Just imagine the vulnerabilities. Of course, with python and
java eliminated, how are you using App Engine at all?

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Efficient Tag/Labels design using App Engine

2010-03-22 Thread Killian

Hi Robert,

Thank you very much for your feedback, you've raised my attention on
points I hadn't considered into account.

 Are all of these entities global or are some or all of them user-specific?
LO will be user specific since their Knowledge attribute will
correspond to a specific user's knowledge of this LO
For tags I had originally thought of creating tags which were user
specific (so users could tag any of their LO whatever way they feel
suits them best) and then have a global Tag which would summarize the
usual LOs tagged with tag T1 and make these common LO/Tag
relationships available to anyone starting and wishing to retrieve
LO's related to a particular tag. So once LO's  from the global tag T1
are imported to a user tag T1, this user can change the membership of
LO's within it's own tag T1 and this won't affect the global
membership T1 tag, unless a large amount of users make the same
change.

 How often do the entities relationships change, are LOs tagged / untagged 
 often?
The task of tagging LO's will be performed quite often, however each
LO's will not be tagged/untagged very often. Very similar to emails in
Gmail, tagging emails is done on a regular basis, however each email
will not be tagged/untagged very often

 Are LOs frequently added and deleted?
Again this is similar to Gmail. New LO will be added regularly (like
new emails received in an inbox), however they should be rarely
deleted.

 Are deletes or adds more often?
There will be more adds than deletes

 Same questions for Tags.
Adds are more often

 Is it important that the Imp values are sequential?
I'm not sure what you mean exactly, but the aim is simply to
prioritise LO to be presented to a user, hence they need to be
ordered, but if for instance 2 LO's had the same sequence number, it
would cause to much problem, unless this happened often.

 How many Tags would a typical LO have?
Less that 10

 How many LOs would a typical Tag have?
A lot ! In the order of 1000's

 Your second option may not be that bad if tag deletes are not that
 common.  You would be able to use the task queue or something to
 handle removing the Tag in the background.

For the first option though, I was thinking, since the knowledge value
of a users LO will not change very fast, maybe I could simply have a
cursor which initially goes through the word arraylist in the tag
entity, querying for LO's user knowledge. The cursor would go down the
list look for the next LO's with low knowledge. So whenever another
request for these tagged LO's with low knowledge is sent, the query
just starts at the cursor index (assuming once a LO is learned it
isn't forgotten of course)

 Depending on the answers to the questions above, you could also
 denormalize the structures so that you have a LOTag kind.
 Then you would be able to query that object easily, and if it is heavily
 denormailized you may not need a lot of additional queries.

Ok so, just to make sure I understand what you mean by LOTag, this
entity for instance would be a tuple of the kind:
LOTag1{LO_Key,TagName, TagOrder}?

Thanks very much for you help :-)









On Mar 22, 4:14 am, Robert Kluin robert.kl...@gmail.com wrote:
 Killian,
Are all of these entities global or are some or all of them
 user-specific?  How often do the entities relationships change, are
 LOs tagged / untagged often?  Are LOs frequently added and deleted?
 Are deletes or adds more often? Same questions for Tags.  Is it
 important that the Imp values are sequential?  How many Tags would a
 typical LO have?  How many LOs would a typical Tag have?

   Your second option may not be that bad if tag deletes are not that
 common.  You would be able to use the task queue or something to
 handle removing the Tag in the background.

   Depending on the answers to the questions above, you could also
 denormalize the structures so that you have a LOTag kind.  Then you
 would be able to query that object easily, and if it is heavily
 denormailized you may not need a lot of additional queries.

 Just some thoughts,
   Robert

 On Mon, Mar 22, 2010 at 4:36 AM, Killian killian.levac...@gmail.com wrote:
  Hi everyone,

  I'm trying to build a (seemingly) relatively simple 'Gmail like' label
  functionality using the app engine and am concerned about making
  proper use of app engine specific characteristics as much as possible
  to avoid fan out issues and make the code scalable.

  Here is the problem:
  I have a very large number of Learning Object entities LO each
  possessing a knowledge value K (between 0% and 100%). Each of these LO
  can be tagged with several tags T (a tag can contain a very large nbr
  of LO). For the moment this is exactly like gmail labels. The only
  difference is that each of these tags order their LO differently
  according to their importance Imp within each tag. So for instance,
  LO1 could be tagged with T1 and T2 and have importances Imp5 and
  Imp155 within each tag (an importance of 0 represents the most

[google-appengine] Re: Accessing gmail's atom feed using the google accounts API google.appengine.api.users

2010-03-22 Thread romuwild

xoopit?

On Mar 22, 4:24 am, Tom Wu service.g2...@gmail.com wrote:
 xoopit

 2010/3/22 romuwild rom...@gmail.com



  is there any way to let a user sign in using the google account API
  and then access its gmail atom feed without any relogin?

  thanks!

  --
  You received this message because you are subscribed to the Google Groups
  Google App Engine group.
  To post to this group, send email to google-appeng...@googlegroups.com.
  To unsubscribe from this group, send email to
  google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2Bunsubscrib 
  e...@googlegroups.com
  .
  For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: storage consumption in dashboard for local development server?

2010-03-22 Thread kostmo

Issue filed:
http://code.google.com/p/googleappengine/issues/detail?id=3001

On Mar 21, 11:21 pm, Robert Kluin robert.kl...@gmail.com wrote:
 I agree, it would be very nice to have something in the local dev
 server that would help estimate the space required on the production
 system.

 If you make an issue for this I will star it, and I am sure a lot of
 people would star it too.

 You might also look at this 
 article:http://code.google.com/appengine/articles/storage_breakdown.html

 Robert

 On Sun, Mar 21, 2010 at 4:25 AM, kostmo kos...@gmail.com wrote:
  I'd like to get a rough estimate of the amount of storage consumed by
  the datastore of a local dev server fully populated with my set of
  data.  Is there some straightforward way to do this?  The local
  dashboard lacks the storage/quota statistics offered for deployed
  (live) applications.

  --
  You received this message because you are subscribed to the Google Groups 
  Google App Engine group.
  To post to this group, send email to google-appeng...@googlegroups.com.
  To unsubscribe from this group, send email to 
  google-appengine+unsubscr...@googlegroups.com.
  For more options, visit this group 
  athttp://groups.google.com/group/google-appengine?hl=en.



-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: ~7 GB of ghost data???

2010-03-22 Thread homunq

OK, I guess I'm guilty on all counts.

Clearly, I can fix that moving forward, though it will cost me a lot
of CPU to fix the data I've already entered. But as a short-term
stopgap, is there any way to delete entire default indexes for a given
property? (I mean, anything besides setting indexed=False and then
touching each entity one-by-one). You can vacuum custom indexes - can
you do it with indexes created by default?

Thanks,
Jameson

On 22 mar, 03:42, Nick Johnson (Google) nick.john...@google.com
wrote:
 Hi,

 The discrepancy between datastore stats volume and stored data is generally
 due to indexing overhead, which is not included in the datastore stats. This
 can be very high for entities with many properties, or with long entity and
 property names or entity keys. Do you have reason to suppose that's not the
 case in your situation?

 -Nick Johnson





 On Sun, Mar 21, 2010 at 3:39 AM, homunq jameson.qu...@gmail.com wrote:
  Something is wrong. My app is showing with 7.42GB of total stored
  data, but only 615 MB of datastore. There is only one version string
  uploaded, which is almost 150MB, and nothing in the blobstore. This
  discrepancy has been getting worse - several hours ago (longer than
  the period since datastore statistics were updated, if you're
  wondering), there were the same 615 MB in the datastore, and only
  3.09GB of total stored data. (at that time, my theory was that it
  was old uploads of tweaks to the same version - but the numbers have
  gone far, far beyond that explanation now.) It's not some exploding
  index; the only non-default index I have is on an entity type with
  just 33 entities.

  Here's the line from my dashboard:
  Total Stored Data        $0.005/GByte-day                82%     7.42 of
  9.00 GBytes
  $0.04 / $0.04

  And here is the word from my datastore statistics:
  Last updated    Total number of entities        Size of all entities
  1:32:13 ago     232,867 615 MBytes
  (metadata 11%, if that matters)

  Please, can someone help me figure out this issue? I'd be happy to
  share any info or code which would help track this down. My app id is
  vulahealth.

  --
  You received this message because you are subscribed to the Google Groups
  Google App Engine group.
  To post to this group, send email to google-appeng...@googlegroups.com.
  To unsubscribe from this group, send email to
  google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2Bunsubscrib 
  e...@googlegroups.com
  .
  For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.

 --
 Nick Johnson, Developer Programs Engineer, App Engine
 Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number:
 368047

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: HELP! 7 GB of ghost data - billable but not in datastore, blobs, or versions.

2010-03-22 Thread homunq

Sorry, this thread is a duplicate of a href=http://groups.google.com/
group/google-appengine/browse_thread/thread/2a116e341b97c6fd/
6ad663cd210032b2?lnk=gstthat other one/a. I got impatient and
reposted before the other one got moderated. Useful word from Google
along with my response is over there - basically, I ask, is there any
way to bulk delete default indexes.

On 22 mar, 05:44, Wooble geoffsp...@gmail.com wrote:
 On Mar 22, 4:20 am, homunq jameson.qu...@gmail.com wrote:

  I have no custom indexes. I have only two entities. 99.9% of the data
  is in the Expando which has about 45 stringproperties (initally all
  indexed, now around 30 are; many frequently empty string); 4 integer
  properties; 4 dates; 4 lists (about 4-8 members typically); from 3-18
  expando string lists (of 20 types total; an average of 6 lists with 2
  members each); and 2 expando strings (mostly but not always present).

 For each indexed property, there will be 2 index entries (one in the
 forward index and one in the reverse index), each of which contains
 the actual data, the name of the property, the name of your
 application, and if there's a parent entity information about the
 parent will be in the key as well.  This quickly adds up.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] i18n and verbose_name

2010-03-22 Thread Emilis Kuke

Hello!
 I spend more than few hours using google and reading django and
google app engine documentation, but I couldn't find solution for my
problem. So I expect maybe somebody could helps me.

So I have model:
class Advertisement(db.Model):
  name = db.StringProperty(required=True,verbose_name=_(u'First
name'))

ModelForm:
class GiftForm(djangoforms.ModelForm):
  class Meta:
model = Advertisement

And template form

{{form.as_p}}


The problem is that First name label translates only when I change
settings.py LANGUAGE_CODE value. All other parts of view works
correctly out of the box. So I suppose that problem isn't django. Any
suggestions?

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] How can I reset my application?

2010-03-22 Thread Snoopy

i just uploaded wrong files to my application..so,is there any way to
delete them or reset that to undeveloped?

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] questions on datastore (lowercase comparison / one character String )

2010-03-22 Thread vchalmel

Hi !

I wonder how to compare a String stored in a datastore entity (field
login) to an entry (also named login) considering their lowercase
version :
 - I tried : SELECT count(this) FROM +Gestionnaire.class.getName()+
WHERE LowerCase(login)== LowerCase('+login+') which throws an
exception

I have another problem with this query, I don't want to exclude the
one-character login like, maybe 1, but this also fires an exception
(  java.lang.Character cannot be cast to
com.google.appengine.api.datastore.Key )

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] new gae member

2010-03-22 Thread Gleidson - MG - Brasil

hello, I am Brazilian and participate in this nearby group.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] how to solve ClassNotPersistenceCapableException

2010-03-22 Thread aptest1 actiprocess

I have developed a application in my local system its running
correctly. after uploading to the GAE its giving following exception:
115.108.122.17 - - [22/Mar/2010:02:41:51 -0700] POST /smartstore/
greet HTTP/1.1 200 134 http://smartstoreact.appspot.com/smartstore/
5CBF907BC63196D7B1E2BE1458031519.cache.html Mozilla/4.0 (compatible;
MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 2.0.50727; .NET CLR
3.0.4506.2152; .NET CLR 3.5.30729),gzip(gfe)
smartstoreact.appspot.comW 03-22 02:41AM 51.000 [smartstoreact/
9.340710510195639511].stderr:
org.datanucleus.jdo.exceptions.ClassNotPersistenceCapableException:
The class The class com.atp.smartstore.DB.config.User is not
persistable. This means that it either hasnt been enhanced, or that
the enhanced version of the file is not in the CLASSPATH (or is hidden
by an unenhanced version), or the Meta-Data/annotations for the class
are not found. is not persistable. This means that it either hasnt
been enhanced, or that the enhanced version of the file is not in the
CLASSPATH (or is hidden by an unenhanced version), or the Meta-Data
for the class is not found.

W 03-22 02:41AM 51.001 [smartstoreact/9.340710510195639511].stderr:
at
org.datanucleus.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:
241)

W 03-22 02:41AM 51.001 [smartstoreact/9.340710510195639511].stderr:
at
org.datanucleus.jdo.JDOPersistenceManager.newObjectIdInstance(JDOPersistenceManager.java:
1571)

W 03-22 02:41AM 51.001 [smartstoreact/9.340710510195639511].stderr:
at
org.datanucleus.jdo.JDOPersistenceManager.getObjectById(JDOPersistenceManager.java:
1767)

W 03-22 02:41AM 51.001 [smartstoreact/9.340710510195639511].stderr:
at
org.datanucleus.store.appengine.jdo.DatastoreJDOPersistenceManager.getObjectById(DatastoreJDOPersistenceManager.java:
71)

W 03-22 02:41AM 51.001 [smartstoreact/9.340710510195639511].stderr:
at com.atp.smartstore.DB.DAO.UserDAO.getUser(UserDAO.java:29)

W 03-22 02:41AM 51.001 [smartstoreact/9.340710510195639511].stderr:
at com.atp.smartstore.service.impl.UserImpl.getUser(UserImpl.java:26)

W 03-22 02:41AM 51.001 [smartstoreact/9.340710510195639511].stderr:
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

W 03-22 02:41AM 51.001 [smartstoreact/9.340710510195639511].stderr:
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)

W 03-22 02:41AM 51.001 [smartstoreact/9.340710510195639511].stderr:
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)

W 03-22 02:41AM 51.001 [smartstoreact/9.340710510195639511].stderr:
at java.lang.reflect.Method.invoke(Unknown Source)

W 03-22 02:41AM 51.001 [smartstoreact/9.340710510195639511].stderr:
at
com.google.apphosting.runtime.security.shared.intercept.java.lang.reflect.Method_
$1.run(Method_.java:165)

W 03-22 02:41AM 51.001 [smartstoreact/9.340710510195639511].stderr:
at java.security.AccessController.doPrivileged(Native Method)

W 03-22 02:41AM 51.001 [smartstoreact/9.340710510195639511].stderr:
at
com.google.apphosting.runtime.security.shared.intercept.java.lang.reflect.Method_.privilegedInvoke(Method_.java:
163)

W 03-22 02:41AM 51.001 [smartstoreact/9.340710510195639511].stderr:
at
com.google.apphosting.runtime.security.shared.intercept.java.lang.reflect.Method_.invoke_(Method_.java:
124)

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] permutations vs Cartesian product in exploding indexes documentation

2010-03-22 Thread kostmo

It is still unclear to me exactly how many index entries will be
produced by including multiple ListProperty's in an entity. In one
post [1], between the inquiring user and the Google rep, the
possibilities for the number of index entries included the power set,
Cartesian product, or the number of unique combinations (this was
the rep's answer, without mention of permutations).

I'd like to verify that the correct combinatorics terminology is being
used in the documentation on exploding indexes [2].

Quote:
the index table must include a row for every permutation of the
values of every property for the index
This is followed by the example:
e2 = MyModel()
e2.x = ['red', 'blue']
e2.y = [1, 2]
the index must store 12 property values: 2 each for the built-in
indexes on x and y, and 2 for each of the 4 permutations of x and y in
the custom index.

The example does not make me completely confident about what type of
combinatoric expression is truly being used, since the sum of the
number of permutations for each property happens to equal the
cardinality of the Cartesian product of the two properties. The
enumeration given in this post [3] seems to suggest that it is
actually the Cartesian product at work.

To clarify this issue, we could use Python 2.6 to illustrate an
example:
 from itertools import product, permutations
 x = ['red', 'blue', 'green']
 y = [1, 2, 3]
 list(permutations(x)) + list(permutations(y))
[('red', 'blue', 'green'), ('red', 'green', 'blue'), ('blue', 'red',
'green'), ('blue', 'green', 'red'), ('green', 'red', 'blue'),
('green', 'blue', 'red'), (1, 2, 3), (1, 3, 2), (2, 1, 3), (2, 3, 1),
(3, 1, 2), (3, 2, 1)]
 list(product(x, y))
[('red', 1), ('red', 2), ('red', 3), ('blue', 1), ('blue', 2),
('blue', 3), ('green', 1), ('green', 2), ('green', 3)]

Is the number of custom index entries that will be generated by an
entity with the two 3-valued properties given above proportional to
the sum of permutations (there are 12) or the Cartesian product (there
are 9)?

Thanks,
Karl

[1] Efficient way to structure my data model
http://groups.google.com/group/google-appengine/browse_thread/thread/326e90024ed53ced/d958fa6df98ba6df
[2] 
http://code.google.com/appengine/docs/python/datastore/queriesandindexes.html#Big_Entities_and_Exploding_Indexes
[3] Size of index in case of exploding index
http://groups.google.com/group/google-appengine/browse_thread/thread/3db5a0338d77d81b/c5f1ea4fa3107b25

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] questions on datastore (lowercase comparison / one character String )

2010-03-22 Thread djidjadji

If you want to compare lowercase you have to store the value lowercase
in the object.

GQL != SQL You can't do more then described on the GQL reference page.

You have to traverse (get) all you current objects and make the field
login lowercase and store the object back and as such update the index
for login-field.

2010/3/21 vchalmel groupeprojeten...@gmail.com:
 Hi !

 I wonder how to compare a String stored in a datastore entity (field
 login) to an entry (also named login) considering their lowercase
 version :
  - I tried : SELECT count(this) FROM +Gestionnaire.class.getName()+
 WHERE LowerCase(login)== LowerCase('+login+') which throws an
 exception

 I have another problem with this query, I don't want to exclude the
 one-character login like, maybe 1, but this also fires an exception
 (  java.lang.Character cannot be cast to
 com.google.appengine.api.datastore.Key )

 --
 You received this message because you are subscribed to the Google Groups 
 Google App Engine group.
 To post to this group, send email to google-appeng...@googlegroups.com.
 To unsubscribe from this group, send email to 
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at 
 http://groups.google.com/group/google-appengine?hl=en.



-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] How can I reset my application?

2010-03-22 Thread djidjadji

Change the version name of the app, upload the new version, make it
default and remove the old version.

2010/3/22 Snoopy tootoo1...@gmail.com:
 i just uploaded wrong files to my application..so,is there any way to
 delete them or reset that to undeveloped?

 --
 You received this message because you are subscribed to the Google Groups 
 Google App Engine group.
 To post to this group, send email to google-appeng...@googlegroups.com.
 To unsubscribe from this group, send email to 
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at 
 http://groups.google.com/group/google-appengine?hl=en.



-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: questions on datastore (lowercase comparison / one character String )

2010-03-22 Thread Gleidson - MG - Brasil

If you're using a relational database it's pretty easy to execute case-
insensitive queries because it's a native feature of the database.  It
usually looks something like this:

 select * from Person where UPPER(lastName) = 'ROSS'

This is a challenge for the App Engine Datastore because we rely on
executing scans over a limited subset of your data and returning
results as we encounter them.  Consider the following strings in
lexicographic order:
...
ROSE
...
ROSS
...
ROSTA
...
Rose
...
Ross
...
Rosta
...
rose
...
ross
...
rosta
...

As you can see there could be an unlimited number of rows in between
'ROSS' and 'Ross', and also between 'Ross' and 'ross' (okay not
unlimited, we do have a limit on indexed string length, but definitely
Large), so if we start scanning at 'ROSS' we might have to skip a huge
number of results before we hit 'Ross', and that doesn't scale.  This
is why the datastore doesn't support case-insensitive queries.

Fortunately it's not hard to implement support for case-insensitive
queries in your application.  Here's the approach: for each field that
you want to query in a case-insensitive way, create a duplicate field
that stores the value of that field in either all upper or lowercase
letters.  Then, have your model object register a pre-persist callback
with your persistence framework (JPA or JDO), and then populate the
duplicate fields inside that callback.  Here's what it looks like.

JPA:
import javax.persistence.PrePersist;
import javax.persistence.PreUpdate;


@Entity
public class Person {
  @Id
  @GeneratedValue(strategy = GenerationType.IDENTITY)
  private Long id;
  private String lastName;
  private String uppercaseLastName;

  public Long getId() {
return id;
  }

  public String getLastName() {
return lastName;
  }

  public void setLastName(String lastName) {
this.lastName = lastName;
  }

  @PrePersist
  @PreUpdate
  public void prePersist() {
if (lastName != null) {
  uppercaseLastName = lastName.toUpperCase();
} else {
  uppercaseLastName = null;
}
  }
}

public ListPerson getPeopleByLastName(String lastName) {
  Query q = em.createQuery(select from Person where uppercaseLastName
= :p);
  q.setParameter(p, lastName.toUpperCase());
  return (ListPerson) q.getResultList();
}


JDO:
import javax.jdo.listener.StoreCallback;

@PersistenceCapable(identityType = IdentityType.APPLICATION)
public class Person implements StoreCallback {
  @PrimaryKey
  @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
  private Long id;
  private String lastName;
  private String uppercaseLastName;

  public Long getId() {
return id;
  }

  public String getLastName() {
return lastName;
  }

  public void setLastName(String lastName) {
this.lastName = lastName;
  }

  public void jdoPreStore() {
if (lastName != null) {
  uppercaseLastName = lastName.toUpperCase();
} else {
  uppercaseLastName = null;
}
  }
}

public ListPerson getPeopleByLastName(String lastName) {
  Query q = pm.newQuery(Person.class, uppercaseLastName == :p);
  return (ListPerson) q.execute(lastName.toUpperCase());
}









On 21 mar, 17:41, vchalmel groupeprojeten...@gmail.com wrote:
 Hi !

 I wonder how to compare a String stored in a datastore entity (field
 login) to an entry (also named login) considering their lowercase
 version :
  - I tried : SELECT count(this) FROM +Gestionnaire.class.getName()+
 WHERE LowerCase(login)== LowerCase('+login+') which throws an
 exception

 I have another problem with this query, I don't want to exclude the
 one-character login like, maybe 1, but this also fires an exception
 (  java.lang.Character cannot be cast to
 com.google.appengine.api.datastore.Key )

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Will GAE ever be open source?

2010-03-22 Thread Patrick Twohig

On Mon, Mar 22, 2010 at 5:47 AM, Wooble geoffsp...@gmail.com wrote:



 On Mar 22, 12:36 am, Iap iap...@gmail.com wrote:
  My concern is that the open-source GAE would also expose the weakness of
  GAE.
  That makes the GAE more vulnerable under attacking.

 I hope you're not coding in a programming language that's open
 source.  Just imagine the vulnerabilities. Of course, with python and
 java eliminated, how are you using App Engine at all?

 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To post to this group, send email to google-appeng...@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com
 .
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.



1995 called.  They want their rhetoric back.



-- 
Patrick H. Twohig.

Namazu Studios
P.O. Box 34161
San Diego, CA 92163-4161

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: ~7 GB of ghost data???

2010-03-22 Thread Patrick Twohig

Hey Nick,

Just out of curiosity, how many properties would it take to get that amount
of wasted space in overhead? Are we talking about entities in the orders of
magnitudes of tens/thousands/hundreds?

On Mon, Mar 22, 2010 at 9:07 AM, homunq jameson.qu...@gmail.com wrote:

OK, I guess I'm guilty on all counts.

Clearly, I can fix that moving forward, though it will cost me a lot
of CPU to fix the data I've already entered. But as a short-term
stopgap, is there any way to delete entire default indexes for a given
property? (I mean, anything besides setting indexed=False and then
touching each entity one-by-one). You can vacuum custom indexes - can
you do it with indexes created by default?

Thanks,
Jameson

On 22 mar, 03:42, Nick Johnson (Google) nick.john...@google.com
wrote:
Hi,

The discrepancy between datastore stats volume and stored data is
generally
due to indexing overhead, which is not included in the datastore stats.
This
can be very high for entities with many properties, or with long entity
and
property names or entity keys. Do you have reason to suppose that's not
the
case in your situation?

-Nick Johnson

On Sun, Mar 21, 2010 at 3:39 AM, homunq jameson.qu...@gmail.com wrote:
Something is wrong. My app is showing with 7.42GB of total stored
data, but only 615 MB of datastore. There is only one version string
uploaded, which is almost 150MB, and nothing in the blobstore. This
discrepancy has been getting worse - several hours ago (longer than
the period since datastore statistics were updated, if you're
wondering), there were the same 615 MB in the datastore, and only
3.09GB of total stored data. (at that time, my theory was that it
was old uploads of tweaks to the same version - but the numbers have
gone far, far beyond that explanation now.) It's not some exploding
index; the only non-default index I have is on an entity type with
just 33 entities.

Here's the line from my dashboard:
Total Stored Data$0.005/GByte-day82% 7.42
of
9.00 GBytes
$0.04 / $0.04

And here is the word from my datastore statistics:
Last updatedTotal number of entitiesSize of all entities
1:32:13 ago 232,867 615 MBytes
(metadata 11%, if that matters)

Please, can someone help me figure out this issue? I'd be happy to
share any info or code which would help track this down. My app id is
vulahealth.

--
You received this message because you are subscribed to the Google
Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.comgoogle-appengine%2Bunsubscrib
e...@googlegroups.com
.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.

--
Nick Johnson, Developer Programs Engineer, App Engine
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration
Number:
368047

--
Patrick H. Twohig.

Namazu Studios
P.O. Box 34161
San Diego, CA 92163-4161

--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: ~7 GB of ghost data???

2010-03-22 Thread Nick Johnson (Google)

Hi Patrick,

An overhead factor of 12 (as observed below) is high, but not outrageous.
With long model names and property names, this could happen with relatively
few indexed properties - on the order of magnitude of tens, at most.

-Nick Johnson

On Mon, Mar 22, 2010 at 8:07 PM, Patrick Twohig
patr...@namazustudios.comwrote:

Hey Nick,

Just out of curiosity, how many properties would it take to get that amount
of wasted space in overhead? Are we talking about entities in the orders of
magnitudes of tens/thousands/hundreds?

On Mon, Mar 22, 2010 at 9:07 AM, homunq jameson.qu...@gmail.com wrote:

OK, I guess I'm guilty on all counts.

Thanks,
Jameson

On 22 mar, 03:42, Nick Johnson (Google) nick.john...@google.com
wrote:
Hi,

The discrepancy between datastore stats volume and stored data is
generally
due to indexing overhead, which is not included in the datastore stats.
This
can be very high for entities with many properties, or with long entity
and
property names or entity keys. Do you have reason to suppose that's not
the
case in your situation?

-Nick Johnson

On Sun, Mar 21, 2010 at 3:39 AM, homunq jameson.qu...@gmail.com
wrote:
Something is wrong. My app is showing with 7.42GB of total stored
data, but only 615 MB of datastore. There is only one version string
uploaded, which is almost 150MB, and nothing in the blobstore. This
discrepancy has been getting worse - several hours ago (longer than
the period since datastore statistics were updated, if you're
wondering), there were the same 615 MB in the datastore, and only
3.09GB of total stored data. (at that time, my theory was that it
was old uploads of tweaks to the same version - but the numbers have
gone far, far beyond that explanation now.) It's not some exploding
index; the only non-default index I have is on an entity type with
just 33 entities.

Here's the line from my dashboard:
Total Stored Data$0.005/GByte-day82% 7.42
of
9.00 GBytes
$0.04 / $0.04

And here is the word from my datastore statistics:
Last updatedTotal number of entitiesSize of all entities
1:32:13 ago 232,867 615 MBytes
(metadata 11%, if that matters)

Please, can someone help me figure out this issue? I'd be happy to
share any info or code which would help track this down. My app id is
vulahealth.

--
You received this message because you are subscribed to the Google
Groups
Google App Engine group.
To post to this group, send email to
google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.comgoogle-appengine%2Bunsubscrib
e...@googlegroups.com
.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.

--
Nick Johnson, Developer Programs Engineer, App Engine
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration
Number:
368047

--
Patrick H. Twohig.

Namazu Studios
P.O. Box 34161
San Diego, CA 92163-4161

--
Nick Johnson, Developer Programs Engineer, App Engine
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number:
368047

--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: ~7 GB of ghost data???

2010-03-22 Thread homunq

OK, after hashing it out on IRC, I see that I have to erase my data
and start again. Since it took me 3 days of CPU quota to add the data,
I want to know if I can erase it quickly.

1. Is the overhead for erasing data (and thus whittling down indexes)
over half the overhead from adding it? Under 10%? Or what? (I don't
need exact numbers, just approximates.

2. If it's more like half - is there some way to just nuke all my data
and start over?

Thanks,
Jameson


On 22 mar, 03:42, Nick Johnson (Google) nick.john...@google.com
wrote:
 Hi,

 The discrepancy between datastore stats volume and stored data is generally
 due to indexing overhead, which is not included in the datastore stats. This
 can be very high for entities with many properties, or with long entity and
 property names or entity keys. Do you have reason to suppose that's not the
 case in your situation?

 -Nick Johnson





 On Sun, Mar 21, 2010 at 3:39 AM, homunq jameson.qu...@gmail.com wrote:
  Something is wrong. My app is showing with 7.42GB of total stored
  data, but only 615 MB of datastore. There is only one version string
  uploaded, which is almost 150MB, and nothing in the blobstore. This
  discrepancy has been getting worse - several hours ago (longer than
  the period since datastore statistics were updated, if you're
  wondering), there were the same 615 MB in the datastore, and only
  3.09GB of total stored data. (at that time, my theory was that it
  was old uploads of tweaks to the same version - but the numbers have
  gone far, far beyond that explanation now.) It's not some exploding
  index; the only non-default index I have is on an entity type with
  just 33 entities.

  Here's the line from my dashboard:
  Total Stored Data        $0.005/GByte-day                82%     7.42 of
  9.00 GBytes
  $0.04 / $0.04

  And here is the word from my datastore statistics:
  Last updated    Total number of entities        Size of all entities
  1:32:13 ago     232,867 615 MBytes
  (metadata 11%, if that matters)

  Please, can someone help me figure out this issue? I'd be happy to
  share any info or code which would help track this down. My app id is
  vulahealth.

  --
  You received this message because you are subscribed to the Google Groups
  Google App Engine group.
  To post to this group, send email to google-appeng...@googlegroups.com.
  To unsubscribe from this group, send email to
  google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2Bunsubscrib 
  e...@googlegroups.com
  .
  For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.

 --
 Nick Johnson, Developer Programs Engineer, App Engine
 Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number:
 368047

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: ~7 GB of ghost data???

2010-03-22 Thread Patrick Twohig

I'd use a cursor on the task queue. Do bulk deletes in blocks of 500 (I
think that's the most keys you can pass to delete on a single call) and it
shouldn't be that hard to wipe it out.

Cheers!

On Mon, Mar 22, 2010 at 1:45 PM, homunq jameson.qu...@gmail.com wrote:

OK, after hashing it out on IRC, I see that I have to erase my data
and start again. Since it took me 3 days of CPU quota to add the data,
I want to know if I can erase it quickly.

1. Is the overhead for erasing data (and thus whittling down indexes)
over half the overhead from adding it? Under 10%? Or what? (I don't
need exact numbers, just approximates.

2. If it's more like half - is there some way to just nuke all my data
and start over?

Thanks,
Jameson

On 22 mar, 03:42, Nick Johnson (Google) nick.john...@google.com
wrote:
Hi,

The discrepancy between datastore stats volume and stored data is
generally
due to indexing overhead, which is not included in the datastore stats.
This
can be very high for entities with many properties, or with long entity
and
property names or entity keys. Do you have reason to suppose that's not
the
case in your situation?

-Nick Johnson

Here's the line from my dashboard:
Total Stored Data$0.005/GByte-day82% 7.42
of
9.00 GBytes
$0.04 / $0.04

And here is the word from my datastore statistics:
Last updatedTotal number of entitiesSize of all entities
1:32:13 ago 232,867 615 MBytes
(metadata 11%, if that matters)

Please, can someone help me figure out this issue? I'd be happy to
share any info or code which would help track this down. My app id is
vulahealth.

--
Nick Johnson, Developer Programs Engineer, App Engine
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration
Number:
368047

--
Patrick H. Twohig.

Namazu Studios
P.O. Box 34161
San Diego, CA 92163-4161

--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: ~7 GB of ghost data???

2010-03-22 Thread Eli Jones

oh man.. well, he's going to be wiping out 7GB of junk... :)

When I went through process of deleting something like 400MB of junk.. it
was not fun

First I started off deleting by __key__ in batches of 500, then I had to
limit down to 200.. then down to 100.. then down to 50.. then down to 10..
then it stopped responding for hours (I could not even fetch(1) from the
Model).

There must be a sanctioned way to remove 100,000s of entities based on how
the datastore is structured. For example, does it make sense to do
something like this.

Use a cursor to:
1. Select __key__ from Model Order By __key__
2. append every 10th (or 100th) result to a list.. and delete that list for
every 100 or 200 or 500 entities added.
3. Once at end of cursor, start over at the beginning.

That way, you wouldn't be deleting everything on the same table at the same
time? The datastore completely died on me when I tried to straight delete
by __key__ using GqlQuery in a loop.. Just kept getting slower and slower.
(I think maybe directly deleting by key_name might work better but I never
had to do a bulk delete again.. so have not tested that theory).

On Mon, Mar 22, 2010 at 5:19 PM, Patrick Twohig
patr...@namazustudios.comwrote:

I'd use a cursor on the task queue. Do bulk deletes in blocks of 500 (I
think that's the most keys you can pass to delete on a single call) and it
shouldn't be that hard to wipe it out.

Cheers!

On Mon, Mar 22, 2010 at 1:45 PM, homunq jameson.qu...@gmail.com wrote:

OK, after hashing it out on IRC, I see that I have to erase my data
and start again. Since it took me 3 days of CPU quota to add the data,
I want to know if I can erase it quickly.

1. Is the overhead for erasing data (and thus whittling down indexes)
over half the overhead from adding it? Under 10%? Or what? (I don't
need exact numbers, just approximates.

2. If it's more like half - is there some way to just nuke all my data
and start over?

Thanks,
Jameson

On 22 mar, 03:42, Nick Johnson (Google) nick.john...@google.com
wrote:
Hi,

The discrepancy between datastore stats volume and stored data is
generally
due to indexing overhead, which is not included in the datastore stats.
This
can be very high for entities with many properties, or with long entity
and
property names or entity keys. Do you have reason to suppose that's not
the
case in your situation?

-Nick Johnson

On Sun, Mar 21, 2010 at 3:39 AM, homunq jameson.qu...@gmail.com
wrote:
Something is wrong. My app is showing with 7.42GB of total stored
data, but only 615 MB of datastore. There is only one version string
uploaded, which is almost 150MB, and nothing in the blobstore. This
discrepancy has been getting worse - several hours ago (longer than
the period since datastore statistics were updated, if you're
wondering), there were the same 615 MB in the datastore, and only
3.09GB of total stored data. (at that time, my theory was that it
was old uploads of tweaks to the same version - but the numbers have
gone far, far beyond that explanation now.) It's not some exploding
index; the only non-default index I have is on an entity type with
just 33 entities.

Here's the line from my dashboard:
Total Stored Data$0.005/GByte-day82% 7.42
of
9.00 GBytes
$0.04 / $0.04

And here is the word from my datastore statistics:
Last updatedTotal number of entitiesSize of all entities
1:32:13 ago 232,867 615 MBytes
(metadata 11%, if that matters)

Please, can someone help me figure out this issue? I'd be happy to
share any info or code which would help track this down. My app id is
vulahealth.

--
You received this message because you are subscribed to the Google
Groups
Google App Engine group.
To post to this group, send email to
google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.comgoogle-appengine%2Bunsubscrib
e...@googlegroups.com
.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.

--
Nick Johnson, Developer Programs Engineer, App Engine
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration
Number:
368047

--
Patrick H. Twohig.

Namazu Studios
P.O. Box 34161
San Diego, CA 92163-4161

--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post

Re: [google-appengine] Re: ~7 GB of ghost data???

2010-03-22 Thread Nick Johnson (Google)

Hi,

On Mon, Mar 22, 2010 at 8:45 PM, homunq jameson.qu...@gmail.com wrote:

OK, after hashing it out on IRC, I see that I have to erase my data
and start again.

Why is that? Wouldn't updating the data be a better option?

Since it took me 3 days of CPU quota to add the data,
I want to know if I can erase it quickly.

1. Is the overhead for erasing data (and thus whittling down indexes)
over half the overhead from adding it? Under 10%? Or what? (I don't
need exact numbers, just approximates.

It should be significantly lower - you can do a keys-only query, and delete
the returned keys.

-Nick Johnson

2. If it's more like half - is there some way to just nuke all my data
and start over?

Thanks,
Jameson

On 22 mar, 03:42, Nick Johnson (Google) nick.john...@google.com
wrote:
Hi,

The discrepancy between datastore stats volume and stored data is
generally
due to indexing overhead, which is not included in the datastore stats.
This
can be very high for entities with many properties, or with long entity
and
property names or entity keys. Do you have reason to suppose that's not
the
case in your situation?

-Nick Johnson

Here's the line from my dashboard:
Total Stored Data$0.005/GByte-day82% 7.42
of
9.00 GBytes
$0.04 / $0.04

And here is the word from my datastore statistics:
Last updatedTotal number of entitiesSize of all entities
1:32:13 ago 232,867 615 MBytes
(metadata 11%, if that matters)

Please, can someone help me figure out this issue? I'd be happy to
share any info or code which would help track this down. My app id is
vulahealth.

--
Nick Johnson, Developer Programs Engineer, App Engine
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration
Number:
368047

--
Nick Johnson, Developer Programs Engineer, App Engine
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number:
368047

--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Can I change the login option of my app if I delete it?

2010-03-22 Thread Mariano Benitez

I understand I cannot change the option while the application is
enabled. Also that the application Id is reserved if I delete the
application. But... can I re-create the application with a different
login option, or the app id is reserved to never be used again?

Thanks!

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Can I change the login option of my app if I delete it?

2010-03-22 Thread djidjadji

To my knowledge the appid can only be once allocated.
If you delete the app the appid is no more available.

2010/3/23 Mariano Benitez mari...@benitez.nu:
 I understand I cannot change the option while the application is
 enabled. Also that the application Id is reserved if I delete the
 application. But... can I re-create the application with a different
 login option, or the app id is reserved to never be used again?

 Thanks!

 --
 You received this message because you are subscribed to the Google Groups 
 Google App Engine group.
 To post to this group, send email to google-appeng...@googlegroups.com.
 To unsubscribe from this group, send email to 
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at 
 http://groups.google.com/group/google-appengine?hl=en.



-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Can I change the login option of my app if I delete it?

2010-03-22 Thread Eli Jones

Here is the specific documentation on app deletion:

http://code.google.com/appengine/kb/adminconsole.html#delete_app

http://code.google.com/appengine/kb/adminconsole.html#delete_appYou cannot
use the appid again, unfortunately.

On Mon, Mar 22, 2010 at 8:25 PM, djidjadji djidja...@gmail.com wrote:

To my knowledge the appid can only be once allocated.
If you delete the app the appid is no more available.

2010/3/23 Mariano Benitez mari...@benitez.nu:
I understand I cannot change the option while the application is
enabled. Also that the application Id is reserved if I delete the
application. But... can I re-create the application with a different
login option, or the app id is reserved to never be used again?

Thanks!

--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Reading a file in GAE Java

2010-03-22 Thread Iain Flynn

Hi there. I have an application that requires me to read in and
transform an XSLT stylesheet, which I keep in the directory /xml in /
war.

I've read the tutorials, and I've included it as a resource file (more
specifically, set that all xsl files are included as resource files),
but when I try to read it in my Java servlet (using the URI /xml/
file.xsl), I get the FilePermission error. I presume I'm getting this
error because it can't find the file (I know that writing files is not
allowed, but I'm opening it in read-mode).

I've looked through all the tutorials and I can't see where I'm going
wrong here. I can use files with my HTML, but getting Java on them
seems to elude me.

I get the feeling I've made a very elementary mistake, but can't for
the life of me figure it out.

Thanks for your time, and sorry if this is the wrong place for this.

- Iain

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] updating to new release of eclipse plugin causing datanucleus errors (again)

2010-03-22 Thread haole

after updating to the newest release (20100319) of the eclipse plugin
(on 3/22), datanucleus enhancer is throwing null pointer exceptions
again. last time, it was due to classpath that was too long. removing
some unneeded jars out of my WEB-INF/lib directory seemed to do the
trick. this time, i cannot figure out what the problem is. i even
tried renaming jars to shorter names to no avail. PLEASE either (1)
fix this, or (2) give some kind of more detailed error message so we
know what is causing this.

furthermore, i cannot figure out how to rollback the google eclipse
plugin to the previous version. is this possible? if so, how?

java.lang.NullPointerException
at
com.google.gdt.eclipse.core.ProcessUtilities.cleanupProcess(ProcessUtilities.java:
367)
at
com.google.gdt.eclipse.core.ProcessUtilities.launchProcessAndActivateOnError(ProcessUtilities.java:
271)
at
com.google.appengine.eclipse.core.orm.enhancement.EnhancerJob.runInWorkspace(EnhancerJob.java:
78)
at
org.eclipse.core.internal.resources.InternalWorkspaceJob.run(InternalWorkspaceJob.java:
38)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:55)

eclipse.buildId=M20100211-1343
java.version=1.6.0_18
java.vendor=Sun Microsystems Inc.
BootLoader constants: OS=win32, ARCH=x86, WS=win32, NL=en_US
Command-line arguments:  -os win32 -ws win32 -arch x86

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] static files not found after upload

2010-03-22 Thread Glenn Blackler

Hello --

My very simple app works fine on the dev server, but once uploaded it
can't access a file in one of my static directories. I have a static
directory HTML with two files in it. The first comes up fine (I
redirect to it from .py script) but the other HTML file always brings
an error when I try to link to it. The error in my log is -- Static
file referenced by handler not found: html/new_task.html.

I searched for others having this same problem, but the only ones I
could find had to do with manage.py and django, which I don't think is
the problem here.

Again, everything works fine on the dev server which is why this is so
frustrating. I appreciate any help.

Here's my app.yaml if it helps :

application: test_app
version: 1
runtime: python
api_version: 1

handlers:
- url: /stylesheets
  static_dir: stylesheets

- url: /html
  static_dir: html

- url: /js
  static_dir: js

- url: /tasks
  script: test2.py

- url: /addnew
  script: test2.py

- url: /.*
  script: home.py

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Reading a file in GAE Java

2010-03-22 Thread Patrick Twohig

I use

Class.getResourceAsStream(/xml/myschema.xsd);

Cheers!

On Mon, Mar 22, 2010 at 10:57 AM, Iain Flynn trancedsha...@gmail.comwrote:

Hi there. I have an application that requires me to read in and
transform an XSLT stylesheet, which I keep in the directory /xml in /
war.

I've read the tutorials, and I've included it as a resource file (more
specifically, set that all xsl files are included as resource files),
but when I try to read it in my Java servlet (using the URI /xml/
file.xsl), I get the FilePermission error. I presume I'm getting this
error because it can't find the file (I know that writing files is not
allowed, but I'm opening it in read-mode).

I've looked through all the tutorials and I can't see where I'm going
wrong here. I can use files with my HTML, but getting Java on them
seems to elude me.

I get the feeling I've made a very elementary mistake, but can't for
the life of me figure it out.

Thanks for your time, and sorry if this is the wrong place for this.

- Iain

--
Patrick H. Twohig.

Namazu Studios
P.O. Box 34161
San Diego, CA 92163-4161

--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: permutations vs Cartesian product in exploding indexes documentation

2010-03-22 Thread kostmo

OK, I just watched Brett Slatkin's I/O talk [1] and he mentions cross
product a couple of times, so it seems that the use of the word
permutation in the docs is incorrect; the number of index entries is
indeed proportional to the Cartesian product, rather than
permutations which would lead to a factorial (n!) number of index
entries.

I've filed a doco issue here [2].

Karl

[1] 
http://code.google.com/events/io/2009/sessions/BuildingScalableComplexApps.html
[2] http://code.google.com/p/googleappengine/issues/detail?id=3003

On Mar 22, 11:50 am, kostmo kos...@gmail.com wrote:
 It is still unclear to me exactly how many index entries will be
 produced by including multiple ListProperty's in an entity. In one
 post [1], between the inquiring user and the Google rep, the
 possibilities for the number of index entries included the power set,
 Cartesian product, or the number of unique combinations (this was
 the rep's answer, without mention of permutations).

 I'd like to verify that the correct combinatorics terminology is being
 used in the documentation on exploding indexes [2].

 Quote:
 the index table must include a row for every permutation of the
 values of every property for the index
 This is followed by the example:
 e2 = MyModel()
 e2.x = ['red', 'blue']
 e2.y = [1, 2]
 the index must store 12 property values: 2 each for the built-in
 indexes on x and y, and 2 for each of the 4 permutations of x and y in
 the custom index.

 The example does not make me completely confident about what type of
 combinatoric expression is truly being used, since the sum of the
 number of permutations for each property happens to equal the
 cardinality of the Cartesian product of the two properties. The
 enumeration given in this post [3] seems to suggest that it is
 actually the Cartesian product at work.

 To clarify this issue, we could use Python 2.6 to illustrate an
 example: from itertools import product, permutations
  x = ['red', 'blue', 'green']
  y = [1, 2, 3]
  list(permutations(x)) + list(permutations(y))

 [('red', 'blue', 'green'), ('red', 'green', 'blue'), ('blue', 'red',
 'green'), ('blue', 'green', 'red'), ('green', 'red', 'blue'),
 ('green', 'blue', 'red'), (1, 2, 3), (1, 3, 2), (2, 1, 3), (2, 3, 1),
 (3, 1, 2), (3, 2, 1)] list(product(x, y))

 [('red', 1), ('red', 2), ('red', 3), ('blue', 1), ('blue', 2),
 ('blue', 3), ('green', 1), ('green', 2), ('green', 3)]

 Is the number of custom index entries that will be generated by an
 entity with the two 3-valued properties given above proportional to
 the sum of permutations (there are 12) or the Cartesian product (there
 are 9)?

 Thanks,
 Karl

 [1] Efficient way to structure my data 
 modelhttp://groups.google.com/group/google-appengine/browse_thread/thread/...
 [2]http://code.google.com/appengine/docs/python/datastore/queriesandinde...
 [3] Size of index in case of exploding 
 indexhttp://groups.google.com/group/google-appengine/browse_thread/thread/...

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: questions on datastore (lowercase comparison / one character String )

2010-03-22 Thread Chau

You can save storage by just quote Max Ross's article...
http://gae-java-persistence.blogspot.com/2009/11/case-insensitive-queries.html

On Mar 23, 12:27 am, Gleidson - MG - Brasil
gleidson.gmo...@gmail.com wrote:
 If you're using a relational database it's pretty easy to execute case-
 insensitive queries because it's a native feature of the database.  It
 usually looks something like this:

  select * from Person where UPPER(lastName) = 'ROSS'

 This is a challenge for the App Engine Datastore because we rely on
 executing scans over a limited subset of your data and returning
 results as we encounter them.  Consider the following strings in
 lexicographic order:
 ...
 ROSE
 ...
 ROSS
 ...
 ROSTA
 ...
 Rose
 ...
 Ross
 ...
 Rosta
 ...
 rose
 ...
 ross
 ...
 rosta
 ...

 As you can see there could be an unlimited number of rows in between
 'ROSS' and 'Ross', and also between 'Ross' and 'ross' (okay not
 unlimited, we do have a limit on indexed string length, but definitely
 Large), so if we start scanning at 'ROSS' we might have to skip a huge
 number of results before we hit 'Ross', and that doesn't scale.  This
 is why the datastore doesn't support case-insensitive queries.

 Fortunately it's not hard to implement support for case-insensitive
 queries in your application.  Here's the approach: for each field that
 you want to query in a case-insensitive way, create a duplicate field
 that stores the value of that field in either all upper or lowercase
 letters.  Then, have your model object register a pre-persist callback
 with your persistence framework (JPA or JDO), and then populate the
 duplicate fields inside that callback.  Here's what it looks like.

 JPA:
 import javax.persistence.PrePersist;
 import javax.persistence.PreUpdate;

 @Entity
 public class Person {
   @Id
   @GeneratedValue(strategy = GenerationType.IDENTITY)
   private Long id;
   private String lastName;
   private String uppercaseLastName;

   public Long getId() {
     return id;
   }

   public String getLastName() {
     return lastName;
   }

   public void setLastName(String lastName) {
     this.lastName = lastName;
   }

   @PrePersist
   @PreUpdate
   public void prePersist() {
     if (lastName != null) {
       uppercaseLastName = lastName.toUpperCase();
     } else {
       uppercaseLastName = null;
     }
   }

 }

 public ListPerson getPeopleByLastName(String lastName) {
   Query q = em.createQuery(select from Person where uppercaseLastName
 = :p);
   q.setParameter(p, lastName.toUpperCase());
   return (ListPerson) q.getResultList();

 }

 JDO:
 import javax.jdo.listener.StoreCallback;

 @PersistenceCapable(identityType = IdentityType.APPLICATION)
 public class Person implements StoreCallback {
   @PrimaryKey
   @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
   private Long id;
   private String lastName;
   private String uppercaseLastName;

   public Long getId() {
     return id;
   }

   public String getLastName() {
     return lastName;
   }

   public void setLastName(String lastName) {
     this.lastName = lastName;
   }

   public void jdoPreStore() {
     if (lastName != null) {
       uppercaseLastName = lastName.toUpperCase();
     } else {
       uppercaseLastName = null;
     }
   }

 }

 public ListPerson getPeopleByLastName(String lastName) {
   Query q = pm.newQuery(Person.class, uppercaseLastName == :p);
   return (ListPerson) q.execute(lastName.toUpperCase());

 }

 On 21 mar, 17:41, vchalmel groupeprojeten...@gmail.com wrote:

  Hi !

  I wonder how to compare a String stored in a datastore entity (field
  login) to an entry (also named login) considering their lowercase
  version :
   - I tried : SELECT count(this) FROM +Gestionnaire.class.getName()+
  WHERE LowerCase(login)== LowerCase('+login+') which throws an
  exception

  I have another problem with this query, I don't want to exclude the
  one-character login like, maybe 1, but this also fires an exception
  (  java.lang.Character cannot be cast to
  com.google.appengine.api.datastore.Key )

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: How to call a URL of your own application?

2010-03-22 Thread dhruvbird

Hello,
And another thing I want to be able to do is figure out the HTTP
return code for that internal URL. Since everything in already a web
hook, I might as well model my app. in a similar fashion. I was
wondering if the appengine guys could work out a hack for urllib where
calls to internal URLs don't cost you an extra request. That way, I
would generously use it for (synchronous) pubsub or some sort of
internal communication which may be externalized later.

Regards,
-Dhruv.

On Mar 22, 6:30 am, Eli Jones eli.jo...@gmail.com wrote:
Depends on how you want the code to work..

If you just want to send of work to get processed by those other handlers,
then firing off a Task (either with Task Queue or Deferred Task) would be
perfect for that.

http://code.google.com/appengine/docs/python/taskqueue/overview.htmlhttp://code.google.com/appengine/articles/deferred.html

for eg: If I have these URLs:
/mail/send/
/chat/send/
/home/

If I want to call them from within the handler of /home/, how can I do
that?

Regards,
-Dhruv.

--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Error if I set required=False for a StringListProperty

2010-03-22 Thread dhruvbird

Hello,
  I have a model with a single attribute that is a StringListProperty.
I get an error if I define it as such:

class Test(db.Expando):
people = db.StringListProperty(required=False, indexed=True)

However, on changing it to:
class Test(db.Expando):
people = db.StringListProperty(required=True, indexed=True,
default=[])

It starts working. Any ideas why this is happening??

Regards,
-Dhruv.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Anyone noticing their requests taking more CPU usage today?

2010-03-22 Thread Blake

My crontabs have been taking around 500-600 cpu seconds, but today
have been up over 1000 cpu seconds.  I'm not asserting that there's an
issue - it's possible it was a change I made, which is why I'm asking
around.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

45 matches

Mail list logo