date:20091203

anyone???

--

You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] Re: how do i use Admins Emailed quota?

2009-12-03 Thread Don Schwarz

http://code.google.com/appengine/docs/java/mail/usingjavamail.html#Features_of_the_Low_level_API

On Thu, Dec 3, 2009 at 8:00 AM, Prashant antsh...@gmail.com wrote:

 anyone???

 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine for Java group.
 To post to this group, send email to
 google-appengine-j...@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com
 .
 For more options, visit this group at
 http://groups.google.com/group/google-appengine-java?hl=en.


--

You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] Re: how do i use Admins Emailed quota?

thanks a lot

--

You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] Re: how do i use Admins Emailed quota?

what if I want to send only one of the admins? there is no way, right?
does that work for XMPP also?

--

You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] Re: Text Search Support for Java

I'll also point out that while it's not on our official roadmap, we have
marked http://code.google.com/p/googleappengine/issues/detail?id=217 as
started. Development is still in the very early stages so the only thing
it's safe to conclude about timeline is that we don't expect it to be ready
within 6 months (the scope of the roadmap). When we have a better sense of
when it will be ready it will appear on the roadmap.

Max

On Wed, Dec 2, 2009 at 3:46 PM, Don Schwarz schwa...@google.com wrote:

FYI, it should be possible to port SearchableModel to Java and run it on
top of our low-level datastore API. We have no immediate plans to do that,
but it would make a nice open-source contribution.

You may want to play around with SearchableModel in Python first to ensure
that its performance characteristics and functionality are acceptable for
your task before you invest the effort, though.

On Wed, Dec 2, 2009 at 5:44 PM, Ikai L (Google) ika...@google.com wrote:

Full text search isn't on our public roadmap:
http://code.google.com/appengine/docs/roadmap.html

The link describes what we plan on releasing in the upcoming months.

On Wed, Dec 2, 2009 at 3:28 PM, steveb steve.buikhui...@gmail.comwrote:

I'm also very interested in this feature. I'm looking at plugging in
Lucene ( http://www.kimchy.org/searchable-google-appengine-with-compass
) to achieve search but I can imagine that there will be lots of
issues with storage, indexing, security, CPU use etc with this
solution.

If we can get an indication of rough roadmap then we can decide
whether to delay and wait or bite off this hairy bit of integration
work.

p.s. would it be possible to bridge between java and python
SearchableModel? I think this would be less total work than plugging
in Lucene. I'd be happy to hear any war stories in this area.

Thanks, Steve

On Dec 3, 7:53 am, lent lentakeu...@gmail.com wrote:
Hi,

This question is targeted at Google guys. Is there any plans to have
limited support for full text search for Java, i.e. something like the
equivalent of SearchableModel in Python, and if so when can this be
expected? When is FULL support for full text search be expected for
appengine (Python and Java)?

Regards,
Len

You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to
google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com
.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.

--
Ikai Lan
Developer Programs Engineer, Google App Engine

--
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to
google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com
.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.

You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] Re: Text Search Support for Java

2009-12-03 Thread Erdinc Yilmazel

For small and readonly indexes you can use lucene and it works without any
issues. All you need to do is generate your index outside the appengine
environment and put your index files in the war package that is deployed.
(Somewhere under WEB-INF would be the best place I think)

Erdinc

On Thu, Dec 3, 2009 at 5:42 PM, Max Ross (Google)
maxr+appeng...@google.commaxr%2bappeng...@google.com
wrote:

Max

On Wed, Dec 2, 2009 at 3:46 PM, Don Schwarz schwa...@google.com wrote:

FYI, it should be possible to port SearchableModel to Java and run it on
top of our low-level datastore API. We have no immediate plans to do that,
but it would make a nice open-source contribution.

You may want to play around with SearchableModel in Python first to ensure
that its performance characteristics and functionality are acceptable for
your task before you invest the effort, though.

On Wed, Dec 2, 2009 at 5:44 PM, Ikai L (Google) ika...@google.comwrote:

Full text search isn't on our public roadmap:
http://code.google.com/appengine/docs/roadmap.html

The link describes what we plan on releasing in the upcoming months.

On Wed, Dec 2, 2009 at 3:28 PM, steveb steve.buikhui...@gmail.comwrote:

If we can get an indication of rough roadmap then we can decide
whether to delay and wait or bite off this hairy bit of integration
work.

p.s. would it be possible to bridge between java and python
SearchableModel? I think this would be less total work than plugging
in Lucene. I'd be happy to hear any war stories in this area.

Thanks, Steve

On Dec 3, 7:53 am, lent lentakeu...@gmail.com wrote:
Hi,

Regards,
Len

You received this message because you are subscribed to the Google
Groups Google App Engine for Java group.
To post to this group, send email to
google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com
.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.

--
Ikai Lan
Developer Programs Engineer, Google App Engine

You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Re: how do i use Admins Emailed quota?

2009-12-03 Thread rnawky

You set them up as an administrator to your App Engine Application

On Dec 1, 11:55 pm, Prashant antsh...@gmail.com wrote:
 Hi,

 whosoever I send a mail it gets counted in Recipients Emailed quota, how
 do I use Admins Emailed quota to send mails to admins.

 Thanks.

--

You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] Re: how do i use Admins Emailed quota?

no, no, I meat to say that, suppose there are 3 admins and I want to send
mail to only one admin using admin quota. this is not possible, rite?

And, does following work for XMPP?

XMPPServiceFactory.getXMPPService()
.sendMessage(
  new MessageBuilder()
.withRecipientJids(new JID(*admins*))
.withBody(some message)
.build());

--

You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] 1.2.8 SDK Prerelease - help us verify!

2009-12-03 Thread Vince Bonfanti

I just started testing with the 1.2.8 prerelease, and I'm getting the
following exception from Queue.add() in code that works in 1.2.6:

java.lang.IllegalStateException: Current enviornment must have the server
url available via the com.google.appengine.server_url_key attribute.

This only happens when Queue.add() is invoked from the servlet init() method
or from a static initializer; if Queue.add() is invoked from a regular
request thread, then it works properly.

Vince

On Tue, Nov 24, 2009 at 9:00 PM, Ikai L (Google) ika...@google.com wrote:

 Hello App Engine Developers,

 As part of our ongoing efforts to improve release quality and
 transparency, we will start prereleasing SDKs for early testing. We
 hope this gives developers a chance to participate in our release
 process by trying out new changes and sending feedback. As of this
 morning, the prerelease SDK for our next release, 1.2.8, is available
 in the familiar download location (note that the filename ends in
 'prerelease.zip'):

 http://code.google.com/p/googleappengine/downloads/list

 If you're interested, please download and give it a try locally with
 your favorite App Engine code. Please note that, as a prerelease, this
 SDK is not yet supported and still subject to change. Thus, please
 don't take critical dependencies or make substantial changes to
 production apps based on this SDK.

 Importantly, this prerelease is purely for the SDK and is intended for
 local testing and development in dev_appserver. The server-side of App
 Engine (our production environment) is not at 1.2.8, so deploying with
 this SDK is not yet supported. In the future, we might enable a
 complete SDK and server test environment for prereleases.

 A few notes on 1.2.8 in particular - this release is primarily for
 servicing and updates in preparation for some exciting feature
 launches we have in the pipeline. The current release notes (still
 subject to change) are included below; these release notes do include
 changes which will only be available on the server side Admin Console
 (non-local) once 1.2.8 is formally released.

 Please try 1.2.8 for local development and send us your feedback!

 Thanks,

 App Engine Team

 Version 1.2.8
 =
  - Support for JAXB. JAXB is included in JDK 1.6 (and App Engine's
 production
servers). If you're using JDK 1.5 with your local dev_appserver,
 you will
need to include the JAXB libraries with your application to use
 it.
  http://code.google.com/p/googleappengine/issues/detail?id=1267
  - Added Quota API (com.google.appengine.api.quota) to match Python
 API.
  - Low-level Memcache API now supports grabTail() and batchIncrement
 ().
  - HTTPResponse object now has getFinalUrl() method for 302
 redirects.
  http://code.google.com/p/googleappengine/issues/detail?id=1464
  - Java Dev Appserver now automatically executes tasks.  If you
 prefer the old
behavior where tasks do not automatically execute you can use the
  -Dtask_queue.disable_auto_task_execution flag when starting the
 server.
  - Additional file extensions permitted when sending mail.
  http://code.google.com/p/googleappengine/issues/detail?id=494
  - Fixed issue with Java mail handler not processing multipart
 messages
correctly.
  - Fixed agent code included in appengine-local-runtime.jar results
 in
RuntimeException.
  http://code.google.com/p/googleappengine/issues/detail?id=2280
  - Fixed issue with sort orders defined on properties that allow
 multiple
values.
  http://code.google.com/p/googleappengine/issues/detail?id=2349
  - Fixed problem with dropped query strings after requiring log-in.
  http://code.google.com/p/googleappengine/issues/detail?id=2225
  - Removed limitation preventing multiple parameters with the same
 name.
  http://code.google.com/p/googleappengine/issues/detail?id=2090
  - Fixed issue with local datastore incorrectly sorting results of
 ancestor queries.
  http://code.google.com/p/googleappengine/issues/detail?id=2177
  - New Index building status page in the Admin Console
  - Task Queue now supports purging queues, and deleting tasks and
 queues via
the Admin Console.
  http://code.google.com/p/googleappengine/issues/detail?id=2159
  http://code.google.com/p/googleappengine/issues/detail?id=1740
  - Over Quota HTTP status code changed from 403 to 503, other to 500.
  - Task Queue now considers all HTTP 2xx status codes to represent
 success.

 ORM Changes

  - Explicitly disallow multiple relationships of the same type
  http://code.google.com/p/datanucleus-appengine/issues/detail?id=154
  - Occasional ArrayOutOfBoundsIndexException
  http://code.google.com/p/datanucleus-appengine/issues/detail?id=156
  - Support inheritance
  http://code.google.com/p/datanucleus-appengine/issues/detail?id=25
  - Support != queries
  - Support IN queries
  http://code.google.com/p/datanucleus-appengine/issues/detail?id=39
  - Inheritance is working for

Re: [appengine-java] Re: how do i use Admins Emailed quota?

2009-12-03 Thread Don Schwarz

On Thu, Dec 3, 2009 at 12:02 PM, Prashant antsh...@gmail.com wrote:

 no, no, I meat to say that, suppose there are 3 admins and I want to send
 mail to only one admin using admin quota. this is not possible, rite?


Correct.


 And, does following work for XMPP?

 XMPPServiceFactory.getXMPPService()
 .sendMessage(
   new MessageBuilder()
 .withRecipientJids(new JID(*admins*))
 .withBody(some message)
 .build());


Not currently, no.  I'm not sure whether that would be a useful feature or
not, but feel free to file an issue in the issue tracker if you have a use
case for it.

--

You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Why no GAE system property?

2009-12-03 Thread Jeff Schnitzer

Why isn't there a system property that can be used to detect if a
program is running on appengine?

This is really a glaring oversight.  I would expect something like
System.getProperty(google.appengine) to be development or
production (or of course null).

Before someone says check the servlet context, realize that this is
grossly inadequate - library writers do not have access to the servlet
context.

I'm trying to get the Hessian RPC library working on appengine so we
have an easy-to-use web services protocol.  One of the first things
the hessian java client does is teardown any keepalive connections by
making a quick request expired at 10ms.  This terminates with a caught
IOException in the normal world, but produces an
ApiDeadlineExceededException in appengine.

I'd like to simply short-circuit this unnecessary behavior when
running on appengine.  I found the SecurityManager check but that
seems unreliable.  Why isn't there a nice easy system property?

Thanks,
Jeff

--

You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] 1.2.8 SDK Prerelease - help us verify!

Hi Vince,

That sounds like a bug in our code that was a side effect of the automatic
task execution that now happens in the dev environment.  Let me see if I can
come up with a workaround for you.

Max
On Thu, Dec 3, 2009 at 10:06 AM, Vince Bonfanti vbonfa...@gmail.com wrote:

 I just started testing with the 1.2.8 prerelease, and I'm getting the
 following exception from Queue.add() in code that works in 1.2.6:

 java.lang.IllegalStateException: Current enviornment must have the server
 url available via the com.google.appengine.server_url_key attribute.

 This only happens when Queue.add() is invoked from the servlet init()
 method or from a static initializer; if Queue.add() is invoked from a
 regular request thread, then it works properly.

 Vince


 On Tue, Nov 24, 2009 at 9:00 PM, Ikai L (Google) ika...@google.comwrote:

 Hello App Engine Developers,

 As part of our ongoing efforts to improve release quality and
 transparency, we will start prereleasing SDKs for early testing. We
 hope this gives developers a chance to participate in our release
 process by trying out new changes and sending feedback. As of this
 morning, the prerelease SDK for our next release, 1.2.8, is available
 in the familiar download location (note that the filename ends in
 'prerelease.zip'):

 http://code.google.com/p/googleappengine/downloads/list

 If you're interested, please download and give it a try locally with
 your favorite App Engine code. Please note that, as a prerelease, this
 SDK is not yet supported and still subject to change. Thus, please
 don't take critical dependencies or make substantial changes to
 production apps based on this SDK.

 Importantly, this prerelease is purely for the SDK and is intended for
 local testing and development in dev_appserver. The server-side of App
 Engine (our production environment) is not at 1.2.8, so deploying with
 this SDK is not yet supported. In the future, we might enable a
 complete SDK and server test environment for prereleases.

 A few notes on 1.2.8 in particular - this release is primarily for
 servicing and updates in preparation for some exciting feature
 launches we have in the pipeline. The current release notes (still
 subject to change) are included below; these release notes do include
 changes which will only be available on the server side Admin Console
 (non-local) once 1.2.8 is formally released.

 Please try 1.2.8 for local development and send us your feedback!

 Thanks,

 App Engine Team

 Version 1.2.8
 =
  - Support for JAXB. JAXB is included in JDK 1.6 (and App Engine's
 production
servers). If you're using JDK 1.5 with your local dev_appserver,
 you will
need to include the JAXB libraries with your application to use
 it.
  http://code.google.com/p/googleappengine/issues/detail?id=1267
  - Added Quota API (com.google.appengine.api.quota) to match Python
 API.
  - Low-level Memcache API now supports grabTail() and batchIncrement
 ().
  - HTTPResponse object now has getFinalUrl() method for 302
 redirects.
  http://code.google.com/p/googleappengine/issues/detail?id=1464
  - Java Dev Appserver now automatically executes tasks.  If you
 prefer the old
behavior where tasks do not automatically execute you can use the
  -Dtask_queue.disable_auto_task_execution flag when starting the
 server.
  - Additional file extensions permitted when sending mail.
  http://code.google.com/p/googleappengine/issues/detail?id=494
  - Fixed issue with Java mail handler not processing multipart
 messages
correctly.
  - Fixed agent code included in appengine-local-runtime.jar results
 in
RuntimeException.
  http://code.google.com/p/googleappengine/issues/detail?id=2280
  - Fixed issue with sort orders defined on properties that allow
 multiple
values.
  http://code.google.com/p/googleappengine/issues/detail?id=2349
  - Fixed problem with dropped query strings after requiring log-in.
  http://code.google.com/p/googleappengine/issues/detail?id=2225
  - Removed limitation preventing multiple parameters with the same
 name.
  http://code.google.com/p/googleappengine/issues/detail?id=2090
  - Fixed issue with local datastore incorrectly sorting results of
 ancestor queries.
  http://code.google.com/p/googleappengine/issues/detail?id=2177
  - New Index building status page in the Admin Console
  - Task Queue now supports purging queues, and deleting tasks and
 queues via
the Admin Console.
  http://code.google.com/p/googleappengine/issues/detail?id=2159
  http://code.google.com/p/googleappengine/issues/detail?id=1740
  - Over Quota HTTP status code changed from 403 to 503, other to 500.
  - Task Queue now considers all HTTP 2xx status codes to represent
 success.

 ORM Changes

  - Explicitly disallow multiple relationships of the same type
  http://code.google.com/p/datanucleus-appengine/issues/detail?id=154
  - Occasional ArrayOutOfBoundsIndexException

Re: [appengine-java] Re: Text Search Support for Java

2009-12-03 Thread Jess Evans

The compass solution is probably sufficient for pet projects. If I
correctly recall, there are still serious scalability issues due to the way
indexes are currently stored and restrictions on app store blob size. The
main issue is initial timeouts due to the GAE design flaw wrt startup /
initial request - timeout. Initializing compass, lucene, jdo, and serving
even a simple request cuts very close to the deadline.

On Wed, Dec 2, 2009 at 6:28 PM, steveb steve.buikhui...@gmail.com wrote:

If we can get an indication of rough roadmap then we can decide
whether to delay and wait or bite off this hairy bit of integration
work.

p.s. would it be possible to bridge between java and python
SearchableModel? I think this would be less total work than plugging
in Lucene. I'd be happy to hear any war stories in this area.

Thanks, Steve

On Dec 3, 7:53 am, lent lentakeu...@gmail.com wrote:
Hi,

Regards,
Len

You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] Why no GAE system property?

2009-12-03 Thread Vince Bonfanti

I agree. My workaround was to create a ServletEventListener and put this in
the contextInitialized() method:

System.setProperty( appengine.server,
event.getServletContext().getServerInfo().contains( Development ) ?
development : production );

Vince

On Thu, Dec 3, 2009 at 1:15 PM, Jeff Schnitzer lhori...@gmail.com wrote:

Why isn't there a system property that can be used to detect if a
program is running on appengine?

This is really a glaring oversight. I would expect something like
System.getProperty(google.appengine) to be development or
production (or of course null).

Before someone says check the servlet context, realize that this is
grossly inadequate - library writers do not have access to the servlet
context.

I'm trying to get the Hessian RPC library working on appengine so we
have an easy-to-use web services protocol. One of the first things
the hessian java client does is teardown any keepalive connections by
making a quick request expired at 10ms. This terminates with a caught
IOException in the normal world, but produces an
ApiDeadlineExceededException in appengine.

I'd like to simply short-circuit this unnecessary behavior when
running on appengine. I found the SecurityManager check but that
seems unreliable. Why isn't there a nice easy system property?

Thanks,
Jeff

You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] Re: Datanucleus Exception - deletePersistent() called recursively

2009-12-03 Thread Jeffrey Goetsch

Here are the Domain classes that have the problem.  Please let me know if
you need more information.

--Jeff

@PersistenceCapable(identityType = IdentityType.APPLICATION)
public class Player {
@PrimaryKey
@Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
private Long key;

@Persistent
private Long userKey;
@Persistent
private String userName;

@Persistent
private Long cityKey;

@Persistent
private Long originalFamilyKey;
@Persistent
private Long currentFamilyKey;

// Money 

@Persistent
private int money = 0;
@Persistent
private int totalMoneyReceived = 0;

//* Store 

@Persistent
private int storeX = -1;
@Persistent
private int storeY = -1;
@Persistent
private String storeName;
@Persistent
private String storeImage;

@Persistent(mappedBy = target)
private SetConvincer convincers = new HashSetConvincer();

//* Troops/Challenge 

@Persistent
private int muscle;
@Persistent
private int dames;

@Persistent(mappedBy = owner)
private SetTroopHire activeTroopHires = new HashSetTroopHire();

@Persistent
private Long activeChallengeAgainstKey;
@Persistent(defaultFetchGroup = true)
private SetLong activeChallengeMoveKeys;

public SetTroopHire getActiveTroopHiresClean() {
IteratorTroopHire hireIterator = activeTroopHires.iterator();
DateTime now = TimeUtil.currentTime();

while (hireIterator.hasNext()) {
TroopHire troopHire = hireIterator.next();
if (troopHire.getAvailableTime().isBefore(now) ||
troopHire.isEmpty()) {
//This is a work around for a GAE Bug 2009.12.1
hireIterator.remove();
}
}
return activeTroopHires;
}


public void unhireTroops(Troops troops, DateTime hireDate) {
//TODO run Nate's Algorithm to check that you can unhire
City city = PM.getObjectById(City.class, cityKey);
boolean found = false;

for (TroopHire troopHire : getActiveTroopHiresClean()) {
if (troopHire.getHireTime().equals(hireDate)) {
found = true;
troopHire.unhireTroops(troops);

muscle -= troops.muscle;
dames -= troops.dames;

addMoney(city.getDeployCost(troops));

break;
}
}

if (!found) {
throw new SlowPlayException(Unhire not legal. Bad
availableTime);
}
}
}

@PersistenceCapable(identityType = IdentityType.APPLICATION)
public class TroopHire implements ComparableTroopHire {
@PrimaryKey
@Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
private Key key;

@Persistent
private Player owner;
@Persistent
private int muscle;
@Persistent
private int dames;
@Persistent
private Date hireTime;
@Persistent
private Date availableTime;
}


On Tue, Dec 1, 2009 at 9:49 AM, Max Ross (Google)
maxr+appeng...@google.commaxr%2bappeng...@google.com
 wrote:

 Could you please post the model objects that are involved?  Also, what
 version of the SDK are you using?

 Thanks,
 Max

 On Mon, Nov 30, 2009 at 2:26 PM, Jeffrey Goetsch jeffg@gmail.comwrote:

 Update:  This appears to happen when I remove the last entry in the Set.

 --Jeff


 On Mon, Nov 30, 2009 at 2:21 PM, Jeffrey Goetsch jeffg@gmail.comwrote:

 I am getting this exception when I try and remove an Owned object from a
 Set.   Let me know if you need more then the stack trace.

 Thanks,
 Jeffrey


 Caused by: org.datanucleus.exceptions.NucleusUserException:
 deletePersistent() called recursively
  at
 org.datanucleus.state.JDOStateManagerImpl.internalDeletePersistent(JDOStateManagerImpl.java:4184)
 at
 org.datanucleus.state.JDOStateManagerImpl.deletePersistent(JDOStateManagerImpl.java:4166)
  at
 org.datanucleus.ObjectManagerImpl.deleteObjectInternal(ObjectManagerImpl.java:1470)
 at
 org.datanucleus.store.mapped.scostore.FKSetStore.remove(FKSetStore.java:527)
  at
 org.datanucleus.store.appengine.DatastoreFKSetStore.remove(DatastoreFKSetStore.java:78)
 at org.datanucleus.sco.backed.Set.remove(Set.java:712)
  at org.datanucleus.sco.backed.Set.remove(Set.java:672)
 at
 org.datanucleus.store.mapped.mapping.PersistenceCapableMapping.preDelete(PersistenceCapableMapping.java:1308)
  at
 org.datanucleus.store.appengine.DependentDeleteRequest.execute(DependentDeleteRequest.java:71)
 at
 org.datanucleus.store.appengine.DatastorePersistenceHandler.deleteObject(DatastorePersistenceHandler.java:533)
  at
 org.datanucleus.state.JDOStateManagerImpl.internalDeletePersistent(JDOStateManagerImpl.java:4198)
 at
 org.datanucleus.state.JDOStateManagerImpl.deletePersistent(JDOStateManagerImpl.java:4166)
  at
 org.datanucleus.ObjectManagerImpl.deleteObjectInternal(ObjectManagerImpl.java:1470)
 at

Re: [appengine-java] Why no GAE system property?

2009-12-03 Thread Toby Reyelts

Yes, I totally agree that the current official way of checking for App
Engine presence is not great for third party libraries which don't even
necessarily run in a servlet container. We've been putting together a
solution for this and hope to make it available soon.

On Thu, Dec 3, 2009 at 1:15 PM, Jeff Schnitzer lhori...@gmail.com wrote:

Why isn't there a system property that can be used to detect if a
program is running on appengine?

This is really a glaring oversight. I would expect something like
System.getProperty(google.appengine) to be development or
production (or of course null).

Before someone says check the servlet context, realize that this is
grossly inadequate - library writers do not have access to the servlet
context.

I'd like to simply short-circuit this unnecessary behavior when
running on appengine. I found the SecurityManager check but that
seems unreliable. Why isn't there a nice easy system property?

Thanks,
Jeff

You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] Why no GAE system property?

2009-12-03 Thread Jeffrey Goetsch

Could you create a Servlet that is loaded on launch, and set this kind of
property? I think this is a workaround until they can do it officially.

--Jeff

On Thu, Dec 3, 2009 at 10:39 AM, Toby Reyelts to...@google.com wrote:

On Thu, Dec 3, 2009 at 1:15 PM, Jeff Schnitzer lhori...@gmail.com wrote:

Why isn't there a system property that can be used to detect if a
program is running on appengine?

This is really a glaring oversight. I would expect something like
System.getProperty(google.appengine) to be development or
production (or of course null).

Before someone says check the servlet context, realize that this is
grossly inadequate - library writers do not have access to the servlet
context.

I'd like to simply short-circuit this unnecessary behavior when
running on appengine. I found the SecurityManager check but that
seems unreliable. Why isn't there a nice easy system property?

Thanks,
Jeff

You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] Why no GAE system property?

2009-12-03 Thread Jeff Schnitzer

On Thu, Dec 3, 2009 at 10:46 AM, Jeffrey Goetsch jeffg@gmail.com wrote:
 Could you create a Servlet that is loaded on launch, and set this kind of
 property?  I think this is a workaround until they can do it officially.

I can't - remember this is in a library, so I would have to force all
the users to setup such a servlet.

However, I've worked around the problem in this particular case.  I
have Hessian (both client and server) working on GAE... which means we
finally have an easy web service stack!  I will submit my patches to
Caucho today.

Jeff

--

You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] 1.2.8 SDK Prerelease - help us verify!

Vince,

As a workaround, try setting the missing environment attribute explicitly
before you add the task to the queue:

ApiProxy.getCurrentEnvironment().getAttributes().put(com.google.appengine.server_url_key,
http://localhost:8080;);

This should only be necessary for tasks that are added when there is no
live request and it should have no impact in prod.

Sorry for the trouble,
Max

On Thu, Dec 3, 2009 at 10:20 AM, Max Ross (Google) 
maxr+appeng...@google.com maxr%2bappeng...@google.com wrote:

 Hi Vince,

 That sounds like a bug in our code that was a side effect of the automatic
 task execution that now happens in the dev environment.  Let me see if I can
 come up with a workaround for you.

 Max

 On Thu, Dec 3, 2009 at 10:06 AM, Vince Bonfanti vbonfa...@gmail.comwrote:

 I just started testing with the 1.2.8 prerelease, and I'm getting the
 following exception from Queue.add() in code that works in 1.2.6:

 java.lang.IllegalStateException: Current enviornment must have the server
 url available via the com.google.appengine.server_url_key attribute.

 This only happens when Queue.add() is invoked from the servlet init()
 method or from a static initializer; if Queue.add() is invoked from a
 regular request thread, then it works properly.

 Vince


 On Tue, Nov 24, 2009 at 9:00 PM, Ikai L (Google) ika...@google.comwrote:

 Hello App Engine Developers,

 As part of our ongoing efforts to improve release quality and
 transparency, we will start prereleasing SDKs for early testing. We
 hope this gives developers a chance to participate in our release
 process by trying out new changes and sending feedback. As of this
 morning, the prerelease SDK for our next release, 1.2.8, is available
 in the familiar download location (note that the filename ends in
 'prerelease.zip'):

 http://code.google.com/p/googleappengine/downloads/list

 If you're interested, please download and give it a try locally with
 your favorite App Engine code. Please note that, as a prerelease, this
 SDK is not yet supported and still subject to change. Thus, please
 don't take critical dependencies or make substantial changes to
 production apps based on this SDK.

 Importantly, this prerelease is purely for the SDK and is intended for
 local testing and development in dev_appserver. The server-side of App
 Engine (our production environment) is not at 1.2.8, so deploying with
 this SDK is not yet supported. In the future, we might enable a
 complete SDK and server test environment for prereleases.

 A few notes on 1.2.8 in particular - this release is primarily for
 servicing and updates in preparation for some exciting feature
 launches we have in the pipeline. The current release notes (still
 subject to change) are included below; these release notes do include
 changes which will only be available on the server side Admin Console
 (non-local) once 1.2.8 is formally released.

 Please try 1.2.8 for local development and send us your feedback!

 Thanks,

 App Engine Team

 Version 1.2.8
 =
  - Support for JAXB. JAXB is included in JDK 1.6 (and App Engine's
 production
servers). If you're using JDK 1.5 with your local dev_appserver,
 you will
need to include the JAXB libraries with your application to use
 it.
  http://code.google.com/p/googleappengine/issues/detail?id=1267
  - Added Quota API (com.google.appengine.api.quota) to match Python
 API.
  - Low-level Memcache API now supports grabTail() and batchIncrement
 ().
  - HTTPResponse object now has getFinalUrl() method for 302
 redirects.
  http://code.google.com/p/googleappengine/issues/detail?id=1464
  - Java Dev Appserver now automatically executes tasks.  If you
 prefer the old
behavior where tasks do not automatically execute you can use the
  -Dtask_queue.disable_auto_task_execution flag when starting the
 server.
  - Additional file extensions permitted when sending mail.
  http://code.google.com/p/googleappengine/issues/detail?id=494
  - Fixed issue with Java mail handler not processing multipart
 messages
correctly.
  - Fixed agent code included in appengine-local-runtime.jar results
 in
RuntimeException.
  http://code.google.com/p/googleappengine/issues/detail?id=2280
  - Fixed issue with sort orders defined on properties that allow
 multiple
values.
  http://code.google.com/p/googleappengine/issues/detail?id=2349
  - Fixed problem with dropped query strings after requiring log-in.
  http://code.google.com/p/googleappengine/issues/detail?id=2225
  - Removed limitation preventing multiple parameters with the same
 name.
  http://code.google.com/p/googleappengine/issues/detail?id=2090
  - Fixed issue with local datastore incorrectly sorting results of
 ancestor queries.
  http://code.google.com/p/googleappengine/issues/detail?id=2177
  - New Index building status page in the Admin Console
  - Task Queue now supports purging queues, and deleting tasks and
 queues via
the Admin Console.

Re: [appengine-java] 1.2.8 SDK Prerelease - help us verify!

2009-12-03 Thread Vince Bonfanti

That fixed it. Thanks.

Vince

On Thu, Dec 3, 2009 at 2:24 PM, Max Ross (Google)
maxr+appeng...@google.commaxr%2bappeng...@google.com
wrote:

Vince,

As a workaround, try setting the missing environment attribute explicitly
before you add the task to the queue:

ApiProxy.getCurrentEnvironment().getAttributes().put(com.google.appengine.server_url_key,
http://localhost:8080;);

This should only be necessary for tasks that are added when there is no
live request and it should have no impact in prod.

Sorry for the trouble,
Max

On Thu, Dec 3, 2009 at 10:20 AM, Max Ross (Google)
maxr+appeng...@google.com maxr%2bappeng...@google.com wrote:

Hi Vince,

That sounds like a bug in our code that was a side effect of the automatic
task execution that now happens in the dev environment. Let me see if I can
come up with a workaround for you.

Max

On Thu, Dec 3, 2009 at 10:06 AM, Vince Bonfanti vbonfa...@gmail.comwrote:

I just started testing with the 1.2.8 prerelease, and I'm getting the
following exception from Queue.add() in code that works in 1.2.6:

java.lang.IllegalStateException: Current enviornment must have the server
url available via the com.google.appengine.server_url_key attribute.

This only happens when Queue.add() is invoked from the servlet init()
method or from a static initializer; if Queue.add() is invoked from a
regular request thread, then it works properly.

Vince

On Tue, Nov 24, 2009 at 9:00 PM, Ikai L (Google) ika...@google.comwrote:

Hello App Engine Developers,

As part of our ongoing efforts to improve release quality and
transparency, we will start prereleasing SDKs for early testing. We
hope this gives developers a chance to participate in our release
process by trying out new changes and sending feedback. As of this
morning, the prerelease SDK for our next release, 1.2.8, is available
in the familiar download location (note that the filename ends in
'prerelease.zip'):

http://code.google.com/p/googleappengine/downloads/list

If you're interested, please download and give it a try locally with
your favorite App Engine code. Please note that, as a prerelease, this
SDK is not yet supported and still subject to change. Thus, please
don't take critical dependencies or make substantial changes to
production apps based on this SDK.

Importantly, this prerelease is purely for the SDK and is intended for
local testing and development in dev_appserver. The server-side of App
Engine (our production environment) is not at 1.2.8, so deploying with
this SDK is not yet supported. In the future, we might enable a
complete SDK and server test environment for prereleases.

A few notes on 1.2.8 in particular - this release is primarily for
servicing and updates in preparation for some exciting feature
launches we have in the pipeline. The current release notes (still
subject to change) are included below; these release notes do include
changes which will only be available on the server side Admin Console
(non-local) once 1.2.8 is formally released.

Please try 1.2.8 for local development and send us your feedback!

Thanks,

App Engine Team

Version 1.2.8
=
- Support for JAXB. JAXB is included in JDK 1.6 (and App Engine's
production
servers). If you're using JDK 1.5 with your local dev_appserver,
you will
need to include the JAXB libraries with your application to use
it.
http://code.google.com/p/googleappengine/issues/detail?id=1267
- Added Quota API (com.google.appengine.api.quota) to match Python
API.
- Low-level Memcache API now supports grabTail() and batchIncrement
().
- HTTPResponse object now has getFinalUrl() method for 302
redirects.
http://code.google.com/p/googleappengine/issues/detail?id=1464
- Java Dev Appserver now automatically executes tasks. If you
prefer the old
behavior where tasks do not automatically execute you can use the
-Dtask_queue.disable_auto_task_execution flag when starting the
server.
- Additional file extensions permitted when sending mail.
http://code.google.com/p/googleappengine/issues/detail?id=494
- Fixed issue with Java mail handler not processing multipart
messages
correctly.
- Fixed agent code included in appengine-local-runtime.jar results
in
RuntimeException.
http://code.google.com/p/googleappengine/issues/detail?id=2280
- Fixed issue with sort orders defined on properties that allow
multiple
values.
http://code.google.com/p/googleappengine/issues/detail?id=2349
- Fixed problem with dropped query strings after requiring log-in.
http://code.google.com/p/googleappengine/issues/detail?id=2225
- Removed limitation preventing multiple parameters with the same
name.
http://code.google.com/p/googleappengine/issues/detail?id=2090
- Fixed issue with local datastore incorrectly sorting results of
ancestor queries.
http://code.google.com/p/googleappengine/issues/detail?id=2177
- New

Re: [appengine-java] 1.2.8 SDK Prerelease - help us verify!

Sure, sorry for the trouble, and thanks for trying out the prerelease!

Max

On Thu, Dec 3, 2009 at 12:12 PM, Vince Bonfanti vbonfa...@gmail.com wrote:

That fixed it. Thanks.

Vince

On Thu, Dec 3, 2009 at 2:24 PM, Max Ross (Google)
maxr+appeng...@google.com maxr%2bappeng...@google.com wrote:

Vince,

As a workaround, try setting the missing environment attribute explicitly
before you add the task to the queue:

ApiProxy.getCurrentEnvironment().getAttributes().put(com.google.appengine.server_url_key,
http://localhost:8080;);

This should only be necessary for tasks that are added when there is no
live request and it should have no impact in prod.

Sorry for the trouble,
Max

On Thu, Dec 3, 2009 at 10:20 AM, Max Ross (Google)
maxr+appeng...@google.com maxr%2bappeng...@google.com wrote:

Hi Vince,

That sounds like a bug in our code that was a side effect of the
automatic task execution that now happens in the dev environment. Let me
see if I can come up with a workaround for you.

Max

On Thu, Dec 3, 2009 at 10:06 AM, Vince Bonfanti vbonfa...@gmail.comwrote:

I just started testing with the 1.2.8 prerelease, and I'm getting the
following exception from Queue.add() in code that works in 1.2.6:

java.lang.IllegalStateException: Current enviornment must have the
server url available via the com.google.appengine.server_url_key attribute.

This only happens when Queue.add() is invoked from the servlet init()
method or from a static initializer; if Queue.add() is invoked from a
regular request thread, then it works properly.

Vince

On Tue, Nov 24, 2009 at 9:00 PM, Ikai L (Google) ika...@google.comwrote:

Hello App Engine Developers,

http://code.google.com/p/googleappengine/downloads/list

Please try 1.2.8 for local development and send us your feedback!

Thanks,

App Engine Team

[appengine-java] Re: Datastore Statistics vs. Quota

2009-12-03 Thread Peter Recore

The datastore stats aren't real time as far as I know.  So part of the
discrepancy might be new entities created after the stats were
generated?

On Dec 3, 4:19 am, Toby tobias.ro...@sunnymail.mobi wrote:
 Hello,

 I just discovered the DataStore statistics.  That is a great thing!
 I got a question though. The Statistics say that the size of all my
 entities is 52Megabyte:

 Last updated    Total number of entities        Size of all entities
 2:46:26 ago     141,896                                 52 MBytes

 On the quota page it says that I stored 0.57 Gigabyte:
 0.57 of 1.00 GBytes

 I do not think that the datastore can have such a big overhead. So
 maybe the statistics are wrong?  Or is there data in the data store
 that I do not see in the statistics?

 Best regards,

 Tobias

--

You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Re: Java heap size

2009-12-03 Thread Philippe Marschall



On Dec 3, 4:51 am, jacek.ambroziak jacek.ambroz...@gmail.com
wrote:
 My application, an XML search engine, likes to manipulate its index
 data structures in memory during indexing.
 Searches are naturally much faster too when index data structures are
 in the heap.
 Unfortunately, I started running into OutOfMemory errors while
 indexing larger numbers of publications,
 but not huge number either: ~ 300. Data (typically byte arrays) I
 manipulate are on the order of 20 MB,
 again a very small mem requirement for this Century (both my dev
 computers are now 4 GB).

 Yes, I could bend over backwards and start using Datastore for some
 form of poor man's virtual memory
 but it wouldn't win all that much more space, would slow down and
 complicate the code.

 So... how much heap space do we have?

http://kohlerm.blogspot.com/2009/04/some-facts-about-java-used-by-googles.html

 Can we affect the size?

Doesn't look like it.

Cheers
Philippe

--

You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] TimeZone

2009-12-03 Thread Nour Ahmed

Can anyone help me to know how to configure the Timezone in the web
application that i run localy on my mmachine?

Thanks in advance!

--

You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Development Console - Datastore

2009-12-03 Thread Ian R.

I am entering data progamatically into the datastore but it is not
showing up in the development console. It appears that the data is
being entered since I am not seeing any errors when the makePersistent
() method is called.

--

You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] Re: oid is not instanceof javax.jdo.identity.ObjectIdentity

2009-12-03 Thread bryce cottam

Okay, sorry for the delay, I haven't been able to focus on this for a few days.

here is the full (including the jar libararies in
case there is an issue with them).

http://www.resmarksystems.com/code/JdoTest.zip

It's a very simple Servlet, by hitting the url hostname/jdotest you
should execute the test case that is failing for me.

I really appreciate your taking  a look at this.  I'm looking forward
to getting it working.

-bryce



On Tue, Dec 1, 2009 at 10:15 PM, Max Ross (Google)
maxr+appeng...@google.com wrote:
 Just give me the smallest amount of compilable, runnable code that
 demonstrates the incorrect behavior.  A unit test is preferable because I
 can just drop it into my own test framework and run it, but I'll take
 whatever format you can manage.

 Thanks,
 Max

 On Tue, Dec 1, 2009 at 1:18 PM, bryce cottam bcot...@gmail.com wrote:

 yeah, I didn't see a TransactionNotActiveException.
 here is my jdoconfig.xml:
   persistence-manager-factory name=transactions-optional
       property name=javax.jdo.PersistenceManagerFactoryClass

 value=org.datanucleus.store.appengine.jdo.DatastoreJDOPersistenceManagerFactory/
       property name=javax.jdo.option.ConnectionURL value=appengine/
       property name=javax.jdo.option.NontransactionalRead
 value=true/
       property name=javax.jdo.option.NontransactionalWrite
 value=true/
       property name=javax.jdo.option.RetainValues value=true/
       property name=datanucleus.appengine.autoCreateDatastoreTxns
 value=true/
   /persistence-manager-factory

 I think the only thing I changed was putting in this line:
       property name=datanucleus.appengine.autoCreateDatastoreTxns
 value=true/

 which may explain why I don't get the TransactionNotActiveException.
 So, for this unit test/servlet, you want me to just post the class
 file here, or strip down all my code and put up all my model classes
 and the test case/servlet code somewhere in a war or something?

  h I just commented out that line, and I still get the same
 behavior (i.e. I don't get the TransactionNotActiveException).  Not
 sure what's going on.

 thanks,
 -bryce

 On Tue, Dec 1, 2009 at 2:04 PM, Max Ross (Google)
 maxr+appeng...@google.com wrote:
  Now I'm starting to suspect something funny going on with your config.
  I
  received a TransactionNotActiveException when I tried to run your code
  without starting the txn, and the fact that you didn't receive that
  exception doesn't make any sense to me.  I can't explain why you see the
  exception and I don't.  The best thing would be for you to put together
  a
  stripped down test case that demonstrates the problem.  An actual unit
  test
  would be ideal, but if you're not a position to write one then just a
  simple
  standalone servlet should suffice.
 
  Thanks,
  Max
 
 
  On Tue, Dec 1, 2009 at 1:00 PM, bryce cottam bcot...@gmail.com wrote:
 
  that is both a type-o and it was missing  :)  It was missing from my
  test code, but it was present in my real code.  So, after putting
  the tx.begin() in the test code, I get the exact same behavior as the
  real code (i.e. nothing gets written to the datastore and I get the
  exception.  Sorry about missing that in the code I posted.
 
  I'm kinda lost on where to go with this and where to look.  I did
  notice that the code actually prompts the exception is in the
  datanucleus core and there is a comment above it that says:
  / TODO Factor out this PersistenceCapable reference
  ((PersistenceCapable)value).jdoCopyKeyFieldsFromObjectId(new
  AppIDObjectIdFieldConsumer(param, om, ps,
       javaTypeMappings), id);
 
  so, I'm not sure if this has soemthing to do with code style
  datanucleus is trying to phase out or what?  Again, I'm not gonna be
  shocked if I'm missing something though.
  thanks!
  -bryce
 
 
  On Tue, Dec 1, 2009 at 1:53 PM, Max Ross (Google)
  maxr+appeng...@google.com wrote:
   I don't see a call to tx.begin() in the code below.  Is that a typo
   or
   is it
   actually missing?
  
   On Tue, Dec 1, 2009 at 12:29 PM, bryce cottam bcot...@gmail.com
   wrote:
  
   thanks so much for checking into this Max.
  
   The data that I'm saving is user generated, so there's not a set
   script for it.  Conceptually what I'm doing is this:
  
                  Bundle bundle = new Bundle();
  
                  bundle.setName(My Bundle);
                  bundle.setDescription(new
   com.google.appengine.api.datastore.Text(My Description));
                  bundle.setPricingModel(PricingModel.PER_GUEST);
  
                  RatePlan ratePlan = new RatePlan();
                  ratePlan.setPricingModel(PricingModel.PER_GUEST);
                  ratePlan.setAdultPrice(new
   java.math.BigDecimal(300.00));
                  ratePlan.setYouthPrice(new
   java.math.BigDecimal(275.00));
  
                  bundle.setRatePlan(ratePlan);
  
                  ListKey activityIds = getActivityIds();
                  int i = 0;
                  for (Key id :

[appengine-java] Re: TimeZone

2009-12-03 Thread m seleron

Hi,

It is likely to possibly solve it by this.
Please refer to the following links.
http://www.javaworld.com/javaworld/jw-10-2003/jw-1003-time.html?page=3

The page in the back and forth might be also related.

thanks.


On 12月4日, 午前4:08, Nour Ahmed salahtimes...@googlemail.com wrote:
 Can anyone help me to know how to configure the Timezone in the web
 application that i run localy on my mmachine?

 Thanks in advance!

--

You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Re: App Engine SDK 1.2.8 released including new Admin Console features

2009-12-03 Thread Peter Recore

I was just looking at the release notes linked to by the App Engine
Blog post about this and I couldn't find any detail about the Improved
java compatibility stuff.  Is there another version of the release
notes with more detail?

The release notes linked to in the blog post were these:
http://code.google.com/p/googleappengine/wiki/SdkReleaseNotes

-peter

App Engine Team wrote:
 The App Engine team has been hard at work tackling our the issues on
 our tracker, tweaking APIs and closing bugs. In addition to a ton of
 bug fixes, 1.2.8 also includes:

 Enhanced Admin Console - Users will notice new tools for managing
 tasks and queues created with the Task Queue API, and more visibility
 into index processing.

 Improved Java Compatibility - This release adds support for new filter
 operators and inheritance to JPA and JDO as well as support for JAXB,
 the single most requested feature for the Java SDK.

 This was also the first release we previewed with developers before
 formally rolling out changes. Thanks very much to all the developers
 that gave us feedback on the preview release.

 1.2.8 is now available for both Python and Java developers. Take a
 look at our release notes for details on all the changes included in
 this release.

--

You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Re: App Engine SDK 1.2.8 released including new Admin Console features

2009-12-03 Thread Peter Recore

Found a better link for us java types:

http://code.google.com/p/googleappengine/wiki/SdkForJavaReleaseNotes


On Dec 3, 10:18 pm, Peter Recore peterrec...@gmail.com wrote:
 I was just looking at the release notes linked to by the App Engine
 Blog post about this and I couldn't find any detail about the Improved
 java compatibility stuff.  Is there another version of the release
 notes with more detail?

 The release notes linked to in the blog post were 
 these:http://code.google.com/p/googleappengine/wiki/SdkReleaseNotes

 -peter

 App Engine Team wrote:
  The App Engine team has been hard at work tackling our the issues on
  our tracker, tweaking APIs and closing bugs. In addition to a ton of
  bug fixes, 1.2.8 also includes:

  Enhanced Admin Console - Users will notice new tools for managing
  tasks and queues created with the Task Queue API, and more visibility
  into index processing.

  Improved Java Compatibility - This release adds support for new filter
  operators and inheritance to JPA and JDO as well as support for JAXB,
  the single most requested feature for the Java SDK.

  This was also the first release we previewed with developers before
  formally rolling out changes. Thanks very much to all the developers
  that gave us feedback on the preview release.

  1.2.8 is now available for both Python and Java developers. Take a
  look at our release notes for details on all the changes included in
  this release.

--

You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Usage of maps/google earth

2009-12-03 Thread Avis developer

Hi,
  Am doing a project on GPS plotting in appengine.. So can you help me
with it. I am from India, so still Google Street is not possible, can
you guys help with me it, anybody doing a related project pls help
me..; I nedd help in mapping services, pl ur help wil be greatly
appreciated..

--

You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Re: Collections usge in Appengine Java

2009-12-03 Thread Avis developer

Sir thanks for the reply , but pl help me sir, am jus a beginner, can
u pls be elaborate, thanks for spending ur time and effort

--

You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[google-appengine] DataStore Total Size Limit?

2009-12-03 Thread Seth

Hello,

Is there any limit to the total size of data we can store in the
DataStore?  I know we can purchase more storage, but wondering if
there's a hard limit.

Thanks!
Seth

--

You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] DataStore Total Size Limit?

2009-12-03 Thread Nick Johnson (Google)

Hi Seth,

There's no hard limit on the amount of datastore storage you can consume, as
long as you have the budget to pay for it.

-Nick Johnson

On Thu, Dec 3, 2009 at 9:02 AM, Seth sethl...@gmail.com wrote:

 Hello,

 Is there any limit to the total size of data we can store in the
 DataStore?  I know we can purchase more storage, but wondering if
 there's a hard limit.

 Thanks!
 Seth

 --

 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To post to this group, send email to google-appeng...@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com
 .
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.





-- 
Nick Johnson, Developer Programs Engineer, App Engine
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number:
368047

--

You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Stuck Index

2009-12-03 Thread Nick Johnson (Google)

Hi Prashant,

The time taken to build an index depends more on the indexes in the queue
ahead of yours than the size of your index. Index building may take up to
12-24 hours at busy times.

-Nick Johnson

On Wed, Dec 2, 2009 at 8:56 PM, Prashant antsh...@gmail.com wrote:

 It's more than an hr. and index still building just for 2 record, I guess
 it is stuck. appid - gaewcms ; index - _Content_ (Uri asc, Weight asc, Added
 desc)

  --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To post to this group, send email to google-appeng...@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com
 .
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.




-- 
Nick Johnson, Developer Programs Engineer, App Engine
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number:
368047

--

You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: DataStore Total Size Limit?

2009-12-03 Thread Seth

Aloha Nick

 There's no hard limit on the amount of datastore storage you can consume, as
 long as you have the budget to pay for it.

Great news!  I appreciate the quick and helpful reply.

Looking forward to trying it out,
Seth

--

You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Jaiku code and exploding indexes

2009-12-03 Thread Nick Johnson (Google)

Hi Carlos,

An index such as this on a single list property is not 'exploding' - the
number of index rows will be equal to the number of list entries. Exploding
indexes occur when you index two or more list properties, or the same one
twice.

-Nick Johnson

On Wed, Dec 2, 2009 at 6:13 PM, Carlos Alberto Machado
carlosalberto...@gmail.com wrote:

Hi,

I've studied the Jaiku code ( http://code.google.com/p/jaikuengine/ )
and noticed a piece of it that can cause an exploding index:

- kind: InboxEntry
properties:
- name: inbox
- name: created_at
direction: desc

'inbox' is a StringListProperty and the list is very large for real
cases (in fact, the number of followers of a user of the service).

How can I know if an index will be too large? What are the reference
values?

As I don't have much knowledge of GAE, I want to know if I'm right.

Thanks!

You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com
.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.

--
Nick Johnson, Developer Programs Engineer, App Engine
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number:
368047

You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: gql not giving full result set

2009-12-03 Thread Dinesh Varadharajan

Ikai,
Thanks for checking that out. We already exported and imported close
to 1 million records to make it work again. we left a sample set of
data for google to reproduce the issue.

However we are concerned about this bug being fixed and if possible
when it was fixed, since this had been happening quite frequently last
month.

On Dec 3, 12:00 am, Ikai L (Google) ika...@google.com wrote:
 Dinesh,

 I see the inconsistency. How long have you been writing to this dataset? My
 suspicion is that the indexes may have been updated incorrectly in a
 previous release due to a bug that we have since addressed. Unfortunately,
 the bug fix may not have retroactively addressed the incorrectly updated
 indexes.

 How large is the dataset? For small datasets, bulk exporting and importing
 will address the issue, but for large datasets, we'll have to look to an
 alternative solution.

 On Tue, Dec 1, 2009 at 3:40 AM, Dinesh Varadharajan 



 dinesh.varadhara...@orangescape.com wrote:
  Ikai,
  Unfortunately we don't have dummy data to showcase this. I have a
  reproducible case in production.

  The app id is os-dev.appspot.com.

  if you execute the query

  SELECT * FROM PrimaryData where ApplicationId =
  'Application_1652c875_be0f_11de_b4a5_a3c424aa5af6' and SheetMetadataId
  = 'Sheet001' and Deleted=False

  it returns 8 records.

  and if you execute

  SELECT * FROM PrimaryData where ApplicationId =
  'Application_1652c875_be0f_11de_b4a5_a3c424aa5af6' and SheetMetadataId
  = 'Sheet001'

   it should at least return 8 records(I am removing a condition). but
  it returns only 4 records.

  Please let me know if you want to be added as developer to os-dev to
  be able to access the datastore.

  Dinesh

  On Dec 1, 12:27 am, Ikai L (Google) ika...@google.com wrote:
   Prashant, do you have sample data you can provide? It's even better if it
   isn't real data.

   On Sun, Nov 29, 2009 at 3:19 AM, Prashant antsh...@gmail.com wrote:
what about this - I had a data store entity with known id, and JDOQL
  simply
failed to retrieve it by id throwing JDOObjectNotFoundException. I
  think
there is some major issue with datastore/indexes.

 --
You received this message because you are subscribed to the Google
  Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
  .
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com
  google-appengine%2bunsubscr...@googlegroups.comgoogle-appengine%252bunsubscr...@googlegroups.com

.
For more options, visit this group at
   http://groups.google.com/group/google-appengine?hl=en.

   --
   Ikai Lan
   Developer Programs Engineer, Google App Engine

  --

  You received this message because you are subscribed to the Google Groups
  Google App Engine group.
  To post to this group, send email to google-appeng...@googlegroups.com.
  To unsubscribe from this group, send email to
  google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com
  .
  For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.

 --
 Ikai Lan
 Developer Programs Engineer, Google App Engine

--

You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Has anyone done a source code control system hosted on App Engine?

2009-12-03 Thread Niklas Rosencrantz

On Wed, Dec 2, 2009 at 11:51 PM, samwyse samw...@gmail.com wrote:
 I'd like to host Hg or something someplace with better uptime than my
 home server.
There's Rietveld Code Review for Subversion svn which syncs to hg
http://code.google.com/p/rietveld/
I too look for good hg handling, somewhat stuck getting recommended
wing, komdo or eclipse asking for drPython,Boa,Eric,PIDA or something
like Tortoise for UNIX class working environment
best regards

--

You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Continuing HTTPErrr 302 when using remote_api after a 200 transactions - need to re-use connection handles

2009-12-03 Thread Tim Hoffman

Hi Ikai


I am using remote_api called from within plone.

Because I could potentially get multiple concurrent calls from plone
to push content I was creating a new connection
(requiring new authentication )  on each request to push content from
plone to app engine.

Unfortunately when I started pushing large amounts of content each
content entity being pushed was creating a new
connection (invoking authentication). When I hit 200 of these in the
space of say 10mins I would get hit by google infrastructure
which would basically stop transactions.  CheckSuccess didn't like
getting a redirect to the authentication stuff.
And I couldn't re-authenticate for about 10 mins anyway.  Looks like I
was running afoul of systems designed to prevent
spamming etc  Any other connection attempts from the same host
using that user id would also fail.
And none of this would be seen in the app engine logs, because the
remote_api auth/connection was being redirected
even before we got to appengine.

I found that if I cached the connection and didn't continually re-
authenticate I had no problems.

Hope that makes sense.

Rgds

Tim
On Dec 3, 3:35 am, Ikai L (Google) ika...@google.com wrote:
 Tim, great find. What do you mean re-use connections? If you let me know,
 I'll update the docs and open an issue to provide a better error message.

 On Tue, Dec 1, 2009 at 8:56 PM, Tim Hoffman zutes...@gmail.com wrote:
  Ok

  Got the the bottom of it all.

  You need to re-use connections.  If you make too many authenticated
  connections then googles higher up infrastructure
  shuts you down for about 10 min.

  It does highlight some problems in the remote_api code as the error
  messages passed back are particularly informative unless you really
  dig. but at least I understand where the problem is.

  T

  On Dec 2, 11:31 am, Tim Hoffman zutes...@gmail.com wrote:
   OK

   Started debugging inside the apiproxy_rpc.py(113)CheckSuccess()
   and what I believe is happening is correct.

   the rpc call MaekSyncCall is getting a redirect to a request to login

   apiproxy_rpc.py(113)CheckSuccess()

   The exception is a HTTPErrror which is a redirect (302)
   to

    'https://www.google.com/a/polytechnic.wa.edu.au/ServiceLogin?
   service=ahpassive=truecontinue=http://psc-prod1.appspot.com/_ah/login
   %3Fcon stuff deleted',

   Obviously requesting me to log in again.

   So somewhere I am running afoul of higher layers in google that watch
   what is going on.
   I am going to need some help from google engineers to get to the
   bottom of this one.

   Rgds

   Tim Hoffman

   On Dec 2, 11:04 am, Tim Hoffman zutes...@gmail.com wrote:

Something I have observerd and can now prove repeatably

Once I get the error 302 inside the remote_api call.  It takes
somewhere between 2 and 10 mins before I can resubmit the transaction.
During that that time, if I try the same call from another process (on
the same box with the same auth details) the call still gets the error
until such time as something in google lets it go.

It really is looking like I am being clobbered by some google watch
dog.

T

On Dec 2, 9:58 am, Tim Hoffman zutes...@gmail.com wrote:

 I am still getting this problem.

 One of the things I have noticed is the errors (302) are being
 propogated by google infrastructure and not recorded any way
 in the apps logs.

 My gut feel is that some other google infrastructure thinks I  am
 doing something I shouldn't requires me to re-authenticate but
 the remote_api proxies can't deal with a 302 inside a Namerror in
 CheckSuccess see stack below.

 Obviously what I am doing is different to the bulkloader client.

 Anyone got any suggestions on how I might go about getting to the
 bottom of this.

 T

 2009-12-02T01:55:49 INFO root Exception sending Rollback:
 Traceback (most recent call last):
   File /opt/ktstudio/zope/instances/prod/swantafe.buildout/
 google_appengine/google/appengine/api/datastore.py, line 1989, in
 RunInTransactionCustomRetries
     tx.handle, resp)
   File /opt/ktstudio/zope/instances/prod/swantafe.buildout/
 google_appengine/google/appengine/api/apiproxy_stub_map.py, line 72,
 in MakeSyncCall
     apiproxy.MakeSyncCall(service, call, request, response)
   File /opt/ktstudio/zope/instances/prod/swantafe.buildout/
 google_appengine/google/appengine/api/apiproxy_stub_map.py, line
  266,
 in MakeSyncCall
     rpc.CheckSuccess()
   File /opt/ktstudio/zope/instances/prod/swantafe.buildout/
 google_appengine/google/appengine/api/apiproxy_rpc.py, line 111, in
 CheckSuccess
     raise self.exception
 NameError: global name 'txdata' is not defined

 --
 2009-12-02T01:55:49 ERROR Zope.SiteErrorLoghttp://
  10.8.0.134:49081/swan/portal_skins/custom/send_uos
 Traceback (innermost last):
   Module ZPublisher.Publish, line 119, in

[google-appengine] Single Column Indexing

2009-12-03 Thread vivpuri

If you disable indexing(via indexed=False) on a model class
properties, and enable it again by removing the indexed=False, do all
the entities for that property get indexed again? If not, what is the
way to get that property indexed, since you cannot really specify
single column index in the index.yaml

--

You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Single Column Indexing

2009-12-03 Thread Paul Kinlan

Hi,

I am pretty sure the only way to do it is to get every entity and put them
back into the datastore.  This will ensure that the entity is saved.

P

2009/12/3 vivpuri v...@vivekpuri.com

 If you disable indexing(via indexed=False) on a model class
 properties, and enable it again by removing the indexed=False, do all
 the entities for that property get indexed again? If not, what is the
 way to get that property indexed, since you cannot really specify
 single column index in the index.yaml

 --

 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To post to this group, send email to google-appeng...@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com
 .
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.




--

You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Multiple datastores or share datastore in one app

2009-12-03 Thread Roy

In an RDBMS world you would actually create a schema for each team,
not a full database (unless you have a morbid need to drive your
sysadmin to an early grave). I did make a feature request for this,
but you know how it is with GAE feature requests.

If you're using LLAPI, there is actually a very simple trick that will
work, Append the team name to the kind.
So instead of a kind called player, you have kinds called player-
spurs, player-w19an, etc.

You can have a very simple customer data access layer which does this
automatically based on a session variable.

I use this trick for testing. Whenever I login using a designated test
username, I append -test to my kind names so that I'm not missing
live and test data in a single dataset.



On Dec 3, 7:53 am, andreas_b andreas.borg...@gmail.com wrote:
 Thanks both of you!

 Very helpful answers.

 Best Regards, Andreas

 On Dec 2, 8:24 pm, Ikai L (Google) ika...@google.com wrote:



  Creating a database per team is a very heavyweight way to address the
  problem of data segregation. This is unnecessary and in general, not a
  recommended best practice, as you would provide data isolation at the
  application layer. The intuitive solution here is to create an entity group
  for a league or team (depending on your transactional needs) and place child
  entities in that group.

  On Tue, Dec 1, 2009 at 10:40 AM, andreas_b andreas.borg...@gmail.comwrote:

   Hi all.

   I'm working on a GWT/GAE project where the idea is to create a portal
   for sport teams. Each sport team can sign up to get an account where
   they can register players, keep track of leagues, matches, statistics
   and so on. Each team should also be able to use their own domain,
   which automatically should load the site with their configuration when
   entered (basically just load the gwt-app with some url-parameter that
   is forwarded to server-side).

   So, coming from a normal SQL-environment, it seems to me that each
   team that signs up should get their own private database for all their
   data. As I understand it, this is not possible with GAE datastore?
   There is a one-to-one mapping between an application and a datastore?

   If this is the case, then what is the best way forward? I guess each
   entity could have a team ID, but it really doesn't seem like a good
   idea. There should be some kind of isolation between the different
   teams' data.

   Registering a new GAE app for each team is not an option either since
   we expect at least hundreds of teams.

   So, is there some way to isolate entities from each other within a GAE
   datastore?
   Also, would it be feasible from a performance point of view to do
   this?

   Or is simply GAE not the right way to go for this kind of web
   offering?

   Thanks in advance.

   BR, Andreas

   --

   You received this message because you are subscribed to the Google Groups
   Google App Engine group.
   To post to this group, send email to google-appeng...@googlegroups.com.
   To unsubscribe from this group, send email to
   google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2Bunsubscrib
e...@googlegroups.com
   .
   For more options, visit this group at
  http://groups.google.com/group/google-appengine?hl=en.

  --
  Ikai Lan
  Developer Programs Engineer, Google App Engine

--

You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Multiple datastores or share datastore in one app

2009-12-03 Thread andreas_b

So I've spent some time now reading more about the datastore and I
find it hard to get a complete picture about how it works from the gae
docs and articles.
In my case, I must make sure there is some implicit isolation of the
different team's data. So after reading more about entity groups, it
seemed like a good idea
to create a Team entity which is the root entity of an entity group
that contains all the team entities like players, matches, forum posts
etc.
But then I also read the article about datastore contention where it
is recommended that the entity groups are kept as small as possible.

It is likely that lots of forum/comment as well as matches/statistics
entities will be created over time, leading to quite a large entity
group in the case described above.
So now I'm thinking that team data should be divided into several
logical entity groups, BUT, there must still be some owned
relationship between all entities that belongs
to a team. Is is possible to have relationships between entity
groups?

I also read that you can specify an app-assigned ID as part of an
entity key. Would it be possible to use a unique team ID as part of
the entity keys?
The goal is of course to let the datastore take care of the security
restraints as much as possible using its build-in features such as
entity groups.
So I'm looking for the optimal way to enforce team data isolation and
avoid any performance/contention pitfalls such as with too large
entity groups.

If you can recommend any in-depth documentation/sample code on these
matters, I would happily look into it.

Thanks!

BR, Andreas

On Dec 2, 8:24 pm, Ikai L (Google) ika...@google.com wrote:
Creating a database per team is a very heavyweight way to address the
problem of data segregation. This is unnecessary and in general, not a
recommended best practice, as you would provide data isolation at the
application layer. The intuitive solution here is to create an entity group
for a league or team (depending on your transactional needs) and place child
entities in that group.

On Tue, Dec 1, 2009 at 10:40 AM, andreas_b andreas.borg...@gmail.comwrote:

Hi all.

I'm working on a GWT/GAE project where the idea is to create a portal
for sport teams. Each sport team can sign up to get an account where
they can register players, keep track of leagues, matches, statistics
and so on. Each team should also be able to use their own domain,
which automatically should load the site with their configuration when
entered (basically just load the gwt-app with some url-parameter that
is forwarded to server-side).

So, coming from a normal SQL-environment, it seems to me that each
team that signs up should get their own private database for all their
data. As I understand it, this is not possible with GAE datastore?
There is a one-to-one mapping between an application and a datastore?

If this is the case, then what is the best way forward? I guess each
entity could have a team ID, but it really doesn't seem like a good
idea. There should be some kind of isolation between the different
teams' data.

Registering a new GAE app for each team is not an option either since
we expect at least hundreds of teams.

So, is there some way to isolate entities from each other within a GAE
datastore?
Also, would it be feasible from a performance point of view to do
this?

Or is simply GAE not the right way to go for this kind of web
offering?

Thanks in advance.

BR, Andreas

--
Ikai Lan
Developer Programs Engineer, Google App Engine

You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Multiple datastores or share datastore in one app

2009-12-03 Thread andreas_b

I chose the wrong wording in my original post. I didn't mean actual
full databases, but something like a schema yes.
Have a lot to learn when it comes to efficient database usage in any
case :-).

Actually I'm not using the LLAPI for portability reasons. I don't want
to build too many dependencies on a particular web service, so
I chose JDO instead, although LLAPI seems a bit more powerful. I might
actually look into it to gain more flexibility.

Thanks for the tip!

BR, Andreas

On Dec 3, 2:19 pm, Roy roy.smith@googlemail.com wrote:
 In an RDBMS world you would actually create a schema for each team,
 not a full database (unless you have a morbid need to drive your
 sysadmin to an early grave). I did make a feature request for this,
 but you know how it is with GAE feature requests.

 If you're using LLAPI, there is actually a very simple trick that will
 work, Append the team name to the kind.
 So instead of a kind called player, you have kinds called player-
 spurs, player-w19an, etc.

 You can have a very simple customer data access layer which does this
 automatically based on a session variable.

 I use this trick for testing. Whenever I login using a designated test
 username, I append -test to my kind names so that I'm not missing
 live and test data in a single dataset.

 On Dec 3, 7:53 am, andreas_b andreas.borg...@gmail.com wrote:

  Thanks both of you!

  Very helpful answers.

  Best Regards, Andreas

  On Dec 2, 8:24 pm, Ikai L (Google) ika...@google.com wrote:

   Creating a database per team is a very heavyweight way to address the
   problem of data segregation. This is unnecessary and in general, not a
   recommended best practice, as you would provide data isolation at the
   application layer. The intuitive solution here is to create an entity 
   group
   for a league or team (depending on your transactional needs) and place 
   child
   entities in that group.

   On Tue, Dec 1, 2009 at 10:40 AM, andreas_b 
   andreas.borg...@gmail.comwrote:

Hi all.

I'm working on a GWT/GAE project where the idea is to create a portal
for sport teams. Each sport team can sign up to get an account where
they can register players, keep track of leagues, matches, statistics
and so on. Each team should also be able to use their own domain,
which automatically should load the site with their configuration when
entered (basically just load the gwt-app with some url-parameter that
is forwarded to server-side).

So, coming from a normal SQL-environment, it seems to me that each
team that signs up should get their own private database for all their
data. As I understand it, this is not possible with GAE datastore?
There is a one-to-one mapping between an application and a datastore?

If this is the case, then what is the best way forward? I guess each
entity could have a team ID, but it really doesn't seem like a good
idea. There should be some kind of isolation between the different
teams' data.

Registering a new GAE app for each team is not an option either since
we expect at least hundreds of teams.

So, is there some way to isolate entities from each other within a GAE
datastore?
Also, would it be feasible from a performance point of view to do
this?

Or is simply GAE not the right way to go for this kind of web
offering?

Thanks in advance.

BR, Andreas

--

You received this message because you are subscribed to the Google 
Groups
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2Bunsubscrib
 e...@googlegroups.com
.
For more options, visit this group at
   http://groups.google.com/group/google-appengine?hl=en.

   --
   Ikai Lan
   Developer Programs Engineer, Google App Engine

--

You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] how many index

2009-12-03 Thread alf

How many indexes can I create in a application?

I hope create more than 100 kinds and probably more than one index per
king  there are any limit?

thanks

--

You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Multiple datastores or share datastore in one app

2009-12-03 Thread Robert Kluin

Andreas,
  I have a very similar application.  It tracks a league's season, team,
player, and game scores / statistics.  I needed to have isolation between
leagues and within leagues between seasons, teams, players.  Basically the
approach I settled on was adding a league and season property to most of
the models.

  When a user logs in they must select a league they have permissions to
access or manage, I store the league key in a datastore-backed session.  Any
time they access something I verify that the league key matches, and, of
course, any queries I run filter out content from different leagues.  I
continue this approach down to the lower level items (players and games for
example).

  I do use entity groups, but at a much finer level, and they may not really
be needed at all.  For storing statistics I have models like PlayerSeason,
PlayerWeek.  I put Player, PlayerSeason, and PlayerWeek into the same entity
group.  Likewise for the teams and their stats.  I did this so that if a
part of my db.put() that stores all games and related statistics models
fails I do not have players or teams getting mismatched within themselves --
not really sure it helps much though.

Robert



On Thu, Dec 3, 2009 at 9:07 AM, andreas_b andreas.borg...@gmail.com wrote:

 I chose the wrong wording in my original post. I didn't mean actual
 full databases, but something like a schema yes.
 Have a lot to learn when it comes to efficient database usage in any
 case :-).

 Actually I'm not using the LLAPI for portability reasons. I don't want
 to build too many dependencies on a particular web service, so
 I chose JDO instead, although LLAPI seems a bit more powerful. I might
 actually look into it to gain more flexibility.

 Thanks for the tip!

 BR, Andreas

 On Dec 3, 2:19 pm, Roy roy.smith@googlemail.com wrote:
  In an RDBMS world you would actually create a schema for each team,
  not a full database (unless you have a morbid need to drive your
  sysadmin to an early grave). I did make a feature request for this,
  but you know how it is with GAE feature requests.
 
  If you're using LLAPI, there is actually a very simple trick that will
  work, Append the team name to the kind.
  So instead of a kind called player, you have kinds called player-
  spurs, player-w19an, etc.
 
  You can have a very simple customer data access layer which does this
  automatically based on a session variable.
 
  I use this trick for testing. Whenever I login using a designated test
  username, I append -test to my kind names so that I'm not missing
  live and test data in a single dataset.
 
  On Dec 3, 7:53 am, andreas_b andreas.borg...@gmail.com wrote:
 
   Thanks both of you!
 
   Very helpful answers.
 
   Best Regards, Andreas
 
   On Dec 2, 8:24 pm, Ikai L (Google) ika...@google.com wrote:
 
Creating a database per team is a very heavyweight way to address the
problem of data segregation. This is unnecessary and in general, not
 a
recommended best practice, as you would provide data isolation at the
application layer. The intuitive solution here is to create an entity
 group
for a league or team (depending on your transactional needs) and
 place child
entities in that group.
 
On Tue, Dec 1, 2009 at 10:40 AM, andreas_b 
 andreas.borg...@gmail.comwrote:
 
 Hi all.
 
 I'm working on a GWT/GAE project where the idea is to create a
 portal
 for sport teams. Each sport team can sign up to get an account
 where
 they can register players, keep track of leagues, matches,
 statistics
 and so on. Each team should also be able to use their own domain,
 which automatically should load the site with their configuration
 when
 entered (basically just load the gwt-app with some url-parameter
 that
 is forwarded to server-side).
 
 So, coming from a normal SQL-environment, it seems to me that each
 team that signs up should get their own private database for all
 their
 data. As I understand it, this is not possible with GAE datastore?
 There is a one-to-one mapping between an application and a
 datastore?
 
 If this is the case, then what is the best way forward? I guess
 each
 entity could have a team ID, but it really doesn't seem like a good
 idea. There should be some kind of isolation between the different
 teams' data.
 
 Registering a new GAE app for each team is not an option either
 since
 we expect at least hundreds of teams.
 
 So, is there some way to isolate entities from each other within a
 GAE
 datastore?
 Also, would it be feasible from a performance point of view to do
 this?
 
 Or is simply GAE not the right way to go for this kind of web
 offering?
 
 Thanks in advance.
 
 BR, Andreas
 
 --
 
 You received this message because you are subscribed to the Google
 Groups
 Google App Engine group.
 To post to this group, send email to
 google-appeng...@googlegroups.com.
 To

Re: [google-appengine] Google Dashboard System Status page link ---server error 500

2009-12-03 Thread Jon McAlister

Yep, I see what you mean, looking into it. Thanks for the report.

On Wed, Dec 2, 2009 at 10:55 PM, oldcomputer computer...@yahoo.com wrote:
 says report,, gives server 500 error at system status page

 --

 You received this message because you are subscribed to the Google Groups 
 Google App Engine group.
 To post to this group, send email to google-appeng...@googlegroups.com.
 To unsubscribe from this group, send email to 
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at 
 http://groups.google.com/group/google-appengine?hl=en.




--

You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] @Lob

2009-12-03 Thread HKHAIRANE

Hey everyBody, i try to persist my video  1MB with @Lob , this
doesn't persist my field in my Class VideoSequence

otherwise all other fiels are saved in database, the code is:


@Entity
public class VideoSequence implements Serializable {

private static final long serialVersionUID = 1L;
@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
Key id;
@Version
long version;

int sequenceNumber;
@Lob
byte[] videoBlock;

public int getSequenceNumber() {
return sequenceNumber;
}

public void setSequenceNumber(int sequenceNumber) {
this.sequenceNumber = sequenceNumber;
}

public Key getId() {
return id;
}

public static VideoSequence newVideoSequence() {
VideoSequence videoSequence = new VideoSequence();
return videoSequence;
}

public byte[] getVideoBlock() {
return videoBlock;
}

public void setVideoBlock(byte[] videoBlock) {
this.videoBlock = videoBlock;
}


any ideas

Thanks in advance

--

You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Google Dashboard System Status page link ---server error 500

2009-12-03 Thread Jon McAlister

Fixed now. Apologies for the outage.

On Thu, Dec 3, 2009 at 8:29 AM, Jon McAlister jon...@google.com wrote:
 Yep, I see what you mean, looking into it. Thanks for the report.

 On Wed, Dec 2, 2009 at 10:55 PM, oldcomputer computer...@yahoo.com wrote:
 says report,, gives server 500 error at system status page

 --

 You received this message because you are subscribed to the Google Groups 
 Google App Engine group.
 To post to this group, send email to google-appeng...@googlegroups.com.
 To unsubscribe from this group, send email to 
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at 
 http://groups.google.com/group/google-appengine?hl=en.





--

You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Creating a generated key with an ancestor in JPA

2009-12-03 Thread markm208

Hello,
I am trying to create an auto-generated key with an ancestor. I can
see how to do it at the low level in python but I want to do it in
java with JPA. Can anyone point me to an example?

--

You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: gql not giving full result set

2009-12-03 Thread Johann C. Rocholl

My previous response to this thread is not showing up on Google
Groups, sorry if this is a double post.

I think I'm seeing the same problem, and I made a simple page to
reproduce it. My App ID is scoretool, and the test page is at /dns/
test/ on the appspot domain for my App ID. I'm not including the URL
to prevent this message being marked as spam.

The page shows the results of a simple query, once using the ascending
__key__ index and once the new descending __key__ index that was
created less than 4 days ago. Both queries should return the same
results, but they don't. My dataset is large (total of 2 million
items, using 15 GB including metadata). My full source code is
available at /jcrocholl/scoretool on github.com.

Please let me know if I can do anything else to help diagnose this
issue.

--

You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: gql not giving full result set

2009-12-03 Thread Johann C. Rocholl

I think I'm seeing the same problem. I have recently (in the last 3
days) added descending indexes on __key__ for two models, and they
should return very similar entries because for each domains_domain
model instance, there should be a dns_lookup instance. It works okay
for __key__ ascending and for a separate string property called
backwards both ascending and descending, but the new descending
__key__ indexes return only partial results. It seems that maybe 40%
of items are missing from the descending __key__ indexes.

I have a rather large dataset (2,203,599 items, 15.7 GB including
metadata). The App ID is scoretool. You can see the problem if you
visit http://scoretool.appspot.com/dns/cron/ and hit refresh until you
get Selector: before [random string] and the text Created [number]
missing DNS lookups: followed by (not really).

My complete source code is available:
http://github.com/jcrocholl/scoretool/blob/master/dns/views.py
http://github.com/jcrocholl/scoretool/blob/master/prefixes/selectors.py

Please let me know if I can do anything else to help diagnose this
problem.

Johann

--

You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Any ZK Users on Google App Engine

2009-12-03 Thread rsft rsft

I had developed a Google App Engine application using the ZK Ajax
framework.  Everything is good, however I did notice some performance
issues and I suspect that it could be due to the fact that a lot of
rendering is done on the Server side.

For instance when a browser make an initial request to a zul page, the
zk Engine probably renders the page from scratch.  I'm sure one
possible optimization is to cache the rendered result in the Google
Cache and when a request is made simply push the generated HTML to the
browser.

I'm working on this on and off, but I'm wondering if there's any other
ZK users out there and probably we can share notes on some of the
possible optimizations.

--

You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Has anyone done a source code control system hosted on App Engine?

2009-12-03 Thread lior harsat

you can try unfuddle.

On Thu, Dec 3, 2009 at 1:51 AM, samwyse samw...@gmail.com wrote:

 I'd like to host Hg or something someplace with better uptime than my
 home server.

 --

 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To post to this group, send email to google-appeng...@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com
 .
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.




--

You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Empty value of a Key field

2009-12-03 Thread Henry

The method which I have to retrieve the information of and announcer
is this one...

private Announcer getAnnouncerFromEmailAddress(String emailAddress)
throws UnknownUserException {
Announcer announcerLoaded = null;
// Creating a PersistentManager to interact with the DataBase
PersistenceManager pm = PMF.get().getPersistenceManager();
// Query to check if the current user exists
Query query = pm.newQuery(Announcer.class);
// Set the filter
query.setFilter(emailAddress == emailAddressParam);
// Declare the parameter
query.declareParameters(String emailAddressParam);
// Run the query and check the result
try {
ListAnnouncer results = (ListAnnouncer) 
query.execute
(emailAddress);
if (!results.isEmpty()) {
announcerLoaded = results.get(0);
} else {
throw new UnknownUserException();
}
} finally {
pm.close();
}
return announcerLoaded;
}

after execute the query, the field country is null.

On Dec 2, 7:13 pm, Ikai L (Google) ika...@google.com wrote:
 Do you have a reproducible test case or some code you can post?



 On Tue, Dec 1, 2009 at 5:02 AM, Henry enricrequ...@gmail.com wrote:
  Hi Guys,

  Im new at Google App Engine, I've the follow problem:
  When Im recovering a persistent class Announcer, it has one field
  (country) with NO VALUE...I thought that must to be for a bad mapping
  with the annotations creating the proper class, but I don't know how
  to do to get again the Key that I've stored. When Im storing the Key
  (country), it has value, I can see it in the Google App Engine
  DataViewer.
  This is my class with the annotations:

  @PersistenceCapable(identityType = IdentityType.APPLICATION)
  public class Announcer {

        �...@primarykey
        �...@persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
         private Long id;
        �...@persistent
         private String emailAddress;
        �...@persistent
         private String password;
        �...@persistent
         private String firstName;
        �...@persistent
         private String lastName;
        �...@persistent
         private Key country;

  Anyone could help me?
  Im using Java and JDO.
  Thanks for advance.

  --

  You received this message because you are subscribed to the Google Groups
  Google App Engine group.
  To post to this group, send email to google-appeng...@googlegroups.com.
  To unsubscribe from this group, send email to
  google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com
  .
  For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.

 --
 Ikai Lan
 Developer Programs Engineer, Google App Engine

--

You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Continuing HTTPErrr 302 when using remote_api after a 200 transactions - need to re-use connection handles

Tim,

Ah, great find. If I'm to understand you correctly, you were using a
different process off App Engine to log into App Engine, hitting the login
page N times in a span of M minutes. This resulted in you not being able to
authenticate anymore, but after you cached the auth cookie, it started
working fine.

On Thu, Dec 3, 2009 at 3:01 AM, Tim Hoffman zutes...@gmail.com wrote:

 Hi Ikai


 I am using remote_api called from within plone.

 Because I could potentially get multiple concurrent calls from plone
 to push content I was creating a new connection
 (requiring new authentication )  on each request to push content from
 plone to app engine.

 Unfortunately when I started pushing large amounts of content each
 content entity being pushed was creating a new
 connection (invoking authentication). When I hit 200 of these in the
 space of say 10mins I would get hit by google infrastructure
 which would basically stop transactions.  CheckSuccess didn't like
 getting a redirect to the authentication stuff.
 And I couldn't re-authenticate for about 10 mins anyway.  Looks like I
 was running afoul of systems designed to prevent
 spamming etc  Any other connection attempts from the same host
 using that user id would also fail.
 And none of this would be seen in the app engine logs, because the
 remote_api auth/connection was being redirected
 even before we got to appengine.

 I found that if I cached the connection and didn't continually re-
 authenticate I had no problems.

 Hope that makes sense.

 Rgds

 Tim
 On Dec 3, 3:35 am, Ikai L (Google) ika...@google.com wrote:
  Tim, great find. What do you mean re-use connections? If you let me know,
  I'll update the docs and open an issue to provide a better error message.
 
  On Tue, Dec 1, 2009 at 8:56 PM, Tim Hoffman zutes...@gmail.com wrote:
   Ok
 
   Got the the bottom of it all.
 
   You need to re-use connections.  If you make too many authenticated
   connections then googles higher up infrastructure
   shuts you down for about 10 min.
 
   It does highlight some problems in the remote_api code as the error
   messages passed back are particularly informative unless you really
   dig. but at least I understand where the problem is.
 
   T
 
   On Dec 2, 11:31 am, Tim Hoffman zutes...@gmail.com wrote:
OK
 
Started debugging inside the apiproxy_rpc.py(113)CheckSuccess()
and what I believe is happening is correct.
 
the rpc call MaekSyncCall is getting a redirect to a request to login
 
apiproxy_rpc.py(113)CheckSuccess()
 
The exception is a HTTPErrror which is a redirect (302)
to
 
 'https://www.google.com/a/polytechnic.wa.edu.au/ServiceLogin?
service=ahpassive=truecontinue=
 http://psc-prod1.appspot.com/_ah/login
%3Fcon stuff deleted',
 
Obviously requesting me to log in again.
 
So somewhere I am running afoul of higher layers in google that watch
what is going on.
I am going to need some help from google engineers to get to the
bottom of this one.
 
Rgds
 
Tim Hoffman
 
On Dec 2, 11:04 am, Tim Hoffman zutes...@gmail.com wrote:
 
 Something I have observerd and can now prove repeatably
 
 Once I get the error 302 inside the remote_api call.  It takes
 somewhere between 2 and 10 mins before I can resubmit the
 transaction.
 During that that time, if I try the same call from another process
 (on
 the same box with the same auth details) the call still gets the
 error
 until such time as something in google lets it go.
 
 It really is looking like I am being clobbered by some google watch
 dog.
 
 T
 
 On Dec 2, 9:58 am, Tim Hoffman zutes...@gmail.com wrote:
 
  I am still getting this problem.
 
  One of the things I have noticed is the errors (302) are being
  propogated by google infrastructure and not recorded any way
  in the apps logs.
 
  My gut feel is that some other google infrastructure thinks I  am
  doing something I shouldn't requires me to re-authenticate but
  the remote_api proxies can't deal with a 302 inside a Namerror in
  CheckSuccess see stack below.
 
  Obviously what I am doing is different to the bulkloader client.
 
  Anyone got any suggestions on how I might go about getting to the
  bottom of this.
 
  T
 
  2009-12-02T01:55:49 INFO root Exception sending Rollback:
  Traceback (most recent call last):
File /opt/ktstudio/zope/instances/prod/swantafe.buildout/
  google_appengine/google/appengine/api/datastore.py, line 1989,
 in
  RunInTransactionCustomRetries
  tx.handle, resp)
File /opt/ktstudio/zope/instances/prod/swantafe.buildout/
  google_appengine/google/appengine/api/apiproxy_stub_map.py, line
 72,
  in MakeSyncCall
  apiproxy.MakeSyncCall(service, call, request, response)
File /opt/ktstudio/zope/instances/prod/swantafe.buildout/

[google-appengine] Re: Datastore is slow on queries involving many entities, but a smallish dataset

2009-12-03 Thread Julian Namaro


Eric,

Sorry my previous answer was wrong. The query time increases with the
number of entities fetched faster than I thought.
From a small benchmark in Python average time for your query is
150-200ms.
This is for fetching 130 entities of about 400 bytes, with 5
properties each, using an inequality filter.

But still, one second is definitively too long. Hope the Java gurus
can help you!




On Dec 3, 5:03 am, Eric Rannaud eric.rann...@gmail.com wrote:
 Crossposting to App Engine Java group: the original thread is 
 athttp://groups.google.com/group/google-appengine/browse_thread/thread/...

 In a few words: I have a problem with reasonable queries taking a very
 long time (several seconds). These queries return 128 entities, from a
 total of 500,000 entities of that type in the datastore. Each entity
 is about 400 bytes.



 On Tue, Dec 1, 2009 at 6:49 PM, Stephen sdea...@gmail.com wrote:
  On Dec 1, 9:12 pm, Eric Rannaud eric.rann...@gmail.com wrote:
  On Tue, Dec 1, 2009 at 11:02 AM, Stephen sdea...@gmail.com wrote:
  On Dec 1, 9:55 am, Eric Rannaud eric.rann...@gmail.com wrote:
  SELECT * FROM MessageS where id = 0  id  128 order by id

  Calendar c = Calendar.getInstance();
  long t0 = c.getTimeInMillis();
  qmsgr = (ListMessageS) qmsg.execute(lo, hi);
  System.err.println(getCMIdRange:qmsg:  + (c.getTimeInMillis() - 
  t0));

  Are you fetching all 128 entities in one batch? If you don't, the
  result is fetched in batches of 20, incurring extra disk reads and rpc
  overhead.

  Not sure how you do that with the Java API, but with python you pass
  '128' to the .fetch() method of a query object.

  As far as I can tell, there is no such equivalent in the Java API. The

  Something like this..?

  DatastoreService datastore =
 DatastoreServiceFactory.getDatastoreService();

  Query query = new Query(MessageS);
  query.addFilter(id, Query.FilterOperator.GREATER_THAN_OR_EQUAL, 0);

  ListEntity messages = datastore.prepare(query)
 .asList(FetchOptions.Builder.withLimit(128));

  You might also have to tweak chunkSize and/or prefetchSize, or ask on
  the Java list.

 I did some tests with the code you proposed. The performance remains
 essentially the same as with the JDO API, i.e. between 1 and 4 second
 per execute/prepare statement (2.5s on average).

 DatastoreService datastore =
 DatastoreServiceFactory.getDatastoreService();
 Query query = new Query(MessageS);
 query.addFilter(id, Query.FilterOperator.GREATER_THAN_OR_EQUAL, lo);
 query.addFilter(id, Query.FilterOperator.LESS_THAN, hi);

 long t0 = Calendar.getInstance().getTimeInMillis();

 ListEntity r = datastore.prepare(query)
 .asList(FetchOptions.Builder
 .withLimit(128)
 .prefetchSize(128)
 .chunkSize(128));

 System.err.println(LOW:getCMIdRange:qmsg: 
+ (Calendar.getInstance().getTimeInMillis() - t0)
+   + r.size());

 Thanks.

--

You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Query with key range and a composite index

2009-12-03 Thread n...@apption.com

I'm trying to perform a query on multiple properties of an entity, as
well as a key range.  I have been testing the system with only 1 or 2
properties, and a key range on some of the queries.  The time for the
queries seem to be very dependent on whether or not there is a key
range when using a composite index.  I've been timing the queries,
here are some typical times. All queries have a limit of 200.

Select __key__ from indexClass WHERE A = :a
returned 200 objects in 59 ms

Select __key__ from indexClass WHERE A = :a' AND __key__ = :minKey
AND __key__ = :maxKey
returned 200 objects in 61 ms

This seems normal, but when i add a second field, the following
happens:
Note that i have an index built with status 'Serving' on properties A
and B

Select __key__ from indexClass WHERE A = :a' AND B = :b
returned 91 objects in 37ms

Select __key__ from indexClass WHERE A = :a' AND B = :b AND __key__
= :minKey AND __key__ = :maxKey
returned 29 objects in 440ms

To ensure that the index is built and working properly, i ran the
query:
Select __key__ from indexClass WHERE A = :a' AND B = :b ORDER BY A
which works correctly, so I don't think appengine could be performing
a merge-join here.

Any ideas why the last query is slow?  I could always add a field for
the keyName and index that to be used for key range queries, but I
don't think I should have to.

Thanks,
Nick

--

You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Multiple datastores or share datastore in one app

Andreas,

The datastore provides virtually no security mechanisms for data
partitioning within the same application. The responsibility for access
control belongs in your application layer.

On Thu, Dec 3, 2009 at 8:28 AM, Robert Kluin robert.kl...@gmail.com wrote:

 Andreas,
   I have a very similar application.  It tracks a league's season, team,
 player, and game scores / statistics.  I needed to have isolation between
 leagues and within leagues between seasons, teams, players.  Basically the
 approach I settled on was adding a league and season property to most of
 the models.

   When a user logs in they must select a league they have permissions to
 access or manage, I store the league key in a datastore-backed session.  Any
 time they access something I verify that the league key matches, and, of
 course, any queries I run filter out content from different leagues.  I
 continue this approach down to the lower level items (players and games for
 example).

   I do use entity groups, but at a much finer level, and they may not
 really be needed at all.  For storing statistics I have models like
 PlayerSeason, PlayerWeek.  I put Player, PlayerSeason, and PlayerWeek into
 the same entity group.  Likewise for the teams and their stats.  I did this
 so that if a part of my db.put() that stores all games and related
 statistics models fails I do not have players or teams getting mismatched
 within themselves -- not really sure it helps much though.

 Robert




 On Thu, Dec 3, 2009 at 9:07 AM, andreas_b andreas.borg...@gmail.comwrote:

 I chose the wrong wording in my original post. I didn't mean actual
 full databases, but something like a schema yes.
 Have a lot to learn when it comes to efficient database usage in any
 case :-).

 Actually I'm not using the LLAPI for portability reasons. I don't want
 to build too many dependencies on a particular web service, so
 I chose JDO instead, although LLAPI seems a bit more powerful. I might
 actually look into it to gain more flexibility.

 Thanks for the tip!

 BR, Andreas

 On Dec 3, 2:19 pm, Roy roy.smith@googlemail.com wrote:
  In an RDBMS world you would actually create a schema for each team,
  not a full database (unless you have a morbid need to drive your
  sysadmin to an early grave). I did make a feature request for this,
  but you know how it is with GAE feature requests.
 
  If you're using LLAPI, there is actually a very simple trick that will
  work, Append the team name to the kind.
  So instead of a kind called player, you have kinds called player-
  spurs, player-w19an, etc.
 
  You can have a very simple customer data access layer which does this
  automatically based on a session variable.
 
  I use this trick for testing. Whenever I login using a designated test
  username, I append -test to my kind names so that I'm not missing
  live and test data in a single dataset.
 
  On Dec 3, 7:53 am, andreas_b andreas.borg...@gmail.com wrote:
 
   Thanks both of you!
 
   Very helpful answers.
 
   Best Regards, Andreas
 
   On Dec 2, 8:24 pm, Ikai L (Google) ika...@google.com wrote:
 
Creating a database per team is a very heavyweight way to address
 the
problem of data segregation. This is unnecessary and in general, not
 a
recommended best practice, as you would provide data isolation at
 the
application layer. The intuitive solution here is to create an
 entity group
for a league or team (depending on your transactional needs) and
 place child
entities in that group.
 
On Tue, Dec 1, 2009 at 10:40 AM, andreas_b 
 andreas.borg...@gmail.comwrote:
 
 Hi all.
 
 I'm working on a GWT/GAE project where the idea is to create a
 portal
 for sport teams. Each sport team can sign up to get an account
 where
 they can register players, keep track of leagues, matches,
 statistics
 and so on. Each team should also be able to use their own domain,
 which automatically should load the site with their configuration
 when
 entered (basically just load the gwt-app with some url-parameter
 that
 is forwarded to server-side).
 
 So, coming from a normal SQL-environment, it seems to me that each
 team that signs up should get their own private database for all
 their
 data. As I understand it, this is not possible with GAE datastore?
 There is a one-to-one mapping between an application and a
 datastore?
 
 If this is the case, then what is the best way forward? I guess
 each
 entity could have a team ID, but it really doesn't seem like a
 good
 idea. There should be some kind of isolation between the different
 teams' data.
 
 Registering a new GAE app for each team is not an option either
 since
 we expect at least hundreds of teams.
 
 So, is there some way to isolate entities from each other within a
 GAE
 datastore?
 Also, would it be feasible from a performance point of view to do
 this?
 
 Or is simply GAE not the right way to go for

Re: [google-appengine] @Lob

Have you looked into using com.google.appengine.api.datastore.Blob?

http://code.google.com/appengine/docs/java/datastore/dataclasses.html

http://code.google.com/appengine/docs/java/javadoc/com/google/appengine/api/datastore/Blob.html

On Thu, Dec 3, 2009 at 9:46 AM, HKHAIRANE hichamkhair...@gmail.com wrote:

 Hey everyBody, i try to persist my video  1MB with @Lob , this
 doesn't persist my field in my Class VideoSequence

 otherwise all other fiels are saved in database, the code is:


 @Entity
 public class VideoSequence implements Serializable {

private static final long serialVersionUID = 1L;
@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
Key id;
@Version
long version;

int sequenceNumber;
@Lob
byte[] videoBlock;

public int getSequenceNumber() {
return sequenceNumber;
}

public void setSequenceNumber(int sequenceNumber) {
this.sequenceNumber = sequenceNumber;
}

public Key getId() {
return id;
}

public static VideoSequence newVideoSequence() {
VideoSequence videoSequence = new VideoSequence();
return videoSequence;
}

public byte[] getVideoBlock() {
return videoBlock;
}

public void setVideoBlock(byte[] videoBlock) {
this.videoBlock = videoBlock;
}


 any ideas

 Thanks in advance

 --

 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To post to this group, send email to google-appeng...@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com
 .
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.





-- 
Ikai Lan
Developer Programs Engineer, Google App Engine

--

You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Weird memcached issue

Is there an error message when you try to store data? Do you see any
information in the logs?

On Wed, Dec 2, 2009 at 5:49 PM, naan kaz...@gmail.com wrote:

 My app id is echofonsync and the issue remains. I can get a result
 from memcache.get_stats() now, but can not store new data. Thanks.

 On Dec 2, 5:05 pm, Ikai L (Google) ika...@google.com wrote:
  Is this issue still occurring for you? If so, let me know your
 application
  ID.
 
  Michael, let me know if this is causing you problems in production. I can
  attempt to migrate your application to a different pool, but this is not
  something I'd advise if you're running in production and things are
 working
  fine.
 
 
 
  On Wed, Dec 2, 2009 at 4:26 PM, naan kaz...@gmail.com wrote:
   Hi,
 
   This happens for me too. I can't store any data into memcache. Also, I
   can't get any results with memcache.get_stats(). The function returns
   None.
 
   On Dec 2, 1:45 pm, Michael m...@mzlab.net wrote:
The problem with consistency is: the same key maps to the different
values (when I'm hitting another memcached server), so randomly I'm
getting old value from memcache and it produces weird effects on my
application (like new message appearing, though they were marked as
read).
 
Is it going to be fixed soon?
 
On Dec 2, 11:33 pm, Michael m...@mzlab.net wrote:
 
 At the time of the first post it was going for about 10 minutes.
 
 It's still happening.
 
   --
 
   You received this message because you are subscribed to the Google
 Groups
   Google App Engine group.
   To post to this group, send email to google-appengine@googlegroups.com
 .
   To unsubscribe from this group, send email to
   google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com
 google-appengine%2bunsubscr...@googlegroups.comgoogle-appengine%252bunsubscr...@googlegroups.com
 
   .
   For more options, visit this group at
  http://groups.google.com/group/google-appengine?hl=en.
 
  --
  Ikai Lan
  Developer Programs Engineer, Google App Engine

 --

 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To post to this group, send email to google-appeng...@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com
 .
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.





-- 
Ikai Lan
Developer Programs Engineer, Google App Engine

--

You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Weird memcached issue

We've just tried this and it seems to work for us. What are you storing in
Memcache? How are you generating the keys?

On Thu, Dec 3, 2009 at 10:31 AM, Ikai L (Google) ika...@google.com wrote:

Is there an error message when you try to store data? Do you see any
information in the logs?

On Wed, Dec 2, 2009 at 5:49 PM, naan kaz...@gmail.com wrote:

My app id is echofonsync and the issue remains. I can get a result
from memcache.get_stats() now, but can not store new data. Thanks.

On Dec 2, 5:05 pm, Ikai L (Google) ika...@google.com wrote:
Is this issue still occurring for you? If so, let me know your
application
ID.

Michael, let me know if this is causing you problems in production. I
can
attempt to migrate your application to a different pool, but this is not
something I'd advise if you're running in production and things are
working
fine.

On Wed, Dec 2, 2009 at 4:26 PM, naan kaz...@gmail.com wrote:
Hi,

This happens for me too. I can't store any data into memcache. Also, I
can't get any results with memcache.get_stats(). The function returns
None.

On Dec 2, 1:45 pm, Michael m...@mzlab.net wrote:
The problem with consistency is: the same key maps to the different
values (when I'm hitting another memcached server), so randomly I'm
getting old value from memcache and it produces weird effects on my
application (like new message appearing, though they were marked as
read).

Is it going to be fixed soon?

On Dec 2, 11:33 pm, Michael m...@mzlab.net wrote:

At the time of the first post it was going for about 10 minutes.

It's still happening.

You received this message because you are subscribed to the Google
Groups
Google App Engine group.
To post to this group, send email to
google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com
google-appengine%2bunsubscr...@googlegroups.comgoogle-appengine%252bunsubscr...@googlegroups.com

.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.

--
Ikai Lan
Developer Programs Engineer, Google App Engine

You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Request was aborted after waiting too long...

Mark,

Yes, it should take roughly the same amount of time. Do you notice that it's
more likely to happen at certain times of the day rather than others?

On Wed, Dec 2, 2009 at 12:49 PM, Mark markrobertdav...@gmail.com wrote:

I am using BlazeDS - a remoting service for Flex. I guess it takes a
while to initialize - although not sure why it is slow to init only a
small %age of the time - you would think it'd either always fail or
always succeed.

On Dec 2, 11:33 am, Ikai L (Google) ika...@google.com wrote:
What are you doing on application startup that may be expensive?

On Tue, Dec 1, 2009 at 4:04 PM, Mark markrobertdav...@gmail.com wrote:
Hi,

Could someone please help me? I have a relatively simple app - but
I am seeing that around 5% of the time, my app fails with error
'Request was aborted after waiting too long to attempt to service your
request. Most likely, this indicates that you have reached your
simultaneous active request limit. This is almost always due to
excessively high latency in your app. '

This only happens if the application has not been in use for a few
minutes. I can refresh the web page, and my app always works
immediately. So I guess the problem is happening when GAE cycles my
app out and then tries to wake it back up.

Does anyone have any tips to get around this error? Is it caused
because my app is somehow too slow to initialize?

My appid is mayihelpyoucam

cheers

Mark

You received this message because you are subscribed to the Google
Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com
google-appengine%2bunsubscr...@googlegroups.comgoogle-appengine%252bunsubscr...@googlegroups.com

.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.

--
Ikai Lan
Developer Programs Engineer, Google App Engine

You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Weird memcached issue

I'm using memcache.set_multi() / add_multi() for caching DataStore
data. Code is something like following:

users = User.all().query().fetch(...)

mapping = {}
for u in users:
mapping[str(u.key())] = u

memcache.set_multi(mapping, namespace='synccache')

It seems that set_multi/add_multi returns correct value and there's no
error logs, error messages, nor exception. However, I can't retrieve
them with memcache.get_multi().

Thanks,
Kazuho

On Dec 3, 10:46 am, Ikai L (Google) ika...@google.com wrote:
We've just tried this and it seems to work for us. What are you storing in
Memcache? How are you generating the keys?

On Thu, Dec 3, 2009 at 10:31 AM, Ikai L (Google) ika...@google.com wrote:

Is there an error message when you try to store data? Do you see any
information in the logs?

On Wed, Dec 2, 2009 at 5:49 PM, naan kaz...@gmail.com wrote:

My app id is echofonsync and the issue remains. I can get a result
from memcache.get_stats() now, but can not store new data. Thanks.

On Dec 2, 5:05 pm, Ikai L (Google) ika...@google.com wrote:
Is this issue still occurring for you? If so, let me know your
application
ID.

On Wed, Dec 2, 2009 at 4:26 PM, naan kaz...@gmail.com wrote:
Hi,

This happens for me too. I can't store any data into memcache. Also, I
can't get any results with memcache.get_stats(). The function returns
None.

Is it going to be fixed soon?

On Dec 2, 11:33 pm, Michael m...@mzlab.net wrote:

At the time of the first post it was going for about 10 minutes.

It's still happening.

You received this message because you are subscribed to the Google
Groups
Google App Engine group.
To post to this group, send email to
google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com
google-appengine%2bunsubscr...@googlegroups.comgoogle-appengine%252bunsubscr...@googlegroups.com

.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.

--
Ikai Lan
Developer Programs Engineer, Google App Engine

You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Weird memcached issue

This is a typo:

for key in cached_mapping:
  println(self, str(obj_map[key]))

Should be:

for key in cached_mapping:
  println(self, str(cached_mapping[key]))

This is tested and working for me.

On Thu, Dec 3, 2009 at 12:04 PM, Ikai L (Google) ika...@google.com wrote:

 What is being returned by get_multi? None or empty dictionary? Is a
 namespace being specified?

 Here's some test code that is working for me when deployed. What might you
 be doing differently from this?

 from google.appengine.ext import webapp
 from google.appengine.ext.webapp.util import run_wsgi_app
 from google.appengine.ext import db

 from google.appengine.api import memcache

 class MyThing(db.Model):
   name = db.StringProperty()

   def __str__(self):
 return MyThing: %s % self.name

 class MemcacheTest(webapp.RequestHandler):
   def get(self):
 stats = memcache.get_stats()

 println(self, bCache Hits:%s/bbr % stats['hits'])
 println(self, bCache Misses:%s/bbrbr % stats['misses'])

 memcache.set(data, My data)
 data = memcache.get(data)
 println(self, Set 'data' - 'My data'. Get 'data' -  + data)

 key_range = range(1000)
 mapping = { }

 for i in key_range:
   mapping[key%d % i] = value %d % i

 memcache.set_multi(mapping)
 println(self, Set %d values using set_multi % len(mapping))

 cached_mapping = memcache.get_multi(mapping.keys())
 println(self, Retrieved %d values using get_multi %
 len(cached_mapping))

 namespace = synccache
 println(self, Testing namespaces with namespace '%s' % namespace)
 memcache.set_multi(mapping, namespace=namespace)
 println(self, Set %d values using set_multi in namespace '%s' % (
 len(mapping), namespace ))

 cached_mapping = memcache.get_multi(mapping.keys(),
 namespace=namespace)
 println(self, Retrieved %d values using get_multi in namespace '%s' %
 ( len(cached_mapping), namespace ))


 println(self, Testing objects)
 obj_map = {}
 for i in key_range:
   thing = MyThing(name=thing %d % i)
   obj_map[thing.name] = thing

 namespace = things
 println(self, Testing namespaces with namespace '%s' % namespace)
 memcache.set_multi(obj_map, namespace=namespace)
 println(self, Set %d values using set_multi in namespace '%s' % (
 len(mapping), namespace ))

 cached_mapping = memcache.get_multi(obj_map.keys(),
 namespace=namespace)
 println(self, Retrieved %d values using get_multi in namespace '%s' %
 ( len(cached_mapping), namespace ))

 for key in cached_mapping:
   println(self, str(obj_map[key]))

 def println(handler, string):
   handler.response.out.write(string + br/)


 application = webapp.WSGIApplication([ ('/cachetest', MemcacheTest),
  ], debug=True)

 def main():
   run_wsgi_app(application)

 if __name__ == __main__:
   main()


 On Thu, Dec 3, 2009 at 11:09 AM, naan kaz...@gmail.com wrote:

 I'm using memcache.set_multi() / add_multi() for caching DataStore
 data. Code is something like following:


  users = User.all().query().fetch(...)

  mapping = {}
  for u in users:
  mapping[str(u.key())] = u

  memcache.set_multi(mapping, namespace='synccache')


 It seems that set_multi/add_multi returns correct value and there's no
 error logs, error messages, nor exception. However, I can't retrieve
 them with memcache.get_multi().

 Thanks,
 Kazuho

 On Dec 3, 10:46 am, Ikai L (Google) ika...@google.com wrote:
  We've just tried this and it seems to work for us. What are you storing
 in
  Memcache? How are you generating the keys?
 
  On Thu, Dec 3, 2009 at 10:31 AM, Ikai L (Google) ika...@google.com
 wrote:
 
 
 
   Is there an error message when you try to store data? Do you see any
   information in the logs?
 
   On Wed, Dec 2, 2009 at 5:49 PM, naan kaz...@gmail.com wrote:
 
   My app id is echofonsync and the issue remains. I can get a result
   from memcache.get_stats() now, but can not store new data. Thanks.
 
   On Dec 2, 5:05 pm, Ikai L (Google) ika...@google.com wrote:
Is this issue still occurring for you? If so, let me know your
   application
ID.
 
Michael, let me know if this is causing you problems in production.
 I
   can
attempt to migrate your application to a different pool, but this
 is not
something I'd advise if you're running in production and things are
   working
fine.
 
On Wed, Dec 2, 2009 at 4:26 PM, naan kaz...@gmail.com wrote:
 Hi,
 
 This happens for me too. I can't store any data into memcache.
 Also, I
 can't get any results with memcache.get_stats(). The function
 returns
 None.
 
 On Dec 2, 1:45 pm, Michael m...@mzlab.net wrote:
  The problem with consistency is: the same key maps to the
 different
  values (when I'm hitting another memcached server), so randomly
 I'm
  getting old value from memcache and it produces weird effects
 on my
  application (like new message appearing, though they were

Re: [google-appengine] Re: Weird memcached issue

What is being returned by get_multi? None or empty dictionary? Is a
namespace being specified?

Here's some test code that is working for me when deployed. What might you
be doing differently from this?

from google.appengine.ext import webapp
from google.appengine.ext.webapp.util import run_wsgi_app
from google.appengine.ext import db

from google.appengine.api import memcache

class MyThing(db.Model):
  name = db.StringProperty()

  def __str__(self):
return MyThing: %s % self.name

class MemcacheTest(webapp.RequestHandler):
  def get(self):
stats = memcache.get_stats()

println(self, bCache Hits:%s/bbr % stats['hits'])
println(self, bCache Misses:%s/bbrbr % stats['misses'])

memcache.set(data, My data)
data = memcache.get(data)
println(self, Set 'data' - 'My data'. Get 'data' -  + data)

key_range = range(1000)
mapping = { }

for i in key_range:
  mapping[key%d % i] = value %d % i

memcache.set_multi(mapping)
println(self, Set %d values using set_multi % len(mapping))

cached_mapping = memcache.get_multi(mapping.keys())
println(self, Retrieved %d values using get_multi %
len(cached_mapping))

namespace = synccache
println(self, Testing namespaces with namespace '%s' % namespace)
memcache.set_multi(mapping, namespace=namespace)
println(self, Set %d values using set_multi in namespace '%s' % (
len(mapping), namespace ))

cached_mapping = memcache.get_multi(mapping.keys(), namespace=namespace)
println(self, Retrieved %d values using get_multi in namespace '%s' %
( len(cached_mapping), namespace ))


println(self, Testing objects)
obj_map = {}
for i in key_range:
  thing = MyThing(name=thing %d % i)
  obj_map[thing.name] = thing

namespace = things
println(self, Testing namespaces with namespace '%s' % namespace)
memcache.set_multi(obj_map, namespace=namespace)
println(self, Set %d values using set_multi in namespace '%s' % (
len(mapping), namespace ))

cached_mapping = memcache.get_multi(obj_map.keys(), namespace=namespace)
println(self, Retrieved %d values using get_multi in namespace '%s' %
( len(cached_mapping), namespace ))

for key in cached_mapping:
  println(self, str(obj_map[key]))

def println(handler, string):
  handler.response.out.write(string + br/)


application = webapp.WSGIApplication([ ('/cachetest', MemcacheTest),
 ], debug=True)

def main():
  run_wsgi_app(application)

if __name__ == __main__:
  main()

On Thu, Dec 3, 2009 at 11:09 AM, naan kaz...@gmail.com wrote:

 I'm using memcache.set_multi() / add_multi() for caching DataStore
 data. Code is something like following:


  users = User.all().query().fetch(...)

  mapping = {}
  for u in users:
  mapping[str(u.key())] = u

  memcache.set_multi(mapping, namespace='synccache')


 It seems that set_multi/add_multi returns correct value and there's no
 error logs, error messages, nor exception. However, I can't retrieve
 them with memcache.get_multi().

 Thanks,
 Kazuho

 On Dec 3, 10:46 am, Ikai L (Google) ika...@google.com wrote:
  We've just tried this and it seems to work for us. What are you storing
 in
  Memcache? How are you generating the keys?
 
  On Thu, Dec 3, 2009 at 10:31 AM, Ikai L (Google) ika...@google.com
 wrote:
 
 
 
   Is there an error message when you try to store data? Do you see any
   information in the logs?
 
   On Wed, Dec 2, 2009 at 5:49 PM, naan kaz...@gmail.com wrote:
 
   My app id is echofonsync and the issue remains. I can get a result
   from memcache.get_stats() now, but can not store new data. Thanks.
 
   On Dec 2, 5:05 pm, Ikai L (Google) ika...@google.com wrote:
Is this issue still occurring for you? If so, let me know your
   application
ID.
 
Michael, let me know if this is causing you problems in production.
 I
   can
attempt to migrate your application to a different pool, but this is
 not
something I'd advise if you're running in production and things are
   working
fine.
 
On Wed, Dec 2, 2009 at 4:26 PM, naan kaz...@gmail.com wrote:
 Hi,
 
 This happens for me too. I can't store any data into memcache.
 Also, I
 can't get any results with memcache.get_stats(). The function
 returns
 None.
 
 On Dec 2, 1:45 pm, Michael m...@mzlab.net wrote:
  The problem with consistency is: the same key maps to the
 different
  values (when I'm hitting another memcached server), so randomly
 I'm
  getting old value from memcache and it produces weird effects on
 my
  application (like new message appearing, though they were marked
 as
  read).
 
  Is it going to be fixed soon?
 
  On Dec 2, 11:33 pm, Michael m...@mzlab.net wrote:
 
   At the time of the first post it was going for about 10
 minutes.
 
   It's still happening.
 
 --
 
 You received this message because you are subscribed to the Google
   Groups
 Google App Engine group.

[google-appengine] django running xp problem

2009-12-03 Thread alexarsh

I have OSX machine running django app with appengine launcher and a
wirtual machine with windows XP running the same app.
The problem is that I have 2 XP machines where I get: No module named
urls. What can be the problem? Maybe some system variables?
The code is exactly the same.

Regards, Arshavski Alexander.

--

You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Weird memcached issue

'users' are array, which is a result of DataStore fetch as I describe
above. Not a dict. The code converts users array into memcacheable
dict. Note that same code works fine on my test GAE instance.

On Dec 3, 12:15 pm, Ikai L (Google) ika...@google.com wrote:
 This is a typo:

     for key in cached_mapping:
       println(self, str(obj_map[key]))

 Should be:

     for key in cached_mapping:
       println(self, str(cached_mapping[key]))

 This is tested and working for me.

 On Thu, Dec 3, 2009 at 12:04 PM, Ikai L (Google) ika...@google.com wrote:



  What is being returned by get_multi? None or empty dictionary? Is a
  namespace being specified?

  Here's some test code that is working for me when deployed. What might you
  be doing differently from this?

  from google.appengine.ext import webapp
  from google.appengine.ext.webapp.util import run_wsgi_app
  from google.appengine.ext import db

  from google.appengine.api import memcache

  class MyThing(db.Model):
    name = db.StringProperty()

    def __str__(self):
      return MyThing: %s % self.name

  class MemcacheTest(webapp.RequestHandler):
    def get(self):
      stats = memcache.get_stats()

      println(self, bCache Hits:%s/bbr % stats['hits'])
      println(self, bCache Misses:%s/bbrbr % stats['misses'])

      memcache.set(data, My data)
      data = memcache.get(data)
      println(self, Set 'data' - 'My data'. Get 'data' -  + data)

      key_range = range(1000)
      mapping = { }

      for i in key_range:
        mapping[key%d % i] = value %d % i

      memcache.set_multi(mapping)
      println(self, Set %d values using set_multi % len(mapping))

      cached_mapping = memcache.get_multi(mapping.keys())
      println(self, Retrieved %d values using get_multi %
  len(cached_mapping))

      namespace = synccache
      println(self, Testing namespaces with namespace '%s' % namespace)
      memcache.set_multi(mapping, namespace=namespace)
      println(self, Set %d values using set_multi in namespace '%s' % (
  len(mapping), namespace ))

      cached_mapping = memcache.get_multi(mapping.keys(),
  namespace=namespace)
      println(self, Retrieved %d values using get_multi in namespace '%s' %
  ( len(cached_mapping), namespace ))

      println(self, Testing objects)
      obj_map = {}
      for i in key_range:
        thing = MyThing(name=thing %d % i)
        obj_map[thing.name] = thing

      namespace = things
      println(self, Testing namespaces with namespace '%s' % namespace)
      memcache.set_multi(obj_map, namespace=namespace)
      println(self, Set %d values using set_multi in namespace '%s' % (
  len(mapping), namespace ))

      cached_mapping = memcache.get_multi(obj_map.keys(),
  namespace=namespace)
      println(self, Retrieved %d values using get_multi in namespace '%s' %
  ( len(cached_mapping), namespace ))

      for key in cached_mapping:
        println(self, str(obj_map[key]))

  def println(handler, string):
    handler.response.out.write(string + br/)

  application = webapp.WSGIApplication([ ('/cachetest', MemcacheTest),
                                       ], debug=True)

  def main():
    run_wsgi_app(application)

  if __name__ == __main__:
    main()

  On Thu, Dec 3, 2009 at 11:09 AM, naan kaz...@gmail.com wrote:

  I'm using memcache.set_multi() / add_multi() for caching DataStore
  data. Code is something like following:

   users = User.all().query().fetch(...)

   mapping = {}
   for u in users:
       mapping[str(u.key())] = u

   memcache.set_multi(mapping, namespace='synccache')

  It seems that set_multi/add_multi returns correct value and there's no
  error logs, error messages, nor exception. However, I can't retrieve
  them with memcache.get_multi().

  Thanks,
  Kazuho

  On Dec 3, 10:46 am, Ikai L (Google) ika...@google.com wrote:
   We've just tried this and it seems to work for us. What are you storing
  in
   Memcache? How are you generating the keys?

   On Thu, Dec 3, 2009 at 10:31 AM, Ikai L (Google) ika...@google.com
  wrote:

Is there an error message when you try to store data? Do you see any
information in the logs?

On Wed, Dec 2, 2009 at 5:49 PM, naan kaz...@gmail.com wrote:

My app id is echofonsync and the issue remains. I can get a result
from memcache.get_stats() now, but can not store new data. Thanks.

On Dec 2, 5:05 pm, Ikai L (Google) ika...@google.com wrote:
 Is this issue still occurring for you? If so, let me know your
application
 ID.

 Michael, let me know if this is causing you problems in production.
  I
can
 attempt to migrate your application to a different pool, but this
  is not
 something I'd advise if you're running in production and things are
working
 fine.

 On Wed, Dec 2, 2009 at 4:26 PM, naan kaz...@gmail.com wrote:
  Hi,

  This happens for me too. I can't store any data into memcache.
  Also, I
  can't get any results with memcache.get_stats(). The function

[google-appengine] Re: Concern about deploying real applications on Google AppEng

2009-12-03 Thread Jake

One simple but less-than-ideal solution is to setup a light weight
reverse http proxy on a server that has a non-blacklisted ip, but has
no restrictions when connecting to GAE. This would definately work,
but there are two major disadvantages:

1. You pay for the 3x the bandwidth. Going into the reverse proxy,
going out, and again into GAE.
2. Additional latency/less scalability.


On Dec 2, 9:37 pm, DBPZ dbpzd...@gmail.com wrote:
 Hi guys,

 The google appeng really gives me so much fun and the free quota is
 very delicious such that some REAL applications may be run within it.
 However, when I considered to do something big here, I found a problem
 that I cannot apply for a dedicated IP address for my applications,
 while most users of my potential applications are from an Asia
 country. Any attempt of deploying my applications behind an IP address
 that is shared with some other unknown users may results in being
 blocked at the network borders because of some offensive to policies
 of that country. If it happens, I even cannot save my applications by
 moving it to another provider because nowhere else provides the same
 sandbox.

 Not only to me, this issue may draw back other serious users
 (especially businesses, which are more valuable to Google) considering
 the global accessibility of their applications. I wanna know if there
 are some solutions to this concern, for example, allowing users to
 choose their deployment points (for example, in the target country of
 the application) or building some tunnels across the network borders
 being supervised by authorities.

--

You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Weird memcached issue

I try to store 100 users object per request. memcache.set_multi seems
working (return an empty array which means all data stored
successfully), but get_multi returns only 9-15 of them. It worked
yesterday morning, then stopped working afternoon. it seems that I
can't store new user data into memcache, but still can get old cached
data with get_multi.

The test code you gave me works fine. And my code works fine too on
another GAE instance. Hmm Does call memcache.flush_all() will
solve the issue?

On Dec 3, 12:04 pm, Ikai L (Google) ika...@google.com wrote:
 What is being returned by get_multi? None or empty dictionary? Is a
 namespace being specified?

 Here's some test code that is working for me when deployed. What might you
 be doing differently from this?

 from google.appengine.ext import webapp
 from google.appengine.ext.webapp.util import run_wsgi_app
 from google.appengine.ext import db

 from google.appengine.api import memcache

 class MyThing(db.Model):
   name = db.StringProperty()

   def __str__(self):
     return MyThing: %s % self.name

 class MemcacheTest(webapp.RequestHandler):
   def get(self):
     stats = memcache.get_stats()

     println(self, bCache Hits:%s/bbr % stats['hits'])
     println(self, bCache Misses:%s/bbrbr % stats['misses'])

     memcache.set(data, My data)
     data = memcache.get(data)
     println(self, Set 'data' - 'My data'. Get 'data' -  + data)

     key_range = range(1000)
     mapping = { }

     for i in key_range:
       mapping[key%d % i] = value %d % i

     memcache.set_multi(mapping)
     println(self, Set %d values using set_multi % len(mapping))

     cached_mapping = memcache.get_multi(mapping.keys())
     println(self, Retrieved %d values using get_multi %
 len(cached_mapping))

     namespace = synccache
     println(self, Testing namespaces with namespace '%s' % namespace)
     memcache.set_multi(mapping, namespace=namespace)
     println(self, Set %d values using set_multi in namespace '%s' % (
 len(mapping), namespace ))

     cached_mapping = memcache.get_multi(mapping.keys(), namespace=namespace)
     println(self, Retrieved %d values using get_multi in namespace '%s' %
 ( len(cached_mapping), namespace ))

     println(self, Testing objects)
     obj_map = {}
     for i in key_range:
       thing = MyThing(name=thing %d % i)
       obj_map[thing.name] = thing

     namespace = things
     println(self, Testing namespaces with namespace '%s' % namespace)
     memcache.set_multi(obj_map, namespace=namespace)
     println(self, Set %d values using set_multi in namespace '%s' % (
 len(mapping), namespace ))

     cached_mapping = memcache.get_multi(obj_map.keys(), namespace=namespace)
     println(self, Retrieved %d values using get_multi in namespace '%s' %
 ( len(cached_mapping), namespace ))

     for key in cached_mapping:
       println(self, str(obj_map[key]))

 def println(handler, string):
   handler.response.out.write(string + br/)

 application = webapp.WSGIApplication([ ('/cachetest', MemcacheTest),
                                      ], debug=True)

 def main():
   run_wsgi_app(application)

 if __name__ == __main__:
   main()



 On Thu, Dec 3, 2009 at 11:09 AM, naan kaz...@gmail.com wrote:
  I'm using memcache.set_multi() / add_multi() for caching DataStore
  data. Code is something like following:

   users = User.all().query().fetch(...)

   mapping = {}
   for u in users:
       mapping[str(u.key())] = u

   memcache.set_multi(mapping, namespace='synccache')

  It seems that set_multi/add_multi returns correct value and there's no
  error logs, error messages, nor exception. However, I can't retrieve
  them with memcache.get_multi().

  Thanks,
  Kazuho

  On Dec 3, 10:46 am, Ikai L (Google) ika...@google.com wrote:
   We've just tried this and it seems to work for us. What are you storing
  in
   Memcache? How are you generating the keys?

   On Thu, Dec 3, 2009 at 10:31 AM, Ikai L (Google) ika...@google.com
  wrote:

Is there an error message when you try to store data? Do you see any
information in the logs?

On Wed, Dec 2, 2009 at 5:49 PM, naan kaz...@gmail.com wrote:

My app id is echofonsync and the issue remains. I can get a result
from memcache.get_stats() now, but can not store new data. Thanks.

On Dec 2, 5:05 pm, Ikai L (Google) ika...@google.com wrote:
 Is this issue still occurring for you? If so, let me know your
application
 ID.

 Michael, let me know if this is causing you problems in production.
  I
can
 attempt to migrate your application to a different pool, but this is
  not
 something I'd advise if you're running in production and things are
working
 fine.

 On Wed, Dec 2, 2009 at 4:26 PM, naan kaz...@gmail.com wrote:
  Hi,

  This happens for me too. I can't store any data into memcache.
  Also, I
  can't get any results with memcache.get_stats(). The function
  returns
  None.

  On Dec 2, 1:45 pm, Michael

Re: [google-appengine] Re: Weird memcached issue

Are you doing the get_multi and set_multi within the same request, or are
the items disappearing afterwards? There's a 100mb limit to the number of
items you can be storing in Memcache, but I'm not sure this is what you're
hitting - the least recently used objects would be getting expired. What
kind of data is stored in the User object?

-- Forwarded message --
From: naan kaz...@gmail.com
Date: Thu, Dec 3, 2009 at 12:51 PM
Subject: [google-appengine] Re: Weird memcached issue
To: Google App Engine google-appengine@googlegroups.com

I try to store 100 users object per request. memcache.set_multi seems
working (return an empty array which means all data stored
successfully), but get_multi returns only 9-15 of them. It worked
yesterday morning, then stopped working afternoon. it seems that I
can't store new user data into memcache, but still can get old cached
data with get_multi.

The test code you gave me works fine. And my code works fine too on
another GAE instance. Hmm Does call memcache.flush_all() will
solve the issue?

On Dec 3, 12:04 pm, Ikai L (Google) ika...@google.com wrote:
 What is being returned by get_multi? None or empty dictionary? Is a
 namespace being specified?

 Here's some test code that is working for me when deployed. What might you
 be doing differently from this?

 from google.appengine.ext import webapp
 from google.appengine.ext.webapp.util import run_wsgi_app
 from google.appengine.ext import db

 from google.appengine.api import memcache

 class MyThing(db.Model):
   name = db.StringProperty()

   def __str__(self):
 return MyThing: %s % self.name

 class MemcacheTest(webapp.RequestHandler):
   def get(self):
 stats = memcache.get_stats()

 println(self, bCache Hits:%s/bbr % stats['hits'])
 println(self, bCache Misses:%s/bbrbr % stats['misses'])

 memcache.set(data, My data)
 data = memcache.get(data)
 println(self, Set 'data' - 'My data'. Get 'data' -  + data)

 key_range = range(1000)
 mapping = { }

 for i in key_range:
   mapping[key%d % i] = value %d % i

 memcache.set_multi(mapping)
 println(self, Set %d values using set_multi % len(mapping))

 cached_mapping = memcache.get_multi(mapping.keys())
 println(self, Retrieved %d values using get_multi %
 len(cached_mapping))

 namespace = synccache
 println(self, Testing namespaces with namespace '%s' % namespace)
 memcache.set_multi(mapping, namespace=namespace)
 println(self, Set %d values using set_multi in namespace '%s' % (
 len(mapping), namespace ))

 cached_mapping = memcache.get_multi(mapping.keys(),
namespace=namespace)
 println(self, Retrieved %d values using get_multi in namespace '%s'
%
 ( len(cached_mapping), namespace ))

 println(self, Testing objects)
 obj_map = {}
 for i in key_range:
   thing = MyThing(name=thing %d % i)
   obj_map[thing.name] = thing

 namespace = things
 println(self, Testing namespaces with namespace '%s' % namespace)
 memcache.set_multi(obj_map, namespace=namespace)
 println(self, Set %d values using set_multi in namespace '%s' % (
 len(mapping), namespace ))

 cached_mapping = memcache.get_multi(obj_map.keys(),
namespace=namespace)
 println(self, Retrieved %d values using get_multi in namespace '%s'
%
 ( len(cached_mapping), namespace ))

 for key in cached_mapping:
   println(self, str(obj_map[key]))

 def println(handler, string):
   handler.response.out.write(string + br/)

 application = webapp.WSGIApplication([ ('/cachetest', MemcacheTest),
  ], debug=True)

 def main():
   run_wsgi_app(application)

 if __name__ == __main__:
   main()

 On Thu, Dec 3, 2009 at 11:09 AM, naan kaz...@gmail.com wrote:
  I'm using memcache.set_multi() / add_multi() for caching DataStore
  data. Code is something like following:

   users = User.all().query().fetch(...)

   mapping = {}
   for u in users:
   mapping[str(u.key())] = u

   memcache.set_multi(mapping, namespace='synccache')

  It seems that set_multi/add_multi returns correct value and there's no
  error logs, error messages, nor exception. However, I can't retrieve
  them with memcache.get_multi().

  Thanks,
  Kazuho

  On Dec 3, 10:46 am, Ikai L (Google) ika...@google.com wrote:
   We've just tried this and it seems to work for us. What are you
storing
  in
   Memcache? How are you generating the keys?

   On Thu, Dec 3, 2009 at 10:31 AM, Ikai L (Google) ika...@google.com
  wrote:

Is there an error message when you try to store data? Do you see any
information in the logs?

On Wed, Dec 2, 2009 at 5:49 PM, naan kaz...@gmail.com wrote:

My app id is echofonsync and the issue remains. I can get a result
from memcache.get_stats() now, but can not store new data. Thanks.

On Dec 2, 5:05 pm, Ikai L (Google) ika...@google.com wrote:
 Is this issue still occurring for you? If so, let me know your
application

[google-appengine] Question on using TextProperty()

2009-12-03 Thread Charlie

I am new to using Google App Engine and am having trouble with
textproperty(). How do I keep the formatting when someone submits
something?

example:
Directions:
1.Turn left
2. Turn right
3. Go straight

Then I submit, and when I view it on the page, it turns out like this.

Directions:
1.Turn left 2. Turn right 3. Go straight.

Thanks for help.

--

You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Weird memcached issue

No, they are not in the same request. get_multi is called within 5
minutes after set_multi is called. User object is firly small (less
than 1k) and total number of items in memcache is less than 50k, so it
won't be more than 100MB. User object contains strings, integers and
integer list. No blobs, nor string list. Again, all set_multi call
seems success, no error, no exception occurred.

Thanks,
Kazuho

On Dec 3, 1:14 pm, Ikai L (Google) ika...@google.com wrote:
 Are you doing the get_multi and set_multi within the same request, or are
 the items disappearing afterwards? There's a 100mb limit to the number of
 items you can be storing in Memcache, but I'm not sure this is what you're
 hitting - the least recently used objects would be getting expired. What
 kind of data is stored in the User object?

 -- Forwarded message --
 From: naan kaz...@gmail.com
 Date: Thu, Dec 3, 2009 at 12:51 PM
 Subject: [google-appengine] Re: Weird memcached issue
 To: Google App Engine google-appengine@googlegroups.com

 I try to store 100 users object per request. memcache.set_multi seems
 working (return an empty array which means all data stored
 successfully), but get_multi returns only 9-15 of them. It worked
 yesterday morning, then stopped working afternoon. it seems that I
 can't store new user data into memcache, but still can get old cached
 data with get_multi.

 The test code you gave me works fine. And my code works fine too on
 another GAE instance. Hmm Does call memcache.flush_all() will
 solve the issue?

 On Dec 3, 12:04 pm, Ikai L (Google) ika...@google.com wrote:
  What is being returned by get_multi? None or empty dictionary? Is a
  namespace being specified?

  Here's some test code that is working for me when deployed. What might you
  be doing differently from this?

  from google.appengine.ext import webapp
  from google.appengine.ext.webapp.util import run_wsgi_app
  from google.appengine.ext import db

  from google.appengine.api import memcache

  class MyThing(db.Model):
    name = db.StringProperty()

    def __str__(self):
      return MyThing: %s % self.name

  class MemcacheTest(webapp.RequestHandler):
    def get(self):
      stats = memcache.get_stats()

      println(self, bCache Hits:%s/bbr % stats['hits'])
      println(self, bCache Misses:%s/bbrbr % stats['misses'])

      memcache.set(data, My data)
      data = memcache.get(data)
      println(self, Set 'data' - 'My data'. Get 'data' -  + data)

      key_range = range(1000)
      mapping = { }

      for i in key_range:
        mapping[key%d % i] = value %d % i

      memcache.set_multi(mapping)
      println(self, Set %d values using set_multi % len(mapping))

      cached_mapping = memcache.get_multi(mapping.keys())
      println(self, Retrieved %d values using get_multi %
  len(cached_mapping))

      namespace = synccache
      println(self, Testing namespaces with namespace '%s' % namespace)
      memcache.set_multi(mapping, namespace=namespace)
      println(self, Set %d values using set_multi in namespace '%s' % (
  len(mapping), namespace ))

      cached_mapping = memcache.get_multi(mapping.keys(),
 namespace=namespace)
      println(self, Retrieved %d values using get_multi in namespace '%s'
 %
  ( len(cached_mapping), namespace ))

      println(self, Testing objects)
      obj_map = {}
      for i in key_range:
        thing = MyThing(name=thing %d % i)
        obj_map[thing.name] = thing

      namespace = things
      println(self, Testing namespaces with namespace '%s' % namespace)
      memcache.set_multi(obj_map, namespace=namespace)
      println(self, Set %d values using set_multi in namespace '%s' % (
  len(mapping), namespace ))

      cached_mapping = memcache.get_multi(obj_map.keys(),
 namespace=namespace)
      println(self, Retrieved %d values using get_multi in namespace '%s'
 %
  ( len(cached_mapping), namespace ))

      for key in cached_mapping:
        println(self, str(obj_map[key]))

  def println(handler, string):
    handler.response.out.write(string + br/)

  application = webapp.WSGIApplication([ ('/cachetest', MemcacheTest),
                                       ], debug=True)

  def main():
    run_wsgi_app(application)

  if __name__ == __main__:
    main()

  On Thu, Dec 3, 2009 at 11:09 AM, naan kaz...@gmail.com wrote:
   I'm using memcache.set_multi() / add_multi() for caching DataStore
   data. Code is something like following:

    users = User.all().query().fetch(...)

    mapping = {}
    for u in users:
        mapping[str(u.key())] = u

    memcache.set_multi(mapping, namespace='synccache')

   It seems that set_multi/add_multi returns correct value and there's no
   error logs, error messages, nor exception. However, I can't retrieve
   them with memcache.get_multi().

   Thanks,
   Kazuho

   On Dec 3, 10:46 am, Ikai L (Google) ika...@google.com wrote:
We've just tried this and it seems to work for us. What are you
 storing
   in
Memcache? How are you

Re: [google-appengine] Received ClassNotFoundException deserializing a byte array.

What does the class you are trying to persist look like?

On Wed, Dec 2, 2009 at 1:23 AM, bartatamas bartata...@gmail.com wrote:

 Hi!

 I've deployed a new version of my web application and it doesn't work
 now because of this exception:

 Uncaught exception from servlet
 Received ClassNotFoundException deserializing a byte array.
 org.datanucleus.exceptions.NucleusException: Received
 ClassNotFoundException deserializing a byte array.
at
 org.datanucleus.store.appengine.SerializationManager$1.deserialize
 (SerializationManager.java:146)
at org.datanucleus.store.appengine.SerializationManager.deserialize
 (SerializationManager.java:171)
at
 org.datanucleus.store.appengine.DatastoreFieldManager.deserializeFieldValue
 (DatastoreFieldManager.java:321)
at
 org.datanucleus.store.appengine.DatastoreFieldManager.fetchObjectField
 (DatastoreFieldManager.java:297)
at org.datanucleus.state.AbstractStateManager.replacingObjectField
 (AbstractStateManager.java:1197)
at bt.web.reminder.database.Event.jdoReplaceField(Event.java)
at bt.web.reminder.database.Event.jdoReplaceFields(Event.java)
at org.datanucleus.state.JDOStateManagerImpl.replaceFields
 (JDOStateManagerImpl.java:2772)
at org.datanucleus.state.JDOStateManagerImpl.replaceFields
 (JDOStateManagerImpl.java:2791)
at
 org.datanucleus.store.appengine.DatastorePersistenceHandler.fetchObject
 (DatastorePersistenceHandler.java:443)
at org.datanucleus.state.JDOStateManagerImpl.loadUnloadedFields
 (JDOStateManagerImpl.java:1560)
at org.datanucleus.jdo.state.Hollow.transitionRetrieve(Hollow.java:
 161)
at org.datanucleus.state.AbstractStateManager.retrieve
 (AbstractStateManager.java:484)
at org.datanucleus.state.JDOStateManagerImpl.preSerialize
 (JDOStateManagerImpl.java:4455)
at bt.web.reminder.database.Event.jdoPreSerialize(Event.java)
at bt.web.reminder.database.Event.writeObject(Event.java)
at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at java.io.ObjectStreamClass.invokeWriteObject(Unknown Source)
at java.io.ObjectOutputStream.writeSerialData(Unknown Source)
at java.io.ObjectOutputStream.writeOrdinaryObject(Unknown Source)
at java.io.ObjectOutputStream.writeObject0(Unknown Source)
at java.io.ObjectOutputStream.writeObject(Unknown Source)
at java.util.ArrayList.writeObject(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at java.io.ObjectStreamClass.invokeWriteObject(Unknown Source)
at java.io.ObjectOutputStream.writeSerialData(Unknown Source)
at java.io.ObjectOutputStream.writeOrdinaryObject(Unknown Source)
at java.io.ObjectOutputStream.writeObject0(Unknown Source)
at java.io.ObjectOutputStream.defaultWriteFields(Unknown Source)
at java.io.ObjectOutputStream.defaultWriteObject(Unknown Source)
at bt.web.reminder.database.Account.writeObject(Account.java)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at java.io.ObjectStreamClass.invokeWriteObject(Unknown Source)
at java.io.ObjectOutputStream.writeSerialData(Unknown Source)
at java.io.ObjectOutputStream.writeOrdinaryObject(Unknown Source)
at java.io.ObjectOutputStream.writeObject0(Unknown Source)
at java.io.ObjectOutputStream.writeObject(Unknown Source)
at java.util.HashMap.writeObject(Unknown Source)
at sun.reflect.GeneratedMethodAccessor37.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at java.io.ObjectStreamClass.invokeWriteObject(Unknown Source)
at java.io.ObjectOutputStream.writeSerialData(Unknown Source)
at java.io.ObjectOutputStream.writeOrdinaryObject(Unknown Source)
at java.io.ObjectOutputStream.writeObject0(Unknown Source)
at java.io.ObjectOutputStream.writeObject(Unknown Source)
at com.google.appengine.api.memcache.MemcacheSerialization.serialize
 (MemcacheSerialization.java:257)
at com.google.appengine.api.memcache.MemcacheServiceImpl.put
 (MemcacheServiceImpl.java:314)
at com.google.appengine.api.memcache.stdimpl.GCache.put(GCache.java:
 164)


 Do you know what should I do? What does this message mean:

 Caused by:

[google-appengine] Re: Has anyone done a source code control system hosted on App Engine?

2009-12-03 Thread samwyse

That's exactly what I want.  (How did I miss it?)  Thanks!

On Dec 3, 5:11 am, Stephen sdea...@gmail.com wrote:
 On Dec 2, 11:51 pm, samwyse samw...@gmail.com wrote:

  I'd like to host Hg or something someplace with better uptime than my
  home server.

 http://hg-repos.appspot.com/mercurial_appengine/

 See also:

 http://code.google.com/events/io/2009/sessions/MercurialBigTable.html

--

You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Help with an accurate counter with no contention...?

2009-12-03 Thread Consultuning

The crux of the matter here is that you're requiring two things that
seem to be contradictory. You can have accuracy at the expense of
contention, and the only way of avoiding contention completely is at
the expense of accuracy. I don't have a theoretical proof of this on
hand, but in the general case, any update of a shared resource that
requires accuracy needs some kind of locking to avoid race conditions.

You can try to minimize contentions at the expense of other resources.
The task queue is reducing contention in the counters, at the expense
of deferred delays and creating contention in the queue operations.
And if you take a second look at it, the bookeeping needed to maintain
the task queue is more or less equivalent, if not higher in terms
resources than the sharded approach that adds a new shard after a
couple of timeouts.

On 30 nov, 00:22, Eli Jones eli.jo...@gmail.com wrote:
 I think that.. Since you require exactness on the counters, your
 primary option is the sharded counter option without memcache.

 You could/should modify the code so that.. If you get two timeouts in
 a row when trying to update a counter, then it just creates a new
 shard for the user.  This way, busy users would get more shards than
 non-busy users.

 Now, I am presuming that you will not end up with more than 1,000
 counter shards per user (I don't see how you could have that much
 going on for one user every second that that many shards would be
 needed).  If that happens, then, like you say, you'll have to page
 through the shards.

 When it comes to counters, they are usually discussing them in the
 context of counting total visits to a site or something like that..
 The task queue method, from my recollection, is used to aggregate
 memcached counters on a timed interval.

 The sub-case of having a sharded counter per user should work fine, I think.

 If you create your counter so it creates a new shard for a user after
 2 or 3 timeouts on a write, you should be fine (in my estimation).  In
 that case, you are guaranteed to get all counts written to the
 datastore (barring code, system failure)

 On 11/29/09, peterk peter.ke...@gmail.com wrote:

  I am slightly leaning toward going back to good old sharded
  counters...

  The reason I was looking for alternatives was because of some things
  Brett Slatkin said in his Google IO presentation on task queues - that
  really sharded counters were just a band-aid at the time, and that
  they still have problems with time-outs.

  But I guess as long as you keep enough shards for a given counter, it
  shouldn't time out? We can presumably catch contention warnings and
  within a number of retries create a new shard if necessary for an
  update to write to so we never lose one.

  I do agree with Brett that it's a bit heavy-handed, but if it
  guarantees accuracy...

  As far as I see it, the options seem to be this:

  Write-Behind Cache:
  + Control over DB write rate - 100 counter updates can be one DB write
  with this etc.
  + No redundancy in counter DB storage
  + Lower latency in the update request
  - Busy counters could keep a task in the queue constantly. Lots of
  busy counters (e.g. one for every user) = lots of task entries =
  problem with daily limit and lagging db update due to 10 task/sec
  invocation limit (?)
  - Lots of busy counters = thrashing of the cache and higher potential
  for a counter to disappear from cache before its saved (?)

  Sharded Counters:
  + No impact on task queue quota
  + More reliable, no dependency on a cache
  + Can maybe scale better with larger number of counters since you're
  not dealing with a limited cache size and task quota (?)
  + Can still deal with contention...
  -...as long as you manage the number of shards. Contention (and thus a
  dropped update) possible if number of shards is too low for a counter
  - More storage space required (n entities for n shards vs one counter
  entity)
  - DB write on every update; more latency in the request
  - Problem with reading counter total if number of shards goes beyond
  1000? Thus limited to 1000 shards = ~5000 counter updates/sec? Maybe
  could be overcome this during such busy times with a single
  total_count entity that aggregates all shards every now and then via a
  task (i.e. a task to read the shard values, up to n shards 1000, and
  write the aggregate to a single total_count entity). Count total would
  lag for a bit, but probably OK if it was being updated this much.

  The examples we were thinking of where we write deltas to the
  datastore would be more reliable than the memcache approach I think,
  but would still suffer from the problem of using an awful lot of tasks
  with many counters (and thus also being limited to updating ten
  counters per second because of the 10 task/sec run limit on the
  queues?). If you had a hundred thousand active counters, that means
  potentially tends of thousands of counters with tasks in queues at a
  given time,

Re: [google-appengine] Question on using TextProperty()

2009-12-03 Thread Robert Kluin

Just a guess here, but try wrapping the contents in a pre tag.  Probably
they newlines are not getting displayed by the browser, have you viewed
source to see if they are there?

print 'pre%s/pre' % mymodel.directions

Robert


On Thu, Dec 3, 2009 at 2:52 PM, Charlie trick...@gmail.com wrote:

 I am new to using Google App Engine and am having trouble with
 textproperty(). How do I keep the formatting when someone submits
 something?

 example:
 Directions:
 1.Turn left
 2. Turn right
 3. Go straight

 Then I submit, and when I view it on the page, it turns out like this.

 Directions:
 1.Turn left 2. Turn right 3. Go straight.

 Thanks for help.

 --

 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To post to this group, send email to google-appeng...@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com
 .
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.




--

You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Has anyone done a source code control system hosted on App Engine?

2009-12-03 Thread Robert Kluin

There is someone doing improvements and bug fixes to mercurial_appengine on
bitbucket:
http://bitbucket.org/durin42/mercurial-appengine/

Robert


On Thu, Dec 3, 2009 at 5:23 PM, samwyse samw...@gmail.com wrote:

 That's exactly what I want.  (How did I miss it?)  Thanks!

 On Dec 3, 5:11 am, Stephen sdea...@gmail.com wrote:
  On Dec 2, 11:51 pm, samwyse samw...@gmail.com wrote:
 
   I'd like to host Hg or something someplace with better uptime than my
   home server.
 
  http://hg-repos.appspot.com/mercurial_appengine/
 
  See also:
 
  http://code.google.com/events/io/2009/sessions/MercurialBigTable.html

 --

 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To post to this group, send email to google-appeng...@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com
 .
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.




--

You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Continuing HTTPErrr 302 when using remote_api after a 200 transactions - need to re-use connection handles

2009-12-03 Thread Tim Hoffman

Hi Ikai

Not exactly

I am using this to connect to the remote_api

def connect(appid,auth_func,host):
remote_api_stub.ConfigureRemoteDatastore(appid, '/remote_api',
auth_func, host)

Its seems you never get a handle back on which you make the call, it
just hooks up the remote data store under the hood to all the
db api methods.

Each time I was starting a new set of pushes connect was being called.
I now track if I have called ConfigureRemoteDatastore and don't call
it again.

That way the current connection is re-used.

Regards

Tim

On Dec 4, 2:06 am, Ikai L (Google) ika...@google.com wrote:
 Tim,

 Ah, great find. If I'm to understand you correctly, you were using a
 different process off App Engine to log into App Engine, hitting the login
 page N times in a span of M minutes. This resulted in you not being able to
 authenticate anymore, but after you cached the auth cookie, it started
 working fine.

 On Thu, Dec 3, 2009 at 3:01 AM, Tim Hoffman zutes...@gmail.com wrote:
  Hi Ikai

--

You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Weird memcached issue