[appengine-java] Does a new front end instance class allow for a larger Java XMX Heap Size?

[Scott Murphy]

If I choose a larger instance class, will it give me a larger Heap?  I 
tried this with a back end and found that I could not use the extra memory 
because the heap still stayed pretty small and my app would throw 
OutOfMemory exceptions when I got to around 256MB

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine-java/-/QXUrumbE-dwJ.
To post to this group, send email to google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] New to this, need help setting up files for droid app

[Scott Boynton]

I'm using AI Inventor and I have an app that needs to download files. I 
need the file names to never change but I need to update the contents of 
the files. Mostly text files and a PIC file. I added App Engine but I can 
not find a way to upload files. I'm used to a simple FTP server. Am I using 
the wrong service?
I need some step by step for the first time.
Thanks
Scott
 

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine-java/-/vriY1u8M1pgJ.
To post to this group, send email to google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] Re: Objectify-Appengine 2.1 released, supports Partial Indexes

[Scott Hernandez]

It is supported in appengine. Every time you set a property in an
Entity (appengine Entity) you can specify if that property is indexed
or not. The partial indexing support just lets you control that on a
per instance basis instead of having the field always indexed or not.

On Fri, Aug 12, 2011 at 11:14 AM, Tobias  wrote:
> Hi There,
> Can you tell me a little about how partial indexes are implemented in
> objectify? Is it supported in the underlying API or is it solely implemented
> in objectify?
> Kind regards Tobias.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine for Java" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/google-appengine-java/-/AcFUXARv3sEJ.
> To post to this group, send email to google-appengine-java@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine-java+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine-java?hl=en.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Major *MAJOR* Problems with Always On

[Scott Murphy]

I can no longer successfully deploy an application.  It starts up 3 
instances, 2/3 start fine.  But the one in the middle starts real slow and 
recycles every minute.  This started occurring less than a week ago and I 
don't know what to do.

Is there any paid support for App Engine.  This is not a problem with my 
application, because old versions that used to work fine do not.  This 
problem *ONLY* occurs on the middle instance every time I start a new 
version.

As you can see below, the middle one keeps resetting.  

*Instances *[image: 
help]
QPS*Latency*RequestsErrorsAgeMemoryAvailability0.050700.7 ms900:09:27123.0 
MBytes[image: Resident Icon]Resident0.1170.0 ms500:00:4743.5 MBytes[image: 
Resident Icon]Resident0.08398.0 ms1530:09:27124.8 MBytes[image: Resident 
Icon]Resident

When I first start the application, instances 1 and 3 quickly go to 110 
MBytes of memory within 15 seconds and settle.  Instance 2 (middle instance) 
SLOWLY grows in memory and never makes it past 72 MBytes before it dies.


Is there any SLA for App Engine??? I created a support ticket and have 
gotten no response.

http://code.google.com/p/googleappengine/issues/detail?id=5253

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine-java/-/korlO_jwMfIJ.
To post to this group, send email to google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Re: Multiple issues with eclipse 1.5.1 update

[Scott]

#1 and #2 Actually turned out to be an issue with the Appwrench
plugin. It does not seem to be compatible with the 1.5.1 SDK plugin. I
have notified the Appwrench developers of the
problem.

#3 Was just me having a mental hiccup. The correct switch is --
address.

On Jun 22, 7:51 pm, Scott  wrote:
> I updated the Google Plugin and App Engine SDK bundles for Eclipse
> today and am having a number of problems. First my configuration:
>
> OS:
> Ubuntu Linux 11.04
>
> Eclipse:
> Version: Helios Service Release 2
> Build id: 20110218-0911
>
> Plugins:
> Google Plugin for Eclipse 3.6   2.3.2.r36v201106211634
> com.google.gdt.eclipse.suite.e36.feature.feature.group
> Google App Engine Java SDK 1.5.1        1.5.1.r36v201106211634
> com.google.appengine.eclipse.sdkbundle.e36.feature.feature.group
>
> Issues:
> 1. Following update restart my app engine projects won't build. The
> error log has: java.lang.IllegalArgumentException: Unable to find /
> home/eeb/opt/eclipse/plugins/
> com.google.appengine.eclipse.sdkbundle_1.5.1.r36v201106211634/
> appengine-sdk-plugin.jar/lib/shared
>
> There is additional stack if needed.
>
> 2. I go to the preferences for the app engine sdk and attempt to set
> an SDK I get: Failed to initialize App Engine SDK at /home/eeb/opt/
> eclipse/plugins/
> com.google.appengine.eclipse.sdkbundle_1.5.1.r36v201106211634/
> appengine-java-sdk-1.5.1
>
> 3. After following the work around below, the --server option seems to
> be ignored in the debug configuration, and the development only binds
> to localhost. I have to test devices against the server, so this is a
> critical problem.
>
> Workaround:
> - Uninstalled the App Engine Java SDK plugin. I can now set the app
> engine sdk and run my development server. The result is an info
> message and stack trace at the top: INFO: Unable to 
> accesshttps://0.0.0.0/api/updatecheck?runtime=java&release=1.5.1×tamp=...['1.0']
>
> Can someone try and reproduce these same issues? If you can let me
> know, and I will open a bug ticket.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Multiple issues with eclipse 1.5.1 update

[Scott]

I updated the Google Plugin and App Engine SDK bundles for Eclipse
today and am having a number of problems. First my configuration:

OS:
Ubuntu Linux 11.04

Eclipse:
Version: Helios Service Release 2
Build id: 20110218-0911

Plugins:
Google Plugin for Eclipse 3.6   2.3.2.r36v201106211634
com.google.gdt.eclipse.suite.e36.feature.feature.group
Google App Engine Java SDK 1.5.11.5.1.r36v201106211634
com.google.appengine.eclipse.sdkbundle.e36.feature.feature.group


Issues:
1. Following update restart my app engine projects won't build. The
error log has: java.lang.IllegalArgumentException: Unable to find /
home/eeb/opt/eclipse/plugins/
com.google.appengine.eclipse.sdkbundle_1.5.1.r36v201106211634/
appengine-sdk-plugin.jar/lib/shared

There is additional stack if needed.

2. I go to the preferences for the app engine sdk and attempt to set
an SDK I get: Failed to initialize App Engine SDK at /home/eeb/opt/
eclipse/plugins/
com.google.appengine.eclipse.sdkbundle_1.5.1.r36v201106211634/
appengine-java-sdk-1.5.1

3. After following the work around below, the --server option seems to
be ignored in the debug configuration, and the development only binds
to localhost. I have to test devices against the server, so this is a
critical problem.

Workaround:
- Uninstalled the App Engine Java SDK plugin. I can now set the app
engine sdk and run my development server. The result is an info
message and stack trace at the top: INFO: Unable to access
https://0.0.0.0/api/updatecheck?runtime=java&release=1.5.1×tamp=1308100600&api_versions=['1.0']

Can someone try and reproduce these same issues? If you can let me
know, and I will open a bug ticket.



-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Re: After upgrade to GAE SDK 1.5.0 - Cache is not working

[Scott]

I am having the same issue this:

 CacheFactory factory =
net.sf.jsr107cache.CacheManager.getInstance().getCacheFactory();

Throws a CacheException with no stack trace and the message:

net.sf.jsr107cache.CacheException: Could not find class:
'com.google.appengine.api.memcache.jsr107cache.GCacheFactory'




On May 11, 6:03 pm, Don Schwarz  wrote:
> What is the NullPointerException that you are getting?
>
> You shouldn't need to change your GCacheFactory.  Nothing changed with that
> in 1.5.0, and the new package you are using is not correct.
>  com.google.appengine.api.memcache.jsr107cache.GCacheFactory is the one you
> want.  It is in appengine-jsr107cache-1.5.0.jar.
>
>
>
>
>
>
>
> On Wed, May 11, 2011 at 5:00 PM, luka  wrote:
> > I have recently upgraded from GAE 1.4.3 to 1.5.0 and the MemCache service I
> > work with stop working.
> > When I try to reach it, I get NullPointerException
>
> > I saw that the package of "GCacheFactory" was changed
> > to "com.google.appengine.api.memcache.stdimpl" although besides of that I am
> > not sure what is wrong.
>
> > I have a simple cache implementation using Spring beans which worked well
> > during the last year:
>
> > (Please help, my service has customers waiting for solution)
>
> > import java.util.HashMap;
> > import java.util.Map;
> > import java.util.logging.Level;
> > import java.util.logging.Logger;
>
> > import com.google.appengine.api.memcache.stdimpl.GCacheFactory;
>
> > import net.sf.jsr107cache.Cache;
> > import net.sf.jsr107cache.CacheException;
> > import net.sf.jsr107cache.CacheManager;
>
> > public class OnLoad {
> >  private static Logger logger =
> > Logger.getLogger("com.lugo.server.utils.OnLoad");
>
> > Cache cache;
> >  /**
> >  * Create Cache ( MemCache )
> >  */
> >  public void onStartup(){
> > try {
> > Map props = new HashMap();
> >  int expire = 3600 * 192 ; // 8 Days
> >         props.put(GCacheFactory.EXPIRATION_DELTA, expire);
> >  cache = CacheManager.getInstance().getCacheFactory().createCache(props);
> >         } catch (CacheException e) {
> >         logger.log(Level.WARNING,"onStartup()", e);
> >         }
> > }
> >  public Cache getCache(){
> > return cache;
> >  }
>
> > }
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Google App Engine for Java" group.
> > To post to this group, send email to
> > google-appengine-java@googlegroups.com.
> > To unsubscribe from this group, send email to
> > google-appengine-java+unsubscr...@googlegroups.com.
> > For more options, visit this group at
> >http://groups.google.com/group/google-appengine-java?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Re: Single session per user

[Scott]

Finally got it all figured out. You have to delete both the memcache
session entry and the datastore record. The sessions are not managed
this way othe dev server it seems, so it won't work in that context.

Here is the code:

 if(foundUser.getSessionId() !=
null) {

MemcacheService ms =
MemcacheServiceFactory.getMemcacheService();

String mkey = "_ahs" + 
foundUser.getSessionId();
if(ms.contains(mkey)){
log.info("user already 
logged in sending logout.");
ms.delete(mkey);

DatastoreService ds =
DatastoreServiceFactory.getDatastoreService();

com.google.appengine.api.datastore.Query q = new
com.google.appengine.api.datastore.Query("_ah_SESSION");
Key key = 
KeyFactory.createKey("_ah_SESSION","_ahs" +
foundUser.getSessionId());

q.addFilter(Entity.KEY_RESERVED_PROPERTY, FilterOperator.EQUAL,
key);

Entity e = 
ds.prepare(q).asSingleEntity();
if(e != null) {
Response r = 
new Response(RESPONSES.LOGGED_OUT);
SendMessageTask 
smt = new
SendMessageTask(pm.detachCopy(foundUser), r, false);
TaskOptions 
taskOptions = withUrl(Constants.MATCH_QUEUE);

defer(smt,"matchQueue", taskOptions);


com.google.appengine.api.datastore.Transaction txn =
ds.beginTransaction();
try {


ds.delete(key);


txn.commit();
} finally {
if 
(txn.isActive()) {

txn.rollback();
}
}
}
}

}



Here is what is going on:
* I check to see if the user found in the datastore has a session id
saved.
* I check the memcache to see if the "_ahs" + session id object exist.
* I delete the memcache object.
* I delete the entity in "_ah_SESSION".
* I send a message via a channel to the logged in device telling it to
log out. If this device is off or the browser is closed, this message
will just expire.

This is a pretty inefficient way of going about this, but I don't
expect it to happen all that often.






On Feb 27, 10:51 am, Scott  wrote:
> Thanks for the response Didier!
>
> I know I can store the session id, but how do I then retrieve the
> session object by id? I have been looking at ways to access
> _ah_SESSION entities using the low level datastore api like this:
>
>            DatastoreService ds =
> DatastoreServiceFactory.getDatastoreService();
>            com.google.appengine.api.datastore.Query q = new
> com.google.appengine.api.datastore.Query("_ah_SESSION");
>
> I can set a filter to return a single object matching a key value, but
> is the session id the key value? This is difficult to test, because it
> seems the _ah_SESSION entities are not created on the dev server.
>
> I have also read that session objects could be stored in the memcache.
> I have been trying this as well using the following code:
>
>             MemcacheService ms =
> MemcacheServiceFactory.getMemcacheService();
>             ms.get(?);
>
> I am again not certain what to use for the key value in the ms.get()
> call. I know the key is prefixed with _ahs, but I don't know the rest.
> Is this prefix appended to the session id in some way? This is also
> difficult to test on the dev server, because the sessions don't seem
> to be stored in the memcache.
>
> Does anyone know the difference in session management on the dev
> server vs the production

[appengine-java] Re: Single session per user

[Scott]

Thanks for the response Didier!

I know I can store the session id, but how do I then retrieve the
session object by id? I have been looking at ways to access
_ah_SESSION entities using the low level datastore api like this:

   DatastoreService ds =
DatastoreServiceFactory.getDatastoreService();
   com.google.appengine.api.datastore.Query q = new
com.google.appengine.api.datastore.Query("_ah_SESSION");

I can set a filter to return a single object matching a key value, but
is the session id the key value? This is difficult to test, because it
seems the _ah_SESSION entities are not created on the dev server.

I have also read that session objects could be stored in the memcache.
I have been trying this as well using the following code:

MemcacheService ms =
MemcacheServiceFactory.getMemcacheService();
ms.get(?);

I am again not certain what to use for the key value in the ms.get()
call. I know the key is prefixed with _ahs, but I don't know the rest.
Is this prefix appended to the session id in some way? This is also
difficult to test on the dev server, because the sessions don't seem
to be stored in the memcache.

Does anyone know the difference in session management on the dev
server vs the production server?

Thanks,
Scott



On Feb 27, 2:15 am, Didier Durand  wrote:
> Hi,
>
> Yes, the simplest way is to store the session id in the datastore.
>
> You can get a session id via javax.servlet.http .HttpSession.getId();
>
> regards
>
> didier
>
> On Feb 26, 7:51 pm, Scott  wrote:
>
>
>
>
>
>
>
> > Howdy,
>
> > I want to only allow my users to have a single session at a time. The
> > act of logging in should invalidate a session that might have been
> > created from another devices as well as sending that device a notice
> > via the channel that the session had ended.
>
> > I actually had this working on my dev server. I used a singleton to
> > hold a copy of the session object and channel information, and when
> > the user logged in from a different device, I would invalidate and
> > send a message. The problem is that since GAE the production server is
> > distributed, I have no guarantee of the same JVM on each call.
>
> > Reading through docs and group post, the only possibility I could
> > think of was to retrieve the session object from the datastore or
> > memcache. Is it possible to read the session object from the either of
> > these locations? If so what is the key and object type?
>
> > Thanks a lot,
> > Scott

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Single session per user

[Scott]

Howdy,

I want to only allow my users to have a single session at a time. The
act of logging in should invalidate a session that might have been
created from another devices as well as sending that device a notice
via the channel that the session had ended.

I actually had this working on my dev server. I used a singleton to
hold a copy of the session object and channel information, and when
the user logged in from a different device, I would invalidate and
send a message. The problem is that since GAE the production server is
distributed, I have no guarantee of the same JVM on each call.

Reading through docs and group post, the only possibility I could
think of was to retrieve the session object from the datastore or
memcache. Is it possible to read the session object from the either of
these locations? If so what is the key and object type?

Thanks a lot,
Scott

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Re: Channel API problem

[Scott]

The Channel API is working for me, but I don't see anything you are
doing wrong in your code. Here is a link to another discussion
containing the code that worked for me
https://groups.google.com/d/topic/google-appengine-java/GfJQsf8OQ0I/discussion


On Dec 16, 5:26 am, Tarun  wrote:
> I have been trying to use the newly launched Channel API on SDK 1.4 in
> my project. I try to push a message to client which actually pops up a
> notification but I am getting the ChannelFailureException.
> Here is the code:
> TestServlet Where the token is created:
> public void doGet(HttpServletRequest req, HttpServletResponse resp)
>                         throws ServletException, IOException {
>                 PrintWriter out = resp.getWriter();
>                 resp.setContentType("text/html");
>                 // Creating the userID on the basis of the session id
>                 String userId = req.getSession().getId();
>                 // Starting the ChannelService of the channel API
>                 ChannelService channelService = ChannelServiceFactory
>                                 .getChannelService();
>                 // Creating the client token
>                 String token = channelService.createChannel(userId);
>                 req.getSession().setAttribute("token", token);
>
>                 out.println("");
>                 out.println("");
>                 out.println("This is a servlet which will be used to 
> try the
> new Channel API");
>                 out.println("");
>                 out.println("");
>                 out.println("Click Here to 
> recieve a
> notification on the next page using channel API");
>                 out.println("");
>         }
>
> }
>
> Channel API.jsp
> 
>     channel = new goog.appengine.Channel('<%= userId%>');
>     socket = channel.open();
>     //socket.onopen = onOpened;
>     socket.onmessage = onMessage;
>     socket.onerror = onError;
>     socket.onclose = onClose;
>     onMessage = function (message){
>         $.pnotify({
>                         pnotify_title: 'Regular Notice',
>                         pnotify_text: message.data
>                 });
>     };
>
>     onError = function (description,code){
>         $.pnotify({
>                         pnotify_title: 'Uh Oh!',
>                         pnotify_text: 'something went wrong',
>                         pnotify_type: 'error',
>                         pnotify_hide: false
>                 });
>     };
>  
>
> Send Message servlet:
> public void doGet(HttpServletRequest req, HttpServletResponse resp)
>                         throws ServletException, IOException {
>
>                 String userId = (String) 
> req.getSession().getAttribute("token");
>                 ChannelService channelService =
> ChannelServiceFactory.getChannelService();
>                 channelService.sendMessage(new ChannelMessage(userId,
>                                 "This is the message that has been sent"));
>
>         }
>
> }
>
> The stack trace I got
> Uncaught exception from servlet
> com.google.appengine.api.channel.ChannelFailureException: An
> unexpected error occurred.
>         at
> com.google.appengine.api.channel.ChannelServiceImpl.sendMessage(ChannelServ 
> iceImpl.java:
> 59)
>
> Can you tell me if there is something wrong with the code or with the
> Channel API?
>
> Thanks in advance,
> Tarun

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Channel socket close error

[Scott]

Howdy,

When I close my socket from a javascript based client by calling the
socket.close() method the following error is always caught in the
socket.onerror function:

500 - Client connection with ID connection-17 not found

Do I need to do something different to not get this error or can I
safely ignore it?

Thanks,
Scott

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Re: 1.4.0 Preview and Channel API

[Scott]

I do not have the docs. Can't those be generated off the jar?

I noticed a constant network activity icon while running the app on my
iPhone, which is probably associated with the polling. I am worried
how this will affect battery life. When I have a chance I am going to
dig a bit deeper into the javascript source. and see if I can figure
out more about that.

James M wrote:
> Thanks for the tip.  I added the call to the controller to create the
> ChannelService.  However, what's odd is that my implementation is like
> your first attempt.  I had to use the channelId returned from
> ChannelService in the JavaScript when opening the socket.
>
> The other odd thing I'm seeing is that this is polling.  Do you happen
> to have the javadocs for the ChannelAPI?  So far I haven't turned up
> anything specific to Java.
>
> Thanks again,
> James
>
>
>
> On Nov 29, 12:01 pm, Scott  wrote:
> > Is the HTML snippet the full client code? If so you need to call the
> > createChannel service from the client prior to opening the socket.
> > Also what does the "?key=dev" do for you in your script import?
> >
> > Good luck!
> >
> > Scott
> >
> > On Nov 29, 7:20 am, James M  wrote:
> >
> >
> >
> >
> >
> >
> >
> > > Hi Scott,
> >
> > > I'm trying to follow your example but am getting stuck.  I've created
> > > the following controllers:
> >
> > > /** create the channel **/
> > >  public ModelAndView createChannel(HttpServletRequest request,
> > > HttpServletResponse response) {
> > >           ChannelService channelService =
> > > ChannelServiceFactory.getChannelService();
> > >       channelService.createChannel("coupoz");
> >
> > >       return constructModelAndView("example");
> >
> > >   }
> >
> > > /** send a message **/
> > >   public ModelAndView sendMessage(HttpServletRequest request,
> > > HttpServletResponse response) {
> > >       ChannelService channelService =
> > >           ChannelServiceFactory.getChannelService();
> > >                       channelService.sendMessage(new
> > >           ChannelMessage("coupoz","test"));
> >
> > >           return constructModelAndView("example");
> > >   }
> >
> > > // NOTE:  both controllers return a JSON string.
> >
> > > I also have an HTML file:
> >
> > > 
> > > 
> > > 
> > > 
> > >          var channel = new goog.appengine.Channel("coupoz");
> > >          var   socket = channel.open();
> > >             socket.onopen = function() {
> > >                 console.info("socket opened...");
> > >             }
> > >             socket.onmessage = function(evt) {
> > >                  var o = JSON.parse(evt.data);
> > >                  alert(o);
> > >             }
> > > 
> >
> > > 
> >
> > > 
> > > 
> > > 
> >
> > > My steps to test are as follows:
> >
> > > * Loadhttp://localhost:/business/createChannel.do
> > > * Loadhttp://localhost:/channel.html
> > > * Loadhttp://localhost:/business/sendMessage.do
> >
> > > Here is the stack trace that I get when loading the channel.html file:
> >
> > > WARNING: /_ah/channel/dev
> > > com.google.appengine.api.channel.dev.LocalChannelFailureException:
> > > Channel for application key null not found.
> > >         at
> > > com.google.appengine.api.channel.dev.ChannelManager.getChannel(ChannelManag
> > >  er.java:
> > > 58)
> > >         at
> > > com.google.appengine.api.channel.dev.ChannelManager.getClientChannel(Channe
> > >  lManager.java:
> > > 73)
> > >         at
> > > com.google.appengine.api.channel.dev.ChannelManager.connectClient(ChannelMa
> > >  nager.java:
> > > 122)
> > >         at
> > > com.google.appengine.api.channel.dev.LocalChannelServlet.doGet(LocalChannel
> > >  Servlet.java:
> > > 64)
> > >         at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
> > >         at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
> > >         at 
> > > org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:
> > > 511)
> > >         at org.mortbay.jetty.servlet.ServletHandler
> > > $CachedChain.doFilter(Servle

[appengine-java] Re: 1.4.0 Preview and Channel API

[Scott]

Is the HTML snippet the full client code? If so you need to call the
createChannel service from the client prior to opening the socket.
Also what does the "?key=dev" do for you in your script import?

Good luck!

Scott

On Nov 29, 7:20 am, James M  wrote:
> Hi Scott,
>
> I'm trying to follow your example but am getting stuck.  I've created
> the following controllers:
>
> /** create the channel **/
>  public ModelAndView createChannel(HttpServletRequest request,
> HttpServletResponse response) {
>           ChannelService channelService =
> ChannelServiceFactory.getChannelService();
>       channelService.createChannel("coupoz");
>
>       return constructModelAndView("example");
>
>   }
>
> /** send a message **/
>   public ModelAndView sendMessage(HttpServletRequest request,
> HttpServletResponse response) {
>       ChannelService channelService =
>           ChannelServiceFactory.getChannelService();
>                       channelService.sendMessage(new
>           ChannelMessage("coupoz","test"));
>
>           return constructModelAndView("example");
>   }
>
> // NOTE:  both controllers return a JSON string.
>
> I also have an HTML file:
>
> 
> 
> 
> 
>          var channel = new goog.appengine.Channel("coupoz");
>          var   socket = channel.open();
>             socket.onopen = function() {
>                 console.info("socket opened...");
>             }
>             socket.onmessage = function(evt) {
>                  var o = JSON.parse(evt.data);
>                  alert(o);
>             }
> 
>
> 
>
> 
> 
> 
>
> My steps to test are as follows:
>
> * Loadhttp://localhost:/business/createChannel.do
> * Loadhttp://localhost:/channel.html
> * Loadhttp://localhost:/business/sendMessage.do
>
> Here is the stack trace that I get when loading the channel.html file:
>
> WARNING: /_ah/channel/dev
> com.google.appengine.api.channel.dev.LocalChannelFailureException:
> Channel for application key null not found.
>         at
> com.google.appengine.api.channel.dev.ChannelManager.getChannel(ChannelManag 
> er.java:
> 58)
>         at
> com.google.appengine.api.channel.dev.ChannelManager.getClientChannel(Channe 
> lManager.java:
> 73)
>         at
> com.google.appengine.api.channel.dev.ChannelManager.connectClient(ChannelMa 
> nager.java:
> 122)
>         at
> com.google.appengine.api.channel.dev.LocalChannelServlet.doGet(LocalChannel 
> Servlet.java:
> 64)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
>         at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:
> 511)
>         at org.mortbay.jetty.servlet.ServletHandler
> $CachedChain.doFilter(ServletHandler.java:1166)
>         at
> com.google.appengine.api.blobstore.dev.ServeBlobFilter.doFilter(ServeBlobFi 
> lter.java:
> 58)
>         at org.mortbay.jetty.servlet.ServletHandler
> $CachedChain.doFilter(ServletHandler.java:1157)
>         at
> com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(Trans 
> actionCleanupFilter.java:
> 43)
>         at org.mortbay.jetty.servlet.ServletHandler
> $CachedChain.doFilter(ServletHandler.java:1157)
>         at
> com.google.appengine.tools.development.StaticFileFilter.doFilter(StaticFile 
> Filter.java:
> 122)
>         at org.mortbay.jetty.servlet.ServletHandler
> $CachedChain.doFilter(ServletHandler.java:1157)
>         at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:
> 388)
>         at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:
> 216)
>         at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:
> 182)
>         at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:
> 765)
>         at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:
> 418)
>         at
> com.google.apphosting.utils.jetty.DevAppEngineWebAppContext.handle(DevAppEn 
> gineWebAppContext.java:
> 70)
>         at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:
> 152)
>         at com.google.appengine.tools.development.JettyContainerService
> $ApiProxyHandler.handle(JettyContainerService.java:349)
>         at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:
> 152)
>         at org.mortbay.jetty.Server.handle(Server.java:326)
>         at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:
> 542)
>         at org.mortbay.jetty.HttpConnection
&

[appengine-java] Re: 1.4.0 Preview and Channel API

[Scott]

I found the problem and now pushing to the client works!!

In the controller that creates the channel, I just need create the
service and use the same id on the client. Whatever is returned by the
create channel method is not the id that I need to use to send the
message.

So this:

>                 ChannelService channelService =
> ChannelServiceFactory.getChannelService();
>
>                 String channelId = channelService.createChannel("coupoz");
>
>                 return new ChannelModel(channelId);

becomes:
ChannelService channelService =
ChannelServiceFactory.getChannelService();

channelService.createChannel("coupoz");


return OK_RESPONSE;

On the client, I just change:

>          var channel = new goog.appengine.Channel(channelId);

to be:
   var channel = new goog.appengine.Channel("coupoz");

The push then works great, which is really all I needed, but I am
interested in how to receive though.


On Nov 24, 8:52 pm, Scott  wrote:
> I am trying to use the Channel API in the 1.4.0 preview, but I am not
> using GWT. Here is what I have done so far:
>
> 1. Using the Spring framework I have a controller that listens for a
> channel create request form a client to create a new channel. The
> controller creates the channel using the following code:
>
>                 ChannelService channelService =
> ChannelServiceFactory.getChannelService();
>
>                 String channelId = channelService.createChannel("coupoz");
>
>                 return new ChannelModel(channelId);
>
> 2. The client, which imports the "_ah/channel/jsapi" script, then
> creates a channel and opens a socket using the channel id returned by
> the controller. Here is the client code:
>
>          var channel = new goog.appengine.Channel(channelId);
>          var   socket = channel.open();
>             socket.onopen = function() {}
>             socket.onmessage = function(evt) {
>                  var o = JSON.parse(evt.data);
>                  alert(o);
>             }
>
> 3. I then have a button that I can push from the client that calls
> another controller that sends a message to the client via the channel.
> I am sending back the channel id in the call from the client. Here is
> the code:
>
>             ChannelService channelService =
> ChannelServiceFactory.getChannelService();
>             channelService.sendMessage(new
> ChannelMessage(channelId,"test"));
>
> This seems bassackwards but when I tried just sending from the client
> using socket.send and got the error below. Plus I have no idea to
> listen for the send on the server side.
>
> WARNING: /_ah/channel/dev
> com.google.appengine.api.channel.dev.LocalChannelFailureException:
> Channel for application key null not found.
>
> When I try pushing to the client I get the following error:
> com.google.appengine.api.channel.ChannelFailureException: An
> unexpected error occurred.
> .
> .
> .
> Caused by: com.google.apphosting.api.ApiProxy$ApplicationException:
> ApplicationError: 2:
>         at
> com.google.appengine.api.channel.dev.LocalChannelService.sendChannelMessage 
> (LocalChannelService.java:
> 91)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> .
> .
> .
>
> Any tips or suggestions?
>
> Thanks!

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] 1.4.0 Preview and Channel API

[Scott]

I am trying to use the Channel API in the 1.4.0 preview, but I am not
using GWT. Here is what I have done so far:

1. Using the Spring framework I have a controller that listens for a
channel create request form a client to create a new channel. The
controller creates the channel using the following code:

ChannelService channelService =
ChannelServiceFactory.getChannelService();

String channelId = channelService.createChannel("coupoz");

return new ChannelModel(channelId);

2. The client, which imports the "_ah/channel/jsapi" script, then
creates a channel and opens a socket using the channel id returned by
the controller. Here is the client code:

 var channel = new goog.appengine.Channel(channelId);
 var   socket = channel.open();
socket.onopen = function() {}
socket.onmessage = function(evt) {
 var o = JSON.parse(evt.data);
 alert(o);
}

3. I then have a button that I can push from the client that calls
another controller that sends a message to the client via the channel.
I am sending back the channel id in the call from the client. Here is
the code:

ChannelService channelService =
ChannelServiceFactory.getChannelService();
channelService.sendMessage(new
ChannelMessage(channelId,"test"));

This seems bassackwards but when I tried just sending from the client
using socket.send and got the error below. Plus I have no idea to
listen for the send on the server side.

WARNING: /_ah/channel/dev
com.google.appengine.api.channel.dev.LocalChannelFailureException:
Channel for application key null not found.

When I try pushing to the client I get the following error:
com.google.appengine.api.channel.ChannelFailureException: An
unexpected error occurred.
.
.
.
Caused by: com.google.apphosting.api.ApiProxy$ApplicationException:
ApplicationError: 2:
at
com.google.appengine.api.channel.dev.LocalChannelService.sendChannelMessage(LocalChannelService.java:
91)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
.
.
.


Any tips or suggestions?

Thanks!





-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] 1.3.8 upgrade delete datastore contents

[Scott]

After upgrading my Java SDK to 1.3.8 and restarting the server, the
development console shows my datastore as empty. After checking the
log I found this error:

java.io.InvalidClassException:
com.google.appengine.api.datastore.dev.LocalDatastoreService$Extent;
local class incompatible: stream classdesc serialVersionUID =
-7335090543304226540, local class serialVersionUID =
3822635795418372340

I don't remember experiencing with previous upgrades. Does anyone else
see this problem?

Thanks,
Scott

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Re: [objectify-appengine] any limitation on the versions for a give app-id

[Scott Hernandez]

Yes, I think it is 10. You can delete old versions so you can upload
new ones, but you can only have a limited number at a time.

On Sat, Jul 17, 2010 at 9:42 AM, aswath satrasala
 wrote:
> I am planning to update the application every month.  Hence, I want to give
> a different version no for the new upload to the appengine.
> Is there any limit on the number of versions that can be uploaded to
> appengine for the given app-id.
>
> -Aswath
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Re: 500 Error Caused by: java.lang.NoSuchMethodException: org.codehaus.groovy.runtime.dgm$692.()

[Scott]

Are you guys including the Httpbuilder jar?  Because my application
works perfectly until I add that monster, then I get the same errors
as you.  If I then remove it, they go away.

I have tried deploying several times with the only difference being
the HttpBuilder jar, thus I have concluded that that is the culprit.

On May 5, 6:42 am, Christian Goudreau 
wrote:
> I have the same problem from time to time... It's anying every 60 secondes
> my app throw that message.
>
> Christian
>
>
>
> On Wed, May 5, 2010 at 2:29 AM, Daisy  wrote:
> > A few minutes after I posted the previous message, the app started to
> > work.  But that is a testing app, and the real one's app id is
> > shijinxiaochu, which still has the same problem.  Could somebody
> > please fix that as well?  Thanks.
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Google App Engine for Java" group.
> > To post to this group, send email to
> > google-appengine-j...@googlegroups.com.
> > To unsubscribe from this group, send email to
> > google-appengine-java+unsubscr...@googlegroups.com
> > .
> > For more options, visit this group at
> >http://groups.google.com/group/google-appengine-java?hl=en.
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Google App Engine for Java" group.
> To post to this group, send email to google-appengine-j...@googlegroups.com.
> To unsubscribe from this group, send email to 
> google-appengine-java+unsubscr...@googlegroups.com.
> For more options, visit this group 
> athttp://groups.google.com/group/google-appengine-java?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] Eclipse plugin - prevent auto-copy of jars

[Scott Hernandez]

There was  recent discussion about dependencies (valid for Objectify or any
datastore based impl.)  but the eclipse plugin question has not been
answered well.

http://groups.google.com/group/objectify-appengine/browse_thread/thread/d464550b97cee2e3/45c70bea305822fd

I suspect that if you enable the plugin, it will always copy the files into
your lib dir :(

You can disable it and use ant/command-line to deploy, as a last ditch
option.

On Sun, Apr 4, 2010 at 11:25 AM, Krishna wrote:

> Hi,
>
> I'm using Objectify for persistence and I'd really like to remove all
> jdo /  jpa related jars from my lib directory but eclipse keeps
> transferring them back. I also tried modifying the contents of the
> com.google.appengine.eclipse.core file under the .setting directory
> but I guess eclipse overwrites my changes on startup and then copies
> over the jars anyway. I've seen discussions where people have used the
> appcfg tool to manually deploy instead of doing so from within eclipse
> but I'm hoping a better solution  exists.
>
> This blogpost
> http://www.answercow.com/2010/03/google-app-engine-cold-start-guide-for.html
> claims that startup times reduce by ~400ms by removing these jars and
> I really love speedups especially when they are free :-).
>
> Perhaps someone from Google can comment on whether removing these jars
> will actually speedup apps even if the app does not directly use
> them.
>
> Also, I am not sure which jars need to be retained for sure and I'd
> really appreciate it if someone could could give me a pointer
> regarding this. I am pretty new to Java so apologies if this is a
> naive question.
>
> Thanks so much,
>
> Krishna
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine for Java" group.
> To post to this group, send email to
> google-appengine-j...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine-java+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/google-appengine-java?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] allowMultipleRelationsOfSameType = true

[scott]

I have upgraded to AE SDK 1.3.2, and am still having problems
retrieving a class that contains multiple properties of the same type,
i.e.

class A {

class B x;
class B y;

}

With the allowMultipleRelationsOfSameType= true, the persistence is
OK, but the retrieve returns duplicate value for properties of the
same type (even though it was saved with different values for x & y).

Is there a fix planned for this?

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] Objectify - Twig - approaches to persistence

[Scott Hernandez]

I appreciate your concern for Objectify staying clean, and consistent.
It *is* good to have frameworks that offer different feature sets and
ways in which they implement them.

On Sun, Mar 14, 2010 at 4:27 AM, John Patterson  wrote:
> On 13 Mar 2010, at 11:00, Jeff Schnitzer wrote:
>>
>> since you can (and IMNSHO probably should) always disable automatic
>> activation and refresh the graph manually.
>
> Although I dislike premature optimisations such as this note that you can
> configure Activation to not activate any Objects by default.
>
>> After you explained the concept of uninitialized entities (the brief
>> blurb in your docs really isn't enough), I actually rather like your
>> solution!  I might even implement something similar in Objectify.  But
>> I really think you need to document the hell out of the issues
>> surrounding them.  It is very very easy to corrupt data.
>
> Despite all the talk about differences in "philosophy" and the "down
> playing" of the features in Twig I can see from the Objectify mailing list
> that discussion has already started on how it would be possible to add
> support for direct references and async parallel commands.
>
> Be very careful not to end up with an API that bulges with features that are
> tacked on as an after thought.  The Objectify Query API as it is now would
> need to explode with permutations.  Perhaps it is best to stick to the goal
> of being a more usable object capable interface to the low-level API?
>  Objectify does this very well.

I have been arguing for direct references since the beginning of
Objectify. The problem has always been one of how to implement it. The
difficulties have always been conceptual and performance related, not
actually making it work ;)

Some of the questions that come up are: how deep to you go down the
object graph, what do you do about uninitialized objects (which might
only have a key/id[+parent]), when do you get the
children/referenced-objects, what transaction do you use (the same as
the initial get, or can you specify one at each level), how do you
deal with puts (do you cascade, or require the user to explicitly put
what they've changed), do you support automatic "dirty detection"
(related to the last question), are there any special behaviors with
delete, does the implementation seem obvious (and clear) to the user?

> When Twig started as not much more than a thin wrapper around the low-level
> interface and grew in complexity as more high level Object oriented features
> were added it became obvious that mixing direct references and low-level
> Keys and Queries in the same API is just not manageable.  So I stripped out
> almost all the low-level references and the result is a very clean, focused
> API.
>
> Adding a collection of helper functions to Objectify for things like merging
> OR queries would very soon become unorganised and make exploring the API
> difficult.

"Or" support should be trivial, as you have said. You just add an "Or"
method to our query object (maybe changing the current "filter" method
to "and"). All the work goes on below the covers.

> You should probably make a decision about the design goals of Objectify and
> stay true to it.  There is always the option of building a higher-level
> interface on top of Objectify - now that would be interesting!  In Twig you
> have the option to drop down to use the low-level datastore service if you
> really need to - but the necessity for this has decreased a lot with the
> final release 1.0
>
> Although this our-answer-to-the-great-question-is-the-only-logical-one
> banter is terrific fun it might make sense to work on some functionality in
> common.  Scott and I briefly mentioned making a common profiling ApiProxy to
> help understand the performance of your app.  I've made a quick start on
> this but it really is a feature that makes sense not just for Twig.

We have a lot of room to create some common shared utility libraries
(for performance tuning/collection, async operations, mapping
properties, etc.). In fact, it would be a great step forward if we
could make sure the mapping for our objects follow the same naming
scheme. For things like Embedded it is important that we represent the
entity mapping the same so that people can changes from
twig/objectify/other-framework without having to re-process all their
entities. John, what do you think about that?

> If the feature sets of Twig and Objectify end up converging over time -
> albeit with differences in default settings - it begs the question if there
> might be some way to cooperate on higher level functionality also.

It does strike me that much of our framework functionality is common,
and therefor is probably a wasted effort (if we

Re: [appengine-java] Re: Objectify - Twig - approaches to persistence

[Scott Hernandez]

Nacho, this may be a silly question, have you used the datastore api
or app-engine?

On Fri, Mar 12, 2010 at 2:19 PM, Nacho Coloma  wrote:
>> Because the datastore has no schema the interfaces need to define the schema
>> themselves - as Java objects and some annotations.  So if you are already
>> defining the schema at the data model level it makes sense to define changes
>> there too.
>>
>> Do you mean use the low-level api instead to alter the data?  In that case
>> you still need to configure the interface to read/write the new format - so
>> may as well reuse it, no?
>
> This is the point were we differ. To me it makes all sense to prepare
> a low-level API for schema migration, but I would not bind it to the
> current mappings. I mean, the possibilities here are just too many,
> and the up-to-date persistence mappings are not relevant to do the
> migration. They seem to be more of a hassle than helpful for that
> purpose.

Okay, I'll bit. How do you do schema migration (both on a live system,
and in a batch way), in the low-level api?. In the datastore every
entity has its own schema. There is no way to alter the schema without
altering the data that is stored in an entity.

> I may be wrong, so I am just asking. I was comparing the examples
> given here with what would be the equivalent of porting to GAE the
> typical schema upgrade script, and didn't see much benefit in it.

What "update script" are you talking about? The only way to update
data in the datastore is write it back to the datastore; There is no
DDL, or metadata language, or data transformation system for the
datastore, that I know about.

>> I do remember something in HIbernate that could generate "update table"
>> scripts from changed config files.  But I guess with all back ends they had
>> to support it was a much tougher job.
>
> AFAIK they have something to create the DDL, but then you should use
> your own database tools to create the upgrade script.

They can create schema, and migrate schema automatically (generating
the alter scripts without dropping the tables), assuming the types of
changes you do are supported. I've found hibernate to do a good job in
creating and migrating schema automatically for development, and if
careful, production too. It is considered good practice to generate
the migration scripts from hibernate, and then hand-fix them to be
safe.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] Objectify - Twig - approaches to persistence

[Scott Hernandez]

I hope the general public is enjoying this discussion. There *are*
lots of useful points in this thread, really :) It isn't just about
Twig and Objectify; it is clearly coming down to "philosophical
differences".

On Fri, Mar 12, 2010 at 6:52 AM, John Patterson  wrote:
>
> On 12 Mar 2010, at 13:01, Scott Hernandez wrote:
>>
>> We have a different idea about live systems and managing
>> upgrades/deployments. To answer the question below, you can always
>> upgrade the data in place because you will always need a way to load
>> that data into the current object representation. It just may be that
>> you need to take the system offline to migrate the data.
>
> In my app taking the system off-line while re-processing the data is not an
> option.  The actual reprocessing can take days.
>
> I see your point though.  My data is mainly static so I don't have the
> issues you describe of keeping the new version in sync with the live
> version.  However, I can think of several solutions to this while still
> having the safety advantage of independent "tables" in the datastore.
>
> The simple way would be to, during the migration period, create both a
> Person and a PersonNew whenever a user makes an edit.
>
> Another way would be to do this transparently by creating a simple "forked
> translator" that stored the Person as two Entities instead of one.  I won't
> go into details but it is certainly not a major extension.
>
> I would probably go for the first method if it was just for my app - but
> this is the kind of feature that could be pushed into the framework to make
> it easier for others to benefit too.

Right, that brings up another issue. Code maintenance. You are now
changing the types (two versions are represented by two different java
classes in your project at the same time) of your pojos. If you pass
those objects through many layers of code (Entities -> DAO ->
Service-Framework -> GWT/JS) you might need to either abstract the
implementation or change the objects across all the layers of your
application. This may be a good practice in large, versioned systems,
but in small apps requiring that schema changes may require doubling
your entity classes, and possibly changing your interfaces with a new
(temporary) classes will probably only make things more complicated.
The code is going to get more complicated, and possibly in the wrong
places.

I would assume you do something like this when versioning an object:
1) Duplicate the class code 2) renaming it to something temporary
(with the old version number), and 3) incrementing the version on the
original class. Now what? It gets complicated. You now have two
versions of an object, where only the old one has data until you do a
mass migration. If you do just in time migration you have all kinds of
issues and you are back to a solution like we have come up with in
Objectify. If you wait and do a full data migration then you have your
data offline until that migration is done.

It seems like the better solution if you are concerned about
data-loss, and creating backups, would be having a goal of in-place
migration with a backup option. The framework could create a temporary
(backup) copy of the old state (call it -migration-x, or
something else unlikely to be used). Then you can clean-up your backup
data when things go well, or you can reprocess the backup data if
there are problems (the first n times).

Also, it isn't that there is no testing for me, using Objectify. You
can easily write unit tests that populate the local datastore with the
old version of your data and then runs tests for the migration
(in-place, or otherwise). Either way you need to be careful when
migrating data from one schema to another. It is the same in sql, or
any database. It would be much more helpful if app-engine had some
sort of backup, or snapshot, for us to leverage.

>>> The versions remain completely separate.  Modifying live data in place
>>> gives
>>> me heart pains.
>>
>> Duplicating data (during upgrades) is unacceptable, for my app. It may
>> be safer to leave the old data, but it is not always possible.
>
> There is no maximum stored data on App Engine.  If you delete the data after
> one day is that still a problem?
>

There may be no limit, but there is a cost. The more data you have to
move, the slower it will be. I don't disagree, full data migration is
probably best. If you can take the system down for a short enough
period, why not reprocess and migrate all the data?

>> I have
>> a *lot* of data, that was costly to generate (both in terms of network
>> and cpu) and to store. Jeff, and others on the Objectify list, have
>> spent a lot of time working on a solution with the goal of keeping the
>> app up during an upgra

Re: [appengine-java] Objectify - Twig - approaches to persistence

[Scott Hernandez]

Okay, time to chime in. There are some pretty different philosophies
driving these frameworks.

On Fri, Mar 12, 2010 at 4:56 AM, John Patterson  wrote:
>
> On 12 Mar 2010, at 05:45, Jeff Schnitzer wrote:
>
> How, in Twig, do you rename a field?
>
> I'm glad you brought this up.  I've found simple renames are better handled
> at the low level - no need to rehydrate them into instances just to rename a
> property.
> But for typesafe changes large or small Twig supports data migration in a
> much safer, more flexible way than Objectify.  Read on for details.

We have a different idea about live systems and managing
upgrades/deployments. To answer the question below, you can always
upgrade the data in place because you will always need a way to load
that data into the current object representation. It just may be that
you need to take the system offline to migrate the data. Objectify's
goal is to not require taking your system down to make data (schema)
changed.

> Cool, the @AlsoLoad is quite a neat feature.  Although very limited to
> simple naming changes and nothing structural.  All this is based on a
> dangerous assumption that you can modify "live" data in place.  Hardly
> bullet proof.
> The Twig solution is to create a new version of the type (v2) and process
> your changes while leaving the live data completely isolated and safe.  Then
> after you have tested your changes you bump up the version number of your
> live app.
> So basically:
> 1 - Create your new class PersonNew and set its version to 2
> @Version(2)
> class PersonNew {
>     int age;
>    String fullName;
> }
> 2 - Configure the "Kind name" to be the same e.g. "P" (short kind names save
> DS space)
> 3 - Load your old Person instances and for each store a new PersonNew
> instance.
> Your data is now upgraded.
> 4 - Now you can just replace Person with PersonNew in your code and deploy a
> new version.
> 5 - Test the version.  This crucial step is not possible with Objectifies
> solution.
> 6 - Only when you are happy your changes are correct make the new version
> live.  The new data version will be used and you can delete your old data
> when you want.
> So you see the Twig method of data migration is much safer than Objectifies.
>  It has the following advantages:
> 1 - You don't eve have a "half-updated" schema
> 2 - Tt works for major structural changes rather than just trivial field
> renames.
> 3 - You can test the new data structure before going live
> 4 - You can revert back to you old data structure
> The versions remain completely separate.  Modifying live data in place gives
> me heart pains.

Duplicating data (during upgrades) is unacceptable, for my app. It may
be safer to leave the old data, but it is not always possible. I have
a *lot* of data, that was costly to generate (both in terms of network
and cpu) and to store. Jeff, and others on the Objectify list, have
spent a lot of time working on a solution with the goal of keeping the
app up during an upgrade (in simple upgrades), without needing to
migrate all the existing data first, which would require downtime.

Testing is important, but either way you need to transform the data
from the old schema to the new schema, with code. Objectify gives you
ways to do this with a live system. It means you can run in a
configuration with various schemas of data (stored instances) that all
transform to the current (live) version. It may be that you change the
schema so much that you must take the system off-line to migrate data
(mostly related to getting your queries working across schemas), but
maybe not.

[snip]
> Hollow, stateful entities with lifecycles are pretty magical.
> Cascading loads and saves are pretty magical.  POJOs and Keys are
> simple.
>
> There really is nothing magical about an uninitialised instance.  Sounds
> like you are just on a witch hunt.
> John

If the documentation was *very* clear about this maybe, but as Jeff
said, Hibernates biggest complaint was from new users who got LazyInit
exceptions. This is the same thing you have created, without the
benefit of having an exception. You may just get passed an instance of
an uninitialized object if you aren't careful.

Wooppsss. Why is my data gone?... Oh, I passed the wrong
(uninitialized) object around, and then saved it.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] Re: confusion about entities group and transaction.

[Scott Hernandez]

As I understand it, you never need (or can) prefix you appid to the
key. If you could it would open up the option to get data from other
appids, and that is not possible. That is also a huge security hole,
if it existed :(

You are correct that you need to include the full ancestry to define
the key for an object. You can have entities with the same kind, and
id/name, but they must have different keys (because they belong to a
different parents).

I don't work with JDO anymore so I can' help with the code needed to
do this. I'm sure someone out there can help.

On Thu, Mar 11, 2010 at 12:57 PM, Ian Marshall  wrote:
> You're welcome!
>
> Yes, I believe that, as you have done, the key has to hold the full
> entity path, from the entity group root, through children, to the
> entity. If the entity is its own root (not in your case) then only one
> element is needed.
>
> I believe that the key also has info about the app, so that the key is
> unique throughout BigTable throughout the world: app - [root] -
> [child] - ... - [child] - "end" entity.   Here "[]" means prsent only
> if exists.
>
> I came across your problem a while ago when trying to retrieve
> entities by constructing their keys and then finding object by ID. I
> found out (by re-reading the right part of the GAE/J docs again
> properly) that full key chaining was required.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] Slim3: Global Transaction support, Fast spin-up and HOT reloading

[Scott Hernandez]

There is some interesting stuff in the release, but I've browsed the
Global Transaction stuff and I'm a little scared.
(http://sites.google.com/site/slim3appengine/slim3-datastore/transactions/global-transaction)

It seems that in the code it writes out a task in a queue to close the
commits, possibly. I'm pretty sure transactions should not be
implemented using queues, for any reason. The complexity of storing
temporary lock/tx objects in the datastore, time spent during that
transaction, and then pushing them through with a task (maybe much
later) seems like a dangerous plan.

Supporting multi-entity-group transactions needs to be implemented at
the datastore and not in a wrapper like that, IMO. There is no way to
rollback any committed transactions from any other entity-group, and
there are many other issues as well ... there is too much application
specific behavior that can't be assumed.

I can think of many reason this is a bad idea, and only very few where
this implementation will work well.

What happens if you have 3 transactions (1 transaction over 3 entity
groups) and the first two entity group transactions commit and the
third fails during commit? Is there a way to roll the first two back?

What happens if your user expect transactions to work like a database
where you have transaction isolation with gets/queries and updated the
value of the same entity multiple times from repeated gets? (think of
multiple "get, incr. object property, put" where the object only gets
incremented by the last amount, not the sum of the increments)

Where does the user get an error if the transaction fails later when
the task runs from the queue? (sure, that is application specific, but
dangerous all the same as the current models don't allow for implicit
non-synchronous operations).

These seem to be issues with any kind of custom transaction framework
on top of appengine, not just specific to the slim3 framework.

-Scott

On Tue, Mar 9, 2010 at 1:22 PM, nicolas melendez  wrote:
> cute framework
> NM
>
> On Tue, Mar 9, 2010 at 4:17 PM, Ikai L (Google)  wrote:
>>
>> Hi Yasuo,
>>
>> Wow, this look neat! I'll go ahead and add this to open source projects.
>>
>> One question, though: do you have any unit testing examples? E.g. how
>> would I mock out the datastore or write a Controller test?
>>
>> On Tue, Mar 9, 2010 at 12:42 AM, Yasuo Higa  wrote:
>> > Hi all,
>> >
>> > We are pleased to announce the release of Slim3 RC1.
>> >
>> > Slim3 is a full-stack MVC framework optimized for Google App
>> > Engine/Java.
>> > Our main concept is "Simple" and "Less Is More". "Less is more" means
>> > simplicity and clarity lead to good design.
>> >
>> > The main features of Slim3 are as follows:
>> >
>> > * Global Transactions
>> > * Faster than JDO/JPA
>> > * Fast spin-up
>> > * HOT reloading
>> > * Type safe query
>> >
>> > You can find more information about Slim3 here:
>> > http://slim3.org
>> >
>> > Thanks,
>> >
>> > Yasuo Higa
>> >
>> > --
>> > You received this message because you are subscribed to the Google
>> > Groups "Google App Engine for Java" group.
>> > To post to this group, send email to
>> > google-appengine-j...@googlegroups.com.
>> > To unsubscribe from this group, send email to
>> > google-appengine-java+unsubscr...@googlegroups.com.
>> > For more options, visit this group at
>> > http://groups.google.com/group/google-appengine-java?hl=en.
>> >
>> >
>>
>>
>>
>> --
>> Ikai Lan
>> Developer Programs Engineer, Google App Engine
>> http://googleappengine.blogspot.com | http://twitter.com/app_engine
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Google App Engine for Java" group.
>> To post to this group, send email to
>> google-appengine-j...@googlegroups.com.
>> To unsubscribe from this group, send email to
>> google-appengine-java+unsubscr...@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/google-appengine-java?hl=en.
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine for Java" group.
> To post to this group, send email to google-appengine-j...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine-java+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine-java?hl=en.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] Re: Can pm.makePersistentAll() help me write 12,000 objects?

[Scott Hernandez]

Guillermo,

Taskqueue items can only be 10K
(http://code.google.com/appengine/docs/java/taskqueue/overview.html#Quotas_and_Limits).
The basic idea is that if you have more data than that you put it into
an entity (in the data-store) and have the task pull it out and
process it. It might be that you can persist those 12K entities in a
lot of large entities (that are still under 1mByte each), but that is
a lot of work for something that will still probably fail. I guess it
all depends where the cost is on the puts (indexing, raw writes by
bytes, number of items).

And if your comeback is "memcache", well, I won't even start
discussing using a non-persistent, volatile, store like that for
temporary storage while you write them to the datastore... in batches
using the taskqueue/cron/etc.

Really, there needs to be something that can handle the write volume.
On Thu, Feb 25, 2010 at 12:08 PM, Guillermo Schwarz
 wrote:
> I think there is a way to grab big chunks of oprations, put them in a
> queue to be done asynchronously and that would be it.
>
> My take would be that using proxies it would be easy to queue any long
> operation transparently. I've done that with EJBs in the past, I don't
> see the reason a QueingProxy couldn't be written.
>
> Cheers,
> Guillermo.
>
> On 25 feb, 17:02, Jeff Schnitzer  wrote:
>> I don't think the original poster had a requirement for synchronous
>> writes; he just didn't want to do the writes asynchronously because it
>> involved a lot more code.
>>
>> I'm also perfectly fine with asynchronous writes and a very lax
>> interpretation of consistency.  I don't even mind writing extra code.
>> The thing I worry about is the feasibility of a heavy write load and
>> the total cost of it.
>>
>> Unfortunately I really can't describe in detail what I want to do (I
>> normally laugh at this kind of secrecy, but in this case it's
>> warranted).  For the game mechanic I'm thinking about, the
>> average-case scenario is not very far from the worst-case scenario.
>> Just a little detail:
>>
>>  * There is no requirement that all of a user's friends must be
>> playing the game or even have installed the app to receive points.
>> Welcome to the world of social gaming, you can play without even
>> without knowing it!
>>  * There are *lots* of FB users that have > 1k friends.  Probably
>> millions.  More active FB users are likely to have more friends... and
>> more likely to use my app.
>>  * Points can be assigned to multiple layers, so the # of updates is
>> (layers * friends).
>>  * Tens of thousands of people play this "game". It could become
>> hundreds of thousands very soon.  If I'm lucky, millions.
>>
>> I would love to implement this game mechanic, but I just can't.
>> Asynchronous or not, it's *way* too expensive on appengine.  When it
>> comes time to implement this feature (and it's going to come, I can
>> see the winds blowing), I'm probably going to have to move my scoring
>> system out of appengine.  Which is a bit ironic, because one of the
>> main advantages of appengine is scalability.
>>
>> I would *love* to see some sort of super-super-lax and
>> super-super-cheap consistency option for BigTable.  Or even an
>> alternative key/value datastore that simply works like a persistent
>> version of memcached.  Something that would let me sustain 10k
>> writes/sec without bankrupting me.
>>
>> Jeff
>>
>> On Thu, Feb 25, 2010 at 11:16 AM, Ikai L (Google)  wrote:
>>
>> > Jeff, point taken, but the original poster has been asking for three
>> > different requirements:
>> > - requirement to do all writes synchronously
>> > - sub-some-couple-hundred-millisecond writes
>> > - 12k entities being written
>>
>> > This just won't scale well if it's common. Messaging users can be done
>> > asynchronously, as can the portion crediting friends. I understand the
>> > argument that you may want to do this during the lifecycle of the request 
>> > so
>> > the original user gets some kind of feedback backed by a strongly 
>> > consistent
>> > datastore, but this just isn't done. Feedback is usually faked out
>> > optimistically, assuming that the writes will all be successful with some
>> > cached layer being the only part of the stack being updated inside the
>> > request. Thinking of worse case scenarios is a good thought exercise, but
>> > it's also a bit too optimistic to design a product assuming all of a Users'
>> > friends will play the game and engineer to meet that unrealistic
>> > expectation. What are the standard and slightly non-standard use cases? I'd
>> > probably look at a solution where I can store the data somewhere associated
>> > with the original user for any users not already in the datastore, then
>> > retrieve and generate a score for any of that User's friends on first
>> > access. Facebook's developer admin tool has some pretty good statistics 
>> > such
>> > as bounce rate, block rate and invitation accept rate that can be used to
>> > tune this design.
>> > Slightly off top

Re: [appengine-java] Re: Local datastore is empty after migrating to 1.3.1

[Scott Hernandez]

I'm pretty sure the "your datastore is empty" message on the admin
data viewer pages doesn't mean that his query, in his application
code, returned no results.

While I don't disagree with you that the data is still in there I'm
pretty what you just described is documented in the behavior of
indexes in the data-store pretty thoroughly. If you don't index a
property (or don't store it) you can not sort or search for it.

As an example of your data is there, but completely invisible, try
changing your app-id in your appengine-web.xml and looking at the
admin page data viewer after a restart.

On Thu, Feb 25, 2010 at 12:14 AM, Valters Vingolds  wrote:
> I suspect the data is still there, but somehow not visible to your
> queries.
> For example, following happened in my case: I had a query that
> retrieves my (root) entities, but they are ordered by sequence_id . In
> 1.3.0., this query worked fine even if sequence_id is un-initialized
> (is null). With 1.3.1., somehow entities with null sequence_id do not
> show up in query results - and in development I had neglected to set
> this field at all. So, getting empty results back: and this may lead
> you to think that data has been lost.
>
> Though I found out that if I remove "order by" from the query, all
> results show up - the data is still there.
>
> Now, from correctness standpoint, excluding "null" entities from
> "order by" query is ok - as you can't compare against null (per SQL
> rules) - so such rows may be dropped. Anyway, what I did, I wrote a
> small migration script to load all my entities, initialize sequence_id
> = 0 and save them. And it's now working fine with 1.3.1.
>
> On Feb 17, 9:06 pm, mcrady  wrote:
>> Same here.  If I switch back to 1.3.0 the local data is still there.
>>
>> Any workarounds to get to 1.3.1 with the local data?
>>
>> On Feb 13, 10:45 am, Brian  wrote:
>>
>> > My remote datastore kept the data, mylocalone was wiped. Pretty easy
>> > to make a script to load a DB with test data, not really a good idea
>> > to rely on yourlocaldatastore...
>>
>> > On Feb 12, 6:32 pm, mably  wrote:
>>
>> > > Same thing happened to me.
>>
>> > > Got an emptylocaldatastore after updating to 1.3.1.
>>
>> > > Hopefully I didn't have much data in it.  But it might not be the case
>> > > for everybody.
>>
>> > > On 12 fév, 09:35, Al Murauski  wrote:
>>
>> > > > The Development Console tells "The datastore is empty" after updating
>> > > > to 1.3.1.
>>
>> > > > Any ideas why? I need mylocaldata back.
>>
>> > > > The local_db file is not emoty and contain sthe data it contained
>> > > > before 1.3.1 migration.
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Google App Engine for Java" group.
> To post to this group, send email to google-appengine-j...@googlegroups.com.
> To unsubscribe from this group, send email to 
> google-appengine-java+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/google-appengine-java?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] custom authentication when using Google Apps domain

[Scott Hernandez]

It is very simple to know if you need to use google apps or not.
Here is the article about it:
http://code.google.com/appengine/articles/auth.html

1.) Do you want to authenticate users from a specific google apps
domain? (Note: Saying yes will exclude gmail and general google
accounts)

2.) Do you need to have a custom domain name?

If neither of there is true then you don't need google apps. If either
is true then you do.

If you *just* want a custom domain name then you will need to create a
"free" google apps account for your domain-name mappings; it will not
need to have any users created (your users will not be in there
because they will come from the google accounts authentication
domain).

So, to answer your question, create a free google apps account. Add
your domain mapping, and never go in there again.

Now, when it comes to security, and authorization, you will have to
write your own using some kind of application-based security (like a
filter). You will need to maintain your own roles (and membership).
This will most likely mean creating security entities keyed to their
google email address from the current logged in user. As I said above,
you will not need to pay for that authentication service (even with a
custom domain name).

Good luck to you :)

On Wed, Feb 24, 2010 at 11:54 PM, Stephan Hartmann  wrote:
> You can use Google Apps Standard accounts or common Google Accounts (e.g.
> Goggle mail) as well (you have to choose when you setup your app).
> If you choose not to use Google Apps Accounts for your app, you still can
> assign it a subdomain of your Google Apps domain.
>
> For implementing your own custom authentication i'd try
> http://securityfilter.sourceforge.net/
> It has a deployment descriptor-like configuration.
> If you choose SecurityFilter, you should take care of static files because
> GAE will serve them directly bypassing servlet filters.
>
> Regards,
> Stephan
>
>
> 2010/2/24 Houston startup coder 
>>
>> We need users to be able to access our app on our domain, so I went
>> into my Dashboard and clicked "Add New URL" to set this up, and then
>> added a CNAME at GoDaddy.  We use Google Apps for our domain
>> internally, but do not want the users of our App Engine app accessing
>> Mail, Documents, Sites, etc.  However, my understanding is that the
>> only way I could point a subdomain at our App Engine app was to use
>> Google Apps and "Add New URL".
>>
>> We're going to have an installed desktop application access our app
>> via ClientLogin, and it's fine if that needs to talk to
>> .appspot.com in order to use HTTPS.  The plan is for the
>> installed application to hide the credentials it uses to safely
>> communicate with our App Engine application behind the scenes.
>>
>> The only time our users will directly interact with the App Engine app
>> is when they login via a web browser.  For that, we'd like to use
>> custom authentication because I don't want to have to pay $50/user/
>> year for them to access via a Google Apps Premier account.  But it
>> sounds like this means we can't safeguard portions of the site with
>> the deployment descriptor if we're using custom authentication:
>>
>>
>> http://code.google.com/appengine/docs/java/config/webxml.html#Security_and_Authentication
>>
>> Is this true?  How can we handle this?
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Google App Engine for Java" group.
>> To post to this group, send email to
>> google-appengine-j...@googlegroups.com.
>> To unsubscribe from this group, send email to
>> google-appengine-java+unsubscr...@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/google-appengine-java?hl=en.
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine for Java" group.
> To post to this group, send email to google-appengine-j...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine-java+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine-java?hl=en.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] Re: Local datastore is empty after migrating to 1.3.1

[Scott Hernandez]

I got a read error when I tried running with my existing (v1.3.0)
data-store file; I created a new one (by renaming the old one) and
restarting. I suspect the internal format is different and the old
format cannot be read. Whether this is by design or a bug is another
question.

On Tue, Feb 23, 2010 at 10:26 AM, David  wrote:
> I just upgraded to 1.3.1 and also hit this issue.  I hope someone from
> the GAE team will soon comment on this behavior.  I'm also curious how
> common a problem this is. Have other people upgraded to 1.3.1 and
> maintained their local data store?
>
> On Feb 17, 3:06 pm, mcrady  wrote:
>> Same here.  If I switch back to 1.3.0 the local data is still there.
>>
>> Any workarounds to get to1.3.1with the local data?
>>
>> On Feb 13, 10:45 am, Brian  wrote:
>>
>> > My remotedatastorekept the data, mylocalone was wiped. Pretty easy
>> > to make a script to load a DB with test data, not really a good idea
>> > to rely on yourlocaldatastore...
>>
>> > On Feb 12, 6:32 pm, mably  wrote:
>>
>> > > Same thing happened to me.
>>
>> > > Got an emptylocaldatastoreafterupdating to1.3.1.
>>
>> > > Hopefully I didn't have much data in it.  But it might not be the case
>> > > for everybody.
>>
>> > > On 12 fév, 09:35, Al Murauski  wrote:
>>
>> > > > The Development Console tells "Thedatastoreisempty"afterupdating
>> > > > to1.3.1.
>>
>> > > > Any ideas why? I need mylocaldata back.
>>
>> > > > The local_db file is not emoty and contain sthe data it contained
>> > > > before1.3.1migration.
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] Editing records in datastore (using JAVA)

[Scott Hernandez]

If you would like a very simple interface to the app engine datastore
(not JPA/JDO) then Objectify might be for you. It doesn't sound like
you need to use JDO/JPA. Here is an example of exactly what you want:
http://code.google.com/p/objectify-appengine/wiki/IntroductionToObjectify#Basic_Operations:_Get,_Put,_Delete

Notice that the Car is created, stored, then (can be) retrieved,
changed, and stored again:

Car porche = new Car("2FAST", "red");
ofy.put(porche);

... you can use the same instance, or get it at another time; it will
be the same object

// Change some data and write it
porche.color = "blue";
ofy.put(porche);

The simplicity is that you only store (put) and retrieve (get) objects
from the datastore. There is no disconnected state, or any issues
therein (no merging or reconnecting existing objects to some manager).
It is simple, easy, and works with regular (POJO) objects. There isn't
any special gotchas or explicit order to storing parent or children
(just that you do :) . it works just as you should expect a simple
persistence framework to work.

On Thu, Feb 18, 2010 at 7:55 AM, Manjoor  wrote:
> I have been searching for sample java program to add, edit and delete
> records. I found many example showing how to add and delete records
> but not a single about editing. Do anyone have a sample source link to
> show how to edit a record ???
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Google App Engine for Java" group.
> To post to this group, send email to google-appengine-j...@googlegroups.com.
> To unsubscribe from this group, send email to 
> google-appengine-java+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/google-appengine-java?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] A List property - Can we filter 'null' and '[null]' values separately ?

[Scott Hernandez]

To summarize the issue, you cannot search for a null (list) if you
store a null value in any instance of that (list) property. This is
due to the way indexing is done for lists/arrays in the datastore.

The short answer is no. You will have to store another property as a
marker that you have a null list/array rather than a list/array that
contains a null entry.

In Objectify (http://code.google.com/p/objectify-appengine) we have a
feature that doesn't store null entries for @Embedded
collection/arrays; instead we store another property which keeps track
of which entries in that collection are null. Then you can search for
any collection that has null entries, but at the moment you can't
search for a null/empty collection (since we don't store them, or any
marker property indicating the difference between null/empty for that
collection). We have plans to add support for differentiating between
null/empty collections in queries, and it will be pretty easy to use,
but we are waiting on more people "needing" that feature.

Here are some discussions relating directly to this issue on the
Objectify group:
http://groups.google.com/group/objectify-appengine/browse_thread/thread/c05efb6943387eb4

On Fri, Feb 19, 2010 at 7:36 AM, vbart  wrote:
> Hello,
>
> I'm using low-level datastore API and its Query class to filter
> entities.
> I put two entities to datastore: the first has set a property 'list'
> to null and the second has the same property set to [null] (a list
> containing one null value).
>
> The following query will find both of them - that's ok:
>  new Query("MyEntity").addFilter( "list", Query.FilterOperator.EQUAL,
> null)
>
> But is there a way (using GAE API only) to filter only the first
> entity or only the second one ?
> The full code is on my blog: 
> http://vaclavb.blogspot.com/2010/02/lists-and-nulls-in-google-app-engine.html
>
> Vaclav
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Google App Engine for Java" group.
> To post to this group, send email to google-appengine-j...@googlegroups.com.
> To unsubscribe from this group, send email to 
> google-appengine-java+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/google-appengine-java?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Delete data in the datastore

[Scott]

I would like to delete about 1200 records from the datastore but the
below syntax doesn't work.

DELETE FROM table_name
WHERE some_column=some_value


Any ideas?

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] Converting Entity to Object

[Scott Hernandez]

http://code.google.com/p/objectify-appengine

(it seems it got truncated from John's message)

On Tue, Feb 2, 2010 at 10:32 AM, John Patterson  wrote:
> You can check out Twig or Objectify which all aim to do what you are asking
> about.
>
> Twig is higher level - more like an Object database built on top of the
> datastore in which you use the native low-level Queries.  Objectify is more
> low level in that you handle relationships between objects manually with
> Keys but has great docs.  Both make use of generics to make your code more
> maintainable.
>
> http://code.google.com/p/twig-persist/
>
> On 2 Feb 2010, at 18:08, Alexandru Farcas wrote:
>
>> Is there a way of converting dynamically an Entity to a specific
>> Object ?
>> For example:
>> I have some Contact objects and User, Invoice etc, witch are
>> PersistanceCapable.
>> So i can make queries (javax.jdo.Query) that return lists of those
>> kind of objects (Contact, User, Invoice etc) or i can use app engine
>> datastore.Query to get lists of Entity objects.
>> 1. Is there a simple method of converting an Entity object to a
>> Contact for example? but without doing smth like :
>>       map = entitiy.getProperties();
>>       Contact c = new Contact();
>>       c.setProperty1(map.get("property1"));
>>       c.setProperty2()
>>       ... etc
>>
>>     I want to know this because i want to use Query from the
>> datastore package and not javax.jdo and it doesn't seem a good
>> solution to make a method for each kind of entity. (I could use java
>> reflection to create the objects, but i think it is very complicated
>> since i don't know for example the types of fields and it is probably
>> a costly method).
>>
>> 2. Is the Query class form app engine datastore package more powerful
>> (or more suited) than the one from javax.jdo? Witch would you
>> recommend ?
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Google App Engine for Java" group.
>> To post to this group, send email to
>> google-appengine-j...@googlegroups.com.
>> To unsubscribe from this group, send email to
>> google-appengine-java+unsubscr...@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/google-appengine-java?hl=en.
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine for Java" group.
> To post to this group, send email to google-appengine-j...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine-java+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine-java?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Secure JSP

[Scott]

I have 1 jsp in my app that I want to secure and make people login
with their google accounts in order to see the page.  Is this possible
and if so how?

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Restrict which google accounts can access the app

[Scott]

I read that you can make people login to their google account in order
gain access to your app, but is there a way to restrict access to only
certain google account??

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] loop through all entries in memcache

[Scott Hernandez]

You can try the getTail(int) method but I don't think that will do
what you want. It is a destructive operation and will cost you
bandwidth and time.
http://code.google.com/appengine/docs/java/javadoc/com/google/appengine/api/memcache/MemcacheService.html

On Tue, Jan 26, 2010 at 9:01 AM, John Patterson  wrote:
> You can't.  You put things in and can _maybe_ take them out again later as
> long as you know the key.  Thats pretty much it.
>
> On 26 Jan 2010, at 23:28, Andrei wrote:
>
>> how do i get list of all keys for memcache?

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Running queries in the Data Viewer of the Datastore

[Scott]

Is it possible to run update/insert queries in the data viewer?  I
have tried a few different variations of them and can't seem to get it
to work.

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Re: Task queue concurrency

[Scott Hernandez]

The docs here seem to indicate that dequeuing happens at 5/sec
(default and 10 max). 
http://code.google.com/appengine/docs/java/taskqueue/overview.html

On Oct 27, 8:41 am, James Cooper  wrote:
> Hi there,
>
> Last night I experimented with task queues to see what level of
> concurrency I could achieve when running on the live environment.
>
> Summary of the test app:
>   - Bulk load 30,000 entities of a given type (3 properties / entity
> object).
>   - Command line job I ran from my PC that hit an URL to queue the
> entries
>      - This program was multi-threaded so I could simulate a bit of
> load (10 concurrent threads)
>   - Queueing URL created a task queue entry within the same app
>   - 2nd URL handled the task queue request and stored entity to the
> Datastore
>
> I watched the task queue dashboard for a few minutes and observed a
> few things:
>   - Enqueue rate quickly outpaced dequeue rate
>       - I was enqueing at about 12 requests / second, but dequeuing at
> 4 requests / second
>   - GAE did not appear to increase the dequeue rate over time in
> response to my queue depth
>
> Result: It took a very long time to dequeue 30,000 tasks (over 2
> hours).  It seemed that GAE was running one instance of my app.
>
> Expected: Much higher throughput.
>
> Is this expected behavior?  It seems that given the 30 second request
> limit that task queues are an important way to increase throughput
> (ala MapReduce).  But the "swarm" of app instances never seemed to
> arrive.
>
> thanks
>
> -- James
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---

[appengine-java] Increasing Jetty max form size for JSF application

[Scott Stevenson]

I'm hitting problems with the maximum form size imposed by Jetty.  By
default it is set to 200,000 which doesn't seem to be enough for my
client-side state saving JSF 2.0 application.

java.lang.IllegalStateException: Form too large257423>20
 [java] at org.mortbay.jetty.Request.extractParameters
(Request.java:1276)
 [java] at org.mortbay.jetty.Request.getParameter(Request.java:
641)

Is there some way to increase this maximum form size?

Using JSF server-side state saving doesn't work (serialization
errors).  It's possibly related to issue 1700 so client state saving
is my only option at the moment.

 http://code.google.com/p/googleappengine/issues/detail?id=1700

Thanks.

Scott.




--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---