Re: [google-appengine] Re: Best strategy for near real time updates?

[Pertti Kellomäki]

Hi,

On Sun, May 5, 2013 at 10:20 PM, Jeff Schnitzer  wrote:

> This has come up a number of times in the past. Timestamps are based on
> the clock on the server running your Python code. Google makes no official
> guarantees about clock skew in the cluster, although presumably they are
> all NTP synchronized. In practice, 30s of overlap is going to work 99.9% of
> the time.
>

Thanks, that's plenty reliable for my purposes. I realize this may be well
trodden water, apologies for that. I am fairly new to app engine so I need
to feel my way around a bit.
-- 
Pertti

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [google-appengine] Re: Best strategy for near real time updates?

[Jeff Schnitzer]

This has come up a number of times in the past. Timestamps are based on the
clock on the server running your Python code. Google makes no official
guarantees about clock skew in the cluster, although presumably they are
all NTP synchronized. In practice, 30s of overlap is going to work 99.9% of
the time. I wouldn't trust anything less than 10s through - people I trust
have observed multiple seconds of skew in the system.

Jeff


On Sun, May 5, 2013 at 12:13 PM, Pertti Kellomäki <
pertti.kellom...@gmail.com> wrote:

> Hi again Vinny,
>
> A related question about implementing pull updates. I would like to pull
> only the updates since the previous pull, but if possible I would like to
> avoid recording which updates have been delivered to each client.
>
> How much can I trust the datastore timestamps (i.e. auto_now)? The hrd is
> eventually consistent, and from what I've read "eventually" means "within a
> few seconds". Would I be in the clear if I post the server side timestamp T
> returned by the previous poll, and the server replies with all updates with
> timestamps from say T-30s up to now? So if the client polls once a minute,
> it polls 90 second time windows at 60 second intervals. Any duplicates
> would be dealt with by the client.
> --
> Pertti
>
>
>  --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to google-appengine+unsubscr...@googlegroups.com.
> To post to this group, send email to google-appengine@googlegroups.com.
> Visit this group at http://groups.google.com/group/google-appengine?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [google-appengine] Re: Best strategy for near real time updates?

[Pertti Kellomäki]

Hi again Vinny,

A related question about implementing pull updates. I would like to pull
only the updates since the previous pull, but if possible I would like to
avoid recording which updates have been delivered to each client.

How much can I trust the datastore timestamps (i.e. auto_now)? The hrd is
eventually consistent, and from what I've read "eventually" means "within a
few seconds". Would I be in the clear if I post the server side timestamp T
returned by the previous poll, and the server replies with all updates with
timestamps from say T-30s up to now? So if the client polls once a minute,
it polls 90 second time windows at 60 second intervals. Any duplicates
would be dealt with by the client.
-- 
Pertti

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [google-appengine] Re: Best strategy for near real time updates?

[Pertti Kellomäki]

Hi Vinny,

Essentially, if you want to push updates in less than a handful of seconds,
> use the channel API. For anything else, AJAX/JSONP pull updating is
> completely fine.
>

Thanks for your insight. In my case updates are relatively infrequent and
latency is not a big problem, so I guess I'll save a few bucks and
implement pull updating.
-- 
Pertti

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

[google-appengine] Re: Best strategy for near real time updates?

[Ray]

Use PubNub :)

On Sunday, May 5, 2013 12:32:48 AM UTC+8, Pertti Kellomäki wrote:
>
> Hi all,
>
> I am building an app where users can post comments and the like, which 
> should be pushed to selected other users in near real time. My current 
> prototype uses the  channel API and it is working fine, but I am a tad 
> worried about the relatively high cost of opening channels.
>
> What would be the best strategy for implementing updates like this? The 
> updates need not be instant, so e.g. polling once a minute would be 
> sufficient. On the other hand it does not feel right to right to do polling 
> when there is a mechanism specifically for pushing updates.
> -- 
> Pertti
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

[google-appengine] Re: Best strategy for near real time updates?

[Vinny P]

Hello Pertti,

Polling once a minute is quite slow, actually. You don't need the channel 
API for updates that are infrequent as once/minute.The Channel API is meant 
for realtime communications (i.e. seconds/less-than-second difference 
between messages) - for example, if you were implementing a chat app. 

Essentially, if you want to push updates in less than a handful of seconds, 
use the channel API. For anything else, AJAX/JSONP pull updating is 
completely fine.


-
-Vinny P
Technology & Media Advisor
Chicago, IL

My Go side project: http://invalidmail.com/


On Saturday, May 4, 2013 11:32:48 AM UTC-5, Pertti Kellomäki wrote:
>
> Hi all,
>
> I am building an app where users can post comments and the like, which 
> should be pushed to selected other users in near real time. My current 
> prototype uses the  channel API and it is working fine, but I am a tad 
> worried about the relatively high cost of opening channels.
>
> What would be the best strategy for implementing updates like this? The 
> updates need not be instant, so e.g. polling once a minute would be 
> sufficient. On the other hand it does not feel right to right to do polling 
> when there is a mechanism specifically for pushing updates.
> -- 
> Pertti
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

[google-appengine] Re: Best strategy

[Vinny P]

Honestly? It really doesn't matter what language you write in. If you were 
doing something complicated like image processing or heavy mathematical 
analysis, then we could talk about which libraries are available for each 
language and decide based on that. If all you're doing is datastore 
manipulation, any language is more than capable of handling that.

IMO, consider using Go. I've been dabbling in it for simple side projects, 
and it's quite a fun language. It also seems to be the future of AppEngine.

-Vinny

On Tuesday, March 12, 2013 6:32:43 PM UTC-5, victo...@gmail.com wrote:
>
> Thanks for the replies! I am now more confused than I was before on which 
> language I should use:P I am using datastore and to server static files and 
> that is it. There seems to be better things about python in regards to 
> class loading, but java seems to be potentially faster for other things and 
> more portable. I think the best route for me would be to use Python because 
> I will be loading the Datastore class MANY times. 
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

[google-appengine] Re: Best strategy

[timh]

I have also found (in python) that startup time is much quicker if you are 
only loading core appengine modules.  Even one non core module seemed to 
heavily affect startup times (especially on M/S when gae was having a bad 
hair day ;-).  I found early on that having a fast path to cached stuff 
that didn't involve any 3rd party modules usually meant a very quick 
startup time even on the worst M/S days.  If the cache lookup failed then 
start the full stack (ad that was as lazy as possible, ie don't load CRUD 
form definitions unless the user was logged in.)

All these strategies (which could be considered over-optimisations) tended 
to make even bad days bearable on python with M/S ;-)

I have since moved to HRD which has meant I haven't had to resort to some 
of these strategies.

T

On Thursday, March 14, 2013 2:22:50 AM UTC+8, Renzo Nuccitelli wrote:
>
>  I am part of team anti "Java/GAE". There are a lot of reason because I 
> prefer Python, but the main on is the cold start problem already mentioned. 
> The fact that the cold start time more o less proportional to the number of 
> class you have in you project make Java not first citizen on GAE. Just to 
> make it clear, it is not a Java problem, is a GAE-Java interaction problem.
>
>   About it, it's not true that Python can have this problem. You can avoid 
> it with the right approach. If you could make your code to be read in a 
> lazy way, cold start would be not a problem. I just wrote a small 
> framework, Zenwarch  to do this 
> and the result is that my cold start is constant and small all the time, 
> even when the number of script files grows.
>
>  And even webapp2 has LazyHandlers (
> http://webapp-improved.appspot.com/guide/routing.html?highlight=routing%20string)
>  
> So you can avoid the cold start.
>
>  So, for small projects, use whatever language you like. But if you think 
> your app can get bigger and you want to use java, use AWS.
>
>
>
> On Tuesday, March 12, 2013 8:36:39 PM UTC-3, Carl Schroeder wrote:
>>
>> This has been documented to death on these forums with resident instances 
>> for paid apps and posted logs with missing warmup requests. 
>>
>> Also, "Routinely" is a very loose description. Does that mean that 
>> sometimes your app loads faster than 5 seconds and sometimes slower?
>> What instance size are you talking? F4s or F1?
>>
>> If you have stumbled upon something magical that accesses the user & 
>> datastore apis and never takes longer than 5 seconds to service requests in 
>> an F1, I suggest you post some code here. You would close two open issues 
>> for Google and make the people who starred them really happy.
>>
>>
>>
>> On Tuesday, March 12, 2013 11:05:21 AM UTC-7, Vinny P wrote:
>>>
>>>
>>> On Tuesday, March 12, 2013 12:18:24 PM UTC-5, Carl Schroeder wrote:

 The sentiment is not anti-Java, it is anti GAE/Java. Writing "Hello 
 world" to the datastore takes 4-5 seconds to cold start load on an F2 in 
 GAE/Java. That is on good days, it could take twice as long on stormy 
 days. 
 The more the application grows, the worse this gets.

 Why risk 5+ second response times? Wait until Google has fixed the Java 
 instance loading time issue before you embark on writing Java apps for GAE.
 FYI, I used the low-level API and no frameworks. Latency was untenable.


>>> I can't say that I share your experiences with GAE/J, I find that I can 
>>> routinely cold start instances + datastore requests in less than 5 seconds, 
>>> although I'm recently having issues with memcache slowness...
>>>
>>> True, GAE/J has problems. But so do the Python and Go runtimes. I'm 
>>> subscribed to the google-appengine-go Google group, and I see as many 
>>> complaints there, as I do on this group about Python/Java.  Giving the OP a 
>>> balanced view of the pros/cons of each runtime is the point of this thread, 
>>> and I think it's fair to let him know what he's looking at.
>>>
>>> @OP: if you're interested in looking more about Java performance, may I 
>>> recommend this thread (it's a bit of a long read, to give you fair 
>>> warning): 
>>> https://groups.google.com/d/msg/google-appengine/sA3o-PTAckc/okDqu5aE-78J
>>>  
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [google-appengine] Re: Best strategy

[Victor Trot]

Thank you so much for the links and ideas Renzo. I did not expect this
thread to become a Java vs Python war :P


On Wed, Mar 13, 2013 at 11:22 AM, Renzo Nuccitelli  wrote:

>  I am part of team anti "Java/GAE". There are a lot of reason because I
> prefer Python, but the main on is the cold start problem already mentioned.
> The fact that the cold start time more o less proportional to the number of
> class you have in you project make Java not first citizen on GAE. Just to
> make it clear, it is not a Java problem, is a GAE-Java interaction problem.
>
>   About it, it's not true that Python can have this problem. You can avoid
> it with the right approach. If you could make your code to be read in a
> lazy way, cold start would be not a problem. I just wrote a small
> framework, Zenwarch  to do this
> and the result is that my cold start is constant and small all the time,
> even when the number of script files grows.
>
>  And even webapp2 has LazyHandlers (
> http://webapp-improved.appspot.com/guide/routing.html?highlight=routing%20string)
> So you can avoid the cold start.
>
>  So, for small projects, use whatever language you like. But if you think
> your app can get bigger and you want to use java, use AWS.
>
>
>
> On Tuesday, March 12, 2013 8:36:39 PM UTC-3, Carl Schroeder wrote:
>>
>> This has been documented to death on these forums with resident instances
>> for paid apps and posted logs with missing warmup requests.
>>
>> Also, "Routinely" is a very loose description. Does that mean that
>> sometimes your app loads faster than 5 seconds and sometimes slower?
>> What instance size are you talking? F4s or F1?
>>
>> If you have stumbled upon something magical that accesses the user &
>> datastore apis and never takes longer than 5 seconds to service requests in
>> an F1, I suggest you post some code here. You would close two open issues
>> for Google and make the people who starred them really happy.
>>
>>
>>
>> On Tuesday, March 12, 2013 11:05:21 AM UTC-7, Vinny P wrote:
>>>
>>>
>>> On Tuesday, March 12, 2013 12:18:24 PM UTC-5, Carl Schroeder wrote:

 The sentiment is not anti-Java, it is anti GAE/Java. Writing "Hello
 world" to the datastore takes 4-5 seconds to cold start load on an F2 in
 GAE/Java. That is on good days, it could take twice as long on stormy days.
 The more the application grows, the worse this gets.

 Why risk 5+ second response times? Wait until Google has fixed the Java
 instance loading time issue before you embark on writing Java apps for GAE.
 FYI, I used the low-level API and no frameworks. Latency was untenable.


>>> I can't say that I share your experiences with GAE/J, I find that I can
>>> routinely cold start instances + datastore requests in less than 5 seconds,
>>> although I'm recently having issues with memcache slowness...
>>>
>>> True, GAE/J has problems. But so do the Python and Go runtimes. I'm
>>> subscribed to the google-appengine-go Google group, and I see as many
>>> complaints there, as I do on this group about Python/Java.  Giving the OP a
>>> balanced view of the pros/cons of each runtime is the point of this thread,
>>> and I think it's fair to let him know what he's looking at.
>>>
>>> @OP: if you're interested in looking more about Java performance, may I
>>> recommend this thread (it's a bit of a long read, to give you fair
>>> warning): https://groups.**google.com/d/msg/google-**
>>> appengine/sA3o-PTAckc/**okDqu5aE-78J
>>>
>>>
>>  --
> You received this message because you are subscribed to a topic in the
> Google Groups "Google App Engine" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/google-appengine/NjF11Uy-4EM/unsubscribe?hl=en
> .
> To unsubscribe from this group and all its topics, send an email to
> google-appengine+unsubscr...@googlegroups.com.
> To post to this group, send email to google-appengine@googlegroups.com.
> Visit this group at http://groups.google.com/group/google-appengine?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

[google-appengine] Re: Best strategy

[Renzo Nuccitelli]

 I am part of team anti "Java/GAE". There are a lot of reason because I 
prefer Python, but the main on is the cold start problem already mentioned. 
The fact that the cold start time more o less proportional to the number of 
class you have in you project make Java not first citizen on GAE. Just to 
make it clear, it is not a Java problem, is a GAE-Java interaction problem.

  About it, it's not true that Python can have this problem. You can avoid 
it with the right approach. If you could make your code to be read in a 
lazy way, cold start would be not a problem. I just wrote a small 
framework, Zenwarch  to do this and 
the result is that my cold start is constant and small all the time, even 
when the number of script files grows.

 And even webapp2 has LazyHandlers (
http://webapp-improved.appspot.com/guide/routing.html?highlight=routing%20string)
 
So you can avoid the cold start.

 So, for small projects, use whatever language you like. But if you think 
your app can get bigger and you want to use java, use AWS.



On Tuesday, March 12, 2013 8:36:39 PM UTC-3, Carl Schroeder wrote:
>
> This has been documented to death on these forums with resident instances 
> for paid apps and posted logs with missing warmup requests. 
>
> Also, "Routinely" is a very loose description. Does that mean that 
> sometimes your app loads faster than 5 seconds and sometimes slower?
> What instance size are you talking? F4s or F1?
>
> If you have stumbled upon something magical that accesses the user & 
> datastore apis and never takes longer than 5 seconds to service requests in 
> an F1, I suggest you post some code here. You would close two open issues 
> for Google and make the people who starred them really happy.
>
>
>
> On Tuesday, March 12, 2013 11:05:21 AM UTC-7, Vinny P wrote:
>>
>>
>> On Tuesday, March 12, 2013 12:18:24 PM UTC-5, Carl Schroeder wrote:
>>>
>>> The sentiment is not anti-Java, it is anti GAE/Java. Writing "Hello 
>>> world" to the datastore takes 4-5 seconds to cold start load on an F2 in 
>>> GAE/Java. That is on good days, it could take twice as long on stormy days. 
>>> The more the application grows, the worse this gets.
>>>
>>> Why risk 5+ second response times? Wait until Google has fixed the Java 
>>> instance loading time issue before you embark on writing Java apps for GAE.
>>> FYI, I used the low-level API and no frameworks. Latency was untenable.
>>>
>>>
>> I can't say that I share your experiences with GAE/J, I find that I can 
>> routinely cold start instances + datastore requests in less than 5 seconds, 
>> although I'm recently having issues with memcache slowness...
>>
>> True, GAE/J has problems. But so do the Python and Go runtimes. I'm 
>> subscribed to the google-appengine-go Google group, and I see as many 
>> complaints there, as I do on this group about Python/Java.  Giving the OP a 
>> balanced view of the pros/cons of each runtime is the point of this thread, 
>> and I think it's fair to let him know what he's looking at.
>>
>> @OP: if you're interested in looking more about Java performance, may I 
>> recommend this thread (it's a bit of a long read, to give you fair 
>> warning): 
>> https://groups.google.com/d/msg/google-appengine/sA3o-PTAckc/okDqu5aE-78J
>>  
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

[google-appengine] Re: Best strategy

[Carl Schroeder]

This has been documented to death on these forums with resident instances 
for paid apps and posted logs with missing warmup requests. 

Also, "Routinely" is a very loose description. Does that mean that 
sometimes your app loads faster than 5 seconds and sometimes slower?
What instance size are you talking? F4s or F1?

If you have stumbled upon something magical that accesses the user & 
datastore apis and never takes longer than 5 seconds to service requests in 
an F1, I suggest you post some code here. You would close two open issues 
for Google and make the people who starred them really happy.



On Tuesday, March 12, 2013 11:05:21 AM UTC-7, Vinny P wrote:
>
>
> On Tuesday, March 12, 2013 12:18:24 PM UTC-5, Carl Schroeder wrote:
>>
>> The sentiment is not anti-Java, it is anti GAE/Java. Writing "Hello 
>> world" to the datastore takes 4-5 seconds to cold start load on an F2 in 
>> GAE/Java. That is on good days, it could take twice as long on stormy days. 
>> The more the application grows, the worse this gets.
>>
>> Why risk 5+ second response times? Wait until Google has fixed the Java 
>> instance loading time issue before you embark on writing Java apps for GAE.
>> FYI, I used the low-level API and no frameworks. Latency was untenable.
>>
>>
> I can't say that I share your experiences with GAE/J, I find that I can 
> routinely cold start instances + datastore requests in less than 5 seconds, 
> although I'm recently having issues with memcache slowness...
>
> True, GAE/J has problems. But so do the Python and Go runtimes. I'm 
> subscribed to the google-appengine-go Google group, and I see as many 
> complaints there, as I do on this group about Python/Java.  Giving the OP a 
> balanced view of the pros/cons of each runtime is the point of this thread, 
> and I think it's fair to let him know what he's looking at.
>
> @OP: if you're interested in looking more about Java performance, may I 
> recommend this thread (it's a bit of a long read, to give you fair 
> warning): 
> https://groups.google.com/d/msg/google-appengine/sA3o-PTAckc/okDqu5aE-78J 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

[google-appengine] Re: Best strategy

[victortrot]

Thanks for the replies! I am now more confused than I was before on which 
language I should use:P I am using datastore and to server static files and 
that is it. There seems to be better things about python in regards to 
class loading, but java seems to be potentially faster for other things and 
more portable. I think the best route for me would be to use Python because 
I will be loading the Datastore class MANY times. 

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

[google-appengine] Re: Best strategy

[Jim]

I would have to defer to others with more recent experience using Python on 
GAE.  I wrote a very early GAE/Python app when GAE first launched, but then 
switched to Java when it became available, mostly for the reasons I cited 
in my earlier post.  My early experience with Python was that it also had 
significant cold start times and could be a pig, especially with the Django 
framework.  But like I said, that was over four years ago and I'm sure 
things have changed a lot since then.



On Monday, March 11, 2013 4:38:02 PM UTC-5, victo...@gmail.com wrote:
>
> Thanks for the reply Jim. The things you mentioned below I am not doing, 
> but in terms of reading strings from the data store very frequently and 
> updating them, which is the sole purpose of my app, would you agree in this 
> scenario python is more efficient than Java?
>
> On Monday, March 11, 2013 10:44:54 AM UTC-7, victo...@gmail.com wrote:
>>
>>
>> Hello, I was wondering if someone could help me choose the best strategy 
>> for my application in terms of design.I want to use the least amount of 
>> resources as I can and optimize my code. My application is essentially 
>> written in html and javascript. I will use java to access the datastore 
>> (unless there is another more efficient way?). So essentially I will be 
>> storing two main things inside the datastore. A notification status (1 
>> single integer, I am expecting about 200,000 of these to be stored) and 
>> content (5000 ascii characters each expecting about 200,000 of these as 
>> well). They will be access quite frequently (most likely 50,000 
>> notifications and content will be read and updated every 30 mins). I have 
>> read much about optimizing from google here, but I am wondering as well 
>> what people would suggest I do to use the least amount fo resources so I 
>> can compare.
>>
>> Thanks!
>> Cheers
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

[google-appengine] Re: Best strategy

[Vinny P]

On Tuesday, March 12, 2013 12:18:24 PM UTC-5, Carl Schroeder wrote:
>
> The sentiment is not anti-Java, it is anti GAE/Java. Writing "Hello world" 
> to the datastore takes 4-5 seconds to cold start load on an F2 in GAE/Java. 
> That is on good days, it could take twice as long on stormy days. The more 
> the application grows, the worse this gets.
>
> Why risk 5+ second response times? Wait until Google has fixed the Java 
> instance loading time issue before you embark on writing Java apps for GAE.
> FYI, I used the low-level API and no frameworks. Latency was untenable.
>
>
I can't say that I share your experiences with GAE/J, I find that I can 
routinely cold start instances + datastore requests in less than 5 seconds, 
although I'm recently having issues with memcache slowness...

True, GAE/J has problems. But so do the Python and Go runtimes. I'm 
subscribed to the google-appengine-go Google group, and I see as many 
complaints there, as I do on this group about Python/Java.  Giving the OP a 
balanced view of the pros/cons of each runtime is the point of this thread, 
and I think it's fair to let him know what he's looking at.

@OP: if you're interested in looking more about Java performance, may I 
recommend this thread (it's a bit of a long read, to give you fair 
warning): 
https://groups.google.com/d/msg/google-appengine/sA3o-PTAckc/okDqu5aE-78J 

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

[google-appengine] Re: Best strategy

[Carl Schroeder]

The sentiment is not anti-Java, it is anti GAE/Java. Writing "Hello world" 
to the datastore takes 4-5 seconds to cold start load on an F2 in GAE/Java. 
That is on good days, it could take twice as long on stormy days. The more 
the application grows, the worse this gets.

Why risk 5+ second response times? Wait until Google has fixed the Java 
instance loading time issue before you embark on writing Java apps for GAE.
FYI, I used the low-level API and no frameworks. Latency was untenable.

On Tuesday, March 12, 2013 10:08:26 AM UTC-7, Vinny P wrote:
>
> There's a lot of anti-Java sentiment here, which is justifiable 
> considering the problems Java has and is having on AppEngine. With that 
> said, Java is only a problem if you're loading in a bunch of libraries 
> (classloading is slow on GAE). If all you're doing is manipulating the 
> datastore,you can do that with the low-level API and skip using the Apache 
> PersistenceManager and other datastore abstractions. 
>
> I would also recommend that you look into Cloud SQL ( 
> https://developers.google.com/cloud-sql/docs/introduction ). While the 
> datastore is a great place to store data, it can also be very spiky in 
> terms of latency. Cloud SQL IMHO seems to be less spiky, and since your 
> data has a clear schema, SQL can work for you. It might also be cheaper, 
> depending on your use patterns.
>
> -Vinny
>
> On Monday, March 11, 2013 12:44:54 PM UTC-5, victo...@gmail.com wrote:
>>
>>
>> Hello, I was wondering if someone could help me choose the best strategy 
>> for my application in terms of design.I want to use the least amount of 
>> resources as I can and optimize my code. My application is essentially 
>> written in html and javascript. I will use java to access the datastore 
>> (unless there is another more efficient way?). So essentially I will be 
>> storing two main things inside the datastore. A notification status (1 
>> single integer, I am expecting about 200,000 of these to be stored) and 
>> content (5000 ascii characters each expecting about 200,000 of these as 
>> well). They will be access quite frequently (most likely 50,000 
>> notifications and content will be read and updated every 30 mins). I have 
>> read much about optimizing from google here, but I am wondering as well 
>> what people would suggest I do to use the least amount fo resources so I 
>> can compare.
>>
>> Thanks!
>> Cheers
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

[google-appengine] Re: Best strategy

[Vinny P]

There's a lot of anti-Java sentiment here, which is justifiable considering 
the problems Java has and is having on AppEngine. With that said, Java is 
only a problem if you're loading in a bunch of libraries (classloading is 
slow on GAE). If all you're doing is manipulating the datastore,you can do 
that with the low-level API and skip using the Apache PersistenceManager 
and other datastore abstractions. 

I would also recommend that you look into Cloud SQL ( 
https://developers.google.com/cloud-sql/docs/introduction ). While the 
datastore is a great place to store data, it can also be very spiky in 
terms of latency. Cloud SQL IMHO seems to be less spiky, and since your 
data has a clear schema, SQL can work for you. It might also be cheaper, 
depending on your use patterns.

-Vinny

On Monday, March 11, 2013 12:44:54 PM UTC-5, victo...@gmail.com wrote:
>
>
> Hello, I was wondering if someone could help me choose the best strategy 
> for my application in terms of design.I want to use the least amount of 
> resources as I can and optimize my code. My application is essentially 
> written in html and javascript. I will use java to access the datastore 
> (unless there is another more efficient way?). So essentially I will be 
> storing two main things inside the datastore. A notification status (1 
> single integer, I am expecting about 200,000 of these to be stored) and 
> content (5000 ascii characters each expecting about 200,000 of these as 
> well). They will be access quite frequently (most likely 50,000 
> notifications and content will be read and updated every 30 mins). I have 
> read much about optimizing from google here, but I am wondering as well 
> what people would suggest I do to use the least amount fo resources so I 
> can compare.
>
> Thanks!
> Cheers

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

[google-appengine] Re: Best strategy

[victortrot]

Thanks, yes, I will only be using Java, Python, or Go for database access, 
nothing else/

On Monday, March 11, 2013 10:44:54 AM UTC-7, victo...@gmail.com wrote:
>
>
> Hello, I was wondering if someone could help me choose the best strategy 
> for my application in terms of design.I want to use the least amount of 
> resources as I can and optimize my code. My application is essentially 
> written in html and javascript. I will use java to access the datastore 
> (unless there is another more efficient way?). So essentially I will be 
> storing two main things inside the datastore. A notification status (1 
> single integer, I am expecting about 200,000 of these to be stored) and 
> content (5000 ascii characters each expecting about 200,000 of these as 
> well). They will be access quite frequently (most likely 50,000 
> notifications and content will be read and updated every 30 mins). I have 
> read much about optimizing from google here, but I am wondering as well 
> what people would suggest I do to use the least amount fo resources so I 
> can compare.
>
> Thanks!
> Cheers

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [google-appengine] Re: Best strategy

[Alpha Scorpii]

go is just very efficient as i have tried. also make use of memcache wherever 
possible to cache datastore query result 

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [google-appengine] Re: Best strategy

[Jeff Schnitzer]

I just can't resist this conversation, sigh.

What do you mean by "efficient"?  In GAE-land there is only "more
expensive" and "less expensive".  Depending on what you are doing,
Java _could_ be less expensive than Python; lots of algorithmic
manipulation (even just a lot of serialization) will burn
significantly more Python-hours than Java-hours.

But if you go Java, and you have a sophisticated app (ie lots of class
files), you will start to bump into long app startup times and
user-facing cold starts, which screws up the UX. Python is not immune
to this problem, just less severe. Go apps appear to be immune... but
then you have to use Go.

The bigger problem is that it sounds like you want to iterate through
50k entities in some sort of aggregation pattern. You may end up
jumping through some hoops to make that perform.

Jeff


On Mon, Mar 11, 2013 at 5:38 PM,   wrote:
> Thanks for the reply Jim. The things you mentioned below I am not doing, but
> in terms of reading strings from the data store very frequently and updating
> them, which is the sole purpose of my app, would you agree in this scenario
> python is more efficient than Java?
>
>
> On Monday, March 11, 2013 10:44:54 AM UTC-7, victo...@gmail.com wrote:
>>
>>
>> Hello, I was wondering if someone could help me choose the best strategy
>> for my application in terms of design.I want to use the least amount of
>> resources as I can and optimize my code. My application is essentially
>> written in html and javascript. I will use java to access the datastore
>> (unless there is another more efficient way?). So essentially I will be
>> storing two main things inside the datastore. A notification status (1
>> single integer, I am expecting about 200,000 of these to be stored) and
>> content (5000 ascii characters each expecting about 200,000 of these as
>> well). They will be access quite frequently (most likely 50,000
>> notifications and content will be read and updated every 30 mins). I have
>> read much about optimizing from google here, but I am wondering as well what
>> people would suggest I do to use the least amount fo resources so I can
>> compare.
>>
>> Thanks!
>> Cheers
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to google-appengine+unsubscr...@googlegroups.com.
> To post to this group, send email to google-appengine@googlegroups.com.
> Visit this group at http://groups.google.com/group/google-appengine?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

[google-appengine] Re: Best strategy

[Carl Schroeder]

It doesn't matter how efficient Java is or is not at some arbitrary 
problem. Do not use Java on GAE. It is not ready for prime time. The 
instance startup scheduler will make your life hell. Use Python or Go.

I just got done porting all of my Java code to Go and Python. I would have 
others avoid this mistake I made.

However, I am also a connoisseur of schadenfreude. So if you ignore my 
advice do go with GAE/Java, be sure and post here about your performance 
problems so I can feed upon your soul. ;)

On Monday, March 11, 2013 2:38:02 PM UTC-7, victo...@gmail.com wrote:
>
> Thanks for the reply Jim. The things you mentioned below I am not doing, 
> but in terms of reading strings from the data store very frequently and 
> updating them, which is the sole purpose of my app, would you agree in this 
> scenario python is more efficient than Java?
>
> On Monday, March 11, 2013 10:44:54 AM UTC-7, victo...@gmail.com wrote:
>>
>>
>> Hello, I was wondering if someone could help me choose the best strategy 
>> for my application in terms of design.I want to use the least amount of 
>> resources as I can and optimize my code. My application is essentially 
>> written in html and javascript. I will use java to access the datastore 
>> (unless there is another more efficient way?). So essentially I will be 
>> storing two main things inside the datastore. A notification status (1 
>> single integer, I am expecting about 200,000 of these to be stored) and 
>> content (5000 ascii characters each expecting about 200,000 of these as 
>> well). They will be access quite frequently (most likely 50,000 
>> notifications and content will be read and updated every 30 mins). I have 
>> read much about optimizing from google here, but I am wondering as well 
>> what people would suggest I do to use the least amount fo resources so I 
>> can compare.
>>
>> Thanks!
>> Cheers
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

[google-appengine] Re: Best strategy

[victortrot]

Thanks for the reply Jim. The things you mentioned below I am not doing, 
but in terms of reading strings from the data store very frequently and 
updating them, which is the sole purpose of my app, would you agree in this 
scenario python is more efficient than Java?

On Monday, March 11, 2013 10:44:54 AM UTC-7, victo...@gmail.com wrote:
>
>
> Hello, I was wondering if someone could help me choose the best strategy 
> for my application in terms of design.I want to use the least amount of 
> resources as I can and optimize my code. My application is essentially 
> written in html and javascript. I will use java to access the datastore 
> (unless there is another more efficient way?). So essentially I will be 
> storing two main things inside the datastore. A notification status (1 
> single integer, I am expecting about 200,000 of these to be stored) and 
> content (5000 ascii characters each expecting about 200,000 of these as 
> well). They will be access quite frequently (most likely 50,000 
> notifications and content will be read and updated every 30 mins). I have 
> read much about optimizing from google here, but I am wondering as well 
> what people would suggest I do to use the least amount fo resources so I 
> can compare.
>
> Thanks!
> Cheers

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [google-appengine] Re: Best strategy

[Joshua Smith]

GAE Java is nothing like the Java being profiled there. It is not the same VM, 
and it runs in a completely different manner.

Also, the described application is I/O bound, so the thing being measured 
there, even if it was something like GAE's VM, would not be relevant to the 
OP's question.

On Mar 11, 2013, at 2:33 PM, victort...@gmail.com wrote:

> I was just looking at benchmarks fro efficient of both programs and Java 
> seems to be much more efficient than Python. For instance I looked up some 
> benchmarks on different OS and machines, and Java came out on top by a lot in 
> every benchmark. For instance: 
> http://benchmarksgame.alioth.debian.org/u64q/benchmark.php?test=all&lang=java&lang2=python3
> 
> On Monday, March 11, 2013 10:44:54 AM UTC-7, victo...@gmail.com wrote:
> 
> Hello, I was wondering if someone could help me choose the best strategy for 
> my application in terms of design.I want to use the least amount of resources 
> as I can and optimize my code. My application is essentially written in html 
> and javascript. I will use java to access the datastore (unless there is 
> another more efficient way?). So essentially I will be storing two main 
> things inside the datastore. A notification status (1 single integer, I am 
> expecting about 200,000 of these to be stored) and content (5000 ascii 
> characters each expecting about 200,000 of these as well). They will be 
> access quite frequently (most likely 50,000 notifications and content will be 
> read and updated every 30 mins). I have read much about optimizing from 
> google here, but I am wondering as well what people would suggest I do to use 
> the least amount fo resources so I can compare.
> 
> Thanks!
> Cheers
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Google App Engine" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to google-appengine+unsubscr...@googlegroups.com.
> To post to this group, send email to google-appengine@googlegroups.com.
> Visit this group at http://groups.google.com/group/google-appengine?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>  
>  

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

[google-appengine] Re: Best strategy

[victortrot]

I was just looking at benchmarks fro efficient of both programs and Java 
seems to be much more efficient than Python. For instance I looked up some 
benchmarks on different OS and machines, and Java came out on top by a lot 
in every benchmark. For instance: 
http://benchmarksgame.alioth.debian.org/u64q/benchmark.php?test=all&lang=java&lang2=python3

On Monday, March 11, 2013 10:44:54 AM UTC-7, victo...@gmail.com wrote:
>
>
> Hello, I was wondering if someone could help me choose the best strategy 
> for my application in terms of design.I want to use the least amount of 
> resources as I can and optimize my code. My application is essentially 
> written in html and javascript. I will use java to access the datastore 
> (unless there is another more efficient way?). So essentially I will be 
> storing two main things inside the datastore. A notification status (1 
> single integer, I am expecting about 200,000 of these to be stored) and 
> content (5000 ascii characters each expecting about 200,000 of these as 
> well). They will be access quite frequently (most likely 50,000 
> notifications and content will be read and updated every 30 mins). I have 
> read much about optimizing from google here, but I am wondering as well 
> what people would suggest I do to use the least amount fo resources so I 
> can compare.
>
> Thanks!
> Cheers

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

[google-appengine] Re: Best strategy

[victortrot]

Thanks for the reply Josh, but why not use Java?

On Monday, March 11, 2013 10:44:54 AM UTC-7, victo...@gmail.com wrote:
>
>
> Hello, I was wondering if someone could help me choose the best strategy 
> for my application in terms of design.I want to use the least amount of 
> resources as I can and optimize my code. My application is essentially 
> written in html and javascript. I will use java to access the datastore 
> (unless there is another more efficient way?). So essentially I will be 
> storing two main things inside the datastore. A notification status (1 
> single integer, I am expecting about 200,000 of these to be stored) and 
> content (5000 ascii characters each expecting about 200,000 of these as 
> well). They will be access quite frequently (most likely 50,000 
> notifications and content will be read and updated every 30 mins). I have 
> read much about optimizing from google here, but I am wondering as well 
> what people would suggest I do to use the least amount fo resources so I 
> can compare.
>
> Thanks!
> Cheers

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.