[google-appengine] How to do full table scans for analysis on App Engine?

[ingo]

hello everyone,

i want to provide detailed reports in my app. those reports require
data aggregated across full tables. in the client i want to plot the
change of values over time. this then requires to select very huge
data sets. however, in the app engine there is the restriction to get
1000 items in one request. so what is the best approach to do queries
with big result sets?

i can imagine that the client could split the big request into 10
smaller ones to collect, merge and display the data. but i think this
is complex, error prone and slow. i would rather like to do this on
the server side. but do i have to split the requests to work around
the 1000 items limit or is there another pattern that developers
should use to handle huge datasets.

basically i want to know two things:
1) how can i retrieve huge amounts of data regarding the get 1000
items limit?
2) is the app engine performance good enough to allow showing reports
on the client side i.e. can app engine retrieve and analyse
(avg,min,max,grouping) huge amounts of data (10k - 1million rows) fast
enough?

one more thing: is it possible to reduce the amount of data that is
returned dramatically by doing all the aggregations (e.g. by grouping
and summing) completely within the datastore? currently i am
experimenting with doing full table scans (but i can only retrieve
1000 items in one request..) and aggregating the data within the
application (on server side).

kind regards,
ingo

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: 500 error accessing http://appengine.google.com/

[ingo]

well, seems to be fixed now :)

could you provide any details about the results of your
investigations?

kind regards,
ingo

On 14 Sep., 14:08, ingo  wrote:
> hello app engine team,
>
> i have the same problem.
>
> in general, i love developing gwt apps for the app engine. but this is
> not the first time i cannot reach the administration console, cannot
> deploy my app or cannot use my app without any warning. there have
> been similar errors in the last weeks (e.g. inaccessible svn
> repository or the 502 bad gateway error).
>
> please reduce the frequency of 502 and 500 errors as soon as possible.
> to me this is very important. already my trust in the platform
> decreased a bit.
>
> kind regards,
> ingo
>
> On 14 Sep., 13:44, James  wrote:
>
> > Right now.  Also can't deploy ("Rolling back update" error).
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: 500 error accessing http://appengine.google.com/

[ingo]

hello app engine team,

i have the same problem.

in general, i love developing gwt apps for the app engine. but this is
not the first time i cannot reach the administration console, cannot
deploy my app or cannot use my app without any warning. there have
been similar errors in the last weeks (e.g. inaccessible svn
repository or the 502 bad gateway error).

please reduce the frequency of 502 and 500 errors as soon as possible.
to me this is very important. already my trust in the platform
decreased a bit.

kind regards,
ingo

On 14 Sep., 13:44, James  wrote:
> Right now.  Also can't deploy ("Rolling back update" error).

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: 500 error accessing http://appengine.google.com/

[ingo]

although it looks like this is solved, still some requests take a lot
longer than others e.g. every ~10th request or so takes more than 8
seconds in my application. all other requests take 50-600ms. i would
too appreciate to get some information about those peaks.

kind regards,
ingo

On 14 Sep., 14:50, mscwd01  wrote:
> I've experienced large increases in CPU time per request since the
> maintenance yesterday. I would like to here some official comment
> regarding this...
> GET and PUTs seem to take much longer than a day or so ago.
>
> On 14 Sep, 13:22, Kenneth  wrote:
>
> > I've been having errors all morning.  Sigh.
>
> > Something's up, and it ain't the datastore:
>
> >http://code.google.com/status/appengine/detail/datastore/2010/09/14#a...
>
> > On Sep 14, 12:55 pm, "Raymond C."  wrote:
>
> > > Cant even connect to my application.
>
> > > On Sep 14, 7:44 pm, James  wrote:
>
> > > > Right now.  Also can't deploy ("Rolling back update" error).
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Performance Tuning applications on GAE

[ingo]

Hello Barada,

well, there are some seriously interesting articles in the appengine
documentation e.g. "Best practices for writing scalable applications"
http://code.google.com/intl/de-DE/appengine/articles/scaling/overview.html

a while ago a made a screencast about how to profile gwt/gae
applications locally, see http://www.youtube.com/watch?v=zUJUSxXOOa4
in the honeycrm project that i am doing it turned out to be very
effective to profile my application locally and optimize it
accordingly, see http://code.google.com/p/honeycrm/.

hope this helps. happy optimizing :)

kind regards,
ingo

On 22 Sep., 15:53, Fredrik Bonander 
wrote:
> Using appstats is a good way to see what's going 
> on.http://code.google.com/appengine/docs/python/tools/appstats.html
>
> ..fredrik
>
> On Sep 22, 2010, at 4:14 AM, Barada Sahu wrote:
>
> > Hi everyone,
> > Is there a set of best practices or results that you have observed so
> > far from your efforts on tuning performance of application on GAE?
>
> > Regards
> > Barada Sahu
>
> > --
> > You received this message because you are subscribed to the Google Groups 
> > "Google App Engine" group.
> > To post to this group, send email to google-appeng...@googlegroups.com.
> > To unsubscribe from this group, send email to 
> > google-appengine+unsubscr...@googlegroups.com.
> > For more options, visit this group 
> > athttp://groups.google.com/group/google-appengine?hl=en.
>
> --
> Fredrik Bonander
> carl.fredrik.bonan...@gmail.com
> +46 70 943 5441
>
> - the infinite power of the creative mind -

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Running additional docker containers alongside my GAE app?

[ingo]

 

I built a small Golang app  
and deployed it to GAE's Flexible Environment. Right now this will create a 
single Golang Docker container remotely at deploy time. For simple projects 
similar to this one what is the best practice to add more dependencies that 
run alongside the app  (like statsd)? I want to continue to run my app in a 
container on GAE. But I want to run publicly hosted Docker images alongside 
it (e.g. this one ). I 
am looking for a way to run those multi-Docker instances still within GAE 
to take advantage of high level of abstraction. And if possible I want to 
avoid having to integrate with a Docker Registry. 


Is this feasible on GAE or do I have to switch over to Google Compute Engine 
 or Google 
Container Engine ?


Thanks!

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/318a4bd9-e6b0-4dfe-a774-6e8988d05bee%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Running additional docker containers alongside my GAE app?

[ingo]

Hey Nick,

thanks for your response! I will give your "runtime: custom" suggestion a 
try to see if I can stay fully within GAE.

Some comments:

   - I tried to avoid installing additional dependencies into the app's 
   Docker container. This might be acceptable for a single additional 
   dependency. But it mixes concerns and won't scale for larger numbers of 
   dependencies.
   - You are saying my app runs on GCE already. I can see how that is the 
   case for the instances themselves. But GAE comes with a few features GCE 
   does not have built-in AFAIK: built-in load balancer, versions, traffic 
   splitting, per-version domain names, direct access to the datastore and 
   memcache, Cloud Debugger/Tracer, log aggregation. What of those things 
   would I loose when migrating my example app from GAE to plain GCE? 
   - How does this problem relate to Google Container Engine? Does this 
   sound like a good use case for Container Engine to you?
   

Thanks,
Ingo

On Wednesday, May 11, 2016 at 10:56:44 AM UTC-7, Nick (Cloud Platform 
Support) wrote:
>
> Hey Ingo,
>
> The Flexible Environment (from this point forward, "Flex") is a kind of 
> bridge between App Engine and Compute Engine. Flex apps run within a Docker 
> container on a Compute Engine instance or pool of instances, and requests 
> to the app via appspot.com are load-balanced to go to any one of these 
> instances (depending on your scaling settings, the pool's size and growth 
> behaviour will be dependent). 
>
> Basically, our SDK (gcloud <https://cloud.google.com/sdk/gcloud/>) allows 
> you to specificy either "vm: true" (in the case of "runtime: 
> go/python27/java/php55/etc.") or "runtime: custom 
> <https://cloud.google.com/appengine/docs/flexible/custom-runtimes/configuring-your-app-with-app-yaml#general>"
>  
> in your app.yaml or appengine-web.xml (for java) config file. 
>
> In the case of "vm: true", your traditional App Engine app will be 
> deployed using a special compatibility runtime to a Flex environment, and 
> in the case of "runtime: custom", the Dockerfile in your app's root 
> directory will be used to build the image of the resulting container which 
> is deployed to the Flex Environment (you would need to specify the base 
> image using the Dockerfile FROM directive, and ensure a web server process 
> is started and listening on the appropriate port). 
>
> So, your Flex app is already running on Compute Engine. Therefore, you 
> could use "runtime: custom", along with the appropriate go compatibility base 
> image 
> <https://cloud.google.com/appengine/docs/flexible/custom-runtimes/build#base_images>,
>  
> and simply specify the install / configuration instructions for statsd in 
> your Dockerfile. Each instance of your application would have an installed 
> and running copy of statsd, in that case. Although you may also want to 
> simply deploy a Compute Engine instance running statsd which is reachable 
> via its internal network IP from your app runtime instances, since that 
> appears to be more in line with how statsd is meant to be deployed, as a 
> standalone service within your network.
>
> Does this help answer your questions? Let me know if anything is unclear 
> or remaining to be discussed. 
>
> Cheers!
>
> Nick
> Cloud Platform Community Support  
>
> On Tuesday, May 10, 2016 at 10:55:27 PM UTC-4, ingo wrote:
>>
>> I built a small Golang app <https://github.com/ingojaeckel/weather-wear> 
>> and deployed it to GAE's Flexible Environment. Right now this will create a 
>> single Golang Docker container remotely at deploy time. For simple projects 
>> similar to this one what is the best practice to add more dependencies that 
>> run alongside the app  (like statsd)? I want to continue to run my app in a 
>> container on GAE. But I want to run publicly hosted Docker images alongside 
>> it (e.g. this one <https://hub.docker.com/r/datadog/docker-dd-agent/>). 
>> I am looking for a way to run those multi-Docker instances still within GAE 
>> to take advantage of high level of abstraction. And if possible I want to 
>> avoid having to integrate with a Docker Registry. 
>>
>>
>> Is this feasible on GAE or do I have to switch over to Google Compute 
>> Engine <https://cloud.google.com/compute/docs/containers/container_vms> 
>> or Google Container Engine <https://cloud.google.com/container-engine/>?
>>
>>
>> Thanks!
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/cef48fff-3f1c-4c43-a149-13dbb8f760a8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Running additional docker containers alongside my GAE app?

[ingo]

Hey Nick/Stephen,

Thank you very much for your responses!

I looked at Stephen's suggestion and created a new service/module within my 
GAE app that runs the docker-dd-agent Docker image. I updated my project on 
Github 
<https://github.com/ingojaeckel/weather-wear/tree/master/modules/docker-dd-agent>
 
accordingly. Now my app consists of two modules:
* A: The default service which acts as the HTTP(S) front-end. This is a 
Golang app running within the flexible environment.
* B: A separate docker-dd-agent service based on a custom Dockerfile 
<https://github.com/ingojaeckel/weather-wear/blob/master/modules/docker-dd-agent/Dockerfile>.
 
This service is not accessible via HTTP.

Now I want that my *default service (A) can communicate with the custom 
runtime (B) over TCP* (port 8125 in this case). But after reading 
https://cloud.google.com/appengine/docs/flexible/custom-runtimes/build#listen_to_port_8080
 
I am *not sure if custom runtimes are supported if they do not provide HTTP 
access*. I disabled the (HTTP based) health check in B via app.yaml 
<https://github.com/ingojaeckel/weather-wear/blob/master/modules/docker-dd-agent/app.yaml>.
 
But is this enough to allow communication from A to B?

Thanks,
Ingo

On Thursday, May 12, 2016 at 11:40:53 AM UTC-7, Stephen wrote:
>
> On Wed, May 11, 2016 at 3:55 AM, ingo > 
> wrote: 
> > 
> > But I want to run 
> > publicly hosted Docker images alongside it (e.g. this one). I am looking 
> for 
> > a way to run those multi-Docker instances still within GAE to take 
> advantage 
> > of high level of abstraction. And if possible I want to avoid having to 
> > integrate with a Docker Registry. 
>
>
> Deploy your custom docker image as a new service (module). You can 
> have up to 20 services: 
>
>   
> https://cloud.google.com/appengine/docs/flexible/go/an-overview-of-app-engine 
>
> Leave your existing code deployed as it is, running as the 'default' 
> service. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/6a6c71b5-afbb-4aef-8220-29d6660db162%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Running additional docker containers alongside my GAE app?

[ingo]

Hey Nick,

Thanks for those details! I will try this right away.

Going forward though, how do you suggest to communicate the external IP 
address from the custom runtime (B) to the Golang runtime A? B is a 
dependency which is based off a public docker container. Are you suggesting 
to add the curl command to the initialization of B and write the output of 
it to a shared location e.g. Memcache/Datastore?

If at all possible I would like to determine the IP address of B from A. It 
is a lot easier for me to change A or customize it's behaviour. Whereas I 
want B to stay very simple - a parameterized public docker image.

Thanks,
Ingo

On Monday, May 16, 2016 at 1:50:51 PM UTC-7, Nick (Cloud Platform Support) 
wrote:
>
> Hey Ingo,
>
> Custom runtimes can listen on any port specified in the app.yaml 
> <https://cloud.google.com/appengine/docs/flexible/custom-runtimes/configuring-your-app-with-app-yaml#network_settings>
>  just 
> like a regular Compute Engine instance, although these connections can't 
> bee made through the appid.appspot.com domain or custom domains routing. 
> You'll need to connect to the IP of the instance itself to be able to 
> connect on a non-8080 port via something other than HTTP. The instance can 
> hit the Compute Engine Metadata API 
> <https://cloud.google.com/compute/docs/metadata> to determine its 
> external IP address like this:
>
> curl -H "Metadata-Flavor: Google" 
> http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/externa
> l-ip
> As for health checks, you don't want to disable those, as that's vital to 
> ensuring your app serves properly 
> <https://cloud.google.com/appengine/docs/flexible/go/how-instances-are-managed#health_checking>
> . 
>
> I hope this answer was helpful!
>
> Sincerely,
>
> Nick
> Cloud Platform Community Support
>
> On Monday, May 16, 2016 at 12:39:48 AM UTC-4, ingo wrote:
>>
>> Hey Nick/Stephen,
>>
>> Thank you very much for your responses!
>>
>> I looked at Stephen's suggestion and created a new service/module within 
>> my GAE app that runs the docker-dd-agent Docker image. I updated my project 
>> on 
>> Github 
>> <https://github.com/ingojaeckel/weather-wear/tree/master/modules/docker-dd-agent>
>>  
>> accordingly. Now my app consists of two modules:
>> * A: The default service which acts as the HTTP(S) front-end. This is a 
>> Golang app running within the flexible environment.
>> * B: A separate docker-dd-agent service based on a custom Dockerfile 
>> <https://github.com/ingojaeckel/weather-wear/blob/master/modules/docker-dd-agent/Dockerfile>.
>>  
>> This service is not accessible via HTTP.
>>
>> Now I want that my *default service (A) can communicate with the custom 
>> runtime (B) over TCP* (port 8125 in this case). But after reading 
>> https://cloud.google.com/appengine/docs/flexible/custom-runtimes/build#listen_to_port_8080
>>  
>> I am *not sure if custom runtimes are supported if they do not provide 
>> HTTP access*. I disabled the (HTTP based) health check in B via app.yaml 
>> <https://github.com/ingojaeckel/weather-wear/blob/master/modules/docker-dd-agent/app.yaml>.
>>  
>> But is this enough to allow communication from A to B?
>>
>> Thanks,
>> Ingo
>>
>> On Thursday, May 12, 2016 at 11:40:53 AM UTC-7, Stephen wrote:
>>>
>>> On Wed, May 11, 2016 at 3:55 AM, ingo  wrote: 
>>> > 
>>> > But I want to run 
>>> > publicly hosted Docker images alongside it (e.g. this one). I am 
>>> looking for 
>>> > a way to run those multi-Docker instances still within GAE to take 
>>> advantage 
>>> > of high level of abstraction. And if possible I want to avoid having 
>>> to 
>>> > integrate with a Docker Registry. 
>>>
>>>
>>> Deploy your custom docker image as a new service (module). You can 
>>> have up to 20 services: 
>>>
>>>   
>>> https://cloud.google.com/appengine/docs/flexible/go/an-overview-of-app-engine
>>>  
>>>
>>> Leave your existing code deployed as it is, running as the 'default' 
>>> service. 
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/16944365-d76f-4741-b4be-012f719b9f16%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Unable to run ManagedVM Python tutorial on GCloud 0.9.68/Docker 1.7.1/app-engine-python 1.9.23

[ingo]

Hi,

I have been following the tutorial on Managed VMs in the docs 
<https://cloud.google.com/appengine/docs/managed-vms/tutorial/step2>. I 
installed boot2docker 1.7.1 on MacOSX 10.10.1. I confirmed that the 
installation was successful by running *boot2docker run hello-world* as 
described on https://docs.docker.com/installation/mac/. 

However, running *gcloud preview app run ./app.yaml* and *gcloud preview 
app deploy ./app.yaml* fails with an exception:

$ gcloud preview app run ./app.yaml

Module [default] found in file [/Users/ingo/git/managedvmtest/app.yaml]

INFO: Looking for the Dockerfile in /Users/ingo/git/managedvmtest

INFO: Looking for the default Dockerfile for runtime [python27]

INFO: Dockerfile for runtime [python27] is found in /Users/ingo/google-cloud
-sdk/lib/googlecloudsdk/appengine/dockerfiles. Copying it into application 
directory.

INFO 2015-07-15 17:01:45,643 application_configuration.py:399] No 
version specified. Generated version id: 20150715t170145

INFO 2015-07-15 17:01:45,643 devappserver2.py:762] Skipping SDK update 
check.

INFO 2015-07-15 17:01:45,800 api_server.py:204] Starting API server at: 
http://localhost:49447

/Users/ingo/google-cloud-sdk/platform/google_appengine/lib/requests/requests/packages/urllib3/util/ssl_.py:79:
 
InsecurePlatformWarning: A true SSLContext object is not available. This 
prevents urllib3 from configuring SSL appropriately and may cause certain 
SSL connections to fail. For more information, see 
https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.

 InsecurePlatformWarning

INFO 2015-07-15 17:01:45,923 api_server.py:629] Applying all pending 
transactions and saving the datastore

INFO 2015-07-15 17:01:45,923 api_server.py:632] Saving search indexes

Traceback (most recent call last):

 File 
"/Users/ingo/google-cloud-sdk/platform/google_appengine/dev_appserver.py", 
line 83, in 

   _run_file(__file__, globals())

 File 
"/Users/ingo/google-cloud-sdk/platform/google_appengine/dev_appserver.py", 
line 79, in _run_file

   execfile(_PATHS.script_file(script_name), globals_)

 File 
"/Users/ingo/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/devappserver2.py",
 
line 1027, in 

   main()

 File 
"/Users/ingo/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/devappserver2.py",
 
line 1020, in main

   dev_server.start(options)

 File 
"/Users/ingo/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/devappserver2.py",
 
line 817, in start

   self._dispatcher.start(options.api_host, apis.port, request_data)

 File 
"/Users/ingo/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/dispatcher.py",
 
line 193, in start

   _module, port = self._create_module(module_configuration, port)

 File 
"/Users/ingo/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/dispatcher.py",
 
line 278, in _create_module

   threadsafe_override=threadsafe_override)

 File 
"/Users/ingo/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/module.py",
 
line 1539, in __init__

   super(ManualScalingModule, self).__init__(**kwargs)

 File 
"/Users/ingo/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/module.py",
 
line 558, in __init__

   self._module_configuration)

 File 
"/Users/ingo/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/module.py",
 
line 259, in _create_instance_factory

   module_configuration=module_configuration)

 File 
"/Users/ingo/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/vm_runtime_factory.py",
 
line 76, in __init__

   timeout=self.DOCKER_D_REQUEST_TIMEOUT_SECS)

 File 
"/Users/ingo/google-cloud-sdk/platform/google_appengine/google/appengine/tools/docker/containers.py",
 
line 742, in NewDockerClient

   client.ping()

 File "/Users/ingo/google-cloud-sdk/./lib/docker/docker/client.py", line 
847, in ping

   return self._result(self._get(self._url('/_ping')))

 File "/Users/ingo/google-cloud-sdk/./lib/docker/docker/client.py", line 
126, in _result

   self._raise_for_status(response)

 File "/Users/ingo/google-cloud-sdk/./lib/docker/docker/client.py", line 
122, in _raise_for_status

   raise errors.APIError(e, response, explanation=explanation)

docker.docker.errors.APIError: 400 Client Error: Bad Request ("client is 
too old, minimum supported API version is 1.12, please upgrade your client 
to a newer version")

*ERROR:* (gcloud.preview.app.run) DevAppSever failed with error code [1]

I have set the DOCKER environment variables in my .bash_profile

export DOCKER_HOST=tcp://192.168.59.103:2376

export DOCKER_CERT_PATH=/Users/ingo/.boot2docker/certs

[google-appengine] Re: Unable to run ManagedVM Python tutorial on GCloud 0.9.68/Docker 1.7.1/app-engine-python 1.9.23

[ingo]

Google just updated the documentation on this 
<https://cloud.google.com/appengine/docs/managed-vms/getting-started#check_your_docker_versions>.
 
You can use the Cloud SDK with 1.7.0 of boot2docker. However, you have to 
use the following command to download the 1.7.0 ISO (instead of just 
grabbing the latest version).

boot2docker download 
--iso-url=https://github.com/boot2docker/boot2docker/releases/download/v1.7.0/boot2docker.iso


Thanks,
Ingo

On Wednesday, July 15, 2015 at 1:15:23 PM UTC-4, pdknsk wrote:
>
> Downgrade to the previous Docker version. I already filed a bug about it 
> some minute ago.
>
> https://code.google.com/p/google-cloud-sdk/issues/detail?id=163
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/704e37cf-d5d0-4a57-9d87-cff23e08a65a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] How to do full table scans for analysis on App Engine?

[Ingo Jaeckel]

hello everyone,

i want to provide detailed reports in my app. those reports require
data aggregated across full tables. in the client i want to plot the
change of values over time. this then requires to select very huge
data sets. however, in the app engine there is the restriction to get
1000 items in one request. so what is the best approach to do queries
with big result sets?

i can imagine that the client could split the big request into 10
smaller ones to collect, merge and display the data. but i think this
is complex, error prone and slow. i would rather like to do this on
the server side. but do i have to split the requests to work around
the 1000 items limit or is there another pattern that developers
should use to handle huge datasets.

basically i want to know two things:
1) how can i retrieve huge amounts of data regarding the get 1000 items limit?
2) is the app engine performance good enough to allow showing reports
on the client side i.e. can app engine retrieve and analyse
(avg,min,max,grouping) huge amounts of data (10k - 1million rows) fast
enough?

kind regards,
ingo

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: How to do full table scans for analysis on App Engine?

[Ingo Jaeckel]

hello again,

one more thing: is it possible to reduce the amount of data that is
returned dramatically by doing all the aggregations (e.g. by grouping
and summing) completely within the datastore? currently i am
experimenting with doing full table scans (but i can only retrieve
1000 items in one request..) and aggregating the data within the
application (on server side).

kind regards,
ingo

2010/7/16 Ingo Jaeckel :
> hello everyone,
>
> i want to provide detailed reports in my app. those reports require
> data aggregated across full tables. in the client i want to plot the
> change of values over time. this then requires to select very huge
> data sets. however, in the app engine there is the restriction to get
> 1000 items in one request. so what is the best approach to do queries
> with big result sets?
>
> i can imagine that the client could split the big request into 10
> smaller ones to collect, merge and display the data. but i think this
> is complex, error prone and slow. i would rather like to do this on
> the server side. but do i have to split the requests to work around
> the 1000 items limit or is there another pattern that developers
> should use to handle huge datasets.
>
> basically i want to know two things:
> 1) how can i retrieve huge amounts of data regarding the get 1000 items limit?
> 2) is the app engine performance good enough to allow showing reports
> on the client side i.e. can app engine retrieve and analyse
> (avg,min,max,grouping) huge amounts of data (10k - 1million rows) fast
> enough?
>
> kind regards,
> ingo
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Boost Extensibility and Simplify Maintainability of Web Applications: Proposals for Google Web Toolkit Plug-in Architectures

[Ingo Jaeckel]

Hello everyone

I wrote a short article proposes a plug-in management architecture for
GWT/AppEngine applications, see
http://honeycrm.googlecode.com/files/BoostExtensibilityAndSimplifyMaintainabilityOfWebApplicationsProposalsForGoogleWebToolkitPlugInArchitectures.pdf.
Now, I would like to invite the community to discuss my proposal.
Thanks in advance to everyone for reading and discussing the proposal.

Abstract: The extensibility of Google Web Toolkit (GWT) applications
running on the Google App Engine platform is currently limited due to
the absence of a plug-in infrastructure. Even though for desktop,
server, and web applications, plug-ins have been existing for decades,
they are not prevalent in GWT today. In this article, I outline the
development environment restrictions that make it a challenge to
provide dynamic plug-in loading support. I assume that the
introduction of a plug-in architecture would dramatically enhance the
extensibility and the maintainability of GWT applications. Therefore,
this article proposes approaches for statically and dynamically loaded
plug-ins for GWT applications.

Kind regards

Ingo

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Boost Extensibility and Simplify Maintainability of Web Applications: Proposals for Google Web Toolkit Plug-in Architectures

[Ingo Jaeckel]

Hello everyone

after discussing the topic with a college, I realized I have to
explain the context of my proposal.

There is no application independent mechanism in GWT to dynamically
load extensions. Consequently, GWT applications either do not support
it or implement plug-in loading on their own. I am convinced this has
to be changed, mainly because of two obervations: The dramatically
increasing popularity of both (1) AppStores, e.g. Apple App Store,
Android Market, and (2) browser/web applications. To support AppStore
concepts in web applications, the GWT/AppEngine community would profit
from having an application independent framework for the management of
application extensions, i.e. plug-ins, apps, or add-ons.

Kind regards

Ingo

2011/1/17 Ingo Jaeckel :
> Hello everyone
>
> I wrote a short article proposes a plug-in management architecture for
> GWT/AppEngine applications, see
> http://honeycrm.googlecode.com/files/BoostExtensibilityAndSimplifyMaintainabilityOfWebApplicationsProposalsForGoogleWebToolkitPlugInArchitectures.pdf.
> Now, I would like to invite the community to discuss my proposal.
> Thanks in advance to everyone for reading and discussing the proposal.
>
> Abstract: The extensibility of Google Web Toolkit (GWT) applications
> running on the Google App Engine platform is currently limited due to
> the absence of a plug-in infrastructure. Even though for desktop,
> server, and web applications, plug-ins have been existing for decades,
> they are not prevalent in GWT today. In this article, I outline the
> development environment restrictions that make it a challenge to
> provide dynamic plug-in loading support. I assume that the
> introduction of a plug-in architecture would dramatically enhance the
> extensibility and the maintainability of GWT applications. Therefore,
> this article proposes approaches for statically and dynamically loaded
> plug-ins for GWT applications.
>
> Kind regards
>
> Ingo
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Google authentification

[Ingo Beutler]

Hi,

is there an easy way to learn how the authentification an authorization 
works in the google products? Especially how the linking and 
synchronization is made.

Cheers,
Ingo

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/c76ddd3a-7f4c-4133-a77c-36a42339ef4d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.