[google-appengine] How is patching done for flexible instances?

[Attila-Mihaly Balazs]

Hello all,

Flexible instances sound great in theory - use runtime you wish as long as 
you can create a docker image for it and listen for HTTP on 8080.

However, one of the reasons I prefer AppEngine (standard) is the fact that 
I don't want to be Ops - don't want to be concerned about updating the 
kernel at 3AM in the morning when the latest exploit comes out. I love that 
G just handles that for me with AppEngine standard.

How does this work with flexible? There is a throw-away line in the 
documentation about "flex instances will be restarted at least once per 
month to apply critical security updates", however I fail to see how this 
could work when - from what I can see - the base image can be anything (ie. 
RedHat, Ubuntu, Arch or even Linux From Scratch, OpenWRT, etc :-)).

To what extent does Google patch the flexible instances? Do I still need to 
recreate the docker image weekly for example to ensure that all the latest 
packages are installed? Or rather - do I need to deploy weekly in this 
case? (since deploying rebuilds the docker image - which is nice but still 
annoying that I need to deploy at least once a week to have my instances be 
up to date).

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/474a1dc9-2b30-43a1-9f91-ca1b9ed671ae%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Connect multiple cloud sql databases to one app engine service

['Jun (Cloud Platform Support)' via Google App Engine]

Hi Mike,

You should be able to connect to multiple Cloud SQL instances through Cloud 
SQL Proxy, and you can specify multiple instances in the instances 
parameter (comma separated). Please take a look at these two posts for more 
details: "A GCE instance connecting to two (or more) Cloud SQL databases 
via multiple cloud-sql-proxy-instances 
"
 
and "Connecting to multiple CloudSQL instances using Cloud sql proxy? 
",
 
which have the similar questions. 

- Jun


On Friday, November 17, 2017 at 12:06:49 AM UTC-5, Mike Hardy wrote:
>
> Hi, we have an app that utilizes two databases. Previously, we had the two 
> databases on the same instance in Cloud SQL, but we want to split the 
> databases into separate instances. We're having issues connecting the 
> databases to our Django app engine service. Is this even possible? If so, 
> how would the app.yaml look?
>
> old app.yaml
> runtime: custom
> env: flex
> entrypoint: gunicorn --timeout 360 --graceful-timeout 360 -b :$8080 
> localinsights.wsgi
>
> beta_settings:
> cloud_sql_instances: 
>
> runtime_config:
>   python_version: 3.6
>
> env_variables:
>   SQLALCHEMY_DATABASE_URI: >-
>   
> postgresql+psycopg2://:/?host=/cloudsql/
>
>
> settings.py
> DATABASES['default'] = {
> 'HOST': '/cloudsql/',
> 'ENGINE': 'psqlextra.backend',
> 'NAME': '',
> 'USER': '',
> 'PASSWORD': ''
> }
> DATABASES['NUMBER_2'] = {
> 'HOST': '/cloudsql/',
> 'ENGINE': 'psqlextra.backend',
> 'NAME': 'NUMBER_2',
> 'USER': '',
> 'PASSWORD': ''
> }
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/7ad41428-ccf4-4ce3-8f35-453ecdab5e0c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Re: PHP Flexible Environment: Write access to directory possible?

['Jörg Hartgen' via Google App Engine]

Dear Takashi,

yes, this could be a solution.

We investigated the code and found that the templates_c directory is 
hardcoded in smarty in just two places:

components/renderers/template_renderer.php
and
components/utils/check_utils.php

We could correct both these places and add the files above to the list of 
excluded files to prevent them from being overwritten when updating the 
code.

In template_renderer.php:

###
smarty = new Smarty();
$this->smarty->template_dir = $templateDirectory;
$this->smarty->compile_dir = $compileDirectory;
}

/**
* @param string $templateName
* @param array $params
* @return string
*/
public function render($templateName, $params) {
foreach($params as $key => &$value) {
if (is_object($value)) {
$this->smarty->assign_by_ref($key, $value);
} else {
$this->smarty->assign($key, $value);
}
}

return $this->smarty->fetch($templateName);
}

}

function GetTemplateRenderer() {
return new SmartyTemplateRenderer();
}


###

In check_utils.php there is only a test if the templates_c directory is 
writeable:

###
function CheckTemplatesCacheFolderIsExistsAndWritable() {
$templatesCacheFolder = 'templates_c';
if (!file_exists($templatesCacheFolder) || 
!is_writable($templatesCacheFolder)) {

header('Content-Type: text/html; charset=UTF-8');

$result = file_get_contents(
'components/templates/templates_c_folder_warning.html');
$result = str_replace('{MESSAGE}', 'Error: the templates_c directory does 
not exist or is not writable', $result);
$result = str_replace('{DETAILS}', 'Please make sure that the templates_c 
directory does exist in the root directory of the generated application and 
it is writable by the web server user.', $result);
echo $result;
exit;
}
}

How can we proceed?

Cheers
Jorg




Am Donnerstag, 16. November 2017 20:23:34 UTC+1 schrieb Takashi Matsuo 
(Google):
>
> Hi Jörg,
>
> The Flex build pipeline set a stcict permission on everything under 
> document_root.
> Is it possible to configure smarty to use a directory outside of the 
> document_root (e.g. /tmp, or /app/smarty_cache)?
>
> Technically it's possible to introduce a flag for not setting the strict 
> permission at all, so let us know if you want this option.
>
> On Thu, Nov 2, 2017 at 1:42 PM 'George (Cloud Platform Support)' via 
> Google App Engine > wrote:
>
>> There is a PHP example 
>>  
>> for the flexible environment in the "Using Cloud Storage" document for PHP 
>> and flex. You may consider checking it for applicable sample code, to adapt 
>> to your particular requirements. 
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Google App Engine" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to google-appengi...@googlegroups.com .
>> To post to this group, send email to google-a...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/google-appengine.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/google-appengine/e8db7b1d-54c1-422f-8772-0ac0728a3776%40googlegroups.com
>>  
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
> -- 
> -- Takashi
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/3f59a530-cf38-4e18-a9f9-2a20096b322a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Connect multiple cloud sql databases to one app engine service

[Mike Hardy]

Hi Jun, thanks for your response, but I am a little confused here. When 
deployed, GAE uses a built in cloud sql proxy connection with Cloud SQL so 
there is no way to specify the instance or any other parameter for cloud 
sql proxy. I think any fix will need to use app.yaml but please correct me 
if I'm wrong. Would love to solve this.

On Friday, November 17, 2017 at 12:59:04 PM UTC-8, Jun (Cloud Platform 
Support) wrote:
>
> Hi Mike,
>
> You should be able to connect to multiple Cloud SQL instances through 
> Cloud SQL Proxy, and you can specify multiple instances in the instances 
> parameter (comma separated). Please take a look at these two posts for more 
> details: "A GCE instance connecting to two (or more) Cloud SQL databases 
> via multiple cloud-sql-proxy-instances 
> "
>  
> and "Connecting to multiple CloudSQL instances using Cloud sql proxy? 
> ",
>  
> which have the similar questions. 
>
> - Jun
>
>
> On Friday, November 17, 2017 at 12:06:49 AM UTC-5, Mike Hardy wrote:
>>
>> Hi, we have an app that utilizes two databases. Previously, we had the 
>> two databases on the same instance in Cloud SQL, but we want to split the 
>> databases into separate instances. We're having issues connecting the 
>> databases to our Django app engine service. Is this even possible? If so, 
>> how would the app.yaml look?
>>
>> old app.yaml
>> runtime: custom
>> env: flex
>> entrypoint: gunicorn --timeout 360 --graceful-timeout 360 -b :$8080 
>> localinsights.wsgi
>>
>> beta_settings:
>> cloud_sql_instances: 
>>
>> runtime_config:
>>   python_version: 3.6
>>
>> env_variables:
>>   SQLALCHEMY_DATABASE_URI: >-
>>   
>> postgresql+psycopg2://:/?host=/cloudsql/
>>
>>
>> settings.py
>> DATABASES['default'] = {
>> 'HOST': '/cloudsql/',
>> 'ENGINE': 'psqlextra.backend',
>> 'NAME': '',
>> 'USER': '',
>> 'PASSWORD': ''
>> }
>> DATABASES['NUMBER_2'] = {
>> 'HOST': '/cloudsql/',
>> 'ENGINE': 'psqlextra.backend',
>> 'NAME': 'NUMBER_2',
>> 'USER': '',
>> 'PASSWORD': ''
>> }
>>
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/ca645dae-b554-494b-a638-e2f6acdab6a0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] How do I turn off the faucet????!!!

[Brian Bonner]

I followed this 
tutorial:  https://cloud.google.com/appengine/docs/flexible/nodejs/quickstart

I really wanted to attempt to deploy a docker-compose configuration to GCP 
to see how it would work, but I scaled it back, since it seemed like it was 
a very manual process.

So, I went through the tutorial on 11/10/2017.   I revisited it yesterday 
for a UX call w/ Google and to my surprise it said I owed $13.06.   Today 
it was $16.


I was panicking .   How do I turn off this faucet???




I was looking everywhere.   I looked at the billing details -- which didn't 
match the dashboard.   It showed app-engine flex core and app-engine flex 
memory. 




So I went to the app-engine dashboard and it showed zilch under the 
instance overview.




So I went to instances, I saw two instances there, but it wouldn't let me 
delete them.  




I was going to disable the project, but the text on the settings page said 
that charges would continue to be incurred. 



How do I stop this thing???

So after the price went up again, I went back to the tutorial and found 
cleanup.   Basically it said I have to delete the project.


Is this really the only way to stop billing?   Delete the project?  
 There's no way to stop the container?   Or something else?   I didn't want 
to do it because of this warning in the tutorial:

*Warning:* Deleting a project has the following consequences:
   
   - If you used an existing project, you'll also delete any other work 
   you've done in the project.
   - You can't reuse the project ID of a deleted project. If you created a 
   custom project ID that you plan to use in the future, you should delete 
   the resources inside the project instead. This ensures that URLs that 
   use the project ID, such as an appspot.com URL, remain available.

As you can see, I tried to delete resources to no avail.   It was so 
frustrating.   

I see on the charges page 
(https://cloud.google.com/appengine/pricing#flexible-environment-instances) 
that most of the charges are related to the vCPU.  How do I turn them off 
when I'm not using them?

Can't google provide a better way of showing where the billing expenses are 
coming from?   Is there a command line way to spit out a detail invoices?

Jordan's post 
here:  
https://groups.google.com/forum/#!searchin/google-appengine/flex$20charges|sort:date/google-appengine/uaiyicw6rhg/ole26VFlBwAJ
  
 seems to indicate that specifying autoscaling vs. manual: 1 will save 
money, but this 
document:  
https://cloud.google.com/appengine/docs/flexible/nodejs/configuring-your-app-with-app-yaml
  
seems to indicate that the minimum # of instances is 1 no matter what:   So 
how would this save money?   It seems so contradictory.

min_num_instances Must be 1 or greater, default is 2 to reduce latency. 
When a service is deployed, it is given the minimum number of instances and 
scales according to traffic.


I thought the tutorial for app engine would end up being free based upon 
this:  https://cloud.google.com/free/  It says app-engine is free.

So why was I charged in the first place?

Thank you,

Brian

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/afb945ba-d7f1-4a36-af02-2e9f146dbfca%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: How do I turn off the faucet????!!!

['Yannick (Cloud Platform Support)' via Google App Engine]

Hello Brian,

First things first, you are running App Engine Flexible instances. To stop 
them from running you should go to the Versions 
 page, select the 
version that currently has instances running and click the STOP button. 
This will shutdown running instances for that version of your service. You 
also have the option of disabling your application 

 so 
that it cannot accrue any charges.

Regarding the free tier of App Engine , 
please note that it as well as the ability to scale down to zero instances 
are only available on the App Engine Standard environment while you are 
using the App Engine Flexible environment. You can view a comparison of 
these environments here 
.

I hope this helped. 

On Friday, November 17, 2017 at 5:33:37 PM UTC-5, Brian Bonner wrote:
>
> I followed this tutorial:  
> https://cloud.google.com/appengine/docs/flexible/nodejs/quickstart
>
> I really wanted to attempt to deploy a docker-compose configuration to GCP 
> to see how it would work, but I scaled it back, since it seemed like it was 
> a very manual process.
>
> So, I went through the tutorial on 11/10/2017.   I revisited it yesterday 
> for a UX call w/ Google and to my surprise it said I owed $13.06.   Today 
> it was $16.
>
>
> I was panicking .   How do I turn off this faucet???
>
>
> 
>
>
> I was looking everywhere.   I looked at the billing details -- which 
> didn't match the dashboard.   It showed app-engine flex core and app-engine 
> flex memory. 
>
>
> 
>
>
> So I went to the app-engine dashboard and it showed zilch under the 
> instance overview.
>
>
> 
>
>
> So I went to instances, I saw two instances there, but it wouldn't let me 
> delete them.  
>
>
> 
>
>
> I was going to disable the project, but the text on the settings page said 
> that charges would continue to be incurred. 
>
>
> 
>
> How do I stop this thing???
>
> So after the price went up again, I went back to the tutorial and found 
> cleanup.   Basically it said I have to delete the project.
>
>
> Is this really the only way to stop billing?   Delete the project?  
>  There's no way to stop the container?   Or something else?   I didn't want 
> to do it because of this warning in the tutorial:
>
> *Warning:* Deleting a project has the following consequences:
>
>- If you used an existing project, you'll also delete any other work 
>you've done in the project.
>- You can't reuse the project ID of a deleted project. If you created 
>a custom project ID that you plan to use in the future, you should 
>delete the resources inside the project instead. This ensures that 
>URLs that use the project ID, such as an appspot.com URL, remain 
>available.
>
> As you can see, I tried to delete resources to no avail.   It was so 
> frustrating.   
>
> I see on the charges page (
> https://cloud.google.com/appengine/pricing#flexible-environment-instances) 
> that most of the charges are related to the vCPU.  How do I turn them off 
> when I'm not using them?
>
> Can't google provide a better way of showing where the billing expenses 
> are coming from?   Is there a command line way to spit out a detail 
> invoices?
>
> Jordan's post here:  
> https://groups.google.com/forum/#!searchin/google-appengine/flex$20charges|sort:date/google-appengine/uaiyicw6rhg/ole26VFlBwAJ
>   
>  seems to indicate that specifying autoscaling vs. manual: 1 will save 
> money, but this document:  
> https://cloud.google.com/appengine/docs/flexible/nodejs/configuring-your-app-with-app-yaml
>   
> seems to indicate that the minimum # of instances is 1 no matter what:   So 
> how would this save money?   It seems so contradictory.
>
> min_num_instances Must be 1 or greater, default is 2 to reduce latency. 
> When a service is deployed, it is given the minimum number of instances and 
> scales according to traffic.
>
>
> I thought the tutorial for app engine would end up being free based upon 
> this:  https://cloud.google.com/free/  It says app-engine is free.
>
> So why was I charged in the first place?
>
> Thank you,
>
> Brian
>
>

-- 
You received this message be

[google-appengine] Re: How is patching done for flexible instances?

['Jun (Cloud Platform Support)' via Google App Engine]

Hi Attila,

Basically Google's management services will apply in-place security patches 
(excludes container image runtime) and any operating system and security 
updates during the restarts on a weekly basis per the doc at "Choosing an 
App Engine Environment 

". 


On Friday, November 17, 2017 at 12:18:02 PM UTC-5, Attila-Mihaly Balazs 
wrote:
>
> Hello all,
>
> Flexible instances sound great in theory - use runtime you wish as long as 
> you can create a docker image for it and listen for HTTP on 8080.
>
> However, one of the reasons I prefer AppEngine (standard) is the fact that 
> I don't want to be Ops - don't want to be concerned about updating the 
> kernel at 3AM in the morning when the latest exploit comes out. I love that 
> G just handles that for me with AppEngine standard.
>
> How does this work with flexible? There is a throw-away line in the 
> documentation about "flex instances will be restarted at least once per 
> month to apply critical security updates", however I fail to see how this 
> could work when - from what I can see - the base image can be anything (ie. 
> RedHat, Ubuntu, Arch or even Linux From Scratch, OpenWRT, etc :-)).
>
> To what extent does Google patch the flexible instances? Do I still need 
> to recreate the docker image weekly for example to ensure that all the 
> latest packages are installed? Or rather - do I need to deploy weekly in 
> this case? (since deploying rebuilds the docker image - which is nice but 
> still annoying that I need to deploy at least once a week to have my 
> instances be up to date).
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/59309454-908b-4329-960a-2eac988558a4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: How is patching done for flexible instances?

[Attila-Mihaly Balazs]

Hello,

Thank you for the reply. Just making sure that I understand correctly: when 
you say "excludes container image runtime", it means that none of the 
following scenarios are covered:

Lets say that I'm running a CentOS based docker image on AppEngine Flex.

1) A security bug is discovered in nginx. There is an update available in 
the repositories. However my instances are not patched until I 
rebuild/redeploy my images (and I have to be careful to rebuild them in 
such a away that Docker doesn't re-use a cached intermediary image which 
would result in the package update step being skipped)

2) Let's say that I'm running Java 9 inside my flex instance, using the 
OpenJDK build. A new version of the build is released fixing a security 
bug. I won't get it, until I manually update my Dockerfile presumably and 
redeploy

3) My webservice is written in Haskell, which gets compiled down to a 
native executable statically linking zlib. Zlib has a vulnerability and 
there is a new version. My webservice won't have that update until I 
rebuild / redeploy it.

Is my understanding correct that in all of the above scenarios the onerous 
task of keeping the different libraries / runtimes updated falls on me? I 
do realize that supporting (2) and (3) is somewhat of a pipedream (since 
there are an almost infinite amount of possible configurations) and even 
(1) can be very complicated since there are a lot of linux distributions 
out there, but please do realize that one important reason for choosing 
Google Appengine is that I don't have time to be ops!

Thank you.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/b4da105e-a6a9-49bd-ac9e-67693a49e779%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.