[google-appengine] Re: Local Development server when opening "Datastore Viewer"

[Khaled Wagdy]

So I moved to ver. 270.0.0 indeed the problem has gone, which indicates 
that there was some string/unicode/ascii related update along the way that 
may have broken its ability to handle Unicode characters. I also tried 
versions 396, 383, 380 all had the same problem and error output. 

If someone can look into this so that we can work with the latest Google 
Cloud CLI versions, it would be great!

Have a nice day,
Khaled

On Friday, August 12, 2022 at 11:56:30 PM UTC+2 Khaled Wagdy wrote:

> Hello, 
>
> I am using Python 2.7 on AppEngine and Datastore. I moved my code & test 
> database file to another laptop, I also made a fresh install of gcloud 
> components, currently at version 397.0.0.
>
> Ever since then, trying to access local datastore viewer "
> http://localhost:8000/datastore?kind=" from the browser causes as 
> ascii error as seen in the screenshot below and server output. I have some 
> Arabic content in the DB, but on the old laptop which was running an old 
> gcloud components version (270.0.0) there was no problem browsing through 
> the entities.
>
> I will try to install the old version 270.0.0 using the command *gcloud 
> components update --version 270.0.0* and see if that resolves it. But I 
> would think working on the latest version is best in case something can be 
> fixed to resolve this matter.
>
> Has there been a breaking update in the local development server and/or 
> datastore emulator that might have been causing this issue? Is there a way 
> around this ascii error to be able to browse through the local DB entities 
> without problems? This is important during debugging, and can save a lot of 
> time.
>
> [image: Screen Shot 2022-08-12 at 18.02.18.png]ERROR2022-08-12 
> 16:05:04,764 webapp2.py:1553] 'ascii' codec can't encode characters in 
> position 0-3: ordinal not in range(128)
> Traceback (most recent call last):
>   File 
> "/Users/khaled/Documents/darna/google-cloud-sdk/platform/google_appengine/lib/webapp2-2.5.1/webapp2.py",
>  
> line 1536, in __call__
> rv = self.handle_exception(request, response, e)
>   File 
> "/Users/khaled/Documents/darna/google-cloud-sdk/platform/google_appengine/lib/webapp2-2.5.1/webapp2.py",
>  
> line 1530, in __call__
> rv = self.router.dispatch(request, response)
>   File 
> "/Users/khaled/Documents/darna/google-cloud-sdk/platform/google_appengine/lib/webapp2-2.5.1/webapp2.py",
>  
> line 1278, in default_dispatcher
> return route.handler_adapter(request, response)
>   File 
> "/Users/khaled/Documents/darna/google-cloud-sdk/platform/google_appengine/lib/webapp2-2.5.1/webapp2.py",
>  
> line 1102, in __call__
> return handler.dispatch()
>   File 
> "/Users/khaled/Documents/darna/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/admin/admin_request_handler.py",
>  
> line 88, in dispatch
> super(AdminRequestHandler, self).dispatch()
>   File 
> "/Users/khaled/Documents/darna/google-cloud-sdk/platform/google_appengine/lib/webapp2-2.5.1/webapp2.py",
>  
> line 572, in dispatch
> return self.handle_exception(e, self.app.debug)
>   File 
> "/Users/khaled/Documents/darna/google-cloud-sdk/platform/google_appengine/lib/webapp2-2.5.1/webapp2.py",
>  
> line 570, in dispatch
> return method(*args, **kwargs)
>   File 
> "/Users/khaled/Documents/darna/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/admin/datastore_viewer.py",
>  
> line 674, in get
> start))
>   File 
> "/Users/khaled/Documents/darna/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/admin/datastore_viewer.py",
>  
> line 625, in _get_entity_template_data
> value = data_type.format(raw_value)
>   File 
> "/Users/khaled/Documents/darna/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/admin/datastore_viewer.py",
>  
> line 200, in format
> return str(value)
> UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-3: 
> ordinal not in range(128)
>
> Thank you,
> Khaled
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/d99465be-7fdb-4632-beb4-4ded2ab757d4n%40googlegroups.com.

[google-appengine] Re: Local Development server when opening "Datastore Viewer"

[Khaled Wagdy]

Thank you very much =D! 340.0.0 worked indeed,

On Wednesday, August 31, 2022 at 2:37:59 PM UTC+2 limin...@gmail.com wrote:

> Had the same issue, but ver. 340.0.0 was proved fine.
>
> On Tuesday, August 16, 2022 at 12:37:15 AM UTC+9 khaled@gmail.com 
> wrote:
>
>> So I moved to ver. 270.0.0 indeed the problem has gone, which indicates 
>> that there was some string/unicode/ascii related update along the way that 
>> may have broken its ability to handle Unicode characters. I also tried 
>> versions 396, 383, 380 all had the same problem and error output. 
>>
>> If someone can look into this so that we can work with the latest Google 
>> Cloud CLI versions, it would be great!
>>
>> Have a nice day,
>> Khaled
>>
>> On Friday, August 12, 2022 at 11:56:30 PM UTC+2 Khaled Wagdy wrote:
>>
>>> Hello, 
>>>
>>> I am using Python 2.7 on AppEngine and Datastore. I moved my code & test 
>>> database file to another laptop, I also made a fresh install of gcloud 
>>> components, currently at version 397.0.0.
>>>
>>> Ever since then, trying to access local datastore viewer "
>>> http://localhost:8000/datastore?kind=" from the browser causes as 
>>> ascii error as seen in the screenshot below and server output. I have some 
>>> Arabic content in the DB, but on the old laptop which was running an old 
>>> gcloud components version (270.0.0) there was no problem browsing through 
>>> the entities.
>>>
>>> I will try to install the old version 270.0.0 using the command *gcloud 
>>> components update --version 270.0.0* and see if that resolves it. But I 
>>> would think working on the latest version is best in case something can be 
>>> fixed to resolve this matter.
>>>
>>> Has there been a breaking update in the local development server and/or 
>>> datastore emulator that might have been causing this issue? Is there a way 
>>> around this ascii error to be able to browse through the local DB entities 
>>> without problems? This is important during debugging, and can save a lot of 
>>> time.
>>>
>>> [image: Screen Shot 2022-08-12 at 18.02.18.png]ERROR2022-08-12 
>>> 16:05:04,764 webapp2.py:1553] 'ascii' codec can't encode characters in 
>>> position 0-3: ordinal not in range(128)
>>> Traceback (most recent call last):
>>>   File 
>>> "/Users/khaled/Documents/darna/google-cloud-sdk/platform/google_appengine/lib/webapp2-2.5.1/webapp2.py",
>>>  
>>> line 1536, in __call__
>>> rv = self.handle_exception(request, response, e)
>>>   File 
>>> "/Users/khaled/Documents/darna/google-cloud-sdk/platform/google_appengine/lib/webapp2-2.5.1/webapp2.py",
>>>  
>>> line 1530, in __call__
>>> rv = self.router.dispatch(request, response)
>>>   File 
>>> "/Users/khaled/Documents/darna/google-cloud-sdk/platform/google_appengine/lib/webapp2-2.5.1/webapp2.py",
>>>  
>>> line 1278, in default_dispatcher
>>> return route.handler_adapter(request, response)
>>>   File 
>>> "/Users/khaled/Documents/darna/google-cloud-sdk/platform/google_appengine/lib/webapp2-2.5.1/webapp2.py",
>>>  
>>> line 1102, in __call__
>>> return handler.dispatch()
>>>   File 
>>> "/Users/khaled/Documents/darna/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/admin/admin_request_handler.py",
>>>  
>>> line 88, in dispatch
>>> super(AdminRequestHandler, self).dispatch()
>>>   File 
>>> "/Users/khaled/Documents/darna/google-cloud-sdk/platform/google_appengine/lib/webapp2-2.5.1/webapp2.py",
>>>  
>>> line 572, in dispatch
>>> return self.handle_exception(e, self.app.debug)
>>>   File 
>>> "/Users/khaled/Documents/darna/google-cloud-sdk/platform/google_appengine/lib/webapp2-2.5.1/webapp2.py",
>>>  
>>> line 570, in dispatch
>>> return method(*args, **kwargs)
>>>   File 
>>> "/Users/khaled/Documents/darna/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/admin/datastore_viewer.py",
>>>  
>>> line 674, in get
>>> start))
>>>   File 
>>> "/Users/khaled/Documents/darna/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/admin/datastore_viewer.py",
>>>  
>>> line 625, in _get_entity_template_data
>>> value = data_type.format(raw_value)
>>>   File 
>>> "/Users/khaled/Documents/darna/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/admin/datastore_viewer.py",
>>>  
>>> line 200, in format
>>> return str(value)
>>> UnicodeEncodeError: 'ascii' codec can't encode characters in position 
>>> 0-3: ordinal not in range(128)
>>>
>>> Thank you,
>>> Khaled
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/6e84f58c-132e-458d-be3f-be901b64784fn%40googlegroups.com.

[google-appengine] Re: How do I get a Python37 app working?

[Khaled Wagdy]

Hello,

I am wondering why you are using *gunicorn* command and not just *python 
main.app*? What web framework are you using?

On Monday, April 8, 2019 at 8:21:50 PM UTC+2, Mark Summerfield wrote:
>
> I've change the `app.yaml` file to match the docs:
> ```
> runtime: python37
>
> handlers:
> - url: /static
>   static_dir: static
>
> - url: /.*
>   script: auto
> ```
> But I now realise that when I try to test the app, I don't use the yaml 
> file at all because this is my command line for local testing:
> `gunicorn -b localhost:8080 --reload --log-level debug main:app`
> So that only knows about main.py; it doesn't know anything about the yaml 
> file. 
>
> So, how do I do _local_ testing using the yaml file to serve up the static 
> files and relying on main.py for the dynamic pages?
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/3d198b3d-2935-49d7-8131-ff2f5f186c9d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Creating a HIPPA Compliant GSuite Accessible Only App-- Billing, HIPAA, & Access Questions

[Khaled Wagdy]

Hi Anna,

That's quite interesting! On the bright side, congrats on making an app 
that eventually became the focus of the company, good job on that!

I have some experience with G Suite and with Google App Engine development 
using Python. I have mainly used Google Datastore (*which should soon be 
automatically migrated to Firebase in Datastore mode*). Let me try to 
address some of the points you mentioned..

   1. What do I need to know about transferring the app to the admin owner 
   once the project is completed? They have added me to their suite, but I 
   will be removed once I am done with the project
   I would rather have them create a project admin generic account rather 
   than an individual account, and use that to create the project. This way 
   they can use the same account after you are done, they would just need to 
   reset its password. Or if you want to keep the current setup, you can 
transfer 
   project ownership to another account 
   

 after 
   you are done. Another option is also to add another user as "owner" to the 
   project from *IAM here ,* 
   more about roles here 
   *https://cloud.google.com/iam/docs/understanding-roles* 
   
   2. When I go into my project console under the cloud SQL it says, "You 
   do not have the required permissions to create a Cloud SQL instance. 
   Contact a project administrator to change these settings." I am the project 
   admin so I am assuming this has to do with the little prompt at the 
   top that is telling me to set up billing..
   You are probably right, as you are creating an instance that is billed, 
   you probably need to enable and setup billing first
   3. Should I be inputting my own billing information? Or should I have 
   the organization do it? And if the latter, how do I have them do that?
   No! Please do not do that, you are right about your concerns being 
   billed from your credit card and that later it may not be sufficient for an 
   entire organization consumption of the app. Aside to that you probably 
   don't need the headache of being billed then trying to invoice the 
   organization for the paid bills! You ask them politely to add their credit 
   card information in the project billing section 
   https://console.cloud.google.com/billing/ (*you may need to choose the 
   project*). Again I would setup another owner account for the 
   organization to be able to manage all that stuff.
   4. Has anyone developed a HIPAA compliant app? If so, I would really 
   appreciate any guidance you could offer me. 
   I haven't really, but I see this documentation here 
   *https://cloud.google.com/security/compliance/hipaa/* 
    which has 
   "Recommended technical best practices" section that may be worth going 
   through as a checklist
   5. creating a GSuite accessible only app.. I have only ever made public 
   apps so I am not sure how to configure settings so that nobody can find the 
   app unless they are a member of the suite.
   I am not sure how you are currently managing authentication in your app, 
   but I believe G suite authentication for a specific domain is done from *App 
   Engine settings ,* 
   check *this stack overflow question* 
   

 
   too
   6. I tried deploying my first firebase version of the app on Google App 
   engine and it worked fine.. but I wasn't able to figure out how to edit my 
   source code once it was deployed.. I ended up just deleting the projects 
   and creating new ones every time I went to test a different version. I know 
   that's probably the absolute wrong thing to do, but I am really new to 
   Google App engine
   No no no! You don't edit your code on GCP, you edit locally and then 
   deploy. When you deploy to app engine you have full control! you can simply 
   deploy using a *gcloud* command from the terminal Every time you deploy 
   your code, a new version is created on the GCP project and traffic is 
   migrated to this new version. You can even access old versions, or have two 
   or more versions alive at the same time and you can choose to split traffic 
   between those two or more versions! See... *Deploying to App Engine* 
   
, 
   *Splitting traffic* 
   
   ,

I hope this helps you a bit, all the best,
Khaled

On Thursday, April 11, 2019 at 10:32:37 PM UTC+2, Anna Nevison wrote:
>
> Hi everyone, 
>
> Thanks to some friendly individuals on the App Maker forum.. I believe I 
> have foun

Re: [google-appengine] GAE python script via GAE Cron service job time limit

[Khaled Wagdy]

As far as I know the maximum is 10 minutes

On Sun, 14 Apr 2019, 15:27 Sunil Kumar,  wrote:

>
> Friends,
>
> I would like to run a python script on GAE python 3 standard which calls a
> rest api and pushes the data to big query. This code runs for about 45 mins
> daily on my onprem server.
>
> Is there a time limit under which a python script has to complete when
> called via GAE cron service.
>
> I get lot of mixed answers . What is the correct answer for this
> particular scenario.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to google-appengine+unsubscr...@googlegroups.com.
> To post to this group, send email to google-appengine@googlegroups.com.
> Visit this group at https://groups.google.com/group/google-appengine.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/google-appengine/401b7adc-a8d3-47b8-9f06-cbce5b04871c%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/CAOKhha3xJ86wp%3DBgmqWXdz_TjA%3DnSuogB-0fzMydzNDiJ%2B8XwA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: The Link To WebApp3 NonPublic Repository

[Khaled Wagdy]

Yeah, it seems like diogoa said that they are using the frameworks 
mentioned in this page 
. No 
mentioning of Webapp3...

As mentioned on the page..

This quickstart demonstrates a simple Python app written with the Flask 
>  web framework that can be deployed to App 
> Engine. Although this sample uses Flask, you can use any web framework that 
> satisfies the requirements above. Alternative frameworks include Django 
> , Pyramid 
> , Bottle , and web.py .


webapp2 did have some advantages like using deferred library easily for 
example, other frameworks may have other advantages though, specially with 
AppEngine moving to support Python3


On Wednesday, April 17, 2019 at 4:03:49 PM UTC+2, Marcus Eagan wrote:
>
> Hello there, 
>
> I've recently looked into using webapp3 with a few wishes: that there is a 
> lightweight framework  for working with Google App Engine that evolved from 
> WebApp2, and that the next generation framework supports python 3. The 
> repository listed at this website does not appear to exist, perhaps because 
> it is private: https://webapp2.readthedocs.io/features.html
>
> This sort of information is critical for developers looking to adopt the 
> platform or increase our existing footprint.
>
> Furthermore, links to documentation for both WebApp2 and WebApp3 are down. 
> Please advise on what I can do about the repository and what's going on 
> with the documentation.
>
> Thanks,
>
> Marcus
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/b79e7df3-d499-493a-b3f6-2aad4d56d38e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: "InvalidSenderError: Unauthorized sender error" suddenly started happening on our project.

[Khaled Wagdy]

Hello, can you check the email sender lists? 
https://console.cloud.google.com/appengine/settings/emailsenders and make 
sure that the account(s) you are sending from are listed there?
Also, I'm curious, are the emails you are sending from a gmail.com or Google 
Apps for work  (GSuite) email address(es)?

On Sunday, April 21, 2019 at 6:44:44 PM UTC+2, Andrea Cimino wrote:
>
>
>
> Il giorno domenica 21 aprile 2019 19:30:27 UTC+3, George (Cloud Platform 
> Support) ha scritto:
>
> Hi George,
>
> unfortunately, at least from my point of view, the documentation is not 
> really clear.
> "All email addresses on the Email API Authorized Senders list need to be 
> valid Gmail or Google-hosted domain accounts. App Administrators can add 
> the following accounts to the list of Authorized Senders".
> Still is not clear what is a Google hosted domain account. 
> I know just that our service was working perfectly fine until 15 days ago 
> and now we have this error.
> If you could please give as good pointers to how to get a "Google hosted 
> domain account" it would be very nice,
> as we are stuck and we don't know how to fix this error. 
>
>> Hello Andrea, 
>>
>> You can find details relevant to your question "what is a google-hosted 
>> domain account" on the "Mail API Overview" documentation page 
>> ,
>>  
>> namely in the "Who can send mail" sub-chapter. 
>>
>> "App Administrators can add the following accounts to the list of 
>> Authorized Senders:
>>
>> Their own email address
>> Any group for which they are an Owner or Manager
>> Applications hosted in a G Suite domain: noreply@[DOMAIN].com, as long as 
>> noreply@[DOMAIN].com is a valid account (user or group)". 
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/045202fd-d29e-4fdf-bab1-a6a2531db3a8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: "InvalidSenderError: Unauthorized sender error" suddenly started happening on our project.

[Khaled Wagdy]

Oh I see, 27 March sounds like a little more than 15 days ago, but it could 
be that the systems picked it up later and therefore you started seeing the 
unauthorized sender error.

You probably need to switch as you said to another provider like Mailgun 
<https://cloud.google.com/appengine/docs/standard/java/mail/mailgun>, 
Mailjet <https://cloud.google.com/appengine/docs/standard/java/mail/mailjet>, 
or SendGrid 
<https://cloud.google.com/appengine/docs/standard/java/mail/sendgrid>.

Best of luck :)

On Monday, April 22, 2019 at 2:38:39 PM UTC+2, Andrea Cimino wrote:
>
> The address is correctly listed (every was working fine until 15 days ago).
> The address is neither a gmail.com or Google Apps for Work (GSuite).
> Some times ago we registered for GSuite to have our email sender working, 
> but 2 years ago this requirement was not needed anymore according to the 
> documentation, so we disabled the GSuite account (2 years ago). I think 
> something the email sender requirements have changed during time, so we 
> falled in this problem.
> Recently we have received a mail from google stating:
>
> Greetings from Google,
>
> It appears you have not used your Google Account managed for the "
> ourdomain.com " domain in the last 365 days.
>
> If you do not take one of the corrective measures described below, Google 
> will delete the account on 27 March 2019 or at a later date.
>
> So this is what happened. So we really need a Gsuite account for sending 
> emails?
> I think at this stage is migrating to other services such as Sendgrid as 
> advised.
>
>
>
> Il giorno lunedì 22 aprile 2019 15:28:54 UTC+3, Khaled Wagdy ha scritto:
>>
>> Hello, can you check the email sender lists? 
>> https://console.cloud.google.com/appengine/settings/emailsenders and 
>> make sure that the account(s) you are sending from are listed there?
>> Also, I'm curious, are the emails you are sending from a gmail.com or Google 
>> Apps for work <https://gsuite.google.com/> (GSuite) email address(es)?
>>
>> On Sunday, April 21, 2019 at 6:44:44 PM UTC+2, Andrea Cimino wrote:
>>>
>>>
>>>
>>> Il giorno domenica 21 aprile 2019 19:30:27 UTC+3, George (Cloud Platform 
>>> Support) ha scritto:
>>>
>>> Hi George,
>>>
>>> unfortunately, at least from my point of view, the documentation is not 
>>> really clear.
>>> "All email addresses on the Email API Authorized Senders list need to be 
>>> valid Gmail or Google-hosted domain accounts. App Administrators can add 
>>> the following accounts to the list of Authorized Senders".
>>> Still is not clear what is a Google hosted domain account. 
>>> I know just that our service was working perfectly fine until 15 days 
>>> ago and now we have this error.
>>> If you could please give as good pointers to how to get a "Google hosted 
>>> domain account" it would be very nice,
>>> as we are stuck and we don't know how to fix this error. 
>>>
>>>> Hello Andrea, 
>>>>
>>>> You can find details relevant to your question "what is a google-hosted 
>>>> domain account" on the "Mail API Overview" documentation page 
>>>> <https://cloud.google.com/appengine/docs/standard/go/mail/#who_can_send_mail>,
>>>>  
>>>> namely in the "Who can send mail" sub-chapter. 
>>>>
>>>> "App Administrators can add the following accounts to the list of 
>>>> Authorized Senders:
>>>>
>>>> Their own email address
>>>> Any group for which they are an Owner or Manager
>>>> Applications hosted in a G Suite domain: noreply@[DOMAIN].com, as long 
>>>> as noreply@[DOMAIN].com is a valid account (user or group)". 
>>>>
>>>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/05d427a5-2bab-42d7-b8db-96bb0448d6ff%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: "InvalidSenderError: Unauthorized sender error" suddenly started happening on our project.

[Khaled Wagdy]

Quick note, Mailgun pricing states the following, if you need to send a lot 
of bulk messages monthly:

As a Google Cloud Platform user, your first *30,000* *messages* *are* *free 
> every month*. See the monthly pricing calculator on the sign up page 
> <http://www.mailgun.com/google> for pricing on additional messages and 
> volume discounts.

 
This is mentioned on this page: Sending Email with Mailgun 
<https://cloud.google.com/appengine/docs/standard/java/mail/mailgun> "Pricing" 
section. You might want to compare the features too in case there are other 
aspects you need to consider in your app.

Good luck,


On Monday, April 22, 2019 at 2:43:56 PM UTC+2, Khaled Wagdy wrote:
>
> Oh I see, 27 March sounds like a little more than 15 days ago, but it 
> could be that the systems picked it up later and therefore you started 
> seeing the unauthorized sender error.
>
> You probably need to switch as you said to another provider like Mailgun 
> <https://cloud.google.com/appengine/docs/standard/java/mail/mailgun>, 
> Mailjet 
> <https://cloud.google.com/appengine/docs/standard/java/mail/mailjet>, or 
> SendGrid 
> <https://cloud.google.com/appengine/docs/standard/java/mail/sendgrid>.
>
> Best of luck :)
>
> On Monday, April 22, 2019 at 2:38:39 PM UTC+2, Andrea Cimino wrote:
>>
>> The address is correctly listed (every was working fine until 15 days 
>> ago).
>> The address is neither a gmail.com or Google Apps for Work (GSuite).
>> Some times ago we registered for GSuite to have our email sender working, 
>> but 2 years ago this requirement was not needed anymore according to the 
>> documentation, so we disabled the GSuite account (2 years ago). I think 
>> something the email sender requirements have changed during time, so we 
>> falled in this problem.
>> Recently we have received a mail from google stating:
>>
>> Greetings from Google,
>>
>> It appears you have not used your Google Account managed for the "
>> ourdomain.com " domain in the last 365 days.
>>
>> If you do not take one of the corrective measures described below, Google 
>> will delete the account on 27 March 2019 or at a later date.
>>
>> So this is what happened. So we really need a Gsuite account for sending 
>> emails?
>> I think at this stage is migrating to other services such as Sendgrid as 
>> advised.
>>
>>
>>
>> Il giorno lunedì 22 aprile 2019 15:28:54 UTC+3, Khaled Wagdy ha scritto:
>>>
>>> Hello, can you check the email sender lists? 
>>> https://console.cloud.google.com/appengine/settings/emailsenders and 
>>> make sure that the account(s) you are sending from are listed there?
>>> Also, I'm curious, are the emails you are sending from a gmail.com or 
>>> Google 
>>> Apps for work <https://gsuite.google.com/> (GSuite) email address(es)?
>>>
>>> On Sunday, April 21, 2019 at 6:44:44 PM UTC+2, Andrea Cimino wrote:
>>>>
>>>>
>>>>
>>>> Il giorno domenica 21 aprile 2019 19:30:27 UTC+3, George (Cloud 
>>>> Platform Support) ha scritto:
>>>>
>>>> Hi George,
>>>>
>>>> unfortunately, at least from my point of view, the documentation is not 
>>>> really clear.
>>>> "All email addresses on the Email API Authorized Senders list need to 
>>>> be valid Gmail or Google-hosted domain accounts. App Administrators can 
>>>> add 
>>>> the following accounts to the list of Authorized Senders".
>>>> Still is not clear what is a Google hosted domain account. 
>>>> I know just that our service was working perfectly fine until 15 days 
>>>> ago and now we have this error.
>>>> If you could please give as good pointers to how to get a "Google 
>>>> hosted domain account" it would be very nice,
>>>> as we are stuck and we don't know how to fix this error. 
>>>>
>>>>> Hello Andrea, 
>>>>>
>>>>> You can find details relevant to your question "what is a 
>>>>> google-hosted domain account" on the "Mail API Overview" documentation 
>>>>> page 
>>>>> <https://cloud.google.com/appengine/docs/standard/go/mail/#who_can_send_mail>,
>>>>>  
>>>>> namely in the "Who can send mail" sub-chapter. 
>>>>>
>>>>> "App Administrators can add the following accounts to the list of 
>>>>> Authorized Senders:
>>>>>
>>>>> Their own email address
>>>>> Any group for which they are an Owner or Manager
>>>>> Applications hosted in a G Suite domain: noreply@[DOMAIN].com, as long 
>>>>> as noreply@[DOMAIN].com is a valid account (user or group)". 
>>>>>
>>>>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/a65213c5-5f47-43d0-88fc-b7fc526eae63%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Save/Export GAE logs for more than 30 days in an easily readable way

[Khaled Wagdy]

Hi All,

Is there a best practice for keeping logs for more than 30 days yet in an 
easily readable format?

Recently we had an issue and wanted to check the logs of the previous 
month. Since the logs are only kept for 30 days in *Log Viewer *we couldn't 
find it there. We have previously setup a sink to BigQuery, but although 
the logs seems to be there, we couldn't find the logs we're looking for! 
Not sure if this is lack of experience with BigQuery or is it that only the 
request logs coming from actual users are the ones kept. We are looking for 
a particular monthly CRON job logs, which we couldn't find in the backed up 
logs on BigQuery for some reason.

We thought we would setup another sink to Cloud storage for the future, but 
it seems to be saved in JSON format, which includes too much information 
that makes it hard to read in critical cases where we want to see normal 
log flow as it appears in *Log Viewer. *

We used to use instructions in the Download logs page 

 using 
the command *appcfg.py request_logs *to download logs periodically, but we 
stopped doing that a while ago. The problem with this method is that 
instead of showing the date and time as they appear on the *Log Viewer *page*, 
*they appear in what I assume to be an Epoch format which still makes it 
hard to read, example:
*1:1565112559.564261** log line...*
*3:1565112559.564692 log line...*
*...*
furthermore this method seems to be on its way to be deprecated as 
mentioned on the download logs page and the new *gcloud* tool does not have 
a download option

*Warning:* The following describes using the appcfg tool to download logs. 
> This tool is now deprecated 
> .
>  
> Currently the replacement, Cloud SDK , 
> does not support this download capability; however, you can view your logs 
> using the gcloud app log commands 
> .


 I am not sure why it is not easy to export log viewer as it is, it's 
really very convenient as is, we just a way to download it straight 
forwardly.

Resources*:*

   1. Export logs documentation 
   
   2. Download logs documentation 
   


*Questions:*

   1. Knowing that we have a sink setup to BigQuery, is there a way to 
   fetch the logs of the previous month CORN job? Was it exported as part of 
   the sink? How can we filter through BigQuery to fetch those logs?
   2. Is there a way to export logs to keep for more than 30 days, without 
   changing the log viewer format?

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/0f5aad1c-dba6-4c88-a2f7-e5da6e4975b2%40googlegroups.com.

[google-appengine] Re: Save/Export GAE logs for more than 30 days in an easily readable way

[Khaled Wagdy]

Hello Anthony,

Thank you for your feedback, I am not quite sure I understood your first 
point. When you say the sink will hold the 31st day, you mean it will *not* 
log the 31st day, right? In our case I think we have setup the sink to 
export the logs daily, since in BigQuery we can choose by day in the drop 
down menu so I think it's good for now. The query we need to set in 
BigQuery is a bit cumbersome since there's a lot of fields and it's 
confusing a bit.

For the second point, all we ask for a is an easily readable format just 
like the *Log Viewer.* Date, event, collected information of each request, 
that's it. The closest format we saw to that was the *appcfg* download 
logs, which, as we mentioned before has this epoch format that is 
challenging to read. We would settle for the download logs ability and we 
can setup a cron job to download logs to a cloud storage bucket for future 
reference. But, also as we mentioned, for some reason this is on its way to 
be deprecated. Would you be able to relay a feature request for gcloud SDK 
team for the gcloud SDK to be able to download logs like appcfg? This time, 
preferably in UTC time rather than epoch time. That would be very much 
appreciated.

Thank you,


On Monday, September 9, 2019 at 9:51:08 PM UTC+2, Anthony (Google Cloud 
Support) wrote:
>
> Hi Khaled,
>
> To answer your questions
>
> 1. Your sink will hold all logs from the day it was initialized. Which 
> means 31 days from now, it will hold the 31st day of the log rather than 
> deleting the log from the 1st day. This also means that any logs prior to 
> the export sink will not be retained and will be removed after 30 days.
>
> 2. In regards to your second inquiry, using export sink is the only way to 
> retain logs over 30 days. That said, as per the following document [1] 
> these are the current formats provided by Stackdriver.
>
> I hope this helps.
>
> [1]:
> https://cloud.google.com/solutions/design-patterns-for-exporting-stackdriver-logging#logging_formats
>
> On Thursday, September 5, 2019 at 11:55:50 AM UTC-4, Khaled Wagdy wrote:
>>
>> Hi All,
>>
>> Is there a best practice for keeping logs for more than 30 days yet in an 
>> easily readable format?
>>
>> Recently we had an issue and wanted to check the logs of the previous 
>> month. Since the logs are only kept for 30 days in *Log Viewer *we couldn't 
>> find it there. We have previously setup a sink to BigQuery, but although 
>> the logs seems to be there, we couldn't find the logs we're looking for! 
>> Not sure if this is lack of experience with BigQuery or is it that only the 
>> request logs coming from actual users are the ones kept. We are looking for 
>> a particular monthly CRON job logs, which we couldn't find in the backed up 
>> logs on BigQuery for some reason.
>>
>> We thought we would setup another sink to Cloud storage for the future, 
>> but it seems to be saved in JSON format, which includes too much 
>> information that makes it hard to read in critical cases where we want to 
>> see normal log flow as it appears in *Log Viewer. *
>>
>> We used to use instructions in the Download logs page 
>> <https://cloud.google.com/appengine/docs/standard/python/tools/downloading-logs>
>>  using 
>> the command *appcfg.py request_logs *to download logs periodically, but 
>> we stopped doing that a while ago. The problem with this method is that 
>> instead of showing the date and time as they appear on the *Log Viewer *
>> page*, *they appear in what I assume to be an Epoch format which still 
>> makes it hard to read, example:
>> *1:1565112559.564261** log line...*
>> *3:1565112559.564692 log line...*
>> *...*
>> furthermore this method seems to be on its way to be deprecated as 
>> mentioned on the download logs page and the new *gcloud* tool does not 
>> have a download option
>>
>> *Warning:* The following describes using the appcfg tool to download 
>>> logs. This tool is now deprecated 
>>> <https://cloud.google.com/appengine/docs/standard/python/sdk-gcloud-migration>.
>>>  
>>> Currently the replacement, Cloud SDK 
>>> <https://cloud.google.com/sdk/docs/>, does not support this download 
>>> capability; however, you can view your logs using the gcloud app log
>>>  commands <https://cloud.google.com/sdk/gcloud/reference/app/logs/>.
>>
>>
>>  I am not sure why it is not easy to export log viewer as it is, it's 
>> really very convenient as is, we just a way to download it straight 
>> forwardly.
>>
>> Resources*:*
>>
>>1. Export logs documentation 
>

[google-appengine] Re: Save/Export GAE logs for more than 30 days in an easily readable way

[Khaled Wagdy]

Thank you Vivak,

I have followed your advise and posted on issuetracker. They pointed me to 
the right direction, I thought I would share it here in case someone else 
would like to download the logs.

The command to use is: *$ gcloud logging read*. I have set the *severity*, 
*timestamp, 
format* and chosen the *fields* I believe would be useful in tracing. I 
also channeled the output to a *file*. One should adjust the parameters 
according to their requirements.


*$ gcloud logging read "severity>=DEBUG AND 
timestamp>=\"2019-09-12T00:00:00Z\"" --format='table[no-heading](timestamp, 
protoPayload.ip, protoPayload.method, protoPayload.resource, 
httpRequest.status, protoPayload.userAgent 
,protoPayload.line[]:format="value[separator=\"\"](severity, 
logMessage)")' >> 20190912.txt*

Resources:

   1. https://cloud.google.com/sdk/gcloud/reference/logging/read
   2. 
   https://cloud.google.com/blog/products/gcp/filtering-and-formatting-fun-with
   3. https://cloud.google.com/logging/docs/view/advanced-queries
   


On Wednesday, September 11, 2019 at 8:44:32 PM UTC+2, Vivak Patel wrote:
>
> Hi Khaled
>
> I believe what Anthony meant to say was that exporting logs using sinks 
> will help mitigate the loss of logs on Log Viewer past the 30 day mark. 
> However, logs prior to the creation of the sink will still be lost.
>
> As for the feature request, we always appreciate users collaborating with 
> our engineers on ideas to improve the product. This discussion group does 
> not allow the moderation of feature requests, which is why I encourage you 
> to create the feature request on Public Issue Tracker[1] as it will provide 
> you visibility on the request.
>
> [1] - https://issuetracker.google.com/
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/811ae3a5-5a18-4cb9-9172-e0caaba0cad0%40googlegroups.com.

Re: [google-appengine] Download Nodejs code from Google Cloud Platform

[Khaled Wagdy]

Ouch! I'm not sure for NodeJS, but did you check the "Source" tab on your
GCP project?

On Thu, 23 Jan 2020, 16:10 Mohd Fuad,  wrote:

> I have recently lost access to my hard drive ( it died) and the only
> source code of my application ( Node JS) is at the Google Cloud Platform.
> Is there a way that i can download my node js project from the GCP?
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to google-appengine+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/google-appengine/26714ec4-6954-4bc6-8cf0-ab03884a6780%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/CAOKhha0ptFKtxCv8gOeqfv_aEhFgto8t0qey%3D7ddZ7gc4-C6VA%40mail.gmail.com.