Hi Nick, Thanks - I had just started doing exactly what you suggest! :-)
I crave your indulgence on one other issue ... when I run a gcloud beta logging command with a LOG_FILTER clause on a Unix system, it works fine. For example: gcloud beta logging read 'timestamp<="2017-03-23T00:00:00Z" AND timestamp>="2017-03-22T00:00:00Z"' --limit=2 However, on Windows, the exact same command doesn't work - I suspect the single and double quotes are part of the problem: gcloud beta logging read 'timestamp<="2017-03-23T00:00:00Z" AND timestamp>="2017-03-22T00:00:00Z"' --limit=2 The filename, directory name, or volume label syntax is incorrect. (A simple "gcloud beta logging read" works just fine on Windows.) I've tried various ways of escaping the quotes, but nothing appears to work. Should I post this on StackOverflow? On Wed, Mar 29, 2017 at 12:37 PM, 'Nick (Cloud Platform Support)' via Google App Engine <google-appengine@googlegroups.com> wrote: > Hey Julian, > > You have two options here: either to wait for the google.cloud library to > get support for V1 logs, or adopt a slightly different approach to writing > the python script. You could use the python subprocess > <https://docs.python.org/2/library/subprocess.html> module to call the > gcloud command (keeping in mind that this is part of the beta command > group, which can change on updates without warning (the only warning being > the release notes)), sending the output into a string, and then parsing the > entries that way. This, while it would mean you'd have your logs in a > python environment (with all the leverage that brings) however would not > use the google.cloud library. > > Cheers, > > Nick > Cloud Platform Community Support > > On Tuesday, March 28, 2017 at 5:51:54 PM UTC-4, Julian Bunn wrote: >> >> Thanks, Nick! >> >> I confirm that the command "gcloud beta logging read" retrieves our logs! >> So this looks very promising ... >> >> Is there a way to call this interface similarly to: >> >> from google.cloud import logging >> from google.cloud.logging import DESCENDING >> def main(): >> client = logging.Client() >> for entry in client.list_entries(order_by=DESCENDING): >> print entry >> if __name__ == '__main__': >> main() >> >> >> >> On Tue, Mar 28, 2017 at 12:39 PM, 'Nick (Cloud Platform Support)' via >> Google App Engine <google-appengine@googlegroups.com> wrote: >> >>> Hey Julian, >>> >>> I've observed the identical error when using the filter >>> 'resource.type="gae_app" AND resource.labels.module_id="default"', >>> where I'd deployed a standard environment app on "default". >>> >>> It appears this is caused by the fact that the standard environment >>> doesn't log to Stackdriver v2 logging, as you observed. The docs show all >>> v2 endpoints are what the library interfaces with. Nonetheless, it should >>> be able to read V1 type logs as well, or at least we can consider this a >>> feature request. >>> >>> In the meantime, the following command pattern can be used, presenting >>> no problem with V1 logs: >>> >>> gcloud beta logging read ${LOG_FILTER} >>> >>> >>> I've created a Public Issue Tracker issue >>> <https://b.corp.google.com/issues/36687054> which you can follow while >>> we work on this. >>> >>> Cheers, >>> >>> Nick >>> Cloud Platform Community Support >>> >>> On Monday, March 27, 2017 at 3:14:13 PM UTC-4, Julian Bunn wrote: >>>> >>>> Hi Nick, >>>> >>>> Thanks ... I do try StackOverflow from time to time for these sorts of >>>> technical issues, but have very limited success. Perhaps I'm using bad >>>> tags. See for example a logging related question I posted there on Friday >>>> about JSON credentials for logging https://stackoverflow. >>>> com/questions/43004904/accessing-gae-log-files-using-google- >>>> cloud-logging-python >>>> >>>> Regarding the v1.RequestLog error that's the topic of this thread: we >>>> do not use StackDriver logging in our GAE deployment, so I wonder whether >>>> cloud.logging only supports StackDriver logs? >>>> >>>> Otherwise, I am at a loss as to why google.cloud.logging can't >>>> understand the log format it fetches from our GAE deployment? >>>> >>>> We've now been without access to our log data for almost a week, so I'm >>>> getting quite desperate to find a solution either with request_logs or >>>> cloud.logging :-) >>>> >>>> Thanks for your help. >>>> >>>> Julian >>>> >>>> >>>> >>>> >>>> On Mon, Mar 27, 2017 at 11:58 AM, 'Nick (Cloud Platform Support)' via >>>> Google App Engine <google-appengine@googlegroups.com> wrote: >>>> >>>>> Hey Julian, >>>>> >>>>> I've been able just now to run the same code fine. What might you mean >>>>> by "the type of logging we use on GAE"? Are you doing anything >>>>> particularly >>>>> odd there? >>>>> >>>>> It's worth mentioning here that this forum isn't meant for technical >>>>> support but rather for general discussion of the platform, services, >>>>> design >>>>> patterns, etc. This question should have been posted to >>>>> stackoverflow.com on a relevant Cloud Platform / App Engine tag. We >>>>> monitor Stack Overflow regularly, so we'll be able to answer there, >>>>> although we can perhaps help debug this a little here before you move to >>>>> post there. >>>>> >>>>> Cheers, >>>>> >>>>> Nick >>>>> Cloud Platform Community Support >>>>> >>>>> >>>>> On Monday, March 27, 2017 at 12:25:45 PM UTC-4, Julian Bunn wrote: >>>>>> >>>>>> Right now we cannot use the request_logs feature for downloading our >>>>>> GAE logs (see my other thread on that problem!), so I thought I would try >>>>>> the newer google.cloud features, and get modern :-) >>>>>> >>>>>> Here is the simple test I tried: >>>>>> >>>>>> from google.cloud import logging >>>>>> from google.cloud.logging import DESCENDING >>>>>> def main(): >>>>>> client = logging.Client() >>>>>> for entry in client.list_entries(order_by=DESCENDING): >>>>>> print entry >>>>>> if __name__ == '__main__': >>>>>> main() >>>>>> >>>>>> When executing this (after setting up the gcloud credentials), there >>>>>> seems to be a problem parsing the log entries: >>>>>> >>>>>> python GetLogsCloud.py >>>>>> Traceback (most recent call last): >>>>>> File "GetLogsCloud.py", line 25, in <module> >>>>>> main() >>>>>> File "GetLogsCloud.py", line 18, in main >>>>>> for entry in client.list_entries(order_by=DESCENDING): >>>>>> File "c:\python27\lib\site-packages\google\cloud\iterator.py", >>>>>> line 219, in _items_iter >>>>>> for item in page: >>>>>> File "c:\python27\lib\site-packages\google\cloud\iterator.py", >>>>>> line 163, in next >>>>>> result = self._item_to_value(self._parent, item) >>>>>> File "c:\python27\lib\site-packages\google\cloud\logging\_gax.py", >>>>>> line 488, in _item_to_entry >>>>>> resource = MessageToDict(entry_pb) >>>>>> File "c:\python27\lib\site-packages\google\protobuf\json_format.py", >>>>>> line 133, in MessageToDict >>>>>> return printer._MessageToJsonObject(message) >>>>>> File "c:\python27\lib\site-packages\google\protobuf\json_format.py", >>>>>> line 164, in _MessageToJsonObject >>>>>> return self._RegularMessageToJsonObject(message, js) >>>>>> File "c:\python27\lib\site-packages\google\protobuf\json_format.py", >>>>>> line 196, in _RegularMessageToJsonObject >>>>>> js[name] = self._FieldToJsonObject(field, value) >>>>>> File "c:\python27\lib\site-packages\google\protobuf\json_format.py", >>>>>> line 230, in _FieldToJsonObject >>>>>> return self._MessageToJsonObject(value) >>>>>> File "c:\python27\lib\site-packages\google\protobuf\json_format.py", >>>>>> line 162, in _MessageToJsonObject >>>>>> return methodcaller(_WKTJSONMETHODS[full_name][0], message)(self) >>>>>> File "c:\python27\lib\site-packages\google\protobuf\json_format.py", >>>>>> line 266, in _AnyMessageToJsonObject >>>>>> sub_message = _CreateMessageFromTypeUrl(type_url) >>>>>> File "c:\python27\lib\site-packages\google\protobuf\json_format.py", >>>>>> line 341, in _CreateMessageFromTypeUrl >>>>>> 'Can not find message descriptor by type_url: >>>>>> {0}.'.format(type_url)) >>>>>> TypeError: Can not find message descriptor by type_url: >>>>>> type.googleapis.com/google.appengine.logging.v1.RequestLog. >>>>>> >>>>>> I'm wondering what the issue is here ... is it because of the type of >>>>>> logging we use on GAE? >>>>>> >>>>>> Thanks! >>>>>> >>>>> -- >>>>> You received this message because you are subscribed to a topic in the >>>>> Google Groups "Google App Engine" group. >>>>> To unsubscribe from this topic, visit https://groups.google.com/d/to >>>>> pic/google-appengine/M01jjjNc1dQ/unsubscribe. >>>>> To unsubscribe from this group and all its topics, send an email to >>>>> google-appengine+unsubscr...@googlegroups.com. >>>>> To post to this group, send email to google-appengine@googlegroups.com >>>>> . >>>>> Visit this group at https://groups.google.com/group/google-appengine. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/google-appengine/cd373f85- >>>>> 74d3-4dab-97d7-2693bf1dc6e2%40googlegroups.com >>>>> <https://groups.google.com/d/msgid/google-appengine/cd373f85-74d3-4dab-97d7-2693bf1dc6e2%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> -- >>> You received this message because you are subscribed to a topic in the >>> Google Groups "Google App Engine" group. >>> To unsubscribe from this topic, visit https://groups.google.com/d/to >>> pic/google-appengine/M01jjjNc1dQ/unsubscribe. >>> To unsubscribe from this group and all its topics, send an email to >>> google-appengine+unsubscr...@googlegroups.com. >>> To post to this group, send email to google-appengine@googlegroups.com. >>> Visit this group at https://groups.google.com/group/google-appengine. >>> To view this discussion on the web visit https://groups.google.com/d/ms >>> gid/google-appengine/9ae8bdb3-fe84-43d0-b58e-4b2894ec8158% >>> 40googlegroups.com >>> <https://groups.google.com/d/msgid/google-appengine/9ae8bdb3-fe84-43d0-b58e-4b2894ec8158%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- > You received this message because you are subscribed to a topic in the > Google Groups "Google App Engine" group. > To unsubscribe from this topic, visit https://groups.google.com/d/ > topic/google-appengine/M01jjjNc1dQ/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > google-appengine+unsubscr...@googlegroups.com. > To post to this group, send email to google-appengine@googlegroups.com. > Visit this group at https://groups.google.com/group/google-appengine. > To view this discussion on the web visit https://groups.google.com/d/ > msgid/google-appengine/775cc91d-5d04-4c2b-bec6- > 139157889b71%40googlegroups.com > <https://groups.google.com/d/msgid/google-appengine/775cc91d-5d04-4c2b-bec6-139157889b71%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at https://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/CAAxkZQs4B3voiM2E9eEO3vnhj9FD9HMhEJ12o1Rb14GFovSTCw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
