[google-appengine] Re: Uploading large image files and video to Google Cloud Storage

[Kaan Soral]

I'm not commenting on the issue in general, as things got a bit edgy, 
however, my .02:

I use App Engine, or other cloud products, as they make things easy for me, 
for multi-part uploads, my expectancy is having wrapper libraries that make 
multi-part uploads dead-simple (JS, Mobile: iOS, Java, Cordova, possibly 
others)

In fact, while you are at it, it could even be a large/inclusive library 
for all high level things combined into one package, and possibly, one 
could hand-pick certain things and get a smaller library if size is a 
concern

One example was the old Channel JS library, while I haven't used it myself, 
design-wise it was a nice library, tackled a complex system in a simple way

>From a cloud users' perspective, multi-part uploads is indeed a bit 
complex, and icky challenges like these consume a lot of time, and in the 
end, when you look back, you realise you spent a considerable amount of 
time on these kind of integrations/challenges and on a larger scale, it 
probably has a high impact

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/7f3dcd49-e213-4def-8c87-efe914c3544d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Uploading large image files and video to Google Cloud Storage

[Richard Cheesmar]

Oh, and look at the date on the stackoverflow post, it's not like you guys 
haven't had time to fix the issue.


On Monday, February 20, 2017 at 6:47:26 PM UTC+3, Richard Cheesmar wrote:
>
> I am using the standard python app engine environment and currently 
> looking at how one goes about uploading multiple large media files to 
> Google Cloud Storage (Public Readable) using App Engine or the Client 
> directly (preferred).
>
> I currently send a bunch of smaller images (max 20 - between 30 and 100k 
> on average), at the same time directly via a POST to the server. These 
> images are provided by the client and put in my projects default bucket. I 
> handle the requests images using a separate thread and write them one at a 
> time to the cloud and then associate them with an ndb object. This is all 
> fine and dandy when the images are small and do not cause the request to 
> run out of memory or invoke a DeadlineExceededError. 
>
> But what is the best approach for large image files of 20mb+ a piece or 
> video files of up to 1GB in size? Are there efficient ways to do this from 
> the client directly, would this be possible via the Json api ,a resumable 
> upload, for example? If so, are there any clear examples of how to do this 
> purely in javascript on the client? I have looked at the docs but it's not 
> intuitively obvious at least to me.
>
> I have been looking at the possibilities for a day or two but nothing hits 
> you with a clear linear description or approach. I notice in the Google 
> Docs there is a way using PHP to upload via a POST direct from the client...
> https://cloud.google.com/appengine/docs/php/googlestorage/user_upload...Is 
> this just relevant to using PHP on app engine or is there an equivalent to 
> createUploadUrl 
> for python or javascript?
>
>
> Anyway, I'll keep exploring but any pointers would be greatly appreciated.
>
>
> Cheers
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/db93b8af-add7-4f00-a35b-1e7edcba315e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Uploading large image files and video to Google Cloud Storage

[Richard Cheesmar]

You have to be kidding me. Why isn't this plastered all over the page for 
the JSON API? Or better still fixed if it's an issue.

I have spent valuable time and energy on this and now I have to start 
again. This is not amusing, in fact it is near damn right negligence on 
Googles part.

Adam, you wrote this in your first reply... "A cleaner approach is to use 
the Google API JavaScript Client Library 
 
to upload the file using the JSON API 
."

This makes me want to shout and scream. Can't a company that makes driver 
less cars get a itzy bitzy api that touts resumeable uploads to work with 
CORS?


On Monday, February 20, 2017 at 6:47:26 PM UTC+3, Richard Cheesmar wrote:
>
> I am using the standard python app engine environment and currently 
> looking at how one goes about uploading multiple large media files to 
> Google Cloud Storage (Public Readable) using App Engine or the Client 
> directly (preferred).
>
> I currently send a bunch of smaller images (max 20 - between 30 and 100k 
> on average), at the same time directly via a POST to the server. These 
> images are provided by the client and put in my projects default bucket. I 
> handle the requests images using a separate thread and write them one at a 
> time to the cloud and then associate them with an ndb object. This is all 
> fine and dandy when the images are small and do not cause the request to 
> run out of memory or invoke a DeadlineExceededError. 
>
> But what is the best approach for large image files of 20mb+ a piece or 
> video files of up to 1GB in size? Are there efficient ways to do this from 
> the client directly, would this be possible via the Json api ,a resumable 
> upload, for example? If so, are there any clear examples of how to do this 
> purely in javascript on the client? I have looked at the docs but it's not 
> intuitively obvious at least to me.
>
> I have been looking at the possibilities for a day or two but nothing hits 
> you with a clear linear description or approach. I notice in the Google 
> Docs there is a way using PHP to upload via a POST direct from the client...
> https://cloud.google.com/appengine/docs/php/googlestorage/user_upload...Is 
> this just relevant to using PHP on app engine or is there an equivalent to 
> createUploadUrl 
> for python or javascript?
>
>
> Anyway, I'll keep exploring but any pointers would be greatly appreciated.
>
>
> Cheers
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/871a3e5d-bb70-4bc1-a896-7e3b2bc2ea39%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Appengine Endpoints Api Explorer 'time to execute'

[Rajesh Gupta]

Hi,
I am testing the deployed endpoints. Its a simple api with 3 fieds
I am testing using the API explorer console.

Please find the attached screen shot.

In that screen, I fill all the fields and do 'Execute'
The results is displayed along with the 'time to execute'

This time shown varies very widely from 300ms to 6000ms.  why?

I observe the logs on the server side, and the logs time is consistently
between 100ms - 300ms

Why is the 'time to execute' shown on the API's explorer screen varies so
widely?

Real point is, this wide time variation happens in the production
application also.

-- 
Regards,
Rajesh
*www.VeersoftSolutions.com *
*www.GainERP.com *
*Accounting/Inventory/Orders/Sales/Purchase on Google Cloud Platform and
Mobile*

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/CA%2BS7ijbPBd2wY2QGHFpJmH%3DcnHU0ADVJyQAb6_4OtnPU7KyrXQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: How to use the same G. Cloud Datastore for apps in standard and flexible environments?

['Nick (Cloud Platform Support)' via Google App Engine]

Hey Juan,

You're right to point out that our documentation could be better here. 
We're currently working on expanding the documentation on the Datastore 
emulator. Now, as to your main question:

*google-cloud-datastore*

You can use the google-cloud-datastore 
 library to interact 
with the datastore emulator from both standard and flexible environment 
dev_appserver apps. 

*environment variables*

Just be sure to set the environment variables properly by running $(gcloud 
beta emulators datastore env-init). Be sure to run the command exactly like 
that, with the $() included, since this will tell the shell to run the 
commands which are actually output by gcloud beta emulators datastore 
env-init. The commands, which you could also examine and then run yourself, 
one by one, are going to look like this:

export DATASTORE_DATASET=my-project-id
export DATASTORE_EMULATOR_HOST=localhost:8715
export DATASTORE_EMULATOR_HOST_PATH=localhost:8715/datastore
export DATASTORE_HOST=http://localhost:8715
export DATASTORE_PROJECT_ID=my-project-id

*and you're good to go*

When this is done, the python client library linked above will connect to 
the local emulator. See the documentation on the client library (the link 
was above) for the details of coding.

*a small note about the datastore admin viewer in dev*

Currently, the dev_appserver.py admin panel (localhost:8000 usually) 
"datastore" section does not connect properly to the emulator. This is 
documented in this Public Issue Tracker issue I've just filed 
, and you 
can star that issue to receive updates. I hope this has been helpful to 
you, and feel free to ask any further questions you have. I'll be happy to 
assist.

Cheers,

Nick
Cloud Platform Community Support

On Thursday, February 16, 2017 at 7:59:32 AM UTC-5, Juan Antonio Fernández 
Sánchez wrote:
>
> I'm working in a project that use two apps, one deployed in App Engine 
> standard environment and other in flexible env.
>
>
> One service of the app in the standard env and the app in the flex env. 
> must be access to Google Cloud Datastore 
> .
>
> If I use in *local* the emulated datastore that run the dev_server in 
> standard environment, the app in the flex env. can't access to this 
> datastore and neither use the ndb ORM library like the service in standard 
> env.
>
>
> So, to have consistency in the way to access to data, I don't what 
> solution would be better to solve this.
>
> I've been thinking if I could use Google Cloud SDK (gcloud tool) to run 
> emulators 
> datastore and access to this using Cloud Datastore Cient Libraries 
>  ( instead 
> of Google Datastore NDB Client Library 
>  ) in both apps (to 
> use exactly the same way to connect) I think that this way is most easy to 
> develop after in cloud platform.
>
>
> I'm finding some problems to connect with this emulated datastore and I 
> would like to know if my approach is good to follow fighting with the 
> problems or change to another way or idea to run my entire project in local.
>
>
> Any help or idea will be welcome.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/f6d1ef7f-5329-41bd-906a-7048b1163e3a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: How to download data from my datastore to the local machine

['Alex (Cloud Platform Support)' via Google App Engine]

To add up to Nicola’s reply, you may be able to download your data to your 
local machine by opening the local Datastore viewer as proposed on this 
Issue Tracker thread 
. To do 
so, you will have to access the viewer locally at “
http://localhost:8000/_ah/admin/datastore” and get the whole data flushed 
to you local disk.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/bd4708ad-feca-47e8-93bc-2008fd5a8a16%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Uploading large image files and video to Google Cloud Storage

['Adam (Cloud Platform Support)' via Google App Engine]

Have a look at this Stack Overflow answer 

 
for more details. This is an issue specific to the JSON API. In your case, 
does the 'Origin: ' header differ from the initial POST and subsequent PUT 
requests?

On Tuesday, February 28, 2017 at 11:40:33 AM UTC-5, Richard Cheesmar wrote:
>
> Update:
>
> Ok, so after much deliberation about how to simplify the approach, finally 
> I get a method together that I think I can live with, but there is still 
> one problem.
>
> What I am doing to upload a video from the client direct (via localhost) 
> is:
>
> 1. I generate an access token and build a POST request for the initial 
> upload request to the JSON api.
> 2. I make the POST call which successfully returns a location.
> 3. I return the location to the client via a json object
> 4. I make the PUT call in the returned location which starts the upload - 
> this is done using standard ajax via Jquery.
> 5. The video is actually arriving at the bucket but the ajax call returns 
> with an error at the end of the upload
>
> Cross-Origin Request Blocked: The Same Origin Policy disallows reading 
> the remote resource at https://
> www.googleapis.com/upload/storage/v1/b/my-bucket/o?uploadType=resumable=test-vid_id=generate-id.
>  
> (Reason: CORS header ‘Access-Control-Allow-Origin’ missing).
>
>
> Seems strange to me as the video actually gets to the bucket...
>
> For the time being I have added the * to CORS headers on the bucket by 
> using the Google Cloud shell and some instructions from here 
> https://bitmovin.com/faq/how-do-i-set-up-cors-for-my-google-cloud-storage-bucket/
> and this all seems fine. Well it must be otherwise the PUT wouldn't put 
> the video, if the CORS headers were not in place.
>
> I have no idea why this is, can someone enlighten me?
>
>
> On Monday, February 20, 2017 at 6:47:26 PM UTC+3, Richard Cheesmar wrote:
>>
>> I am using the standard python app engine environment and currently 
>> looking at how one goes about uploading multiple large media files to 
>> Google Cloud Storage (Public Readable) using App Engine or the Client 
>> directly (preferred).
>>
>> I currently send a bunch of smaller images (max 20 - between 30 and 100k 
>> on average), at the same time directly via a POST to the server. These 
>> images are provided by the client and put in my projects default bucket. I 
>> handle the requests images using a separate thread and write them one at a 
>> time to the cloud and then associate them with an ndb object. This is all 
>> fine and dandy when the images are small and do not cause the request to 
>> run out of memory or invoke a DeadlineExceededError. 
>>
>> But what is the best approach for large image files of 20mb+ a piece or 
>> video files of up to 1GB in size? Are there efficient ways to do this from 
>> the client directly, would this be possible via the Json api ,a resumable 
>> upload, for example? If so, are there any clear examples of how to do this 
>> purely in javascript on the client? I have looked at the docs but it's not 
>> intuitively obvious at least to me.
>>
>> I have been looking at the possibilities for a day or two but nothing 
>> hits you with a clear linear description or approach. I notice in the 
>> Google Docs there is a way using PHP to upload via a POST direct from the 
>> client...
>> https://cloud.google.com/appengine/docs/php/googlestorage/user_upload...Is 
>> this just relevant to using PHP on app engine or is there an equivalent to 
>> createUploadUrl 
>> for python or javascript?
>>
>>
>> Anyway, I'll keep exploring but any pointers would be greatly appreciated.
>>
>>
>> Cheers
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/26350cef-7d03-4f43-9f1f-b1f08a42f579%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: from .appspot.com to .appspot-preview.com

['Adam (Cloud Platform Support)' via Google App Engine]

Please see the main Groups thread 

 
for updates on the redirect issue which provides more details. Traffic will 
be redirected back to "appspot.com" and is only going to 
"appspot-preview.com" on a temporary basis.

On Monday, February 20, 2017 at 1:20:47 PM UTC-5, Zip Zit wrote:
>
> Adam.   that word "preview" makes me very nervous.  It implies that there 
> are limited services, or that the object viewed is not production ready, or 
> the object viewed in this guise is for temporary use only...
>
> What is the catch?  Why did they choose that word? 
>
>
>
>
>
> . On Saturday, February 18, 2017 at 11:25:33 AM UTC-8, Adam (Cloud 
> Platform Support) wrote:
>>
>> A change was made causing some App Engine Flexible apps to redirect to 
>> appspot-preview.com. If you have any client applications that point to 
>> the old domain, you'll need to update them to use the new URL.
>>
>>
>>>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/7ac3595d-4e6f-43dd-8c1c-d76e4601969e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Uploading large image files and video to Google Cloud Storage

[Richard Cheesmar]

Update:

Ok, so after much deliberation about how to simplify the approach, finally 
I get a method together that I think I can live with, but there is still 
one problem.

What I am doing to upload a video from the client direct (via localhost) is:

1. I generate an access token and build a POST request for the initial 
upload request to the JSON api.
2. I make the POST call which successfully returns a location.
3. I return the location to the client via a json object
4. I make the PUT call in the returned location which starts the upload - 
this is done using standard ajax via Jquery.
5. The video is actually arriving at the bucket but the ajax call returns 
with an error at the end of the upload

Cross-Origin Request Blocked: The Same Origin Policy disallows reading the 
remote resource at 
https://www.googleapis.com/upload/storage/v1/b/my-bucket/o?uploadType=resumable=test-vid_id=generate-id.
 
(Reason: CORS header ‘Access-Control-Allow-Origin’ missing).


Seems strange to me as the video actually gets to the bucket...

For the time being I have added the * to CORS headers on the bucket by 
using the Google Cloud shell and some instructions from here 
https://bitmovin.com/faq/how-do-i-set-up-cors-for-my-google-cloud-storage-bucket/
and this all seems fine. Well it must be otherwise the PUT wouldn't put the 
video, if the CORS headers were not in place.

I have no idea why this is, can someone enlighten me?


On Monday, February 20, 2017 at 6:47:26 PM UTC+3, Richard Cheesmar wrote:
>
> I am using the standard python app engine environment and currently 
> looking at how one goes about uploading multiple large media files to 
> Google Cloud Storage (Public Readable) using App Engine or the Client 
> directly (preferred).
>
> I currently send a bunch of smaller images (max 20 - between 30 and 100k 
> on average), at the same time directly via a POST to the server. These 
> images are provided by the client and put in my projects default bucket. I 
> handle the requests images using a separate thread and write them one at a 
> time to the cloud and then associate them with an ndb object. This is all 
> fine and dandy when the images are small and do not cause the request to 
> run out of memory or invoke a DeadlineExceededError. 
>
> But what is the best approach for large image files of 20mb+ a piece or 
> video files of up to 1GB in size? Are there efficient ways to do this from 
> the client directly, would this be possible via the Json api ,a resumable 
> upload, for example? If so, are there any clear examples of how to do this 
> purely in javascript on the client? I have looked at the docs but it's not 
> intuitively obvious at least to me.
>
> I have been looking at the possibilities for a day or two but nothing hits 
> you with a clear linear description or approach. I notice in the Google 
> Docs there is a way using PHP to upload via a POST direct from the client...
> https://cloud.google.com/appengine/docs/php/googlestorage/user_upload...Is 
> this just relevant to using PHP on app engine or is there an equivalent to 
> createUploadUrl 
> for python or javascript?
>
>
> Anyway, I'll keep exploring but any pointers would be greatly appreciated.
>
>
> Cheers
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/44fada75-766e-4579-a4e8-cfe609607f51%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Production SDK stuck at 1.9.48?

['George (Cloud Platform Support)' via Google App Engine]

What is the output of the gcloud info command? 

Have you run gcloud components update recently? 

Providing whole lines of logs might help us in identifying the proper 
context of the error. 

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/5209c206-4543-4565-9bf0-eb295659ce57%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] 10mib request limit being hit with call to sendgrid api - Is there another way to make the sendgrid api call with attachments?

[Rob Curtis]

Hi, 

We have an application where a list attachments must be sent in a single 
mail to 1 address. 
Sendgrid has 30MB limit on attachment size, but Appengine has a 10MB limit 
on request size.
The sendgrid api requires base64 encoding of the files, which results in 
the request size being larger than 10MB (if the files are large)


   1. Is there an alternative method for attaching files to a mail in 
   sendgrid?
   2. Is there means to change the request size in AppEngine or use some 
   mechanism for calling the sendgrid API?
   
Thanks
Rob

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/9047aaf7-d566-458d-a18e-7bcbe87dab5e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: [Stackdriver Trace] Cannot have multiple spans with the same name in a trace

['Nick (Cloud Platform Support)' via Google App Engine]

Hey Adrian,

Not to worry, I ought to have updated here to let you know - we're working 
on diagnosing the issue and preparing an internal report for it, and will 
post a public issue tracker thread and link it here once we've got that 
squared away. Shouldn't be too much longer. Thanks for checking! In future, 
you can post directly there, but it's no problem for us to create a thread 
this way in this case.

Cheers,

Nick
Cloud Platform Community Support

On Tuesday, February 28, 2017 at 9:06:24 AM UTC-5, Adrian Bogatu wrote:
>
> Are there any updates regarding this issue? Do you want me to post it 
> directly to Google Code?
>
> On Friday, February 24, 2017 at 5:51:28 PM UTC+2, Nick (Cloud Platform 
> Support) wrote:
>>
>> Hey Adrian,
>>
>> While opening new projects on Google Code was discontinued, we continue 
>> to operate Public Issue Trackers for many of our projects there as the 
>> official forums for issue reporting, and we monitor them daily. Here are a 
>> few of them:
>>
>> App Engine 
>> Compute Engine 
>> 
>> Cloud SDK 
>> Cloud SQL 
>> BigQuery 
>> Cloud Platform 
>> 
>>
>> Cheers,
>>
>> Nick
>> Cloud Platform Community Support
>>
>> On Friday, February 24, 2017 at 4:05:05 AM UTC-5, Adrian Bogatu wrote:
>>>
>>> Hi, Nick,
>>>
>>> Thanks for the answer. Honestly, at first, I wanted to post on Google 
>>> Code, but I know it has been discontinued in 2016 and I wasn't sure it was 
>>> tracked anymore.
>>>
>>> Thanks,
>>> Adrian
>>>
>>> On Thursday, February 23, 2017 at 10:16:04 PM UTC+2, Nick (Cloud 
>>> Platform Support) wrote:

 Hey Adrian,

 Thanks for reporting this. I've just about reproduced this behaviour 
 and will shortly post a link to a Public Issue Tracker 
  thread 
 you can follow for updates. 

 In general, posts like this, which look like a post that belongs in the 
 Public Issue Tracker, should be posted there. This Google Groups forum is 
 meant for more general and high level discussion of the App Engine 
 platform 
 and services rather than specific-issue technical reporting. We monitor 
 there regularly and will quickly get the issue acknowledged and tracked.

 I'll update this thread with a link to the thread once it's made, or if 
 I need any further clarification on reproducing this behaviour.

 Cheers,

 Nick
 Cloud Platform Community Support

 On Thursday, February 23, 2017 at 11:13:27 AM UTC-5, Adrian Bogatu 
 wrote:
>
> Hi,
>
> Situation: When sending a trace containing 2 spans (a root-span and a 
> child span) that have the same span name, the two spans are merged 
> together 
> and only one span is displayed. In addition, some of the fields are 
> overwritten and information is lost.
>
> See example in the attachment (print screen from developer console in 
> google cloud platform), where two spans both having the name "get_test" 
> are 
> sent in a trace to Stackdriver Trace: one client root-span and one server 
> child span. The different annotations of both the spans are merged, but 
> some shared annotations are overwritten by the client root-span (e.g. 
> zipkin.io/span.kind is "client"). I expected to have 2 spans: one 
> with the tag-value "client" and one "server".
>
> This issue is because, I am guessing, the backend considers the span's 
> uniqueness based on the span's name. I don't think that's correct, 
> instead 
> the unique *span id* should be used to track spans.
>
> There are many cases when one wants to use the same name for multiple 
> spans in a trace (e.g. span_name="get_users" to be the same for client 
> and 
> server; standard annotations in libraries like span_name="mysql-client" 
> when you have multiple sequential MySQL request). In those cases, only 
> one 
> span with that specified name will exist, and annotations will be lost.
>
> I used both a custom stackdriver trace client in Go (that uses Go 
> package cloudtrace/v1 
> ) 
> and the relatively new google's zipkin collector for stackdriver trace 
> .
>
> Thanks,
> Adrian
>


-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to 

[google-appengine] Re: [Stackdriver Trace] Cannot have multiple spans with the same name in a trace

[Adrian Bogatu]

Are there any updates regarding this issue? Do you want me to post it 
directly to Google Code?

On Friday, February 24, 2017 at 5:51:28 PM UTC+2, Nick (Cloud Platform 
Support) wrote:
>
> Hey Adrian,
>
> While opening new projects on Google Code was discontinued, we continue to 
> operate Public Issue Trackers for many of our projects there as the 
> official forums for issue reporting, and we monitor them daily. Here are a 
> few of them:
>
> App Engine 
> Compute Engine 
> 
> Cloud SDK 
> Cloud SQL 
> BigQuery 
> Cloud Platform 
> 
>
> Cheers,
>
> Nick
> Cloud Platform Community Support
>
> On Friday, February 24, 2017 at 4:05:05 AM UTC-5, Adrian Bogatu wrote:
>>
>> Hi, Nick,
>>
>> Thanks for the answer. Honestly, at first, I wanted to post on Google 
>> Code, but I know it has been discontinued in 2016 and I wasn't sure it was 
>> tracked anymore.
>>
>> Thanks,
>> Adrian
>>
>> On Thursday, February 23, 2017 at 10:16:04 PM UTC+2, Nick (Cloud Platform 
>> Support) wrote:
>>>
>>> Hey Adrian,
>>>
>>> Thanks for reporting this. I've just about reproduced this behaviour and 
>>> will shortly post a link to a Public Issue Tracker 
>>>  thread 
>>> you can follow for updates. 
>>>
>>> In general, posts like this, which look like a post that belongs in the 
>>> Public Issue Tracker, should be posted there. This Google Groups forum is 
>>> meant for more general and high level discussion of the App Engine platform 
>>> and services rather than specific-issue technical reporting. We monitor 
>>> there regularly and will quickly get the issue acknowledged and tracked.
>>>
>>> I'll update this thread with a link to the thread once it's made, or if 
>>> I need any further clarification on reproducing this behaviour.
>>>
>>> Cheers,
>>>
>>> Nick
>>> Cloud Platform Community Support
>>>
>>> On Thursday, February 23, 2017 at 11:13:27 AM UTC-5, Adrian Bogatu wrote:

 Hi,

 Situation: When sending a trace containing 2 spans (a root-span and a 
 child span) that have the same span name, the two spans are merged 
 together 
 and only one span is displayed. In addition, some of the fields are 
 overwritten and information is lost.

 See example in the attachment (print screen from developer console in 
 google cloud platform), where two spans both having the name "get_test" 
 are 
 sent in a trace to Stackdriver Trace: one client root-span and one server 
 child span. The different annotations of both the spans are merged, but 
 some shared annotations are overwritten by the client root-span (e.g. 
 zipkin.io/span.kind is "client"). I expected to have 2 spans: one with 
 the tag-value "client" and one "server".

 This issue is because, I am guessing, the backend considers the span's 
 uniqueness based on the span's name. I don't think that's correct, instead 
 the unique *span id* should be used to track spans.

 There are many cases when one wants to use the same name for multiple 
 spans in a trace (e.g. span_name="get_users" to be the same for client and 
 server; standard annotations in libraries like span_name="mysql-client" 
 when you have multiple sequential MySQL request). In those cases, only one 
 span with that specified name will exist, and annotations will be lost.

 I used both a custom stackdriver trace client in Go (that uses Go 
 package cloudtrace/v1 
 ) 
 and the relatively new google's zipkin collector for stackdriver trace 
 .

 Thanks,
 Adrian

>>>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/a3ec4b14-793d-4e42-94b8-e449d6be9858%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Production SDK stuck at 1.9.48?

[Attila-Mihaly Balazs]

Hello, 

I see the following in my logs: "app_engine_release=1.9.48". This is for a 
production Python application running in Appengine standard in the EU 
region. I'm wondering why it uses 1.9.48, when 1.9.50 came out more than a 
month ago?

Regards,
Attila

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/ab6dcfd6-1b58-4c1b-a09e-e6f6f76ea50d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Re: "Found more than 100 URLMap entries"

[Joshua Fox]

Thank you. What is the role of

 

Does that count as one URLMap or does that somehow expand so that every JSP
and/or every  file gets counted as one URLMap?

On Mon, Feb 27, 2017 at 11:21 PM, 'Nicholas (Google Cloud Support)' via
Google App Engine  wrote:

> Hey there,
>
> Thanks for providing me with your *appengine-web.xml* and *web.xml*.  At
> the moment, the platform limits the number of *URLMap* handlers to *100*.
> This can be found in the *appinfo.py* file in your own gcloud SDK
> installation (
> */platform/google_appengine/google/appengine/api/appinfo.py*
> on line *2089*).  The definition of *URLMap* can be found in the same
> find with a helpful description on line *917*.
>
> When it comes to Java applications, filter mappings, servlet mappings are
> the main contributors to URLMap handlers.  Basically, any entry that maps a
> URL to an endpoint will be a URLMap.  Looking at some of your servlet
> mappings, you may be able to save some maps in the following ways
>
>- Break down the application into many smaller services.  Your REST
>API servlet(s) for instance could be deploy as a separate service, thus
>taking advantage of the subdomain URL-naming scheme of App Engine services
>like api.your-domain.com/v1/useful-method.  Essentially, the change
>requires that part of the URL be moved from the path to the subdomain.
>- Use a CDN or Cloud Storage to serve static resources.  In your
>appengine-web.xml, I could see several .  Cloud Storage is an
>excellent service for serving static resources.
>- In some cases, a single servlet could serve related paths such as
>/_ah/*.  This example can save you from having 3 single endpoints mappings
>for /_ah/start, /_ah/health, /_ah/stop.
>
> I hope this addresses your questions and helps you trim down some URL
> mappings.
>
> On Friday, February 24, 2017 at 4:02:13 PM UTC-5, Nicholas (Google Cloud
> Support) wrote:
>>
>> There does not seem to be any explicit documentation about the URLMap
>> limitations for GAE nor a breakdown of which XML elements count towards
>> this limit.  The most relevant documentation I could find is the HTTP/S
>> load balancer documentation about URL Maps
>>  and
>> the URLMap resource definition
>> .  These
>> are documented in the context of the a load balancer and not an App Engine
>> application.  The load balancer documentation does not describe which XML
>> elements , , etc. nor which app.yaml
>> handlers count as a URLMap.
>>
>> I can't define what constitutes a URLMap until I dig into this a little
>> deeper.  I suspect this will lead to feedback for our documentation team.
>> For the time being, perhaps we can help identify opportunities to reduce
>> the number of maps you have using more comprehensive regular expressions
>> and such.  If sensitive and not redacted, feel free to reply privately to
>> me here with your web.xml and I'll see what I can do to help.
>>
>> On Thursday, February 23, 2017 at 4:53:29 AM UTC-5, Joshua Fox wrote:
>>>
>>> Where can I find documentation on URLMap count, as reflected in this
>>> error in deployment to GAE?
>>>
>>>   "app.yaml... Found more than 100 URLMap entries in
>>> application configuration"
>>>
>>> What is included in the URLMap count?
>>>
>>> Which elements in *web.xml *are included? *servlet-mapping*,
>>> *security-constraint*, *filter-mapping*, or something else?
>>>
>>> Which elements in *appengine-web.xml ? *
>>>
>>> For some reason, removing * *from
>>> *appengine-web.xml *actually allows deployment for us. Does that path
>>> count as a single URLMap? Or as one per static  file? Per JSP?
>>>
>> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to google-appengine+unsubscr...@googlegroups.com.
> To post to this group, send email to google-appengine@googlegroups.com.
> Visit this group at https://groups.google.com/group/google-appengine.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/google-appengine/b08cff4d-b9f6-4a7b-b42a-
> 0adc3eda0d51%40googlegroups.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>



-- 
*JOSHUA FOX*
Principal Software Architect | Freightos


☏* (Israel): *+972-545691165 | ☏* (US)*:  +1-3123400953* | Skype*:
joshuafox.freightos
Smoother shipping with the world's online freight marketplace.
Online Marketplace

 / In the News