[google-appengine] Re: Uploading large image files and video to Google Cloud Storage

btw here is the client side code to give you a full picture. var xhr = new XMLHttpRequest(); xhr.open("POST", {url_obtained_from_the_server}, true); xhr.onload = function(e) { if (e.target.status < 400) { var location = e.target.getResponseHeader('Location'); this.url = l

[google-appengine] Re: Uploading large image files and video to Google Cloud Storage

Thanks for the clarification. I am trying to prepare a signed URL on the server side so that the client can initiate the request. I have come up with the following Java code but when the client makes the request to the signed URL, it fails. Could you/anyone please take a quick look and tell me

[google-appengine] Re: Uploading large image files and video to Google Cloud Storage

The signed URL that you pass back to the client provides a way for an *unauthenticated* user (to GCS) to upload files *directly* to a GCS bucket. Without this, every user would need an account and permissions setup or all the files would have to go through your own service instead. On Tuesday,

[google-appengine] Re: Uploading large image files and video to Google Cloud Storage

Re: Resumable Uploads, 1. Created a signed URL on https://www.googleapis.com/upload/storage/v1/b/myBucket/o?uploadType=resumable&name=myObject 2.To get a resumable session URI, sent a post request to the signed URL (without the Authorization header) and it responds with an HTTP 401 error b

[google-appengine] Re: Uploading large image files and video to Google Cloud Storage

Just a quick note: we're using FileStack for the uploads (https://www.filestack.com/) and it works pretty well - they have a Google Storage integration and I do believe they also have a mobile solution (though we're not using that ATM). My only gripe is that they require "storage admin" access

[google-appengine] Re: Uploading large image files and video to Google Cloud Storage

Glad to hear you got it working. Regarding my last response, it's not an issue in that its a bug that needs to be fixed, I meant that it's an issue that people encounter when doing resumable uploads with the JSON API. The behavior is mentioned in the callout in the CORS documentation

[google-appengine] Re: Uploading large image files and video to Google Cloud Storage

Was just about to turn the computer off after winding down reading some non tech stuff. I had everything the origin idea. Wow, see what blind crazy fuzzy stress does to the beautiful mind...Messes it right up. Thanks, appreciated. It works. phew. On Monday, February 20, 2017 at 6:47:26 PM UTC+

[google-appengine] Re: Uploading large image files and video to Google Cloud Storage

Ha, then I remembered I could scroll up (duh!) So for Python, you'll have something like this: # Get the origin header that the client sends to your API when it wants to initiate an upload: origin = request.headers.get('origin') # Include it as a header in the GCS signing request: headers = {

[google-appengine] Re: Uploading large image files and video to Google Cloud Storage

If you had everything working other than that JSON + CORS issue (i.e. signing a URL on the server and doing a PUT for the resumable upload) then all you need to do to make it work should be to pass the origin header from the client in that initial signing request. As a general principle, I'd tr

[google-appengine] Re: Uploading large image files and video to Google Cloud Storage

Simon and Adam, I deleted my initial reply, kinda got tetchy. Yes, you are correct I did get rapid responses, this forum has improved dramatically over time. I also accept that i did get pretty tetchy, however, I'm going to write that off as tech stress. I am sure that Google support staff are

[google-appengine] Re: Uploading large image files and video to Google Cloud Storage

Hi, Simon, Yes, you are correct that some help has been forthcoming. Having said that the help was somewhat disparate in nature. On top of which I have numerous stacks to contend with and can't be king of them all. Still, I thought I managed to find a solution and was content, but alas no, and

[google-appengine] Re: Uploading large image files and video to Google Cloud Storage

As with any platform, experience and knowledge is extremely important and has a huge impact on how long it takes to do things. It can be frustrating to run into issues and stressful if we've promised a deliverable based on a tech stack or service we might not be completely familiar with. I thin

[google-appengine] Re: Uploading large image files and video to Google Cloud Storage

Kaan, You're spot on regarding impact. You spend most of your time trying to get over hurdles like these, whilst you should be spending time on the core of the product functionality. I personally think that the web and the cloud have become way to complex in this regard and feel it's up to comp

[google-appengine] Re: Uploading large image files and video to Google Cloud Storage

I'm not commenting on the issue in general, as things got a bit edgy, however, my .02: I use App Engine, or other cloud products, as they make things easy for me, for multi-part uploads, my expectancy is having wrapper libraries that make multi-part uploads dead-simple (JS, Mobile: iOS, Java, C

[google-appengine] Re: Uploading large image files and video to Google Cloud Storage

Oh, and look at the date on the stackoverflow post, it's not like you guys haven't had time to fix the issue. On Monday, February 20, 2017 at 6:47:26 PM UTC+3, Richard Cheesmar wrote: > > I am using the standard python app engine environment and currently > looking at how one goes about uploadi

[google-appengine] Re: Uploading large image files and video to Google Cloud Storage

You have to be kidding me. Why isn't this plastered all over the page for the JSON API? Or better still fixed if it's an issue. I have spent valuable time and energy on this and now I have to start again. This is not amusing, in fact it is near damn right negligence on Googles part. Adam, you

[google-appengine] Re: Uploading large image files and video to Google Cloud Storage

Have a look at this Stack Overflow answer for more details. This is an issue specific to the JSON API. In your case, does the 'Origin: ' header differ from the initial POS

[google-appengine] Re: Uploading large image files and video to Google Cloud Storage

Update: Ok, so after much deliberation about how to simplify the approach, finally I get a method together that I think I can live with, but there is still one problem. What I am doing to upload a video from the client direct (via localhost) is: 1. I generate an access token and build a POST r

[google-appengine] Re: Uploading large image files and video to Google Cloud Storage

Ok, thank you guys, time to just get on with it. On Monday, February 20, 2017 at 5:47:26 PM UTC+2, Richard Cheesmar wrote: > > I am using the standard python app engine environment and currently > looking at how one goes about uploading multiple large media files to > Google Cloud Storage (Pub

[google-appengine] Re: Uploading large image files and video to Google Cloud Storage

To answer your other questions: 1) Does that mean the JSON API will be deprecated at some point? The documentation mentions using the Google Cloud Storage Client libraries, which themselves use the JSON API, so no. 2) Why there is not an equivalent to the CloudStorageTools file upload for other

[google-appengine] Re: Uploading large image files and video to Google Cloud Storage

Ok, looks like you can only do this with signed requests On Monday, February 20, 2017 at 5:47:26 PM UTC+2, Richard Cheesmar wrote: > > I am using the standard python app engine environment and currently > looking at how one goes about uploading multiple large media files to > Google Cloud Storag

[google-appengine] Re: Uploading large image files and video to Google Cloud Storage

Yes, your app creates the signed URL which can be based on whatever authentication you want or already have. That's the point of the signed URLs - the app is giving authorization to upload a file, the user doesn't need to authenticate to the storage system at all. On Friday, 24 February 2017

[google-appengine] Re: Uploading large image files and video to Google Cloud Storage

It looks like JSON api for javascript require some form of client oauth for uploads and this is exactly what I don't want. I want any user using my system to be able to upload direct to my GCS without the necessity to have a Google account. Is this possible? On Monday, February 20, 2017 at 5:4

[google-appengine] Re: Uploading large image files and video to Google Cloud Storage

Tanks, Simon, I'll check it out. On Monday, February 20, 2017 at 5:47:26 PM UTC+2, Richard Cheesmar wrote: > > I am using the standard python app engine environment and currently > looking at how one goes about uploading multiple large media files to > Google Cloud Storage (Public Readable) usin

[google-appengine] Re: Uploading large image files and video to Google Cloud Storage

I'm pretty sure the YouTube uploading uses this (my upload code ended up very similar), so this example should give you something you can adapt: https://developers.google.com/youtube/v3/code_samples/javascript#do_resumable_uploads_with_cors (a newer version could be a bit simpler with newer web

[google-appengine] Re: Uploading large image files and video to Google Cloud Storage

Simon, can you point me to a good example in javascript of coding the resumable upload. On Monday, February 20, 2017 at 5:47:26 PM UTC+2, Richard Cheesmar wrote: > > I am using the standard python app engine environment and currently > looking at how one goes about uploading multiple large media

[google-appengine] Re: Uploading large image files and video to Google Cloud Storage

Just to add my vote for the resumable upload approach. If you have large files, you don't want to use POST because any failure and you need to restart the entire thing. With the resumable upload your AppEngine instance signs the request and gives you a URL that you can then upload to - your fil

[google-appengine] Re: Uploading large image files and video to Google Cloud Storage

Thanks, Adam, yes seen the POST object call to the XML API , had issues testing this from the localhost, is that possible, if so will look into it again. The issue for me with the JSON APi is that i have to use a resumable upload so it

[google-appengine] Re: Uploading large image files and video to Google Cloud Storage

The simplest upload option is the POST object call to the XML API , which accepts form data and allows you to upload a file without any JavaScript using just a tag with the 'action=' attribute set to the Cloud Storage API Endpoint, eg.