[pylons-discuss] Re: API file upload
I was confused on this years ago. The problem is in naming... `curl` is concerned with the HTTP method `POST`. Let's call that `HTTP POST`. If you look at the `curl` documents for the difference on the various `--data-` options. Those options will `HTTP POST` data in different formats and encodings. WebOb's `request.POST` isn't the same as `HTTP POST` though; it's a convenience method that pre-processing structured form data submissions via `HTTP POST` (see https://docs.pylonsproject.org/projects/webob/en/stable/api/request.html#webob.request.BaseRequest.POST) If you look at the source of webob's POST https://github.com/Pylons/webob/blob/master/src/webob/request.py#L749-L797 you'll see that it's trying to parse the raw data such as `self.body_file_raw` and `self.body_file` into form values. Since your `HTTP POST` data isn't a form, you want to avoid all that processing and just operate on the raw data. IIRC `body_file_raw` is better for uploads, because it avoids reading/consuming the body (which may be a stream instead of a static file). I default to `body_file_raw` out of habit. -- You received this message because you are subscribed to the Google Groups "pylons-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discuss+unsubscr...@googlegroups.com. To post to this group, send email to pylons-discuss@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/3d8848d6-87cc-4fab-9e46-eede7457e878%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[pylons-discuss] Re: API file upload
OK, I'm getting there, althought I'm still confused a bit. In WebOb docs I found request.body_file, request.body_file_raw, request.body_file_seekable. In multipart's request.POST, I'm doing: file_obj.seek(0, 2) file_size = file_obj.tell() file_obj.seek(0) Should I be using seekable or raw for this? Zsolt On 17 April 2018 at 15:10, Zsolt Ero wrote: > I've realised the following: > 1. If I don't specify Content-Type, curl defaults to x-www-form-urlencoded > 2. What I thought is the binary file's contents as a string is > actually not working reliably. On an XML upload of a single file I get > thousands of items and request.POST.items() looks like: > [' 2. The binary string I can use is actually request.body. Still, is > there any potential problems with handling this as string and not as a > file object? > > Zsolt > > On 17 April 2018 at 14:24, Zsolt Ero wrote: >> Hi, >> >> I'm trying to implement an API to a website which didn't have an API >> yet. It's purpose will be to allow file uploads from 3rd party native >> apps. >> >> I'd like to implement the API like Dropbox v2 API, just as a good >> reference for API design. >> >> It's upload endpoint has the following specs: >> https://www.dropbox.com/developers/documentation/http/documentation#files-upload >> >> And the following cURL example: >> >> curl -X POST https://content.dropboxapi.com/2/files/upload \ >> --header "Authorization: Bearer " \ >> --header "Dropbox-API-Arg: {\"path\": >> \"/Homework/math/Matrices.txt\",\"mode\": \"add\",\"autorename\": >> true,\"mute\": false}" \ >> --header "Content-Type: application/octet-stream" \ >> --data-binary @local_file.txt >> >> Now my problem is that I've implemented most parts, but if the request >> has --header "Content-Type: application/octet-stream" then WebOb >> doesn't allow using request.POST. It says: >> >> Not an HTML form submission (Content-Type: application/octet-stream) >> >> If I remove that header, I can use request.POST.keys()[0] to read the >> contents of the file as a string. >> >> My question is: >> 1. What am I doing wrong that the Content-Type is not supported? >> 2. Is there any downside of having an up-to-100 MB file as a string? >> Wouldn't the HTML multipart-form-data's file solution use less memory? >> Can I make WebOb handle this kind of uploads like it does multipart >> ones? >> >> Zsolt -- You received this message because you are subscribed to the Google Groups "pylons-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discuss+unsubscr...@googlegroups.com. To post to this group, send email to pylons-discuss@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/CAKw-smBHD9Dg45R1UCu47NTQzYR1H1xdq8_RFyG4E0EKb0zsDw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
[pylons-discuss] Re: API file upload
I've realised the following: 1. If I don't specify Content-Type, curl defaults to x-www-form-urlencoded 2. What I thought is the binary file's contents as a string is actually not working reliably. On an XML upload of a single file I get thousands of items and request.POST.items() looks like: [' wrote: > Hi, > > I'm trying to implement an API to a website which didn't have an API > yet. It's purpose will be to allow file uploads from 3rd party native > apps. > > I'd like to implement the API like Dropbox v2 API, just as a good > reference for API design. > > It's upload endpoint has the following specs: > https://www.dropbox.com/developers/documentation/http/documentation#files-upload > > And the following cURL example: > > curl -X POST https://content.dropboxapi.com/2/files/upload \ > --header "Authorization: Bearer " \ > --header "Dropbox-API-Arg: {\"path\": > \"/Homework/math/Matrices.txt\",\"mode\": \"add\",\"autorename\": > true,\"mute\": false}" \ > --header "Content-Type: application/octet-stream" \ > --data-binary @local_file.txt > > Now my problem is that I've implemented most parts, but if the request > has --header "Content-Type: application/octet-stream" then WebOb > doesn't allow using request.POST. It says: > > Not an HTML form submission (Content-Type: application/octet-stream) > > If I remove that header, I can use request.POST.keys()[0] to read the > contents of the file as a string. > > My question is: > 1. What am I doing wrong that the Content-Type is not supported? > 2. Is there any downside of having an up-to-100 MB file as a string? > Wouldn't the HTML multipart-form-data's file solution use less memory? > Can I make WebOb handle this kind of uploads like it does multipart > ones? > > Zsolt -- You received this message because you are subscribed to the Google Groups "pylons-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discuss+unsubscr...@googlegroups.com. To post to this group, send email to pylons-discuss@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/CAKw-smA8HwKK5o3RNhQPfwAzo-ATgE1zrzj6TrwsLOzZjcka-A%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.