On Thursday, October 15, 2015 at 12:24:48 AM UTC-7, Johannes Rudolph wrote: > > Have you seen/tried > http://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-streaming.html? It > seems to suggest that chunked transfer-encoding is supported, though in a > bit complicated format where on top of chunked transfer-encoding AWS seems > to introduce their own chunked content-encoding layer where each chunk is > signed separately. >
Yes, but I don't know if this works for S3 HTTP POSTs, and I don't need each chunk separately signed. > > >> The point of my original post was that it seems the kaka-http library has >> made a policy decision >> > > WDYM? akka-http supports both `FormData` = www-url-encoding and > `multipart/formdata` encodings, so, I wonder which "policy decision" you > mean? > Perhaps I just couldn't find out how to do it (except very manually as you saw in my code sample). You can easily create a single HTTP entity via Marshal(formData).to[HttpEntity], where that entity will encode all form parameters/values as a query string. How do you, using existing marshallers, create a List[HttpEntity] given a FormData object? The bias is that there is no direct way to do this. I can clearly map over all the fields in the FormData myself and marshall each name/value pair to an HttpEntity. But by default, doing so uses ContentType.`text/plain`. You need your own custom logic to force the content type to NoContentType, which is what we need in this case. > > >> that the only way to convert a FormData to an entity (in a multi-part >> MIME body) is to create a single part, using www-url-encoding, and include >> all fields in query string syntax. This doesn’t work for AWS and therefore >> it seems a limiting policy decision. >> > > Yes, because `scaladsl.model.FormData` models www-url-encoding and not > multipart/formdata. You seem to suggest that there should be a more general > encoding that would allow marshalling to either of both variants. In any > case, there's no policy decision but, as explained above, two different > models for two different kinds of modelling things. > That is the bias I'm talking about. FormData is biased toward www-url-encoding rather than multipart/formdata. Both should be provided. > > It should be equally possible/easy to convert a FormData to a collection >> of parts. >> > > Yes, that could be useful, or otherwise a more general representation of > form data that unifies both kinds. > > >> The requirement that there should be a Content-Type header in these >> parts, also should not be dictated by the framework. >> > > I see that you have to fight a bit to get akka-http to render exactly > what you want, but the reason seems to be mostly that AWS has very specific > rules and interpretations of how to use HTTP. > Actually, AWS is expecting an entity that is exactly what all browsers would generate when you include one or more files in an HTTP form. All the form fields including the files go in separate multi-part mime parts. The files have a Content-Type header and the non-files don't. Now I don't have any issue with the non-strict bias in akka-http. That is the more general, and more async-friendly approach. So it doesn't bother me that you have to do something special to get the various entities to be strict. Here, an expression that maps over the body parts converting each from non-strict to strict and returning a Future[Multipart.FormData] (or whatever it would be) would be fine. akka-http is a general purpose HTTP library and cannot foresee all possible > deviations or additional constraints HTTP APIs try to enforce. So, in the > end, akka-http should make it possible to connect to these APIs (as long as > they support HTTP to a large enough degree) but it may mean that the extra > complexity the API enforces lies in your code. You could see that as a > feature. > I don't disagree with anything you've said there. But there is a bias toward www-url-encode form data. And it would be helpful if akka-http made it as easy to create Multipart.FormData entities. > > That said, I'm pretty sure that there's some sugar missing in the APIs > that would help you build those requests more easily. If you can distill > the generic helpers that are missing from your particular use case we could > discuss how to improve things. > > Here are some things I can see: > > * you can directly build a `Multipart.FormData.Strict` which can be > directly converted into a Strict entity, I guess one problem that we could > solve is that there's only a non-strict FormData.BodyPart.fromFile` which > builds a streamed part to prevent loading the complete file into memory. > There's no `FormData.BodyPart.fromFile` that would actually load the file > and create a strict version of it. We could add that (even if it wouldn't > be recommended to load files into memory...) > * having to run marshalling and unmarshalling manually could be replaced > by some sugar > Yes, that would be good. > * dealing with those chains of `T => Future[U]` functions is cumbersome, > for this and the previous point, we had the simple `pipelining` DSL in > spray which seems to have been lost in parts in the transition to akka-http > Yes. It took me a while to figure out how to change the collections of futures to a future collection of non-futures. But I learned a lot in the process! > > Btw. I don't think your code is too bad, if you break it down into methods > for every step and put it into a utility object, you can reuse it and don't > need to deal with it any more. > Thanks. I'll think about how to do break up that big/nasty for, as well as the original creation of the Name -> Entity map. -- Eric -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
