Thanks. I think I see the problem. This is a bug in Jetty. https://github.com/eclipse/jetty.project/blob/master/jetty-http/src/main/resources/org/eclipse/jetty/http/encoding.properties only includes text/json as mapped to UTF-8. It should also set this for application/json.
We could certainly choose to work around this in Sling, but it should be done at a lower level than the ExportServlet. Regards, Justin On Wed, Nov 29, 2017 at 11:19 PM Chris Millar <ch...@millr.org> wrote: > "Hitting the JCR directly" I.E. No Exporter: > http://localhost:8080/content/bar.0.json for a JCR dump via Sling. > > It looks like this is a result of the Export Servlet not setting any > character encoding, and only setting the content type. Both are needed, > otherwise `charset` defaults to ISO-8859-1 which comes from > javax.servlet.http.HttpServletResponse. > > I have put together a demo project [0] to see the before results, as well > as a fork of sling.model.impl to see the after [1] when character encoding > properly set. > > Install the demo, navigate to [2] and note the charset in Content-Type > header. It's incorrect (ISO-8859-1). > Once done, install the POC branch of models impl (and snapshot > dependencies) to see how the response has changed. > > This is a bare-minimum solution [3]. Really, the ExportServlet should > detect the proper character encoding. This could come from either setting > in the request on the model (passed through the mixin?), or getting it from > other means. > > [0] https://github.com/auniverseaway/sling-model-charset > [1] > > https://github.com/auniverseaway/sling-org-apache-sling-models-impl/tree/POC_Character_Encoding > [2] http://localhost:8080/content/bar.model.json > [3] > > https://github.com/auniverseaway/sling-org-apache-sling-models-impl/blob/POC_Character_Encoding/src/main/java/org/apache/sling/models/impl/ExportServlet.java#L95 > > On Wed, Nov 29, 2017 at 7:04 PM, Justin Edelson <jus...@justinedelson.com> > wrote: > > > I'm not sure what you mean by "Hitting the JCR directly..." since JCR has > > no HTTP support. > > > > In any case, as I said, the ExportServlet doesn't do any special handling > > with the content type and the ITs show that application/json is being > set, > > so I would suggest using a debugger against your instance to see where > the > > charset is being introduced. > > > > Regards, > > Justin > > > > On Wed, Nov 29, 2017 at 7:30 PM Chris Millar <ch...@millr.org> wrote: > > > > > I'm accessing Sling (AEM) directly on port 4502. I can put together a > > test > > > project with Sling and see if it makes a difference. > > > > > > If you look at the getResponseContentType method [0], you can see this > > does > > > not include the charset. > > > > > > Is it getting set somewhere else or being defaulted? Hitting the JCR > > > directly gives me the correct charset. > > > > > > Is there no way I can get to the response object to set this in the > > model? > > > Looking at the Impl, it didn't look like it. > > > > > > [0] > > > > > > https://sling.apache.org/apidocs/sling9/org/apache/sling/api/ > > SlingHttpServletRequest.html#getResponseContentType > > > > > > On Wed, Nov 29, 2017 at 5:14 PM, Justin Edelson < > > jus...@justinedelson.com> > > > wrote: > > > > > > > Hi Chris, > > > > Are you sure that the charset is being added by Sling, i.e. are you > > > > accessing Sling directly or via an Apache server? > > > > > > > > The Export Servlet doesn't actually try to infer anything about the > > > content > > > > type header. It just sets it to request.getResponseContentType(). See > > > > [1]. If you trace that back far enough, it ultimately comes from the > > > > MimeTypeService which I don't believe would ever include a charset. > In > > > the > > > > ITs this comes back as just application/json. > > > > > > > > Regards, > > > > Justin > > > > > > > > [1] > > > > https://github.com/apache/sling-org-apache-sling-models- > > > > impl/blob/master/src/main/java/org/apache/sling/models/ > > > > impl/ExportServlet.java#L110 > > > > [2] > > > > https://github.com/apache/sling-org-apache-sling-models- > > > > integration-tests/blob/master/src/test/java/org/apache/ > > > > sling/models/testing/exporter/ExporterIT.java#L287 > > > > > > > > On Wed, Nov 29, 2017 at 6:45 PM Chris Millar <ch...@millr.org> > wrote: > > > > > > > > > I've constructed a Sling Model Exporter with very basic options > [0], > > > and > > > > it > > > > > seems to be outputting: > > > > > > > > > > Content-Type: application/json; charset=ISO-8859-1 > > > > > > > > > > This breaks the JSON RFC [1] as the default character encoding. It > > > should > > > > > be: > > > > > > > > > > Content-Type: application/json; charset=UTF-8 > > > > > > > > > > I'm hoping there's a solution to this, or that I'm missing > something > > > > > obvious. Any help is greatly appreciated. > > > > > > > > > > [0] > > > https://gist.github.com/auniverseaway/c8b7b9013b5ecdd9c790253d194d17 > > > > b6 > > > > > [1] https://tools.ietf.org/html/rfc7159#section-8.1 > > > > > > > > > > > > > > >
