Rewrite rules for knox (link <https://cwiki.apache.org/confluence/display/KNOX/2017/08/14/Understanding+Rewrite+Rules+for+Apache+Knox#UnderstandingRewriteRulesforApacheKnox-JSONURLRewriteFIlter>) can help me to change header? is it possible?
Le lun. 24 sept. 2018 à 16:43, rabii lamriq <[email protected]> a écrit : > Hello > > Any help please?? > > Le lun. 24 sept. 2018 à 09:31, rabii lamriq <[email protected]> a écrit : > >> Hi, >> >> thanks every One, for your feedbacks, >> >> Yes, ES accept both application/json and application/x-ndjson, there is >> no problem in ES ni in Grafana, when i test my query with curl to send POST >> to ES, it work well ether json or x-ndjson. >> >> However, through Knox Gateway, it work only for x-ndjson, I don't know >> why. so, since Grafana send their POST with only json and not x-ndjson, the >> scenario of Grafana -> Knox -> ES with multi search doesn't work. >> >> I think we have two solution: >> >> 1- make some thing to force Grafana change header of POST and put >> x-ndjson instead of json, or >> 2- rewrite Header by Knox to forwarded with x-ndjson to ES. >> >> If some one have any idea of how to make this or other solution. >> >> >> Regards >> Rabii >> >> Le ven. 21 sept. 2018 à 19:21, Kevin Risden <[email protected]> a >> écrit : >> >>> https://github.com/elastic/elasticsearch/issues/25718 >>> >>> Elasticsearch endpoints should take both application/json and >>> application/x-ndjson. There might be something else going on here. That is >>> why I asked for the Elasticsearch logs. >>> >>> It might not be necessary to rewrite the content type. >>> >>> Kevin Risden >>> >>> >>> On Fri, Sep 21, 2018 at 1:17 PM rabii lamriq <[email protected]> wrote: >>> >>>> Hi all, >>>> >>>> My problem is how to change the content-type=application/json to >>>> application/x-ndjson by rewriting the header in the knox. >>>> >>>> Because grafana send their query with content-type=application/json >>>> and knox return status 400, >>>> >>>> Hower when i reproduce the problem by curl comande, I noticed that if i >>>> replace json by x-ndjson through knox it wirk well. >>>> >>>> Sence i can't change the content-type in grafana, is this possible by >>>> rewriting the header in knox to replace json by x-ndjson. >>>> >>>> Regards >>>> Rabii >>>> >>>> Le ven. 21 sept. 2018 à 18:05, >>>> Kevin Risden >>>> <[email protected]> a écrit : >>>> >>>>> Do you have the error from Elasticsearch logs? It might explain if >>>>> Knox or Grafana is sending a bad request. >>>>> >>>>> Kevin Risden >>>>> >>>>> >>>>> On Fri, Sep 21, 2018 at 12:03 PM Sandeep Moré <[email protected]> >>>>> wrote: >>>>> >>>>>> I don't see any issues from Knox side, what error do you see in >>>>>> Grafana ? I think this could be because Grafana is expecting certain >>>>>> content-type. >>>>>> >>>>>> On Fri, Sep 21, 2018 at 11:37 AM rabii lamriq <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Hi Sandeep >>>>>>> >>>>>>> thanks for your feedback, below is the deboug log for this curl >>>>>>> >>>>>>> 2018-09-18 17:23:23,154 DEBUG hadoop.gateway >>>>>>> (UrlRewriteProcessor.java:rewrite(166)) - Rewrote URL: >>>>>>> https://<host-knox>:8443/gateway/default/es5/<index_name>/_msearch, >>>>>>> direction: IN via explicit rule: ES5/inbound/query to URL: http:// >>>>>>> <host-elast>:9200/<index_name>/_msearch >>>>>>> 2018-09-18 17:23:23,156 DEBUG hadoop.gateway >>>>>>> (ElasticsearchDispatch.java:executeOutboundRequest(79)) - Dispatch >>>>>>> request: >>>>>>> POST http://<host-elast>:9200/<index_name>/_msearch?doAs=<u_user> >>>>>>> 2018-09-18 17:23:23,193 DEBUG hadoop.gateway >>>>>>> (ElasticsearchDispatch.java:executeOutboundRequest(95)) - Dispatch >>>>>>> response >>>>>>> status: 400 >>>>>>> 2018-09-18 17:23:23,194 DEBUG hadoop.gateway >>>>>>> (DefaultDispatch.java:getInboundResponseContentType(202)) - Using >>>>>>> explicit >>>>>>> character set UTF-8 for entity of type application/json >>>>>>> 2018-09-18 17:23:23,195 DEBUG hadoop.gateway >>>>>>> (DefaultDispatch.java:getInboundResponseContentType(210)) - Inbound >>>>>>> response entity content type: application/json; charset=UTF-8 >>>>>>> 2018-09-18 17:23:54,519 DEBUG hadoop.gateway >>>>>>> (GatewayFilter.java:doFilter(116)) - Received request: POST >>>>>>> /es5/<index_name>/_msearch >>>>>>> 2018-09-18 17:23:54,521 INFO hadoop.gateway >>>>>>> (AclsAuthorizationFilter.java:doFilter(85)) - Access Granted: true >>>>>>> >>>>>>> Regards >>>>>>> Rabii >>>>>>> >>>>>>> Le ven. 21 sept. 2018 à 17:17, Sandeep Moré <[email protected]> >>>>>>> a écrit : >>>>>>> >>>>>>>> Hello Rab, >>>>>>>> >>>>>>>> Interesting, do you have the debug logs for this ? they will tell >>>>>>>> us exactly what is happening. >>>>>>>> >>>>>>>> Best, >>>>>>>> Sandeep >>>>>>>> >>>>>>>> On Fri, Sep 21, 2018 at 9:56 AM rabii lamriq <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi All, >>>>>>>>> >>>>>>>>> From Grafana, I Post multisearch query to Elasticsearch using the >>>>>>>>> bulk API with the newline delimited JSON (NDJSON) format. >>>>>>>>> https://www.elastic.co/guide/en/elasticsearch/reference/current/search-multi-search.html >>>>>>>>> like this: >>>>>>>>> >>>>>>>>> curl -u myuser -i -H “Content-Type: application/json” -XPOST " >>>>>>>>> http://host_ES:900/index/_msearch"; --data-binary ’ >>>>>>>>> { first json doc} >>>>>>>>> {second json doc} >>>>>>>>> ’ >>>>>>>>> when bypassing Knox, The query work well either we made json or >>>>>>>>> x-ndjson, >>>>>>>>> However, through Knox gateway, if the >>>>>>>>> content-type=application/json, the query retun this error of 400: >>>>>>>>> >>>>>>>>> HTTP/1.1 400 Bad Request >>>>>>>>> content-type: application/json >>>>>>>>> >>>>>>>>> But when we replace json by x-ndjson like mentioned in the bulk >>>>>>>>> API, it work well. >>>>>>>>> >>>>>>>>> My problem is that Grafana send their query with the >>>>>>>>> content-type=application/json and not x-ndjson, for this, it is >>>>>>>>> possible to >>>>>>>>> make any thing on Knox to solve this problem? >>>>>>>>> >>>>>>>>> Regards >>>>>>>>> Rab >>>>>>>>> >>>>>>>>
