You can try creating a new rewrite rule that drops that problematic header if you don't need all those groups.
Best, Sandeep On Wed, Oct 10, 2018 at 7:01 PM Theyaa Matti <[email protected]> wrote: > Hi David, Kevin, > I have reached out to the spark users community to see if their are > alternatives or any work going on to fix this issue. One other thing I am > trying to test out is the limiting the request header sent from Knox to the > backend services, like spark history ui. > > The main reason this issue appear on our end is the large number of > groups the user belongs to that is pushing the request header to go beyond > 8192. Those groups are not needed in the backend services and I am looking > to see if there is a way for Knox to exclude them when forwarding the > request to the backend? > > Best, > > Theyaa. > > > On Wed, Oct 10, 2018 at 2:26 PM Kevin Risden <[email protected]> wrote: > >> Moving this back to the user list since it was sent without the list: >> >> Question: Is this error coming from the spark history server or the knox >> server? >> >> Answer: It is coming from the spark history UI. Here is what I see in the >> log for spark history ui. >> >> 18/10/10 12:30:29 WARN HttpParser: Header is too large >8192 >> >> 18/10/10 12:30:29 WARN HttpParser: badMessage: 413 for >> HttpChannelOverHttp@2756e900{r=1,c=false,a=IDLE,uri=/?doAs=<user>} >> >> >> With the above information there is probably nothing that can be done >> from Knox. The bad news is that there is no configuration in Spark for this >> yet. See the following: >> * https://issues.apache.org/jira/browse/SPARK-15090 >> * >> https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/ui/JettyUtils.scala#L337 >> * Should see something about: >> * connector.setRequestHeaderSize(requestHeaderSize); >> * connector.setResponseHeaderSize(responseHeaderSize); >> >> Without a configuration to change, the Spark history UI is going to >> default to the 8192 header size. A related change was done for most >> projects to make that header size configurable. >> >> Kevin Risden >> >> >> On Tue, Oct 9, 2018 at 8:45 PM Theyaa Matti <[email protected]> >> wrote: >> >>> Hi David, >>> Thank you for the quick response. I do have that property set >>> with the recommended value in Knox, but the issue still persists only with >>> the Spark History UI. The issue appears only when I enable ssl for knox. If >>> I turn ssl off in Knox the issue disappears, and if I enable it, it shows >>> up right away. >>> >>> Are there any equivalent properties for Spark History UI? since >>> I suspect the issue is with the Spark History UI and not with Knox since >>> all the other UIs work fine with Knox. >>> >>> Best, >>> >>> Theyaa. >>> >>> >>> >>> On Tue, Oct 9, 2018 at 7:11 PM David Villarreal < >>> [email protected]> wrote: >>> >>>> Hi Theyaa, >>>> >>>> Change the size of gateway.httpserver.requestHeaderBuffer property. I >>>> think the default is 8912 (8k) change to 16384. See if that helps. >>>> >>>> For the second problem Request is a replay (34))] this message is often >>>> seen when the timing of one of the servers is off. Make sure you use NTPD >>>> on all servers and they are all in sync. If everything is in sync you can >>>> work around this issue by turning off krb5 replay cache. With the following >>>> parameter >>>> -Dsun.security.krb5.rcache=none >>>> >>>> dav >>>> >>>> >>>> On 10/9/18, 9:01 AM, "Theyaa Matti" <[email protected]> wrote: >>>> >>>> Hi, >>>> I am getting this error message "WARN HttpParser: Header is too >>>> large >>>> >8192" when trying to access the spark history ui through knox. Any >>>> idea >>>> please? >>>> >>>> Also when trying to load the executors page, I get : GSS initiate >>>> failed >>>> [Caused by GSSException: Failure unspecified at GSS-API level >>>> (Mechanism level: >>>> Request is a replay (34))] >>>> >>>> when knox is requesting executorspage-template.html >>>> >>>> appreciate any help here. >>>> >>>> >>>>
