[ https://issues.apache.org/jira/browse/HDFS-10423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15318079#comment-15318079 ]
Nicolae Popa commented on HDFS-10423: ------------------------------------- I have attached the tested patch. The test is using the below script to generate a bloat header for http using the below ad hoc script: It will generate an http request with various sizes to stress the tomcat settings. for size in 1024 2048 4096 8192 8193 16384 32768 44443 44444 44445 65536 65537 ; do echo "################################################################################################################################################################" echo "### Trying with request size: $size" echo "################################################################################################################################################################" echo http_request_size=$size cat <<-EOF>http_request GET /webhdfs/v1/user/npopa?op=liststatus&user.name=npopa HTTP/1.1 Host: localhost:14000 EOF http_request_size_tmp=`wc -c http_request|awk '{print $1}'` export to_fill=`expr $http_request_size - $http_request_size_tmp - 5` # -5 comes from "h: " and new lines before and after the header export RNDSTR=`cat /dev/urandom | env LC_CTYPE=C tr -dc 'a-zA-Z0-9' | fold -w $to_fill| head -n 1` echo "h: $RNDSTR">>http_request echo "">>http_request echo "Request is: " cat http_request echo echo "Request size is: " ls -ltr http_request echo echo "Result is: " nc localhost 14000 < http_request echo done The scenarios and results are: 1) before the patch 2) after the patch 3) with the export HTTPFS_MAX_HTTP_HEADER_SIZE=44444 in etc/hadoop/httpfs-env.sh In before-HTTP-10423.txt we can see it starts failing after a HTTP 8192 request In after-HTTP-10423.txt we can see it starts failing after a HTTP 65536 request (as expected) I also run a test setting export HTTPFS_MAX_HTTP_HEADER_SIZE=44444 in etc/hadoop/httpfs-env.sh and the behavior was as expected. It worked with an HTTP request of 44444 bytes and failed with a request of 44445 bytes (as expected). (see after-HDFS-10423_withCustomHeader44444.txt) For each size below are the expected results: successful result is: Result is: HTTP/1.1 200 OK failed result is: Result is: HTTP/1.1 400 Bad Request > Increase default value of httpfs maxHttpHeaderSize > -------------------------------------------------- > > Key: HDFS-10423 > URL: https://issues.apache.org/jira/browse/HDFS-10423 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs > Affects Versions: 2.6.4 > Reporter: Nicolae Popa > Priority: Minor > Attachments: HDFS-10423.01.patch, after-HDFS-10423.txt, > after-HDFS-10423_withCustomHeader44444.txt, before-HDFS-10423.txt > > > The Tomcat default value of maxHttpHeaderSize is 8k, which is too low for > certain Hadoop workloads in kerberos enabled environments. This JIRA will to > change it to 65536 in server.xml -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org