For #1, have you seen this JIRA ? [SPARK-14867][BUILD] Remove `--force` option in `build/mvn`
On Thu, Apr 28, 2016 at 8:27 PM, Demon King <kdm...@gmail.com> wrote: > BUG 1: > I have installed maven 3.0.2 in system, When I using make-distribution.sh > , it seem not use maven 3.2.2 but use /usr/local/bin/mvn to build spark. So > I add --force option in make-distribution.sh like this: > > line 130: > VERSION=$("$MVN" *--force* help:evaluate -Dexpression=project.version $@ > 2>/dev/null | grep -v "INFO" | tail -n 1) > SCALA_VERSION=$("$MVN"* --force* help:evaluate > -Dexpression=scala.binary.version $@ 2>/dev/null\ > | grep -v "INFO"\ > | tail -n 1) > SPARK_HADOOP_VERSION=$("$MVN" *--force* help:evaluate > -Dexpression=hadoop.version $@ 2>/dev/null\ > | grep -v "INFO"\ > | tail -n 1) > SPARK_HIVE=$("$MVN"* --force* help:evaluate > -Dexpression=project.activeProfiles -pl sql/hive $@ 2>/dev/null\ > | grep -v "INFO"\ > | fgrep --count "<id>hive</id>";\ > # Reset exit status to 0, otherwise the script stops here if the last > grep finds nothing\ > # because we use "set -o pipefail" > echo -n) > > line 170: > BUILD_COMMAND=("$MVN" *--force* clean package -DskipTests $@) > > that will force spark to use build/mvn and solve this problem. > > BUG 2: > > When I run running unit test VersionsSuite, it will hang for one night or > more. I use jstack and lsof and find it try to send a http request. That > seems not be a good item when runing test in terrible network. > > I use jstack and finally find out reason: > > java.lang.Thread.State: RUNNABLE > at java.net.SocketInputStream.socketRead0(Native Method) > at java.net.SocketInputStream.read(SocketInputStream.java:152) > at java.net.SocketInputStream.read(SocketInputStream.java:122) > at java.io.BufferedInputStream.fill(BufferedInputStream.java:235) > at java.io.BufferedInputStream.read1(BufferedInputStream.java:275) > at java.io.BufferedInputStream.read(BufferedInputStream.java:334) > - locked <0x00000007440224d8> (a java.io.BufferedInputStream) > at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:687) > at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:633) > at > sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323) > - locked <0x0000000744022530> (a > sun.net.www.protocol.http.HttpURLConnection) > at > java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:468) > ... > > and I use lsof: > > java 32082 user 247u IPv4 527001934 TCP 8.8.8.8:33233 (LISTEN) > java 32082 user 267u IPv4 527001979 TCP 8.8.8.8:52301 (LISTEN) > java 32082 user 316u IPv4 527001999 TCP *:51993 (LISTEN) > java 32082 user 521u IPv4 527111590 TCP 8.8.8.8:53286 > ->butan141.server4you.de:http (ESTABLISHED) > > This test suite try to connect butan141.server4you.de, The process will > hang when network is terrible . > > >