Re: 回复：Re: Build SPARK from source with SBT failed
I use m1 apple silicon , use java11 from Zulu , and runs SBT based Build Jobs in Kubernetes Best Tufan On Tue, 7 Mar 2023 at 16:11, Sean Owen wrote: > No, it's that JAVA_HOME wasn't set to .../Home. It is simply not finding > javac, in the error. Zulu supports M1. > > On Tue, Mar 7, 2023 at 9:05 AM Artemis User > wrote: > >> Looks like Maven build did find the javac, just can't run it. So it's >> not a path problem but a compatibility problem. Are you doing this on a >> Mac with M1/M2? I don't think that Zulu JDK supports Apple silicon. Your >> best option would be to use homebrew to install the dev tools (including >> OpenJDK) on Mac. On Ubuntu, it seems still the compatibility problem. Try >> to use the apt to install your dev tools, don't do it manually. If you >> manually install JDK, it doesn't install hardware-optimized JVM libraries. >> >> On 3/7/23 8:21 AM, ckgppl_...@sina.cn wrote: >> >> No. I haven't installed Apple Developer Tools. I have installed Zulu >> OpenJDK 11.0.17 manually. >> So I need to install Apple Developer Tools? >> - 原始邮件 ----- >> 发件人：Sean Owen >> 收件人：ckgppl_...@sina.cn >> 抄送人：user >> 主题：Re: Build SPARK from source with SBT failed >> 日期：2023年03月07日 20点58分 >> >> This says you don't have the java compiler installed. Did you install the >> Apple Developer Tools package? >> >> On Tue, Mar 7, 2023 at 1:42 AM wrote: >> >> Hello, >> >> I have tried to build SPARK source codes with SBT in my local dev >> environment (MacOS 13.2.1). But it reported following error: >> [error] java.io.IOException: Cannot run program >> "/Library/Java/JavaVirtualMachines/zulu-11.jdk/Contents/bin/javac" (in >> directory "/Users/username/spark-remotemaster"): error=2, No such file or >> directory >> >> [error] at >> java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1128) >> >> [error] at >> java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1071) >> >> [error] at >> scala.sys.process.ProcessBuilderImpl$Simple.run(ProcessBuilderImpl.scala:75) >> [error] at >> scala.sys.process.ProcessBuilderImpl$AbstractBuilder.run(ProcessBuilderImpl.scala:106) >> >> I need to export JAVA_HOME to let it run successfully. But if I use maven >> then I don't need to export JAVA_HOME. I have also tried to build SPARK >> with SBT in Ubuntu X86_64 environment. It reported similar error. >> >> The official SPARK >> documentation haven't mentioned export JAVA_HOME operation. So I think >> this is a bug which needs documentation or scripts change. Please correct >> me if I am wrong. >> >> Thanks >> >> Liang >> >> >>
Re: 回复：Re: Build SPARK from source with SBT failed
No, it's that JAVA_HOME wasn't set to .../Home. It is simply not finding javac, in the error. Zulu supports M1. On Tue, Mar 7, 2023 at 9:05 AM Artemis User wrote: > Looks like Maven build did find the javac, just can't run it. So it's not > a path problem but a compatibility problem. Are you doing this on a Mac > with M1/M2? I don't think that Zulu JDK supports Apple silicon. Your > best option would be to use homebrew to install the dev tools (including > OpenJDK) on Mac. On Ubuntu, it seems still the compatibility problem. Try > to use the apt to install your dev tools, don't do it manually. If you > manually install JDK, it doesn't install hardware-optimized JVM libraries. > > On 3/7/23 8:21 AM, ckgppl_...@sina.cn wrote: > > No. I haven't installed Apple Developer Tools. I have installed Zulu > OpenJDK 11.0.17 manually. > So I need to install Apple Developer Tools? > - 原始邮件 - > 发件人：Sean Owen > 收件人：ckgppl_...@sina.cn > 抄送人：user > 主题：Re: Build SPARK from source with SBT failed > 日期：2023年03月07日 20点58分 > > This says you don't have the java compiler installed. Did you install the > Apple Developer Tools package? > > On Tue, Mar 7, 2023 at 1:42 AM wrote: > > Hello, > > I have tried to build SPARK source codes with SBT in my local dev > environment (MacOS 13.2.1). But it reported following error: > [error] java.io.IOException: Cannot run program > "/Library/Java/JavaVirtualMachines/zulu-11.jdk/Contents/bin/javac" (in > directory "/Users/username/spark-remotemaster"): error=2, No such file or > directory > > [error] at > java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1128) > > [error] at > java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1071) > > [error] at > scala.sys.process.ProcessBuilderImpl$Simple.run(ProcessBuilderImpl.scala:75) > [error] at > scala.sys.process.ProcessBuilderImpl$AbstractBuilder.run(ProcessBuilderImpl.scala:106) > > I need to export JAVA_HOME to let it run successfully. But if I use maven > then I don't need to export JAVA_HOME. I have also tried to build SPARK > with SBT in Ubuntu X86_64 environment. It reported similar error. > > The official SPARK > documentation haven't mentioned export JAVA_HOME operation. So I think > this is a bug which needs documentation or scripts change. Please correct > me if I am wrong. > > Thanks > > Liang > > >
Re: 回复：Re: Build SPARK from source with SBT failed
Looks like Maven build did find the javac, just can't run it. So it's not a path problem but a compatibility problem. Are you doing this on a Mac with M1/M2? I don't think that Zulu JDK supports Apple silicon. Your best option would be to use homebrew to install the dev tools (including OpenJDK) on Mac. On Ubuntu, it seems still the compatibility problem. Try to use the apt to install your dev tools, don't do it manually. If you manually install JDK, it doesn't install hardware-optimized JVM libraries. On 3/7/23 8:21 AM, ckgppl_...@sina.cn wrote: No. I haven't installed Apple Developer Tools. I have installed Zulu OpenJDK 11.0.17 manually. So I need to install Apple Developer Tools? - 原始邮件 - 发件人：Sean Owen 收件人：ckgppl_...@sina.cn 抄送人：user 主题：Re: Build SPARK from source with SBT failed 日期：2023年03月07日 20点58分 This says you don't have the java compiler installed. Did you install the Apple Developer Tools package? On Tue, Mar 7, 2023 at 1:42 AM wrote: Hello, I have tried to build SPARK source codes with SBT in my local dev environment (MacOS 13.2.1). But it reported following error: [error] java.io.IOException: Cannot run program "/Library/Java/JavaVirtualMachines/zulu-11.jdk/Contents/bin/javac" (in directory "/Users/username/spark-remotemaster"): error=2, No such file or directory [error] at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1128) [error] at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1071) [error] at scala.sys.process.ProcessBuilderImpl$Simple.run(ProcessBuilderImpl.scala:75) [error] at scala.sys.process.ProcessBuilderImpl$AbstractBuilder.run(ProcessBuilderImpl.scala:106) I need to export JAVA_HOME to let it run successfully. But if I use maven then I don't need to export JAVA_HOME. I have also tried to build SPARK with SBT in Ubuntu X86_64 environment. It reported similar error. The official SPARK documentation haven't mentioned export JAVA_HOME operation. So I think this is a bug which needs documentation or scripts change. Please correct me if I am wrong. Thanks Liang
回复：Re: Build SPARK from source with SBT failed
No. I haven't installed Apple Developer Tools. I have installed Zulu OpenJDK 11.0.17 manually.So I need to install Apple Developer Tools?- 原始邮件 - 发件人：Sean Owen 收件人：ckgppl_...@sina.cn 抄送人：user 主题：Re: Build SPARK from source with SBT failed 日期：2023年03月07日 20点58分 This says you don't have the java compiler installed. Did you install the Apple Developer Tools package? On Tue, Mar 7, 2023 at 1:42 AM wrote: Hello, I have tried to build SPARK source codes with SBT in my local dev environment (MacOS 13.2.1). But it reported following error:[error] java.io.IOException: Cannot run program "/Library/Java/JavaVirtualMachines/zulu-11.jdk/Contents/bin/javac" (in directory "/Users/username/spark-remotemaster"): error=2, No such file or directory [error] at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1128) [error] at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1071) [error] at scala.sys.process.ProcessBuilderImpl$Simple.run(ProcessBuilderImpl.scala:75) [error] at scala.sys.process.ProcessBuilderImpl$AbstractBuilder.run(ProcessBuilderImpl.scala:106) I need to export JAVA_HOME to let it run successfully. But if I use maven then I don't need to export JAVA_HOME. I have also tried to build SPARK with SBT in Ubuntu X86_64 environment. It reported similar error. The official SPARK documentation haven't mentioned export JAVA_HOME operation. So I think this is a bug which needs documentation or scripts change. Please correct me if I am wrong. Thanks Liang
Re: Build SPARK from source with SBT failed
This says you don't have the java compiler installed. Did you install the Apple Developer Tools package? On Tue, Mar 7, 2023 at 1:42 AM wrote: > Hello, > > I have tried to build SPARK source codes with SBT in my local dev > environment (MacOS 13.2.1). But it reported following error: > [error] java.io.IOException: Cannot run program > "/Library/Java/JavaVirtualMachines/zulu-11.jdk/Contents/bin/javac" (in > directory "/Users/username/spark-remotemaster"): error=2, No such file or > directory > > [error] at > java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1128) > > [error] at > java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1071) > > [error] at > scala.sys.process.ProcessBuilderImpl$Simple.run(ProcessBuilderImpl.scala:75) > [error] at > scala.sys.process.ProcessBuilderImpl$AbstractBuilder.run(ProcessBuilderImpl.scala:106) > > I need to export JAVA_HOME to let it run successfully. But if I use maven > then I don't need to export JAVA_HOME. I have also tried to build SPARK > with SBT in Ubuntu X86_64 environment. It reported similar error. > > The official SPARK > documentation haven't mentioned export JAVA_HOME operation. So I think > this is a bug which needs documentation or scripts change. Please correct > me if I am wrong. > > Thanks > > Liang > >
