Here is related code: val executorMemory = conf.*getSizeAsBytes*("spark.executor.memory") if (executorMemory < minSystemMemory) { throw new IllegalArgumentException(s"Executor memory $executorMemory must be at least " +
On Fri, May 13, 2016 at 12:47 PM, satish saley <satishsale...@gmail.com> wrote: > Hello, > I am running > https://github.com/apache/spark/blob/branch-1.6/examples/src/main/python/pi.py > example, > but facing following exception > > What is the unit of memory pointed out in the error? > > Following are configs > > --master > > local[*] > > --deploy-mode > > client > > --name > > PysparkExample > > --py-files > > py4j-0.9-src.zip,pyspark.zip, > > --verbose > > > pi.py/py4j-0.9-src.zip/py4j/protocol.py", line 308, in get_return_value > > py4j.protocol.Py4JJavaError: An error occurred while calling > None.org.apache.spark.api.java.JavaSparkContext. > > : java.lang.IllegalArgumentException: System memory 186646528 must be at > least 4.718592E8. Please use a larger heap size. > > at > org.apache.spark.memory.UnifiedMemoryManager$.getMaxMemory(UnifiedMemoryManager.scala:193) > > at > org.apache.spark.memory.UnifiedMemoryManager$.apply(UnifiedMemoryManager.scala:175) > > at org.apache.spark.SparkEnv$.create(SparkEnv.scala:354) > > at org.apache.spark.SparkEnv$.createDriverEnv(SparkEnv.scala:193) > > at > org.apache.spark.SparkContext.createSparkEnv(SparkContext.scala:288) > > at org.apache.spark.SparkContext.<init>(SparkContext.scala:457) > > at > org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:59) > > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > > at java.lang.reflect.Constructor.newInstance(Constructor.java:422) > > at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:234) > > at > py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:381) > > at py4j.Gateway.invoke(Gateway.java:214) > > at > py4j.commands.ConstructorCommand.invokeConstructor(ConstructorCommand.java:79) > > at > py4j.commands.ConstructorCommand.execute(ConstructorCommand.java:68) > > at py4j.GatewayConnection.run(GatewayConnection.java:209) > > at java.lang.Thread.run(Thread.java:745) >