Re: Parallelism: behavioural difference in version 1.2 and 2.1!?
Dear Apostolos, Thanks for the response! Our version is built on 2.1, the problem is that the state-of-the-art system I'm trying to compare is built on the version 1.2. So I have to deal with it. If I understand the level of parallelism correctly, --total-executor-cores is set to the number or workers multiplied by the executor core of each worker, in this case, 32 as well. I make use of the similar script in both the cases, so it shouldn't change. Thanks and regards, Jeevan K. Srivatsa On Wed, 29 Aug 2018 at 16:07, Apostolos N. Papadopoulos < papad...@csd.auth.gr> wrote: > Dear Jeevan, > > Spark 1.2 is quite old, and If I were you I would go for a newer version. > > However, is there a parallelism level (e.g., 20, 30) that works for both > installations? > > regards, > > Apostolos > > > > On 29/08/2018 04:55 μμ, jeevan.ks wrote: > > Hi, > > > > I've two systems. One is built on Spark 1.2 and the other on 2.1. I am > > benchmarking both with the same benchmarks (wordcount, grep, sort, etc.) > > with the same data set from S3 bucket (size ranges from 50MB to 10 GB). > The > > Spark cluster I made use of is r3.xlarge, 8 instances, 4 cores each, and > > 28GB RAM. I observed a strange behaviour while running the benchmarks > and is > > as follows: > > > > - When I ran Spark 1.2 version with default partition number > > (sc.defaultParallelism), the jobs would take forever to complete. So I > > changed it to the number of cores, i.e., 32 times 3 = 96. This did a > magic > > and the jobs completed quickly. > > > > - However, when I tried the above magic number on the version 2.1, the > jobs > > are taking forever. Deafult parallelism works better, but not that > > efficient. > > > > I'm having problem to rationalise this and compare both the systems. My > > question is: what changes were made from 1.2 to 2.1 with respect to > default > > parallelism for this behaviour to occur? How can I have both versions > behave > > similary on the same software/hardware configuration so that I can > compare? > > > > I'd really appreciate your help on this! > > > > Cheers, > > Jeevan > > > > > > > > -- > > Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ > > > > - > > To unsubscribe e-mail: user-unsubscr...@spark.apache.org > > > > -- > Apostolos N. Papadopoulos, Associate Professor > Department of Informatics > Aristotle University of Thessaloniki > Thessaloniki, GREECE > tel: ++0030312310991918 > email: papad...@csd.auth.gr > twitter: @papadopoulos_ap > web: http://delab.csd.auth.gr/~apostol > > > - > To unsubscribe e-mail: user-unsubscr...@spark.apache.org > >
Re: java.nio.file.FileSystemException: /tmp/spark- .._cache : No space left on device
Hi Venkata, On a quick glance, it looks like a file-related issue more so than an executor issue. If the logs are not that important, I would clear /tmp/spark-events/ directory and assign a suitable permission (e.g., chmod 755) to that and rerun the application. chmod 755 /tmp/spark-events/ Thanks and regards, Jeevan K. Srivatsa On Fri, 17 Aug 2018 at 15:20, Polisetti, Venkata Siva Rama Gopala Krishna < vpolise...@spglobal.com> wrote: > Hi > > Am getting below exception when I Run Spark-submit in linux machine , can > someone give quick solution with commands > > Driver stacktrace: > > - Job 0 failed: count at DailyGainersAndLosersPublisher.scala:145, took > 5.749450 s > > org.apache.spark.SparkException: Job aborted due to stage failure: Task 4 > in stage 0.0 failed 4 times, most recent failure: Lost task 4.3 in stage > 0.0 (TID 6, 172.29.62.145, executor 0): java.nio.file.FileSystemException: > /tmp/spark-523d5331-3884-440c-ac0d-f46838c2029f/executor-390c9cd7-217e-42f3-97cb-fa2734405585/spark-206d92c0-f0d3-443c-97b2-39494e2c5fdd/-4230744641534510169119_cache > -> ./PublishGainersandLosers-1.0-SNAPSHOT-shaded-Gopal.jar: No space left > on device > > at > sun.nio.fs.UnixException.translateToIOException(UnixException.java:91) > > at > sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) > > at sun.nio.fs.UnixCopyFile.copyFile(UnixCopyFile.java:253) > > at sun.nio.fs.UnixCopyFile.copy(UnixCopyFile.java:581) > > at > sun.nio.fs.UnixFileSystemProvider.copy(UnixFileSystemProvider.java:253) > > at java.nio.file.Files.copy(Files.java:1274) > > at > org.apache.spark.util.Utils$.org$apache$spark$util$Utils$$copyRecursive(Utils.scala:625) > > at org.apache.spark.util.Utils$.copyFile(Utils.scala:596) > > at org.apache.spark.util.Utils$.fetchFile(Utils.scala:473) > > at > org.apache.spark.executor.Executor$$anonfun$org$apache$spark$executor$Executor$$updateDependencies$5.apply(Executor.scala:696) > > at > org.apache.spark.executor.Executor$$anonfun$org$apache$spark$executor$Executor$$updateDependencies$5.apply(Executor.scala:688) > > at > scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:733) > > at > scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99) > > at > scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99) > > at > scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230) > > at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40) > > at scala.collection.mutable.HashMap.foreach(HashMap.scala:99) > > at > scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:732) > > at org.apache.spark.executor.Executor.org > $apache$spark$executor$Executor$$updateDependencies(Executor.scala:688) > > at > org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:308) > > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > > at java.lang.Thread.run(Thread.java:745) > > > > -- > > The information contained in this message is intended only for the > recipient, and may be a confidential attorney-client communication or may > otherwise be privileged and confidential and protected from disclosure. If > the reader of this message is not the intended recipient, or an employee or > agent responsible for delivering this message to the intended recipient, > please be aware that any dissemination or copying of this communication is > strictly prohibited. If you have received this communication in error, > please immediately notify us by replying to the message and deleting it > from your computer. S&P Global Inc. reserves the right, subject to > applicable local law, to monitor, review and process the content of any > electronic message or information sent to or from S&P Global Inc. e-mail > addresses without informing the sender or recipient of the message. By > sending electronic message or information to S&P Global Inc. e-mail > addresses you, as the sender, are consenting to S&P Global Inc. processing > any of your personal data therein. >
