Hi Mars, Thank you so much. It worked now.
Regards, Sravya From: Mars Hall [mailto:[email protected]] Sent: Tuesday, October 25, 2016 1:32 AM To: [email protected] Subject: Re: Stack overflow exception with Similar Product engine Lakshmi, Everytime my team encounters a stack overflow with PIO/Spark, it has been resolved by allocating more memory. This setting is based on your runtime RAM. Add opts like the following to the pio commands: pio train -- --driver-memory 1g --executor-memory 4g Make sure the extra double-dash is between the pio opts & spark opts as shown. *Mars On Sun, Oct 23, 2016 at 21:24 Guruju, Lakshmi Sravya <[email protected]<mailto:[email protected]>> wrote: Hi, I am using PredictionIo Similar Product engine. I was able to build the engine successfully on my MAC machine. Also train the data successfully with the data size of 50 user events, 50 item events and 50 view events. But when I import data around 5000 user events, 5000 item events, and 5000 view events and try to train the same, I am getting Stack overflow exception. Just FYI, am using Java SDK to import the data as a batch of 50 events at a time. Below is the part of exception trace. Also I found something related to this over stack overflow (http://stackoverflow.com/questions/31484460/spark-gives-a-stackoverflowerror-when-training-using-als ), But as am new to spark, I did not get where to provide the fix. Could somebody help me in resolving it. [ERROR] [Executor] Exception in task 1.0 in stage 692.0 (TID 540) [ERROR] [Executor] Exception in task 0.0 in stage 692.0 (TID 539) [ERROR] [Executor] Exception in task 3.0 in stage 692.0 (TID 542) [WARN] [TaskSetManager] Lost task 0.0 in stage 692.0 (TID 539, localhost): java.lang.StackOverflowError at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2296) at java.io.ObjectInputStream$BlockDataInputStream.peek(ObjectInputStream.java:2589) at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2599) at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1506) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1774) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:371) at scala.collection.immutable.$colon$colon.readObject(List.scala:362) at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) Regards, Sravya
