[MLlib] StackOverflow Error spark mllib 1.6.1 FpGrowth Algorithm for Association rule generation

[Maitray Thaker]

Hi,
I am getting stackoverflow error when I run FpGrowth algorithm on my
21 million transactions with a low support, since I want almost every
products association with other product. I know the problem is caused
by the recursive lineage of the algorithm, but I don't know how to get
around this problem. I also don't know that the RDD checkpointing is
done internally or not in the algorithm. Please suggest a solution
thanks.

-
To unsubscribe e-mail: user-unsubscr...@spark.apache.org

StackOverflow Error spark mllib 1.6.1 FpGrowth Algorithm for Association rule generation

[Maitray Thaker]

Hi,
I am getting stackoverflow error when I run FpGrowth algorithm on my
21 million transactions with a low support, since I want almost every
products association with other product. I know the problem is caused
by the recursive lineage of the algorithm, but I don't know how to get
around this problem. I also don't know that the RDD checkpointing is
done internally or not in the algorithm. Please suggest a solution
thanks.

-
To unsubscribe e-mail: user-unsubscr...@spark.apache.org

Re: Already subscribed to user@spark.apache.org

[Maitray Thaker]

On Mon, Nov 7, 2016 at 1:26 PM,  wrote:

> Hi! This is the ezmlm program. I'm managing the
> user@spark.apache.org mailing list.
>
> Acknowledgment: The address
>
>maitraytha...@gmail.com
>
> was already on the user mailing list when I received
> your request, and remains a subscriber.
>
>
> --- Administrative commands for the user list ---
>
> I can handle administrative requests automatically. Please
> do not send them to the list address! Instead, send
> your message to the correct command address:
>
> To subscribe to the list, send a message to:
>
>
> To remove your address from the list, send a message to:
>
>
> Send mail to the following for info and FAQ for this list:
>
>
>
> Similar addresses exist for the digest list:
>
>
>
> To get messages 123 through 145 (a maximum of 100 per request), mail:
>
>
> To get an index with subject and author for messages 123-456 , mail:
>
>
> They are always returned as sets of 100, max 2000 per request,
> so you'll actually get 100-499.
>
> To receive all messages with the same subject as message 12345,
> send a short message to:
>
>
> The messages should contain one line or word of text to avoid being
> treated as sp@m, but I will ignore their content.
> Only the ADDRESS you send to is important.
>
> You can start a subscription for an alternate address,
> for example "john@host.domain", just add a hyphen and your
> address (with '=' instead of '@') after the command word:
> 

Fwd: Spark optimization problem

[Maitray Thaker]

Hi,
I have a query regarding spark stage optimization. I have asked the
question in more detail at Stackoverflow, please find the following link:
http://stackoverflow.com/questions/40192302/why-is-
that-two-stages-in-apache-spark-are-computing-same-thing