[jira] [Commented] (SPARK-9273) Add Convolutional Neural network to Spark MLlib
[ https://issues.apache.org/jira/browse/SPARK-9273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15151015#comment-15151015 ] Sean Owen commented on SPARK-9273: -- No, I mean that I expect it will start life as an external package. In any event, I want to at least convene any related discussion in 1 JIRA, not N. CNN is a type of ANN so I don't see a value in discussing it separately. > Add Convolutional Neural network to Spark MLlib > --- > > Key: SPARK-9273 > URL: https://issues.apache.org/jira/browse/SPARK-9273 > Project: Spark > Issue Type: New Feature > Components: MLlib >Reporter: yuhao yang >Assignee: yuhao yang > > Add Convolutional Neural network to Spark MLlib -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-9273) Add Convolutional Neural network to Spark MLlib
[ https://issues.apache.org/jira/browse/SPARK-9273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15150960#comment-15150960 ] Alexander Ulanov commented on SPARK-9273: - [~srowen] Do you mean that CNN will never be merged into Spark ML? > Add Convolutional Neural network to Spark MLlib > --- > > Key: SPARK-9273 > URL: https://issues.apache.org/jira/browse/SPARK-9273 > Project: Spark > Issue Type: New Feature > Components: MLlib >Reporter: yuhao yang >Assignee: yuhao yang > > Add Convolutional Neural network to Spark MLlib -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-9273) Add Convolutional Neural network to Spark MLlib
[ https://issues.apache.org/jira/browse/SPARK-9273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15150613#comment-15150613 ] Asim Jalis commented on SPARK-9273: --- Convolutional Neural Networks are significantly different from Artificial Neural Networks that this should be reopened. Unless the ANN functions provide way to get CNN functionality I don’t see this as a duplicate. > Add Convolutional Neural network to Spark MLlib > --- > > Key: SPARK-9273 > URL: https://issues.apache.org/jira/browse/SPARK-9273 > Project: Spark > Issue Type: New Feature > Components: MLlib >Reporter: yuhao yang >Assignee: yuhao yang > > Add Convolutional Neural network to Spark MLlib -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-9273) Add Convolutional Neural network to Spark MLlib
[ https://issues.apache.org/jira/browse/SPARK-9273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15149794#comment-15149794 ] Sateesh Babu G commented on SPARK-9273: --- Hi Alexander, Thank you very much for your help! Can I use any one of the mentioned implementations for CNN regression? Which one is more suitable? I also found deeplearning4j.org has CNN implementation in Spark but with only one Convoultional and one pooling layer. Do you suggest deeplearning4j.org's CNN implementation in Spark. Best, Sateesh > Add Convolutional Neural network to Spark MLlib > --- > > Key: SPARK-9273 > URL: https://issues.apache.org/jira/browse/SPARK-9273 > Project: Spark > Issue Type: New Feature > Components: MLlib >Reporter: yuhao yang >Assignee: yuhao yang > > Add Convolutional Neural network to Spark MLlib -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-9273) Add Convolutional Neural network to Spark MLlib
[ https://issues.apache.org/jira/browse/SPARK-9273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15149139#comment-15149139 ] Alexander Ulanov commented on SPARK-9273: - Hi [~gsateesh110], Besides the one mentioned by Yuhao, there is SparkNet that allows using Caffe. In future, I plan to switch the present neural network implementation in Spark to tensors, and probably implement CNN, that is easier with tensors: https://github.com/avulanov/spark/tree/mlp-tensor Best regards, Alexander > Add Convolutional Neural network to Spark MLlib > --- > > Key: SPARK-9273 > URL: https://issues.apache.org/jira/browse/SPARK-9273 > Project: Spark > Issue Type: New Feature > Components: MLlib >Reporter: yuhao yang >Assignee: yuhao yang > > Add Convolutional Neural network to Spark MLlib -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-9273) Add Convolutional Neural network to Spark MLlib
[ https://issues.apache.org/jira/browse/SPARK-9273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148342#comment-15148342 ] Sateesh Babu G commented on SPARK-9273: --- Hi All, Is there any CNN implementation for SPARK? Thanks in advance! Best, Sateesh > Add Convolutional Neural network to Spark MLlib > --- > > Key: SPARK-9273 > URL: https://issues.apache.org/jira/browse/SPARK-9273 > Project: Spark > Issue Type: New Feature > Components: MLlib >Reporter: yuhao yang >Assignee: yuhao yang > > Add Convolutional Neural network to Spark MLlib -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-9273) Add Convolutional Neural network to Spark MLlib
[ https://issues.apache.org/jira/browse/SPARK-9273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15003466#comment-15003466 ] yuhao yang commented on SPARK-9273: --- I'd like to hear more opinions on if we should make Pooling Layer a functional layer. Since in some research, the pooling layer can have weights of its own. Reopen the jira since it's an independent feature and under active development. > Add Convolutional Neural network to Spark MLlib > --- > > Key: SPARK-9273 > URL: https://issues.apache.org/jira/browse/SPARK-9273 > Project: Spark > Issue Type: New Feature > Components: MLlib >Reporter: yuhao yang >Assignee: yuhao yang > > Add Convolutional Neural network to Spark MLlib -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-9273) Add Convolutional Neural network to Spark MLlib
[ https://issues.apache.org/jira/browse/SPARK-9273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14995685#comment-14995685 ] yuhao yang commented on SPARK-9273: --- Thanks so much for the helpful suggestions. I have an optimized convolution as you mentioned (GEMM). I'll gather some results to see how much it can improve the speed, and decides which is better. (GEMM will take extra space). > Add Convolutional Neural network to Spark MLlib > --- > > Key: SPARK-9273 > URL: https://issues.apache.org/jira/browse/SPARK-9273 > Project: Spark > Issue Type: New Feature > Components: MLlib >Reporter: yuhao yang > > Add Convolutional Neural network to Spark MLlib -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-9273) Add Convolutional Neural network to Spark MLlib
[ https://issues.apache.org/jira/browse/SPARK-9273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14992447#comment-14992447 ] Alexander Ulanov commented on SPARK-9273: - Hi Yuhao. Sounds good! Thanks for refactoring the code to support ANN internal interface. Also, I was able to run your example. It shows increasing accuracy while training however it is not very fast. There is a good explanation how to use matrices multiplication in convolution: http://cs231n.github.io/convolutional-networks/. Basically, one needs to roll all image patches (regions that will be convolved) the into vectors and stack them together in a matrix. The weights of convolutional layer also should be rolled into vectors and stacked. Multiplying two mentioned matrices provides the convolution result that can be unrolled to 3d matrix, however it would not be necessary for this implementation. We can discuss it offline if you wish. Besides the optimization, there are few more things to be done. It includes unit tests for new layers, gradient test, representing pooling layer as functional layer, and performance comparison with the other implementation of CNN. You can take a look at the tests I've added for MLP https://issues.apache.org/jira/browse/SPARK-11262 and MLP benchmark at https://github.com/avulanov/ann-benchmark. A separate branch/repo for these developments might be a good thing to do. I'll be happy to help you with this. > Add Convolutional Neural network to Spark MLlib > --- > > Key: SPARK-9273 > URL: https://issues.apache.org/jira/browse/SPARK-9273 > Project: Spark > Issue Type: New Feature > Components: MLlib >Reporter: yuhao yang > > Add Convolutional Neural network to Spark MLlib -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-9273) Add Convolutional Neural network to Spark MLlib
[ https://issues.apache.org/jira/browse/SPARK-9273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14991018#comment-14991018 ] yuhao yang commented on SPARK-9273: --- Hi [~avulanov]. I've refactored the CNN in [https://github.com/hhbyyh/mCNN/tree/master/src/communityInterface] according to ANN interface. Also I made an example in [Driver.scala| https://github.com/hhbyyh/mCNN/blob/master/src/communityInterface/Driver.scala] to demo how it can be combined with existing layers to resolve MNist. Right now I'm trying to optimize the Convolution operation, referring to [http://petewarden.com/2015/04/20/why-gemm-is-at-the-heart-of-deep-learning/]. Appreciate if you can shed some light :-). > Add Convolutional Neural network to Spark MLlib > --- > > Key: SPARK-9273 > URL: https://issues.apache.org/jira/browse/SPARK-9273 > Project: Spark > Issue Type: New Feature > Components: MLlib >Reporter: yuhao yang > > Add Convolutional Neural network to Spark MLlib -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-9273) Add Convolutional Neural network to Spark MLlib
[ https://issues.apache.org/jira/browse/SPARK-9273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14737948#comment-14737948 ] Alexander Ulanov commented on SPARK-9273: - Hi Yuhao! I have few comments regarding the interface and the optimization of your implementation. There are two options of optimizing convolutions: using matrix-matrix multiplication and using FFTs. The latter seems a bit more complicated since we don't have optimized parallel FFT in Spark. It also has to support batch data processing. Instead, if one uses matrix-matrix multiplication for convolution, then it can take advantage of native BLAS and batch computations can be supported straightforward. Another benefit is that we would not need to change the current Layer's input/ouptput type (matrix) to tensor. We can store the unwrapped inputs/outputs as vectors within the input/output matrix. Do you think that it is reasonable? > Add Convolutional Neural network to Spark MLlib > --- > > Key: SPARK-9273 > URL: https://issues.apache.org/jira/browse/SPARK-9273 > Project: Spark > Issue Type: New Feature > Components: MLlib >Reporter: yuhao yang > > Add Convolutional Neural network to Spark MLlib -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-9273) Add Convolutional Neural network to Spark MLlib
[ https://issues.apache.org/jira/browse/SPARK-9273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14737963#comment-14737963 ] yuhao yang commented on SPARK-9273: --- Thank a lot for your attention [~avulanov]. I do hope we can join forces during the development. I do prefer to use matrix-matrix multiplication for convolutions. And I will try to stick to the current Layer's input/output. I'll send an update this weekend with a version compatible with current interface. Appreciate if you can help take a look. > Add Convolutional Neural network to Spark MLlib > --- > > Key: SPARK-9273 > URL: https://issues.apache.org/jira/browse/SPARK-9273 > Project: Spark > Issue Type: New Feature > Components: MLlib >Reporter: yuhao yang > > Add Convolutional Neural network to Spark MLlib -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-9273) Add Convolutional Neural network to Spark MLlib
[ https://issues.apache.org/jira/browse/SPARK-9273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14644142#comment-14644142 ] yuhao yang commented on SPARK-9273: --- Hi Joseph. The CNN was implemented based on customer requirement and I read the Layer-based architecture from Alexander Ulanov before writing it, so hopefully it can fit into the new neural network framework. Sorry for not informing the community on time. Add Convolutional Neural network to Spark MLlib --- Key: SPARK-9273 URL: https://issues.apache.org/jira/browse/SPARK-9273 Project: Spark Issue Type: New Feature Components: MLlib Reporter: yuhao yang Add Convolutional Neural network to Spark MLlib -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-9273) Add Convolutional Neural network to Spark MLlib
[ https://issues.apache.org/jira/browse/SPARK-9273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14644148#comment-14644148 ] yuhao yang commented on SPARK-9273: --- Besides, do you happen to know any other implementation of CNN based on Spark? I'd like to make comparisons and make improvement accordingly. Add Convolutional Neural network to Spark MLlib --- Key: SPARK-9273 URL: https://issues.apache.org/jira/browse/SPARK-9273 Project: Spark Issue Type: New Feature Components: MLlib Reporter: yuhao yang Add Convolutional Neural network to Spark MLlib -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-9273) Add Convolutional Neural network to Spark MLlib
[ https://issues.apache.org/jira/browse/SPARK-9273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14644149#comment-14644149 ] yuhao yang commented on SPARK-9273: --- Thanks Alexander. I'll keep you posted for any new changes. Add Convolutional Neural network to Spark MLlib --- Key: SPARK-9273 URL: https://issues.apache.org/jira/browse/SPARK-9273 Project: Spark Issue Type: New Feature Components: MLlib Reporter: yuhao yang Add Convolutional Neural network to Spark MLlib -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-9273) Add Convolutional Neural network to Spark MLlib
[ https://issues.apache.org/jira/browse/SPARK-9273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14644899#comment-14644899 ] Joseph K. Bradley commented on SPARK-9273: -- Thanks for coordinating! It will be great if the two efforts can be combined. [~yuhaoyan] I don't know of other major efforts towards CNNs in Spark. Add Convolutional Neural network to Spark MLlib --- Key: SPARK-9273 URL: https://issues.apache.org/jira/browse/SPARK-9273 Project: Spark Issue Type: New Feature Components: MLlib Reporter: yuhao yang Add Convolutional Neural network to Spark MLlib -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-9273) Add Convolutional Neural network to Spark MLlib
[ https://issues.apache.org/jira/browse/SPARK-9273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14642513#comment-14642513 ] Alexander Ulanov commented on SPARK-9273: - I have not heard about the PR until it was submitted. It would be useful to look at the code, benchmark it and see if it fits our API. Add Convolutional Neural network to Spark MLlib --- Key: SPARK-9273 URL: https://issues.apache.org/jira/browse/SPARK-9273 Project: Spark Issue Type: New Feature Components: MLlib Reporter: yuhao yang Add Convolutional Neural network to Spark MLlib -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-9273) Add Convolutional Neural network to Spark MLlib
[ https://issues.apache.org/jira/browse/SPARK-9273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14638354#comment-14638354 ] Apache Spark commented on SPARK-9273: - User 'hhbyyh' has created a pull request for this issue: https://github.com/apache/spark/pull/7609 Add Convolutional Neural network to Spark MLlib --- Key: SPARK-9273 URL: https://issues.apache.org/jira/browse/SPARK-9273 Project: Spark Issue Type: New Feature Components: MLlib Reporter: yuhao yang Add Convolutional Neural network to Spark MLlib -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-9273) Add Convolutional Neural network to Spark MLlib
[ https://issues.apache.org/jira/browse/SPARK-9273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14638364#comment-14638364 ] yuhao yang commented on SPARK-9273: --- Thanks Sean. I did some checking before creating the jira. The only similar one is 9129, for which I'm not sure if it's an umbrella jira. Add Convolutional Neural network to Spark MLlib --- Key: SPARK-9273 URL: https://issues.apache.org/jira/browse/SPARK-9273 Project: Spark Issue Type: New Feature Components: MLlib Reporter: yuhao yang Add Convolutional Neural network to Spark MLlib -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-9273) Add Convolutional Neural network to Spark MLlib
[ https://issues.apache.org/jira/browse/SPARK-9273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14638328#comment-14638328 ] Sean Owen commented on SPARK-9273: -- I think this is duplicated / subsumed many times by other JIRAs covering ANNs in Spark. Add Convolutional Neural network to Spark MLlib --- Key: SPARK-9273 URL: https://issues.apache.org/jira/browse/SPARK-9273 Project: Spark Issue Type: New Feature Components: MLlib Reporter: yuhao yang Add Convolutional Neural network to Spark MLlib -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
