[jira] [Commented] (SPARK-25994) SPIP: Property Graphs, Cypher Queries, and Algorithms

[ https://issues.apache.org/jira/browse/SPARK-25994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16811163#comment-16811163 ] Saikat Kanjilal commented on SPARK-25994: - [~mju] I added some initial comments, will review PR

[jira] [Commented] (SPARK-25994) SPIP: Property Graphs, Cypher Queries, and Algorithms

[ https://issues.apache.org/jira/browse/SPARK-25994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755272#comment-16755272 ] Saikat Kanjilal commented on SPARK-25994: - [~mju] I would like to help out on this issue on the

[jira] [Commented] (SPARK-18085) SPIP: Better History Server scalability for many / large applications

[ https://issues.apache.org/jira/browse/SPARK-18085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16084361#comment-16084361 ] Saikat Kanjilal commented on SPARK-18085: - [~vanzin] I would be interested in helping with this,

[jira] [Created] (SPARK-19224) [PYSPARK] Python tests organization

Saikat Kanjilal created SPARK-19224: --- Summary: [PYSPARK] Python tests organization Key: SPARK-19224 URL: https://issues.apache.org/jira/browse/SPARK-19224 Project: Spark Issue Type: Test

[jira] [Commented] (SPARK-9487) Use the same num. worker threads in Scala/Python unit tests

[ https://issues.apache.org/jira/browse/SPARK-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15808216#comment-15808216 ] Saikat Kanjilal commented on SPARK-9487: [~srowen] build/tests has passed in jenkins, what do you

[jira] [Commented] (SPARK-9487) Use the same num. worker threads in Scala/Python unit tests

[ https://issues.apache.org/jira/browse/SPARK-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15738477#comment-15738477 ] Saikat Kanjilal commented on SPARK-9487: Then I would suggest keeping it open and focus on a

[jira] [Commented] (SPARK-9487) Use the same num. worker threads in Scala/Python unit tests

[ https://issues.apache.org/jira/browse/SPARK-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15738419#comment-15738419 ] Saikat Kanjilal commented on SPARK-9487: I'm ok closing it actually but it does outline issues

[jira] [Commented] (SPARK-9487) Use the same num. worker threads in Scala/Python unit tests

[ https://issues.apache.org/jira/browse/SPARK-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15738309#comment-15738309 ] Saikat Kanjilal commented on SPARK-9487: [~srowen] I think the above plan is great minus one

[jira] [Commented] (SPARK-9487) Use the same num. worker threads in Scala/Python unit tests

[ https://issues.apache.org/jira/browse/SPARK-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15733342#comment-15733342 ] Saikat Kanjilal commented on SPARK-9487: Given the latest thread on the devlist thoughts

[jira] [Commented] (SPARK-9487) Use the same num. worker threads in Scala/Python unit tests

[ https://issues.apache.org/jira/browse/SPARK-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15687208#comment-15687208 ] Saikat Kanjilal commented on SPARK-9487: I really want to finish the effort that I started for

[jira] [Commented] (SPARK-9487) Use the same num. worker threads in Scala/Python unit tests

[ https://issues.apache.org/jira/browse/SPARK-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15684758#comment-15684758 ] Saikat Kanjilal commented on SPARK-9487: [~srowen] following up, thoughts on how to proceed on

[jira] [Commented] (SPARK-9487) Use the same num. worker threads in Scala/Python unit tests

[ https://issues.apache.org/jira/browse/SPARK-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15680080#comment-15680080 ] Saikat Kanjilal commented on SPARK-9487: Ok guess I spoke too soon :), onto the next set of

[jira] [Comment Edited] (SPARK-9487) Use the same num. worker threads in Scala/Python unit tests

[ https://issues.apache.org/jira/browse/SPARK-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15680080#comment-15680080 ] Saikat Kanjilal edited comment on SPARK-9487 at 11/19/16 11:59 PM: --- Ok

[jira] [Commented] (SPARK-9487) Use the same num. worker threads in Scala/Python unit tests

[ https://issues.apache.org/jira/browse/SPARK-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15679880#comment-15679880 ] Saikat Kanjilal commented on SPARK-9487: Ok fixed the unit test, didnt have to resort to using

[jira] [Commented] (SPARK-9487) Use the same num. worker threads in Scala/Python unit tests

[ https://issues.apache.org/jira/browse/SPARK-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15674543#comment-15674543 ] Saikat Kanjilal commented on SPARK-9487: Sean, I took a look at the code and here it is:

[jira] [Commented] (SPARK-9487) Use the same num. worker threads in Scala/Python unit tests

[ https://issues.apache.org/jira/browse/SPARK-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15665370#comment-15665370 ] Saikat Kanjilal commented on SPARK-9487: I am running the tests by the following command:

[jira] [Commented] (SPARK-9487) Use the same num. worker threads in Scala/Python unit tests

[ https://issues.apache.org/jira/browse/SPARK-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15660237#comment-15660237 ] Saikat Kanjilal commented on SPARK-9487: Understood, so I wanted a fresh look at this from a

[jira] [Comment Edited] (SPARK-9487) Use the same num. worker threads in Scala/Python unit tests

[ https://issues.apache.org/jira/browse/SPARK-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15658479#comment-15658479 ] Saikat Kanjilal edited comment on SPARK-9487 at 11/11/16 11:20 PM: --- ok

[jira] [Commented] (SPARK-9487) Use the same num. worker threads in Scala/Python unit tests

[ https://issues.apache.org/jira/browse/SPARK-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15658479#comment-15658479 ] Saikat Kanjilal commented on SPARK-9487: ok so I've spent the last hour or so doing deeper

[jira] [Commented] (SPARK-9487) Use the same num. worker threads in Scala/Python unit tests

[ https://issues.apache.org/jira/browse/SPARK-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15658144#comment-15658144 ] Saikat Kanjilal commented on SPARK-9487: No they don't which is why I asked, I will dig into these

[jira] [Commented] (SPARK-9487) Use the same num. worker threads in Scala/Python unit tests

[ https://issues.apache.org/jira/browse/SPARK-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15658055#comment-15658055 ] Saikat Kanjilal commented on SPARK-9487: [~srowen] given this is my first patch, I wanted to

[jira] [Commented] (SPARK-9487) Use the same num. worker threads in Scala/Python unit tests

[ https://issues.apache.org/jira/browse/SPARK-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15657646#comment-15657646 ] Saikat Kanjilal commented on SPARK-9487: [~srowen] if there's no further issues I will : 1) start

[jira] [Comment Edited] (SPARK-9487) Use the same num. worker threads in Scala/Python unit tests

[ https://issues.apache.org/jira/browse/SPARK-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15651806#comment-15651806 ] Saikat Kanjilal edited comment on SPARK-9487 at 11/9/16 7:22 PM: - Ok , for

[jira] [Commented] (SPARK-9487) Use the same num. worker threads in Scala/Python unit tests

[ https://issues.apache.org/jira/browse/SPARK-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15651806#comment-15651806 ] Saikat Kanjilal commented on SPARK-9487: Ok , for some odd reason my local branch had the changes

[jira] [Commented] (SPARK-9487) Use the same num. worker threads in Scala/Python unit tests

[ https://issues.apache.org/jira/browse/SPARK-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15651559#comment-15651559 ] Saikat Kanjilal commented on SPARK-9487: Sorry forgot to reply to your other question, from my

[jira] [Commented] (SPARK-9487) Use the same num. worker threads in Scala/Python unit tests

[ https://issues.apache.org/jira/browse/SPARK-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15651553#comment-15651553 ] Saikat Kanjilal commented on SPARK-9487: I definitely want to figure out these test failures as a

[jira] [Commented] (SPARK-9487) Use the same num. worker threads in Scala/Python unit tests

[ https://issues.apache.org/jira/browse/SPARK-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15651496#comment-15651496 ] Saikat Kanjilal commented on SPARK-9487: ok I have moved onto python, I am attaching a log that

[jira] [Commented] (SPARK-9487) Use the same num. worker threads in Scala/Python unit tests

[ https://issues.apache.org/jira/browse/SPARK-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15634342#comment-15634342 ] Saikat Kanjilal commented on SPARK-9487: added local[4] to repl, sparksql, streaming, all tests

[jira] [Commented] (SPARK-9487) Use the same num. worker threads in Scala/Python unit tests

[ https://issues.apache.org/jira/browse/SPARK-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15633695#comment-15633695 ] Saikat Kanjilal commented on SPARK-9487: [~srowen], [~holdenk] what are the next steps to drive

[jira] [Commented] (SPARK-9487) Use the same num. worker threads in Scala/Python unit tests

[ https://issues.apache.org/jira/browse/SPARK-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15620657#comment-15620657 ] Saikat Kanjilal commented on SPARK-9487: Added org.apache.spark.mllib unitTest changes to pull

[jira] [Comment Edited] (SPARK-9487) Use the same num. worker threads in Scala/Python unit tests

[ https://issues.apache.org/jira/browse/SPARK-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15620613#comment-15620613 ] Saikat Kanjilal edited comment on SPARK-9487 at 10/30/16 9:46 PM: --

[jira] [Commented] (SPARK-9487) Use the same num. worker threads in Scala/Python unit tests

[ https://issues.apache.org/jira/browse/SPARK-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15620613#comment-15620613 ] Saikat Kanjilal commented on SPARK-9487: [~srowen] Yes I read through that and adjusted the PR

[jira] [Comment Edited] (SPARK-9487) Use the same num. worker threads in Scala/Python unit tests

[ https://issues.apache.org/jira/browse/SPARK-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15620613#comment-15620613 ] Saikat Kanjilal edited comment on SPARK-9487 at 10/30/16 9:40 PM: --

[jira] [Commented] (SPARK-9487) Use the same num. worker threads in Scala/Python unit tests

[ https://issues.apache.org/jira/browse/SPARK-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15620392#comment-15620392 ] Saikat Kanjilal commented on SPARK-9487: PR attached here:

[jira] [Commented] (SPARK-9487) Use the same num. worker threads in Scala/Python unit tests

[ https://issues.apache.org/jira/browse/SPARK-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15599904#comment-15599904 ] Saikat Kanjilal commented on SPARK-9487: Ping on this, [~holdenk] can you let me know if I can

[jira] [Comment Edited] (SPARK-9487) Use the same num. worker threads in Scala/Python unit tests

[ https://issues.apache.org/jira/browse/SPARK-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587627#comment-15587627 ] Saikat Kanjilal edited comment on SPARK-9487 at 10/19/16 4:24 AM: --

[jira] [Comment Edited] (SPARK-9487) Use the same num. worker threads in Scala/Python unit tests

[ https://issues.apache.org/jira/browse/SPARK-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587627#comment-15587627 ] Saikat Kanjilal edited comment on SPARK-9487 at 10/19/16 4:24 AM: --

[jira] [Commented] (SPARK-9487) Use the same num. worker threads in Scala/Python unit tests

[ https://issues.apache.org/jira/browse/SPARK-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587627#comment-15587627 ] Saikat Kanjilal commented on SPARK-9487: [~holdenk] finally getting time to look at this, so I am

[jira] [Updated] (SPARK-9487) Use the same num. worker threads in Scala/Python unit tests

[ https://issues.apache.org/jira/browse/SPARK-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saikat Kanjilal updated SPARK-9487: --- Attachment: ContextCleanerSuiteResults > Use the same num. worker threads in Scala/Python

[jira] [Updated] (SPARK-9487) Use the same num. worker threads in Scala/Python unit tests

[ https://issues.apache.org/jira/browse/SPARK-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saikat Kanjilal updated SPARK-9487: --- Attachment: HeartbeatReceiverSuiteResults > Use the same num. worker threads in Scala/Python

[jira] [Commented] (SPARK-9487) Use the same num. worker threads in Scala/Python unit tests

[ https://issues.apache.org/jira/browse/SPARK-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15578723#comment-15578723 ] Saikat Kanjilal commented on SPARK-9487: Synched the code, am familiarizing myself first with how

[jira] [Commented] (SPARK-14516) Clustering evaluator

[ https://issues.apache.org/jira/browse/SPARK-14516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15570185#comment-15570185 ] Saikat Kanjilal commented on SPARK-14516: - Hello, New to spark and am interested in helping with

[jira] [Commented] (SPARK-12372) Document limitations of MLlib local linear algebra

[ https://issues.apache.org/jira/browse/SPARK-12372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15570152#comment-15570152 ] Saikat Kanjilal commented on SPARK-12372: - @josephkb, new to contributing to spark, is this

[jira] [Commented] (SPARK-9487) Use the same num. worker threads in Scala/Python unit tests

[ https://issues.apache.org/jira/browse/SPARK-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15570122#comment-15570122 ] Saikat Kanjilal commented on SPARK-9487: Hello All, Can I help with this in anyway? Thanks > Use

[jira] [Comment Edited] (SPARK-14212) Add configuration element for --packages option

[ https://issues.apache.org/jira/browse/SPARK-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15570125#comment-15570125 ] Saikat Kanjilal edited comment on SPARK-14212 at 10/12/16 10:53 PM:

[jira] [Commented] (SPARK-14212) Add configuration element for --packages option

[ https://issues.apache.org/jira/browse/SPARK-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15570125#comment-15570125 ] Saikat Kanjilal commented on SPARK-14212: - heldenk@ can I help out with this? > Add

[jira] [Comment Edited] (SPARK-14212) Add configuration element for --packages option

[ https://issues.apache.org/jira/browse/SPARK-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15570125#comment-15570125 ] Saikat Kanjilal edited comment on SPARK-14212 at 10/12/16 10:54 PM:

[jira] [Commented] (SPARK-14302) Python examples code merge and clean up

[ https://issues.apache.org/jira/browse/SPARK-14302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15266029#comment-15266029 ] Saikat Kanjilal commented on SPARK-14302: - Works for me, so what else can I help with? > Python

[jira] [Commented] (SPARK-14302) Python examples code merge and clean up

[ https://issues.apache.org/jira/browse/SPARK-14302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15265118#comment-15265118 ] Saikat Kanjilal commented on SPARK-14302: - And here's the duplication inside the mllib directory:

[jira] [Commented] (SPARK-14302) Python examples code merge and clean up

[ https://issues.apache.org/jira/browse/SPARK-14302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15264207#comment-15264207 ] Saikat Kanjilal commented on SPARK-14302: - Ok I finished my initial assessment of the ml

[jira] [Commented] (SPARK-14302) Python examples code merge and clean up

[ https://issues.apache.org/jira/browse/SPARK-14302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15262825#comment-15262825 ] Saikat Kanjilal commented on SPARK-14302: - Ok thanks for the clarifications, so that means only

[jira] [Commented] (SPARK-14302) Python examples code merge and clean up

[ https://issues.apache.org/jira/browse/SPARK-14302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15262796#comment-15262796 ] Saikat Kanjilal commented on SPARK-14302: - Did you see my question above? Which of the

[jira] [Commented] (SPARK-14302) Python examples code merge and clean up

[ https://issues.apache.org/jira/browse/SPARK-14302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15230434#comment-15230434 ] Saikat Kanjilal commented on SPARK-14302: - Xusen, I didnt hear back on the above question, I will

[jira] [Comment Edited] (SPARK-14302) Python examples code merge and clean up

[ https://issues.apache.org/jira/browse/SPARK-14302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15222649#comment-15222649 ] Saikat Kanjilal edited comment on SPARK-14302 at 4/2/16 2:09 AM: - next

[jira] [Comment Edited] (SPARK-14302) Python examples code merge and clean up

[ https://issues.apache.org/jira/browse/SPARK-14302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15222649#comment-15222649 ] Saikat Kanjilal edited comment on SPARK-14302 at 4/2/16 2:04 AM: - next

[jira] [Commented] (SPARK-14302) Python examples code merge and clean up

[ https://issues.apache.org/jira/browse/SPARK-14302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15222649#comment-15222649 ] Saikat Kanjilal commented on SPARK-14302: - next question, which of the directories should contain

[jira] [Commented] (SPARK-14302) Python examples code merge and clean up

[ https://issues.apache.org/jira/browse/SPARK-14302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15220330#comment-15220330 ] Saikat Kanjilal commented on SPARK-14302: - Ok thanks, so which should I take the python or the

[jira] [Comment Edited] (SPARK-14302) Python examples code merge and clean up

[ https://issues.apache.org/jira/browse/SPARK-14302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15220312#comment-15220312 ] Saikat Kanjilal edited comment on SPARK-14302 at 3/31/16 5:51 PM: -- If I

[jira] [Commented] (SPARK-14302) Python examples code merge and clean up

[ https://issues.apache.org/jira/browse/SPARK-14302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15220312#comment-15220312 ] Saikat Kanjilal commented on SPARK-14302: - If I understand this correctly the goal is just to

[jira] [Commented] (SPARK-14302) Python examples code merge and clean up

[ https://issues.apache.org/jira/browse/SPARK-14302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15220303#comment-15220303 ] Saikat Kanjilal commented on SPARK-14302: - Hi, Can I help with this issue, I've been wanting to