[jira] [Assigned] (SPARK-7551) Don't split by dot if within backticks for DataFrame attribute resolution

2015-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7551: --- Assignee: Apache Spark (was: Wenchen Fan) > Don't split by dot if within backticks for DataF

[jira] [Assigned] (SPARK-7551) Don't split by dot if within backticks for DataFrame attribute resolution

2015-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7551: --- Assignee: Wenchen Fan (was: Apache Spark) > Don't split by dot if within backticks for DataF

[jira] [Commented] (SPARK-7551) Don't split by dot if within backticks for DataFrame attribute resolution

2015-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14539385#comment-14539385 ] Apache Spark commented on SPARK-7551: - User 'cloud-fan' has created a pull request for

[jira] [Commented] (SPARK-7556) User guide update for feature transformer: Binarizer

2015-05-11 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14539382#comment-14539382 ] Liang-Chi Hsieh commented on SPARK-7556: OK. > User guide update for feature tran

[jira] [Commented] (SPARK-7423) spark.ml Classifier predict should not convert vectors to dense format

2015-05-11 Thread George Dittmar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14539311#comment-14539311 ] George Dittmar commented on SPARK-7423: --- Will have a pr for this soon. Just made the

[jira] [Commented] (SPARK-7422) Add argmax to Vector, SparseVector

2015-05-11 Thread George Dittmar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14539314#comment-14539314 ] George Dittmar commented on SPARK-7422: --- Finishing tests for this JIRA with PR inbou

[jira] [Updated] (SPARK-2870) Thorough schema inference directly on RDDs of Python dictionaries

2015-05-11 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-2870: --- Issue Type: Sub-task (was: Improvement) Parent: SPARK-6116 > Thorough schema inference direct

[jira] [Assigned] (SPARK-7545) Bernoulli NaiveBayes should validate data

2015-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7545: --- Assignee: Leah McGuire (was: Apache Spark) > Bernoulli NaiveBayes should validate data > ---

[jira] [Assigned] (SPARK-7545) Bernoulli NaiveBayes should validate data

2015-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7545: --- Assignee: Apache Spark (was: Leah McGuire) > Bernoulli NaiveBayes should validate data > ---

[jira] [Commented] (SPARK-7545) Bernoulli NaiveBayes should validate data

2015-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14539272#comment-14539272 ] Apache Spark commented on SPARK-7545: - User 'leahmcguire' has created a pull request f

[jira] [Commented] (SPARK-7321) Add Column expression for conditional statements (if, case)

2015-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14539268#comment-14539268 ] Apache Spark commented on SPARK-7321: - User 'rxin' has created a pull request for this

[jira] [Created] (SPARK-7557) User guide update for feature transformer: HashingTF

2015-05-11 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-7557: Summary: User guide update for feature transformer: HashingTF Key: SPARK-7557 URL: https://issues.apache.org/jira/browse/SPARK-7557 Project: Spark Is

[jira] [Commented] (SPARK-7556) User guide update for feature transformer: Binarizer

2015-05-11 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14539266#comment-14539266 ] Joseph K. Bradley commented on SPARK-7556: -- [~viirya] Would you be able to add a

[jira] [Created] (SPARK-7556) User guide update for feature transformer: Binarizer

2015-05-11 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-7556: Summary: User guide update for feature transformer: Binarizer Key: SPARK-7556 URL: https://issues.apache.org/jira/browse/SPARK-7556 Project: Spark Is

[jira] [Commented] (SPARK-7443) MLlib 1.4 QA plan

2015-05-11 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14539258#comment-14539258 ] Joseph K. Bradley commented on SPARK-7443: -- Note: The Naive Bayes user guide sect

[jira] [Updated] (SPARK-7272) User guide update for PMML model export

2015-05-11 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-7272: - Summary: User guide update for PMML model export (was: User guide section for PMML model

[jira] [Updated] (SPARK-7555) User guide update for ElasticNet

2015-05-11 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-7555: - Summary: User guide update for ElasticNet (was: User guide section for ElasticNet) > Use

[jira] [Updated] (SPARK-7496) User guide update for Online LDA

2015-05-11 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-7496: - Summary: User guide update for Online LDA (was: Update Programming guide with Online LDA)

[jira] [Updated] (SPARK-7272) User guide section for PMML model export

2015-05-11 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-7272: - Summary: User guide section for PMML model export (was: User guide for PMML model export)

[jira] [Commented] (SPARK-2870) Thorough schema inference directly on RDDs of Python dictionaries

2015-05-11 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14539255#comment-14539255 ] Nicholas Chammas commented on SPARK-2870: - Another use case for this feature is en

[jira] [Commented] (SPARK-7555) User guide section for ElasticNet

2015-05-11 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14539256#comment-14539256 ] Joseph K. Bradley commented on SPARK-7555: -- [~dbtsai] I took the liberty of assig

[jira] [Created] (SPARK-7555) User guide section for ElasticNet

2015-05-11 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-7555: Summary: User guide section for ElasticNet Key: SPARK-7555 URL: https://issues.apache.org/jira/browse/SPARK-7555 Project: Spark Issue Type: Documenta

[jira] [Updated] (SPARK-7321) Add Column expression for conditional statements (if, case)

2015-05-11 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-7321: --- Assignee: Chen Song > Add Column expression for conditional statements (if, case) > --

[jira] [Updated] (SPARK-7443) MLlib 1.4 QA plan

2015-05-11 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-7443: - Description: TODO: create JIRAs for each task and assign them accordingly. h2. API * Che

[jira] [Updated] (SPARK-7443) MLlib 1.4 QA plan

2015-05-11 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-7443: - Description: TODO: create JIRAs for each task and assign them accordingly. h2. API * Che

[jira] [Commented] (SPARK-7296) Timeline view for Stage page

2015-05-11 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14539245#comment-14539245 ] Patrick Wendell commented on SPARK-7296: Actually maybe there is a chance we can d

[jira] [Updated] (SPARK-7296) Timeline view for Stage page

2015-05-11 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-7296: --- Target Version/s: 1.4.0, 1.5.0 (was: 1.5.0) > Timeline view for Stage page >

[jira] [Comment Edited] (SPARK-7296) Timeline view for Stage page

2015-05-11 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14539240#comment-14539240 ] Patrick Wendell edited comment on SPARK-7296 at 5/12/15 4:53 AM: ---

[jira] [Updated] (SPARK-7296) Timeline view for Stage page

2015-05-11 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-7296: --- Target Version/s: 1.5.0 (was: 1.4.0) > Timeline view for Stage page > ---

[jira] [Commented] (SPARK-7296) Timeline view for Stage page

2015-05-11 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14539240#comment-14539240 ] Patrick Wendell commented on SPARK-7296: Personally not sure I have time to review

[jira] [Commented] (SPARK-7547) ElasticNet example code

2015-05-11 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14539216#comment-14539216 ] DB Tsai commented on SPARK-7547: Sure! I'll add the example code and documentation in this

[jira] [Updated] (SPARK-7482) Rename some DataFrame API methods in SparkR to match their counterparts in Scala

2015-05-11 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman updated SPARK-7482: - Assignee: Sun Rui > Rename some DataFrame API methods in SparkR to match their cou

[jira] [Updated] (SPARK-7526) Specify ip of RBackend, MonitorServer and RRDD Socket server

2015-05-11 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman updated SPARK-7526: - Assignee: Weizhong > Specify ip of RBackend, MonitorServer and RRDD Socket server

[jira] [Updated] (SPARK-7435) Make DataFrame.show() consistent with that of Scala and pySpark

2015-05-11 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-7435: --- Assignee: Rekha Joshi > Make DataFrame.show() consistent with that of Scala and pySpark >

[jira] [Commented] (SPARK-7435) Make DataFrame.show() consistent with that of Scala and pySpark

2015-05-11 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14539194#comment-14539194 ] Reynold Xin commented on SPARK-7435: I just did it. > Make DataFrame.show() consiste

[jira] [Updated] (SPARK-7482) Rename some DataFrame API methods in SparkR to match their counterparts in Scala

2015-05-11 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman updated SPARK-7482: - Target Version/s: 1.4.0 > Rename some DataFrame API methods in SparkR to match the

[jira] [Commented] (SPARK-7435) Make DataFrame.show() consistent with that of Scala and pySpark

2015-05-11 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14539192#comment-14539192 ] Shivaram Venkataraman commented on SPARK-7435: -- [~srowen] [~pwendell] Could y

[jira] [Resolved] (SPARK-7435) Make DataFrame.show() consistent with that of Scala and pySpark

2015-05-11 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-7435. -- Resolution: Pending Closed Fix Version/s: 1.4.0 Issue resolved by pull re

[jira] [Updated] (SPARK-7227) Support fillna / dropna in R DataFrame

2015-05-11 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman updated SPARK-7227: - Assignee: Sun Rui > Support fillna / dropna in R DataFrame > -

[jira] [Commented] (SPARK-7227) Support fillna / dropna in R DataFrame

2015-05-11 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14539184#comment-14539184 ] Shivaram Venkataraman commented on SPARK-7227: -- [~sunrui] mentioned to me off

[jira] [Commented] (SPARK-7226) Support math functions in R DataFrame

2015-05-11 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14539181#comment-14539181 ] Reynold Xin commented on SPARK-7226: [~shivaram] somebody available to fix this? > Su

[jira] [Commented] (SPARK-7227) Support fillna / dropna in R DataFrame

2015-05-11 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14539180#comment-14539180 ] Reynold Xin commented on SPARK-7227: [~shivaram] somebody available to fix this? > Su

[jira] [Closed] (SPARK-7035) Drop __getattr__ on pyspark.sql.DataFrame

2015-05-11 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-7035. -- Resolution: Won't Fix I'm closing this one for now. We can continue the discussion. > Drop __getattr__

[jira] [Closed] (SPARK-6198) Support "select current_database()"

2015-05-11 Thread Zhongshuai Pei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhongshuai Pei closed SPARK-6198. - Resolution: Won't Fix > Support "select current_database()" > ---

[jira] [Closed] (SPARK-5129) make SqlContext support "select date +/- XX DAYS from table"

2015-05-11 Thread Zhongshuai Pei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhongshuai Pei closed SPARK-5129. - Resolution: Won't Fix > make SqlContext support "select date +/- XX DAYS from table" > -

[jira] [Closed] (SPARK-6768) Do not support "float/double union decimal or decimal(a ,b) union decimal(c, d)"

2015-05-11 Thread Zhongshuai Pei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhongshuai Pei closed SPARK-6768. - Resolution: Fixed > Do not support "float/double union decimal or decimal(a ,b) union decimal(c,

[jira] [Closed] (SPARK-7026) LeftSemiJoin can not work when it has both equal condition and not equal condition.

2015-05-11 Thread Zhongshuai Pei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhongshuai Pei closed SPARK-7026. - Resolution: Duplicate > LeftSemiJoin can not work when it has both equal condition and not equal

[jira] [Resolved] (SPARK-7509) Add drop column to Python DataFrame API

2015-05-11 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-7509. Resolution: Fixed Fix Version/s: 1.4.0 > Add drop column to Python DataFrame API > --

[jira] [Updated] (SPARK-7554) Throw exception when an active StreamingContext is used to create DStreams and output operations

2015-05-11 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-7554: - Priority: Blocker (was: Critical) > Throw exception when an active StreamingContext is used to cr

[jira] [Updated] (SPARK-7554) Throw exception when an active StreamingContext is used to create DStreams and output operations

2015-05-11 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-7554: - Summary: Throw exception when an active StreamingContext is used to create DStreams and output ope

[jira] [Commented] (SPARK-7554) Throw exception when an active StreamingContext is used to create DStreams and output operations

2015-05-11 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14539143#comment-14539143 ] Tathagata Das commented on SPARK-7554: -- Currently, adding DStreams to an active conte

[jira] [Updated] (SPARK-7554) Throw errors when an active StreamingContext is used to create DStreams and output operations

2015-05-11 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-7554: - Component/s: Streaming Target Version/s: 1.4.0 > Throw errors when an active StreamingCon

[jira] [Created] (SPARK-7554) Throw errors when an active StreamingContext is used to create DStreams and output operations

2015-05-11 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-7554: Summary: Throw errors when an active StreamingContext is used to create DStreams and output operations Key: SPARK-7554 URL: https://issues.apache.org/jira/browse/SPARK-7554

[jira] [Assigned] (SPARK-7553) Add methods to maintain a singleton StreamingContext

2015-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7553: --- Assignee: Tathagata Das (was: Apache Spark) > Add methods to maintain a singleton StreamingC

[jira] [Commented] (SPARK-7553) Add methods to maintain a singleton StreamingContext

2015-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14539140#comment-14539140 ] Apache Spark commented on SPARK-7553: - User 'tdas' has created a pull request for this

[jira] [Assigned] (SPARK-7553) Add methods to maintain a singleton StreamingContext

2015-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7553: --- Assignee: Apache Spark (was: Tathagata Das) > Add methods to maintain a singleton StreamingC

[jira] [Updated] (SPARK-7553) Add methods to maintain a singleton StreamingContext

2015-05-11 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-7553: - Description: In a REPL/notebook environment, its very easy to lose a reference to a StreamingCont

[jira] [Updated] (SPARK-7553) Add methods to maintain a singleton StreamingContext

2015-05-11 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-7553: - Description: In a REPL/notebook environment, its very easy to lose a reference to a StreamingCont

[jira] [Updated] (SPARK-7552) Close files correctly when iteration is finished in WAL recovery

2015-05-11 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-7552: --- Labels: backport-needed (was: ) > Close files correctly when iteration is finished in WAL recovery >

[jira] [Commented] (SPARK-7552) Close files correctly when iteration is finished in WAL recovery

2015-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14539128#comment-14539128 ] Apache Spark commented on SPARK-7552: - User 'jerryshao' has created a pull request for

[jira] [Assigned] (SPARK-7552) Close files correctly when iteration is finished in WAL recovery

2015-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7552: --- Assignee: (was: Apache Spark) > Close files correctly when iteration is finished in WAL r

[jira] [Assigned] (SPARK-7552) Close files correctly when iteration is finished in WAL recovery

2015-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7552: --- Assignee: Apache Spark > Close files correctly when iteration is finished in WAL recovery > -

[jira] [Commented] (SPARK-4128) Create instructions on fully building Spark in Intellij

2015-05-11 Thread Christian Kadner (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14539122#comment-14539122 ] Christian Kadner commented on SPARK-4128: - Hi Patrick, I recently set up my Intel

[jira] [Created] (SPARK-7553) Add methods to maintain a singleton StreamingContext

2015-05-11 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-7553: Summary: Add methods to maintain a singleton StreamingContext Key: SPARK-7553 URL: https://issues.apache.org/jira/browse/SPARK-7553 Project: Spark Issue Typ

[jira] [Updated] (SPARK-7553) Add methods to maintain a singleton StreamingContext

2015-05-11 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-7553: - Description: In a REPL/notebook environment, its very easy to lose a reference to a StreamingCont

[jira] [Resolved] (SPARK-7437) Fold "literal in (item1, item2, ..., literal, ...)" into true or false directly

2015-05-11 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-7437. - Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 5972 [https:/

[jira] [Resolved] (SPARK-7411) CTAS parser is incomplete

2015-05-11 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-7411. - Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 5963 [https:/

[jira] [Updated] (SPARK-7324) Add DataFrame.dropDuplicates

2015-05-11 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-7324: Assignee: Reynold Xin > Add DataFrame.dropDuplicates > > >

[jira] [Resolved] (SPARK-7324) Add DataFrame.dropDuplicates

2015-05-11 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-7324. - Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 6066 [https:/

[jira] [Commented] (SPARK-7531) Install GPG on Jenkins machines

2015-05-11 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14539097#comment-14539097 ] Patrick Wendell commented on SPARK-7531: Yep - that one should work. I've actually

[jira] [Updated] (SPARK-6876) DataFrame.na.replace value support for Python

2015-05-11 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-6876: -- Assignee: Adrian Wang > DataFrame.na.replace value support for Python >

[jira] [Resolved] (SPARK-7520) Install Jekyll On Jenkins Machines

2015-05-11 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-7520. Resolution: Fixed Fix Version/s: 1.4.0 All green - awesome thanks [~shaneknapp]! > I

[jira] [Updated] (SPARK-7150) SQLContext.range()

2015-05-11 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-7150: -- Assignee: Adrian Wang > SQLContext.range() > -- > > Key: SPARK-7150 >

[jira] [Updated] (SPARK-7322) Add DataFrame DSL for window function support

2015-05-11 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-7322: -- Assignee: Cheng Hao > Add DataFrame DSL for window function support > --

[jira] [Updated] (SPARK-7320) Add rollup and cube support to DataFrame DSL

2015-05-11 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-7320: -- Assignee: Cheng Hao > Add rollup and cube support to DataFrame DSL > ---

[jira] [Resolved] (SPARK-7331) Create HiveConf per application instead of per query in HiveQl.scala

2015-05-11 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-7331. - Resolution: Fixed Fix Version/s: 1.2.3 Issue resolved by pull request 6036 [https:/

[jira] [Resolved] (SPARK-7538) Kafka stream fails: java.lang.NoClassDefFound com/yammer/metrics/core/Gauge

2015-05-11 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-7538. Resolution: Fixed This was a cross post from the mailing list. The poster closed the thread

[jira] [Updated] (SPARK-7551) Don't split by dot if within backticks for DataFrame attribute resolution

2015-05-11 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-7551: --- Description: DataFrame's resolve: {code} protected[sql] def resolve(colName: String): NamedExpressio

[jira] [Commented] (SPARK-7540) PMML correctness check

2015-05-11 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14539085#comment-14539085 ] Joseph K. Bradley commented on SPARK-7540: -- [~selvinsource] How much of this JIR

[jira] [Resolved] (SPARK-7530) Add API to get the current state of a StreamingContext

2015-05-11 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-7530. -- Resolution: Fixed Fix Version/s: 1.4.0 > Add API to get the current state of a StreamingC

[jira] [Created] (SPARK-7552) Close files correctly when iteration is finished in WAL recovery

2015-05-11 Thread Saisai Shao (JIRA)
Saisai Shao created SPARK-7552: -- Summary: Close files correctly when iteration is finished in WAL recovery Key: SPARK-7552 URL: https://issues.apache.org/jira/browse/SPARK-7552 Project: Spark I

[jira] [Commented] (SPARK-7545) Bernoulli NaiveBayes should validate data

2015-05-11 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14539081#comment-14539081 ] Joseph K. Bradley commented on SPARK-7545: -- OK, I appreciate it! > Bernoulli Nai

[jira] [Commented] (SPARK-7545) Bernoulli NaiveBayes should validate data

2015-05-11 Thread Leah McGuire (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14539078#comment-14539078 ] Leah McGuire commented on SPARK-7545: - Yes, I think I can get it in. > Bernoulli Nai

[jira] [Commented] (SPARK-7545) Bernoulli NaiveBayes should validate data

2015-05-11 Thread Leah McGuire (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14539077#comment-14539077 ] Leah McGuire commented on SPARK-7545: - I think I can get it in :-) On Mon, May 11, 20

[jira] [Issue Comment Deleted] (SPARK-7545) Bernoulli NaiveBayes should validate data

2015-05-11 Thread Leah McGuire (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leah McGuire updated SPARK-7545: Comment: was deleted (was: Yes, I think I can get it in. ) > Bernoulli NaiveBayes should validate d

[jira] [Updated] (SPARK-5893) Add Bucketizer

2015-05-11 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-5893: - Assignee: Xusen Yin (was: Joseph K. Bradley) > Add Bucketizer > -- > >

[jira] [Resolved] (SPARK-5893) Add Bucketizer

2015-05-11 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-5893. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 5980 [https

[jira] [Commented] (SPARK-7413) Time to write shuffle spill files is not captured in ShuffleWriteMetrics

2015-05-11 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14539067#comment-14539067 ] Josh Rosen commented on SPARK-7413: --- Actually, it looks like we sort-of try to do this i

[jira] [Updated] (SPARK-7269) Incorrect aggregation analysis

2015-05-11 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-7269: -- Description: In a case insensitive analyzer (HiveContext), the attribute name captial differences will

[jira] [Commented] (SPARK-7509) Add drop column to Python DataFrame API

2015-05-11 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14538978#comment-14538978 ] Nicholas Chammas commented on SPARK-7509: - Oh, well nevermind then. :) > Add drop

[jira] [Assigned] (SPARK-7509) Add drop column to Python DataFrame API

2015-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7509: --- Assignee: Apache Spark (was: Reynold Xin) > Add drop column to Python DataFrame API > --

[jira] [Commented] (SPARK-7509) Add drop column to Python DataFrame API

2015-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14538977#comment-14538977 ] Apache Spark commented on SPARK-7509: - User 'rxin' has created a pull request for this

[jira] [Assigned] (SPARK-7509) Add drop column to Python DataFrame API

2015-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7509: --- Assignee: Reynold Xin (was: Apache Spark) > Add drop column to Python DataFrame API > --

[jira] [Assigned] (SPARK-7509) Add drop column to Python DataFrame API

2015-05-11 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin reassigned SPARK-7509: -- Assignee: Reynold Xin > Add drop column to Python DataFrame API > -

[jira] [Commented] (SPARK-7549) Support aggregating over nested fields

2015-05-11 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14538965#comment-14538965 ] Nicholas Chammas commented on SPARK-7549: - To provide a motivating example for the

[jira] [Updated] (SPARK-7509) Add drop column to Python DataFrame API

2015-05-11 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-7509: Target Version/s: 1.4.0 I'm targeting this for 1.4.0, though that's optimistic given that we

[jira] [Commented] (SPARK-7550) Support setting the right schema & serde when writing to Hive metastore

2015-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14538936#comment-14538936 ] Apache Spark commented on SPARK-7550: - User 'rxin' has created a pull request for this

[jira] [Assigned] (SPARK-7550) Support setting the right schema & serde when writing to Hive metastore

2015-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7550: --- Assignee: (was: Apache Spark) > Support setting the right schema & serde when writing to

[jira] [Assigned] (SPARK-7550) Support setting the right schema & serde when writing to Hive metastore

2015-05-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7550: --- Assignee: Apache Spark > Support setting the right schema & serde when writing to Hive metast

[jira] [Commented] (SPARK-7548) Add explode expression

2015-05-11 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14538937#comment-14538937 ] Nicholas Chammas commented on SPARK-7548: - To provide a motivating example for the

  1   2   3   >