[jira] [Commented] (SPARK-27352) Apply for translation of the Chinese version, I hope to get authorization!

[ https://issues.apache.org/jira/browse/SPARK-27352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16811835#comment-16811835 ] Teng Peng commented on SPARK-27352: --- I would say go ahead and send a PR to add the lin

[jira] [Commented] (SPARK-27352) Apply for translation of the Chinese version, I hope to get authorization!

[ https://issues.apache.org/jira/browse/SPARK-27352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16811765#comment-16811765 ] Teng Peng commented on SPARK-27352: --- Correct me if I am wrong. I do not think any auth

[jira] [Created] (SPARK-24907) Migrate JDBC data source to DataSource API v2

Teng Peng created SPARK-24907: - Summary: Migrate JDBC data source to DataSource API v2 Key: SPARK-24907 URL: https://issues.apache.org/jira/browse/SPARK-24907 Project: Spark Issue Type: New Featu

[jira] [Updated] (SPARK-23507) Migrate file-based data sources to data source v2

[ https://issues.apache.org/jira/browse/SPARK-23507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Teng Peng updated SPARK-23507: -- Issue Type: Umbrella (was: Improvement) > Migrate file-based data sources to data source v2 > ---

[jira] [Issue Comment Deleted] (SPARK-18755) Add Randomized Grid Search to Spark ML

[ https://issues.apache.org/jira/browse/SPARK-18755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Teng Peng updated SPARK-18755: -- Comment: was deleted (was: [~yuhaoyan] Is this what you are looking for: after we build the grid, we

[jira] [Updated] (SPARK-22911) Migrate structured streaming sources to new DataSourceV2 APIs

[ https://issues.apache.org/jira/browse/SPARK-22911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Teng Peng updated SPARK-22911: -- Issue Type: Umbrella (was: Improvement) > Migrate structured streaming sources to new DataSourceV2 AP

[jira] [Commented] (SPARK-24516) PySpark Bindings for K8S - make Python 3 the default

[ https://issues.apache.org/jira/browse/SPARK-24516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16508945#comment-16508945 ] Teng Peng commented on SPARK-24516: --- +1. > PySpark Bindings for K8S - make Python 3 t

[jira] [Updated] (SPARK-21199) Its not possible to impute Vector types

[ https://issues.apache.org/jira/browse/SPARK-21199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Teng Peng updated SPARK-21199: -- Component/s: (was: Spark Core) ML > Its not possible to impute Vector types > ---

[jira] [Issue Comment Deleted] (SPARK-22943) OneHotEncoder supports manual specification of categorySizes

[ https://issues.apache.org/jira/browse/SPARK-22943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Teng Peng updated SPARK-22943: -- Comment: was deleted (was: This issue looks quiet interesting, but can you be more specific about "co

[jira] [Commented] (SPARK-24431) wrong areaUnderPR calculation in BinaryClassificationEvaluator

[ https://issues.apache.org/jira/browse/SPARK-24431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16503217#comment-16503217 ] Teng Peng commented on SPARK-24431: --- [~Ben2018] The article makes sense to me. It seem

[jira] [Comment Edited] (SPARK-24431) wrong areaUnderPR calculation in BinaryClassificationEvaluator

[ https://issues.apache.org/jira/browse/SPARK-24431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16499109#comment-16499109 ] Teng Peng edited comment on SPARK-24431 at 6/2/18 6:48 PM: --- I

[jira] [Comment Edited] (SPARK-24431) wrong areaUnderPR calculation in BinaryClassificationEvaluator

[ https://issues.apache.org/jira/browse/SPARK-24431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16499109#comment-16499109 ] Teng Peng edited comment on SPARK-24431 at 6/2/18 6:47 PM: --- I

[jira] [Comment Edited] (SPARK-24431) wrong areaUnderPR calculation in BinaryClassificationEvaluator

[ https://issues.apache.org/jira/browse/SPARK-24431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16499109#comment-16499109 ] Teng Peng edited comment on SPARK-24431 at 6/2/18 6:47 PM: --- I

[jira] [Comment Edited] (SPARK-24431) wrong areaUnderPR calculation in BinaryClassificationEvaluator

[ https://issues.apache.org/jira/browse/SPARK-24431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16499109#comment-16499109 ] Teng Peng edited comment on SPARK-24431 at 6/2/18 6:46 PM: --- I

[jira] [Commented] (SPARK-24431) wrong areaUnderPR calculation in BinaryClassificationEvaluator

[ https://issues.apache.org/jira/browse/SPARK-24431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16499109#comment-16499109 ] Teng Peng commented on SPARK-24431: --- I am trying to understand this description. What'

[jira] [Issue Comment Deleted] (SPARK-24391) to_json/from_json should support arrays of primitives, and more generally all JSON

[ https://issues.apache.org/jira/browse/SPARK-24391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Teng Peng updated SPARK-24391: -- Comment: was deleted (was: My plan is to follow the Spark-19849 & Spark-21513 to support more primitiv

[jira] [Commented] (SPARK-24391) to_json/from_json should support arrays of primitives, and more generally all JSON

[ https://issues.apache.org/jira/browse/SPARK-24391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16491745#comment-16491745 ] Teng Peng commented on SPARK-24391: --- My plan is to follow the Spark-19849 & Spark-21513

[jira] [Commented] (SPARK-24269) Infer nullability rather than declaring all columns as nullable

[ https://issues.apache.org/jira/browse/SPARK-24269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16482156#comment-16482156 ] Teng Peng commented on SPARK-24269: --- Yes, I understand this rationale too. However, I a

[jira] [Commented] (SPARK-24269) Infer nullability rather than declaring all columns as nullable

[ https://issues.apache.org/jira/browse/SPARK-24269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16481812#comment-16481812 ] Teng Peng commented on SPARK-24269: --- Does it make sense to infer nullability from JSON

[jira] [Commented] (SPARK-22943) OneHotEncoder supports manual specification of categorySizes

[ https://issues.apache.org/jira/browse/SPARK-22943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16461006#comment-16461006 ] Teng Peng commented on SPARK-22943: --- This issue looks quiet interesting, but can you be

[jira] [Commented] (SPARK-23180) RFormulaModel should have labels member

[ https://issues.apache.org/jira/browse/SPARK-23180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16460967#comment-16460967 ] Teng Peng commented on SPARK-23180: --- Can you give me an example for 1. the current work

[jira] [Updated] (SPARK-23171) Reduce the time costs of the rule runs that do not change the plans

[ https://issues.apache.org/jira/browse/SPARK-23171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Teng Peng updated SPARK-23171: -- Issue Type: Improvement (was: Umbrella) > Reduce the time costs of the rule runs that do not change th

[jira] [Updated] (SPARK-23171) Reduce the time costs of the rule runs that do not change the plans

[ https://issues.apache.org/jira/browse/SPARK-23171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Teng Peng updated SPARK-23171: -- Issue Type: Umbrella (was: Improvement) > Reduce the time costs of the rule runs that do not change th

[jira] [Commented] (SPARK-24024) Fix deviance calculations in GLM to handle corner cases

[ https://issues.apache.org/jira/browse/SPARK-24024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16443892#comment-16443892 ] Teng Peng commented on SPARK-24024: --- I will first reproduce the issue he has, check how

[jira] [Created] (SPARK-24024) Fix deviance calculations in GLM to handle corner cases

Teng Peng created SPARK-24024: - Summary: Fix deviance calculations in GLM to handle corner cases Key: SPARK-24024 URL: https://issues.apache.org/jira/browse/SPARK-24024 Project: Spark Issue Type:

[jira] [Commented] (SPARK-23740) Add FPGrowth Param for filtering out very common items

[ https://issues.apache.org/jira/browse/SPARK-23740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16413199#comment-16413199 ] Teng Peng commented on SPARK-23740: --- I suppose `beforehand` means before itemsets been

[jira] [Comment Edited] (SPARK-19208) MultivariateOnlineSummarizer performance optimization

[ https://issues.apache.org/jira/browse/SPARK-19208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16407451#comment-16407451 ] Teng Peng edited comment on SPARK-19208 at 3/21/18 4:44 AM: [

[jira] [Commented] (SPARK-19208) MultivariateOnlineSummarizer performance optimization

[ https://issues.apache.org/jira/browse/SPARK-19208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16407451#comment-16407451 ] Teng Peng commented on SPARK-19208: --- [~timhunter] Has the Jira ticket been opened? I be

[jira] [Commented] (SPARK-23537) Logistic Regression without standardization

[ https://issues.apache.org/jira/browse/SPARK-23537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16387276#comment-16387276 ] Teng Peng commented on SPARK-23537: --- This is a quiet interesting question and I do not

[jira] [Created] (SPARK-23578) Add multicolumn support for Binarizer

Teng Peng created SPARK-23578: - Summary: Add multicolumn support for Binarizer Key: SPARK-23578 URL: https://issues.apache.org/jira/browse/SPARK-23578 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-20133) User guide for spark.ml.stat.ChiSquareTest

[ https://issues.apache.org/jira/browse/SPARK-20133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16260095#comment-16260095 ] Teng Peng commented on SPARK-20133: --- I believe the documentation, including user guide

[jira] [Resolved] (SPARK-22449) Add BIC for GLM

[ https://issues.apache.org/jira/browse/SPARK-22449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Teng Peng resolved SPARK-22449. --- Resolution: Later > Add BIC for GLM > --- > > Key: SPARK-22449 >

[jira] [Issue Comment Deleted] (SPARK-22359) Improve the test coverage of window functions

[ https://issues.apache.org/jira/browse/SPARK-22359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Teng Peng updated SPARK-22359: -- Comment: was deleted (was: [~jiangxb] If I can have 1 test as reference, I will figure out the rests m

[jira] [Commented] (SPARK-22359) Improve the test coverage of window functions

[ https://issues.apache.org/jira/browse/SPARK-22359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16241392#comment-16241392 ] Teng Peng commented on SPARK-22359: --- [~jiangxb] If I can have 1 test as reference, I wi

[jira] [Issue Comment Deleted] (SPARK-11502) Word2VecSuite needs appropriate checks

[ https://issues.apache.org/jira/browse/SPARK-11502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Teng Peng updated SPARK-11502: -- Comment: was deleted (was: I am interested in this one. My plan is to compare the test against 1. othe

[jira] [Commented] (SPARK-20077) Documentation for ml.stats.Correlation

[ https://issues.apache.org/jira/browse/SPARK-20077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16239846#comment-16239846 ] Teng Peng commented on SPARK-20077: --- [~srowen] On this pagehttps://spark.apache.org/doc

[jira] [Updated] (SPARK-22449) Add BIC for GLM

[ https://issues.apache.org/jira/browse/SPARK-22449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Teng Peng updated SPARK-22449: -- Description: Currently, we only have AIC for GLM. BIC is another "similar" criterion widely used and i

[jira] [Created] (SPARK-22449) Add BIC for GLM

Teng Peng created SPARK-22449: - Summary: Add BIC for GLM Key: SPARK-22449 URL: https://issues.apache.org/jira/browse/SPARK-22449 Project: Spark Issue Type: Improvement Components: ML

[jira] [Commented] (SPARK-18755) Add Randomized Grid Search to Spark ML

[ https://issues.apache.org/jira/browse/SPARK-18755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16239389#comment-16239389 ] Teng Peng commented on SPARK-18755: --- [~yuhaoyan] Is this what you are looking for: afte

[jira] [Commented] (SPARK-22433) Linear regression R^2 train/test terminology related

[ https://issues.apache.org/jira/browse/SPARK-22433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16237884#comment-16237884 ] Teng Peng commented on SPARK-22433: --- Thanks for the quick response, Sean. I am glad thi

[jira] [Commented] (SPARK-22433) Linear regression R^2 train/test terminology related

[ https://issues.apache.org/jira/browse/SPARK-22433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16237774#comment-16237774 ] Teng Peng commented on SPARK-22433: --- What I agree with you: be coherent, and we prefer

[jira] [Updated] (SPARK-22433) Linear regression R^2 train/test terminology related

[ https://issues.apache.org/jira/browse/SPARK-22433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Teng Peng updated SPARK-22433: -- Description: Traditional statistics is traditional statistics. Their goal, framework, and terminologie

[jira] [Updated] (SPARK-22433) Linear regression R^2 train/test terminology related

[ https://issues.apache.org/jira/browse/SPARK-22433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Teng Peng updated SPARK-22433: -- Description: Traditional statistics is traditional statistics. Their goal, framework, and terminologie

[jira] [Updated] (SPARK-22433) Linear regression R^2 train/test terminology related

[ https://issues.apache.org/jira/browse/SPARK-22433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Teng Peng updated SPARK-22433: -- Description: Traditional statistics is traditional statistics. Their goal, framework, and terminologie

[jira] [Created] (SPARK-22433) Linear regression R^2 train/test terminology related

Teng Peng created SPARK-22433: - Summary: Linear regression R^2 train/test terminology related Key: SPARK-22433 URL: https://issues.apache.org/jira/browse/SPARK-22433 Project: Spark Issue Type: I