[GitHub] spark pull request: [SPARK-6117] [SQL] add describe function to Da...

2015-03-18 Thread azagrebin
Github user azagrebin commented on the pull request: https://github.com/apache/spark/pull/5073#issuecomment-82857943 I have done one aggregation, splitten it locally into resulting DataFrame supplemented with schema and statistic names. I have also created nested version of standard

[GitHub] spark pull request: [SPARK-6117] [SQL] add describe function to Da...

2015-03-17 Thread azagrebin
Github user azagrebin commented on the pull request: https://github.com/apache/spark/pull/5073#issuecomment-82651661 @rxin Thanks for comments, I have tried to simplify, get rid of nested functions, foldLeft and use expressions to describe statistics. --- If your project is set up

[GitHub] spark pull request: [SPARK-6117] [SQL] add describe function to Da...

2015-03-17 Thread azagrebin
GitHub user azagrebin opened a pull request: https://github.com/apache/spark/pull/5073 [SPARK-6117] [SQL] add describe function to DataFrame for summary statis... Please review my solution for SPARK-6117 You can merge this pull request into a Git repository by running: $ git

[GitHub] spark pull request: [SPARK-6116] [SQL] add describe function to Da...

2015-03-17 Thread azagrebin
Github user azagrebin closed the pull request at: https://github.com/apache/spark/pull/5069 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-6116] [SQL] add describe function to Da...

2015-03-17 Thread azagrebin
Github user azagrebin commented on the pull request: https://github.com/apache/spark/pull/5069#issuecomment-82523575 Sorry wrong issue number --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-6116] [SQL] add describe function to Da...

2015-03-17 Thread azagrebin
GitHub user azagrebin opened a pull request: https://github.com/apache/spark/pull/5069 [SPARK-6116] [SQL] add describe function to DataFrame for summary statis... Please review my solution for SPARK-6116 You can merge this pull request into a Git repository by running: $ git

[GitHub] spark pull request: [SPARK-3340] Deprecate ADD_JARS and ADD_FILES

2015-02-17 Thread azagrebin
Github user azagrebin commented on the pull request: https://github.com/apache/spark/pull/4616#issuecomment-74631484 @andrewor14, thanks for the brackets and the credit, I have actually read the style guide but forgot them, sorry for that. I have also created the JIRA account, my

[GitHub] spark pull request: [SPARK-3340] Deprecate ADD_JARS and ADD_FILES

2015-02-16 Thread azagrebin
Github user azagrebin commented on the pull request: https://github.com/apache/spark/pull/4616#issuecomment-74564112 yeah, really better, thanks for the hint, committed again --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-3340] Deprecate ADD_JARS and ADD_FILES

2015-02-16 Thread azagrebin
Github user azagrebin commented on the pull request: https://github.com/apache/spark/pull/4616#issuecomment-74557764 Hi @andrewor14, thanks for the comment. So the variables need to be supported until some next release and a warning should be logged that they are deprecated (may be

[GitHub] spark pull request: [SPARK-3340] Deprecate ADD_JARS and ADD_FILES

2015-02-15 Thread azagrebin
GitHub user azagrebin opened a pull request: https://github.com/apache/spark/pull/4616 [SPARK-3340] Deprecate ADD_JARS and ADD_FILES I created a patch that disables the environment variables. Thereby scala or python shell log a warning message to notify user about the