[jira] [Commented] (SPARK-10474) TungstenAggregation cannot acquire memory for pointer array after switching to sort-based

2015-09-30 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14938249#comment-14938249 ] Reynold Xin commented on SPARK-10474: - Ah ic. It is possible that the # core estimation is completely

[jira] [Commented] (SPARK-10855) Add a JDBC dialect for Apache Derby

2015-09-30 Thread Rick Hillegas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14938334#comment-14938334 ] Rick Hillegas commented on SPARK-10855: --- I intend to create a pull request for this soon after

[jira] [Resolved] (SPARK-10770) SparkPlan.executeCollect/executeTake should return InternalRow rather than external Row

2015-09-30 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-10770. - Resolution: Fixed Fix Version/s: 1.6.0 > SparkPlan.executeCollect/executeTake should

[jira] [Updated] (SPARK-10851) Exception not failing R applications (in yarn cluster mode)

2015-09-30 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-10851: -- Assignee: Sun Rui > Exception not failing R applications (in yarn cluster mode) >

[jira] [Resolved] (SPARK-10851) Exception not failing R applications (in yarn cluster mode)

2015-09-30 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-10851. --- Resolution: Fixed Fix Version/s: 1.6.0 Target Version/s: 1.6.0 > Exception not

[jira] [Commented] (SPARK-9344) Spark SQL documentation does not clarify INSERT INTO behavior

2015-09-30 Thread Xin Ren (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14938337#comment-14938337 ] Xin Ren commented on SPARK-9344: first time try, is it ok I work on this one? Thanks a lot. :P > Spark

[jira] [Commented] (SPARK-10857) SQL injection bug in JdbcDialect.getTableExistsQuery()

2015-09-30 Thread Suresh Thalamati (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14937732#comment-14937732 ] Suresh Thalamati commented on SPARK-10857: -- One issue I ran into with getSchema() call was even

[jira] [Closed] (SPARK-3862) MultiWayBroadcastInnerHashJoin

2015-09-30 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-3862. -- Resolution: Won't Fix Closing this one for now since I think we can do something better with codegen

[jira] [Commented] (SPARK-3862) MultiWayBroadcastInnerHashJoin

2015-09-30 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14938243#comment-14938243 ] Reynold Xin commented on SPARK-3862: David, Thanks. Let's chat there. Since I created the ticket, I

[jira] [Commented] (SPARK-10515) When killing executor, the pending replacement executors will be lost

2015-09-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14936475#comment-14936475 ] Apache Spark commented on SPARK-10515: -- User 'KaiXinXiaoLei' has created a pull request for this

[jira] [Resolved] (SPARK-10736) Use 1 for all ratings if $(ratingCol) = ""

2015-09-30 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-10736. --- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 8937

[jira] [Commented] (SPARK-10800) Flaky test: org.apache.spark.deploy.StandaloneDynamicAllocationSuite

2015-09-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14936521#comment-14936521 ] Apache Spark commented on SPARK-10800: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Updated] (SPARK-10869) Auto-normalization of semi-structured schema from a dataframe

2015-09-30 Thread Julien Genini (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Genini updated SPARK-10869: -- Target Version/s: (was: 1.5.1) > Auto-normalization of semi-structured schema from a

[jira] [Commented] (SPARK-6028) Provide an alternative RPC implementation based on the network transport module

2015-09-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14936411#comment-14936411 ] Apache Spark commented on SPARK-6028: - User 'zsxwing' has created a pull request for this issue:

[jira] [Commented] (SPARK-10687) Discuss nonparametric survival analysis model

2015-09-30 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14936544#comment-14936544 ] Yanbo Liang commented on SPARK-10687: - [~mengxr] Due to the Cox’s Proportional Hazard model is not

[jira] [Updated] (SPARK-10736) Use 1 for all ratings if $(ratingCol) = ""

2015-09-30 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-10736: -- Assignee: Yanbo Liang > Use 1 for all ratings if $(ratingCol) = "" >

[jira] [Resolved] (SPARK-10811) Minimize array copying cost in Parquet converters

2015-09-30 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-10811. Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 8907

[jira] [Commented] (SPARK-10268) Add @Since annotation to ml.tree

2015-09-30 Thread Hiroshi Takahashi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14936485#comment-14936485 ] Hiroshi Takahashi commented on SPARK-10268: --- [~mengxr] Could you take a look? > Add @Since

[jira] [Assigned] (SPARK-10886) Random RDD creation example fix in MLlib statistics programming guide - mllib-statistics.md

2015-09-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10886: Assignee: (was: Apache Spark) > Random RDD creation example fix in MLlib statistics

[jira] [Commented] (SPARK-9344) Spark SQL documentation does not clarify INSERT INTO behavior

2015-09-30 Thread Xin Ren (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14938811#comment-14938811 ] Xin Ren commented on SPARK-9344: Hi I'll try to fix the docs, to get a sense how the commit process works,

[jira] [Created] (SPARK-10887) Build HashRelation outside of HashJoinNode

2015-09-30 Thread Yin Huai (JIRA)
Yin Huai created SPARK-10887: Summary: Build HashRelation outside of HashJoinNode Key: SPARK-10887 URL: https://issues.apache.org/jira/browse/SPARK-10887 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-10888) Add as.DataFrame as a synonym for createDataFrame

2015-09-30 Thread Narine Kokhlikyan (JIRA)
Narine Kokhlikyan created SPARK-10888: - Summary: Add as.DataFrame as a synonym for createDataFrame Key: SPARK-10888 URL: https://issues.apache.org/jira/browse/SPARK-10888 Project: Spark

[jira] [Commented] (SPARK-10669) Link to each language's API in codetabs in ML docs: spark.mllib

2015-09-30 Thread Xin Ren (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14938897#comment-14938897 ] Xin Ren commented on SPARK-10669: - may I have a try on this one? Thanks a lot > Link to each language's

[jira] [Assigned] (SPARK-10886) Random RDD creation example fix in MLlib statistics programming guide - mllib-statistics.md

2015-09-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10886: Assignee: Apache Spark > Random RDD creation example fix in MLlib statistics programming

[jira] [Commented] (SPARK-10886) Random RDD creation example fix in MLlib statistics programming guide - mllib-statistics.md

2015-09-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14938797#comment-14938797 ] Apache Spark commented on SPARK-10886: -- User 'jayantshekhar' has created a pull request for this

[jira] [Assigned] (SPARK-10665) Connect the local iterators with the planner

2015-09-30 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai reassigned SPARK-10665: Assignee: Yin Huai > Connect the local iterators with the planner >

[jira] [Commented] (SPARK-9344) Spark SQL documentation does not clarify INSERT INTO behavior

2015-09-30 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14938814#comment-14938814 ] Simeon Simeonov commented on SPARK-9344: /cc [~joshrosen] [~andrewor14] > Spark SQL documentation

[jira] [Commented] (SPARK-9344) Spark SQL documentation does not clarify INSERT INTO behavior

2015-09-30 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14938825#comment-14938825 ] Simeon Simeonov commented on SPARK-9344: [~joshrosen] when I logged the bug I was using

[jira] [Created] (SPARK-10886) Random RDD creation example fix in MLlib statistics programming guide - mllib-statistics.md

2015-09-30 Thread Jayant Shekhar (JIRA)
Jayant Shekhar created SPARK-10886: -- Summary: Random RDD creation example fix in MLlib statistics programming guide - mllib-statistics.md Key: SPARK-10886 URL: https://issues.apache.org/jira/browse/SPARK-10886

[jira] [Commented] (SPARK-9344) Spark SQL documentation does not clarify INSERT INTO behavior

2015-09-30 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14938806#comment-14938806 ] Simeon Simeonov commented on SPARK-9344: Are you suggesting to fix the docs or the code? > Spark

[jira] [Commented] (SPARK-9344) Spark SQL documentation does not clarify INSERT INTO behavior

2015-09-30 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14938839#comment-14938839 ] Josh Rosen commented on SPARK-9344: --- [~simeons], I just grepped through Spark's code base looking for

[jira] [Commented] (SPARK-9344) Spark SQL documentation does not clarify INSERT INTO behavior

2015-09-30 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14938819#comment-14938819 ] Josh Rosen commented on SPARK-9344: --- [~simeons], are you using HiveContext or SQLContext? If Spark

[jira] [Commented] (SPARK-10886) Random RDD creation example fix in MLlib statistics programming guide - mllib-statistics.md

2015-09-30 Thread Jayant Shekhar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14938782#comment-14938782 ] Jayant Shekhar commented on SPARK-10886: Am in the process of PR for it. > Random RDD creation

[jira] [Comment Edited] (SPARK-10886) Random RDD creation example fix in MLlib statistics programming guide - mllib-statistics.md

2015-09-30 Thread Jayant Shekhar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14938782#comment-14938782 ] Jayant Shekhar edited comment on SPARK-10886 at 9/30/15 8:27 PM: - Am in

[jira] [Commented] (SPARK-9344) Spark SQL documentation does not clarify INSERT INTO behavior

2015-09-30 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14938918#comment-14938918 ] Simeon Simeonov commented on SPARK-9344: [~joshrosen] Here is the reproducible test case you can

[jira] [Commented] (SPARK-10263) Add @Since annotation to ml.param and ml.*

2015-09-30 Thread Hiroshi Takahashi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14936630#comment-14936630 ] Hiroshi Takahashi commented on SPARK-10263: --- [~mengxr] Could you take a look? > Add @Since

[jira] [Assigned] (SPARK-7869) Spark Data Frame Fails to Load Postgres Tables with JSONB DataType Columns

2015-09-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7869: --- Assignee: (was: Apache Spark) > Spark Data Frame Fails to Load Postgres Tables with

[jira] [Assigned] (SPARK-7869) Spark Data Frame Fails to Load Postgres Tables with JSONB DataType Columns

2015-09-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7869: --- Assignee: Apache Spark > Spark Data Frame Fails to Load Postgres Tables with JSONB DataType

[jira] [Commented] (SPARK-7869) Spark Data Frame Fails to Load Postgres Tables with JSONB DataType Columns

2015-09-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14936695#comment-14936695 ] Apache Spark commented on SPARK-7869: - User '0x0FFF' has created a pull request for this issue:

[jira] [Created] (SPARK-10880) Hive module build test failed

2015-09-30 Thread JIRA
Jean-Baptiste Onofré created SPARK-10880: Summary: Hive module build test failed Key: SPARK-10880 URL: https://issues.apache.org/jira/browse/SPARK-10880 Project: Spark Issue Type:

[jira] [Commented] (SPARK-9776) Another instance of Derby may have already booted the database

2015-09-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14936634#comment-14936634 ] Apache Spark commented on SPARK-9776: - User 'KaiXinXiaoLei' has created a pull request for this issue:

[jira] [Updated] (SPARK-10881) Unable to use custom log4j appender in spark executor

2015-09-30 Thread David Moravek (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Moravek updated SPARK-10881: -- Description: In CoarseGrainedExecutorBackend, log4j is initialized, before userclasspath gets

[jira] [Created] (SPARK-10881) Unable to use custom log4j appender in spark executor

2015-09-30 Thread David Moravek (JIRA)
David Moravek created SPARK-10881: - Summary: Unable to use custom log4j appender in spark executor Key: SPARK-10881 URL: https://issues.apache.org/jira/browse/SPARK-10881 Project: Spark

[jira] [Created] (SPARK-10882) Add the ability to connect to secured mqtt brokers

2015-09-30 Thread Alexandre Touret (JIRA)
Alexandre Touret created SPARK-10882: Summary: Add the ability to connect to secured mqtt brokers Key: SPARK-10882 URL: https://issues.apache.org/jira/browse/SPARK-10882 Project: Spark

[jira] [Commented] (SPARK-10865) [Spark SQL] [UDF] the ceil/ceiling function got wrong return value type

2015-09-30 Thread Alexey Grishchenko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14936799#comment-14936799 ] Alexey Grishchenko commented on SPARK-10865: I don't really think that _floor()_ and

[jira] [Commented] (SPARK-10879) spark on yarn support priority option

2015-09-30 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14936813#comment-14936813 ] Marcelo Vanzin commented on SPARK-10879: Is this related to YARN-1963? It's nice to reference the

[jira] [Assigned] (SPARK-10887) Build HashedRelation outside of HashJoinNode

2015-09-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10887: Assignee: Apache Spark (was: Yin Huai) > Build HashedRelation outside of HashJoinNode >

[jira] [Commented] (SPARK-10887) Build HashedRelation outside of HashJoinNode

2015-09-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14939047#comment-14939047 ] Apache Spark commented on SPARK-10887: -- User 'yhuai' has created a pull request for this issue:

[jira] [Created] (SPARK-10891) Add MessageHandler to KinesisUtils.createStream similar to Direct Kafka

2015-09-30 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-10891: --- Summary: Add MessageHandler to KinesisUtils.createStream similar to Direct Kafka Key: SPARK-10891 URL: https://issues.apache.org/jira/browse/SPARK-10891 Project: Spark

[jira] [Commented] (SPARK-10560) Make StreamingLogisticRegressionWithSGD Python API equals with Scala one

2015-09-30 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14939067#comment-14939067 ] Bryan Cutler commented on SPARK-10560: -- Hi [~yanboliang], I'd be happy to do this unless you have

[jira] [Assigned] (SPARK-10888) Add as.DataFrame as a synonym for createDataFrame

2015-09-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10888: Assignee: (was: Apache Spark) > Add as.DataFrame as a synonym for createDataFrame >

[jira] [Commented] (SPARK-10888) Add as.DataFrame as a synonym for createDataFrame

2015-09-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14938923#comment-14938923 ] Apache Spark commented on SPARK-10888: -- User 'NarineK' has created a pull request for this issue:

[jira] [Created] (SPARK-10889) Upgrade Kinesis Client Library

2015-09-30 Thread Avrohom Katz (JIRA)
Avrohom Katz created SPARK-10889: Summary: Upgrade Kinesis Client Library Key: SPARK-10889 URL: https://issues.apache.org/jira/browse/SPARK-10889 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-9617) Implement json_tuple

2015-09-30 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-9617. - Resolution: Fixed Assignee: Nathan Howell Fix Version/s: 1.6.0 This issue has been

[jira] [Assigned] (SPARK-10887) Build HashedRelation outside of HashJoinNode

2015-09-30 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai reassigned SPARK-10887: Assignee: Yin Huai > Build HashedRelation outside of HashJoinNode >

[jira] [Assigned] (SPARK-10887) Build HashedRelation outside of HashJoinNode

2015-09-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10887: Assignee: Yin Huai (was: Apache Spark) > Build HashedRelation outside of HashJoinNode >

[jira] [Commented] (SPARK-10790) Dynamic Allocation does not request any executors if first stage needs less than or equal to spark.dynamicAllocation.initialExecutors

2015-09-30 Thread Jonathan Kelly (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14939060#comment-14939060 ] Jonathan Kelly commented on SPARK-10790: Thank you for the explanation and for such a quick fix,

[jira] [Updated] (SPARK-10892) Join with Data Frame returns wrong results

2015-09-30 Thread Ofer Mendelevitch (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ofer Mendelevitch updated SPARK-10892: -- Attachment: data.json Data file to run with the code, for reproducing. > Join with

[jira] [Updated] (SPARK-10887) Build HashedRelation outside of HashJoinNode

2015-09-30 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-10887: - Summary: Build HashedRelation outside of HashJoinNode (was: Build HashRelation outside of HashJoinNode)

[jira] [Assigned] (SPARK-10888) Add as.DataFrame as a synonym for createDataFrame

2015-09-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10888: Assignee: Apache Spark > Add as.DataFrame as a synonym for createDataFrame >

[jira] [Updated] (SPARK-10561) Provide tooling for auto-generating Spark SQL reference manual

2015-09-30 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated SPARK-10561: --- Description: Here is the discussion thread: http://search-hadoop.com/m/q3RTtcD20F1o62xE Richard Hillegas

[jira] [Created] (SPARK-10890) "Column count does not match; SQL statement:" error in JDBCWriteSuite

2015-09-30 Thread Rick Hillegas (JIRA)
Rick Hillegas created SPARK-10890: - Summary: "Column count does not match; SQL statement:" error in JDBCWriteSuite Key: SPARK-10890 URL: https://issues.apache.org/jira/browse/SPARK-10890 Project:

[jira] [Commented] (SPARK-10889) Upgrade Kinesis Client Library

2015-09-30 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14939023#comment-14939023 ] Burak Yavuz commented on SPARK-10889: - In addition, KCL 1.4.0 supports de-aggregation of records. >

[jira] [Updated] (SPARK-10337) Views are broken

2015-09-30 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-10337: - Assignee: Wenchen Fan > Views are broken > > > Key: SPARK-10337 >

[jira] [Created] (SPARK-10892) Join with Data Frame returns wrong results

2015-09-30 Thread Ofer Mendelevitch (JIRA)
Ofer Mendelevitch created SPARK-10892: - Summary: Join with Data Frame returns wrong results Key: SPARK-10892 URL: https://issues.apache.org/jira/browse/SPARK-10892 Project: Spark Issue

[jira] [Commented] (SPARK-9344) Spark SQL documentation does not clarify INSERT INTO behavior

2015-09-30 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14939130#comment-14939130 ] Josh Rosen commented on SPARK-9344: --- I think this is the relevant part of the stacktrace: {code}

[jira] [Updated] (SPARK-10893) Lag Analytic function broken

2015-09-30 Thread Jo Desmet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jo Desmet updated SPARK-10893: -- Description: Trying to aggregate with the LAG Analytic function gives the wrong result. In my

[jira] [Created] (SPARK-10893) Lag Analytic function broken

2015-09-30 Thread Jo Desmet (JIRA)
Jo Desmet created SPARK-10893: - Summary: Lag Analytic function broken Key: SPARK-10893 URL: https://issues.apache.org/jira/browse/SPARK-10893 Project: Spark Issue Type: Bug Components:

[jira] [Updated] (SPARK-10893) Lag Analytic function broken

2015-09-30 Thread Jo Desmet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jo Desmet updated SPARK-10893: -- Description: Trying to aggregate with the LAG Analytic function gives the wrong result. In my

[jira] [Updated] (SPARK-10893) Lag Analytic function broken

2015-09-30 Thread Jo Desmet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jo Desmet updated SPARK-10893: -- Description: Trying to aggregate with the LAG Analytic function gives the wrong result. In my

[jira] [Updated] (SPARK-10893) Lag Analytic function broken

2015-09-30 Thread Jo Desmet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jo Desmet updated SPARK-10893: -- Description: Trying to aggregate with the LAG Analytic function gives the wrong result. In my

[jira] [Updated] (SPARK-10893) Lag Analytic function broken

2015-09-30 Thread Jo Desmet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jo Desmet updated SPARK-10893: -- Description: Trying to aggregate with the LAG Analytic function gives the wrong result. In my

[jira] [Updated] (SPARK-10893) Lag Analytic function broken

2015-09-30 Thread Jo Desmet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jo Desmet updated SPARK-10893: -- Description: Trying to aggregate with the LAG Analytic function gives the wrong result. In my

[jira] [Resolved] (SPARK-10807) Add as.data.frame() as a synonym for collect()

2015-09-30 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-10807. --- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request

[jira] [Updated] (SPARK-10807) Add as.data.frame() as a synonym for collect()

2015-09-30 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman updated SPARK-10807: -- Assignee: Oscar D. Lara Yejas > Add as.data.frame() as a synonym for collect()

[jira] [Updated] (SPARK-10893) Lag Analytic function broken

2015-09-30 Thread Jo Desmet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jo Desmet updated SPARK-10893: -- Description: Trying to aggregate with the LAG Analytic function gives the wrong result. In my

[jira] [Updated] (SPARK-10893) Lag Analytic function broken

2015-09-30 Thread Jo Desmet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jo Desmet updated SPARK-10893: -- Description: Trying to aggregate with the LAG Analytic function gives the wrong result. In my

[jira] [Commented] (SPARK-10894) Add 'drop' support for DataFrame's subset function

2015-09-30 Thread Weiqiang Zhuang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14939339#comment-14939339 ] Weiqiang Zhuang commented on SPARK-10894: - I am working on this. > Add 'drop' support for

[jira] [Created] (SPARK-10894) Add 'drop' support for DataFrame's subset function

2015-09-30 Thread Weiqiang Zhuang (JIRA)
Weiqiang Zhuang created SPARK-10894: --- Summary: Add 'drop' support for DataFrame's subset function Key: SPARK-10894 URL: https://issues.apache.org/jira/browse/SPARK-10894 Project: Spark

[jira] [Assigned] (SPARK-10891) Add MessageHandler to KinesisUtils.createStream similar to Direct Kafka

2015-09-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10891: Assignee: Apache Spark > Add MessageHandler to KinesisUtils.createStream similar to

[jira] [Commented] (SPARK-10891) Add MessageHandler to KinesisUtils.createStream similar to Direct Kafka

2015-09-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14939359#comment-14939359 ] Apache Spark commented on SPARK-10891: -- User 'brkyvz' has created a pull request for this issue:

[jira] [Assigned] (SPARK-10891) Add MessageHandler to KinesisUtils.createStream similar to Direct Kafka

2015-09-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10891: Assignee: (was: Apache Spark) > Add MessageHandler to KinesisUtils.createStream

[jira] [Commented] (SPARK-5569) Checkpoints cannot reference classes defined outside of Spark's assembly

2015-09-30 Thread Deming Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14939353#comment-14939353 ] Deming Zhu commented on SPARK-5569: --- I encount the exactly same problem with you, and I think it's an

[jira] [Commented] (SPARK-9472) Consistent hadoop config for streaming

2015-09-30 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14936866#comment-14936866 ] Cody Koeninger commented on SPARK-9472: --- You can get around this by passing in the hadoop context

[jira] [Created] (SPARK-10883) Be able to build each module individually

2015-09-30 Thread JIRA
Jean-Baptiste Onofré created SPARK-10883: Summary: Be able to build each module individually Key: SPARK-10883 URL: https://issues.apache.org/jira/browse/SPARK-10883 Project: Spark

[jira] [Commented] (SPARK-10884) Support prediction on single instance for PredictionModel and ClassificationModel

2015-09-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14936951#comment-14936951 ] Apache Spark commented on SPARK-10884: -- User 'yanboliang' has created a pull request for this issue:

[jira] [Commented] (SPARK-10883) Be able to build each module individually

2015-09-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14936898#comment-14936898 ] Apache Spark commented on SPARK-10883: -- User 'jbonofre' has created a pull request for this issue:

[jira] [Assigned] (SPARK-10883) Be able to build each module individually

2015-09-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10883: Assignee: Apache Spark > Be able to build each module individually >

[jira] [Commented] (SPARK-10883) Be able to build each module individually

2015-09-30 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14936944#comment-14936944 ] Marcelo Vanzin commented on SPARK-10883: How are you trying to build the individual module?

[jira] [Commented] (SPARK-10413) Model should support prediction on single instance

2015-09-30 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14936947#comment-14936947 ] Yanbo Liang commented on SPARK-10413: - [~mengxr] [~josephkb] I found this issue involved too many

[jira] [Assigned] (SPARK-10884) Support prediction on single instance for PredictionModel and ClassificationModel

2015-09-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10884: Assignee: (was: Apache Spark) > Support prediction on single instance for

[jira] [Assigned] (SPARK-10884) Support prediction on single instance for PredictionModel and ClassificationModel

2015-09-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10884: Assignee: Apache Spark > Support prediction on single instance for PredictionModel and >

[jira] [Assigned] (SPARK-10883) Be able to build each module individually

2015-09-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10883: Assignee: (was: Apache Spark) > Be able to build each module individually >

[jira] [Commented] (SPARK-10883) Be able to build each module individually

2015-09-30 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-10883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14936899#comment-14936899 ] Jean-Baptiste Onofré commented on SPARK-10883: -- https://github.com/apache/spark/pull/8949 >

[jira] [Issue Comment Deleted] (SPARK-10883) Be able to build each module individually

2015-09-30 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-10883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Baptiste Onofré updated SPARK-10883: - Comment: was deleted (was: https://github.com/apache/spark/pull/8949) > Be able

[jira] [Created] (SPARK-10884) Support prediction on single instance for PredictionModel and ClassificationModel

2015-09-30 Thread Yanbo Liang (JIRA)
Yanbo Liang created SPARK-10884: --- Summary: Support prediction on single instance for PredictionModel and ClassificationModel Key: SPARK-10884 URL: https://issues.apache.org/jira/browse/SPARK-10884

[jira] [Updated] (SPARK-10884) Support prediction on single instance for PredictionModel and ClassificationModel

2015-09-30 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-10884: Description: Support prediction on single instance for regression and classification related

[jira] [Commented] (SPARK-10058) Flaky test: HeartbeatReceiverSuite: normal heartbeat

2015-09-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14937473#comment-14937473 ] Apache Spark commented on SPARK-10058: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Assigned] (SPARK-10058) Flaky test: HeartbeatReceiverSuite: normal heartbeat

2015-09-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10058: Assignee: Apache Spark (was: Shixiong Zhu) > Flaky test: HeartbeatReceiverSuite: normal

[jira] [Commented] (SPARK-9472) Consistent hadoop config for streaming

2015-09-30 Thread Russell Alexander Spitzer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14937471#comment-14937471 ] Russell Alexander Spitzer commented on SPARK-9472: -- Yeah that's the workaround we

  1   2   >