[jira] [Commented] (SPARK-6931) python: struct.pack('!q', value) in write_long(value, stream) in serializers.py require int(but doesn't raise exceptions in common cases)

[ https://issues.apache.org/jira/browse/SPARK-6931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14499162#comment-14499162 ] Bryan Cutler commented on SPARK-6931: - I just checked and it looks like some int() cas

[jira] [Assigned] (SPARK-6418) Add simple per-stage visualization to the UI

[ https://issues.apache.org/jira/browse/SPARK-6418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6418: --- Assignee: Pradyumn Shroff (was: Apache Spark) > Add simple per-stage visualization to the UI

[jira] [Comment Edited] (SPARK-6962) Netty BlockTransferService hangs in the middle of SQL query

[ https://issues.apache.org/jira/browse/SPARK-6962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14498991#comment-14498991 ] Michael Allman edited comment on SPARK-6962 at 4/16/15 11:54 PM: ---

[jira] [Assigned] (SPARK-6963) Flaky test: o.a.s.ContextCleanerSuite automatically cleanup checkpoint

[ https://issues.apache.org/jira/browse/SPARK-6963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6963: --- Assignee: Guoqiang Li (was: Apache Spark) > Flaky test: o.a.s.ContextCleanerSuite automatica

[jira] [Commented] (SPARK-6963) Flaky test: o.a.s.ContextCleanerSuite automatically cleanup checkpoint

[ https://issues.apache.org/jira/browse/SPARK-6963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14499128#comment-14499128 ] Apache Spark commented on SPARK-6963: - User 'witgo' has created a pull request for thi

[jira] [Updated] (SPARK-2695) Figure out a good way to handle NullType columns.

[ https://issues.apache.org/jira/browse/SPARK-2695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-2695: Assignee: (was: Yin Huai) > Figure out a good way to handle NullType columns. >

[jira] [Updated] (SPARK-5295) Stabilize data types

[ https://issues.apache.org/jira/browse/SPARK-5295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-5295: Assignee: (was: Yin Huai) > Stabilize data types > > > Key: SPARK-5

[jira] [Commented] (SPARK-6962) Netty BlockTransferService hangs in the middle of SQL query

[ https://issues.apache.org/jira/browse/SPARK-6962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14498991#comment-14498991 ] Michael Allman commented on SPARK-6962: --- [~adav]Which logs would be helpful? [~pwend

[jira] [Resolved] (SPARK-6966) JDBC datasources use Class.forName to load driver

[ https://issues.apache.org/jira/browse/SPARK-6966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-6966. - Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 5543 [https:/

[jira] [Resolved] (SPARK-6899) Type mismatch when using codegen

[ https://issues.apache.org/jira/browse/SPARK-6899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-6899. - Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 5517 [https:/

[jira] [Commented] (SPARK-6923) Get invalid hive table columns after save DataFrame to hive table

[ https://issues.apache.org/jira/browse/SPARK-6923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14499141#comment-14499141 ] pin_zhang commented on SPARK-6923: -- Do you means if save data frame to the table that use

[jira] [Resolved] (SPARK-6927) Sorting Error when codegen on

[ https://issues.apache.org/jira/browse/SPARK-6927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-6927. - Resolution: Fixed Issue resolved by pull request 5524 [https://github.com/apache/spark/pul

[jira] [Resolved] (SPARK-4897) Python 3 support

[ https://issues.apache.org/jira/browse/SPARK-4897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-4897. --- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 5173 [https://github.com/

[jira] [Updated] (SPARK-5180) Data source API improvement

[ https://issues.apache.org/jira/browse/SPARK-5180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-5180: Assignee: (was: Yin Huai) > Data source API improvement > --- > >

[jira] [Resolved] (SPARK-4842) Use WeakTypeTags in ScalaReflection

[ https://issues.apache.org/jira/browse/SPARK-4842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-4842. - Resolution: Won't Fix This would break the API and we now support tuples pretty nicely whi

[jira] [Commented] (SPARK-6940) PySpark ML.Tuning Wrappers are missing

[ https://issues.apache.org/jira/browse/SPARK-6940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14498930#comment-14498930 ] Punya Biswal commented on SPARK-6940: - Sorry about the duplicate bug - [~omede] and I

[jira] [Reopened] (SPARK-6216) Check Python version in worker before run PySpark job

[ https://issues.apache.org/jira/browse/SPARK-6216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reopened SPARK-6216: --- This merged patch does not work well if you have different major version on driver or worker. > Check Py

[jira] [Resolved] (SPARK-6911) API for access MapType in DataFrame

[ https://issues.apache.org/jira/browse/SPARK-6911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-6911. - Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 5513 [https:/

[jira] [Assigned] (SPARK-6963) Flaky test: o.a.s.ContextCleanerSuite automatically cleanup checkpoint

[ https://issues.apache.org/jira/browse/SPARK-6963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6963: --- Assignee: Apache Spark (was: Guoqiang Li) > Flaky test: o.a.s.ContextCleanerSuite automatica

[jira] [Updated] (SPARK-5354) When possible, correctly set outputPartitioning for leaf SparkPlans

[ https://issues.apache.org/jira/browse/SPARK-5354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-5354: Assignee: (was: Yin Huai) > When possible, correctly set outputPartitioning for leaf SparkPlans > --

[jira] [Updated] (SPARK-6969) Refresh the cached table when REFRESH TABLE is used

[ https://issues.apache.org/jira/browse/SPARK-6969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-6969: Assignee: (was: Yin Huai) > Refresh the cached table when REFRESH TABLE is used > --

[jira] [Assigned] (SPARK-4675) Find similar products and similar users in MatrixFactorizationModel

[ https://issues.apache.org/jira/browse/SPARK-4675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-4675: --- Assignee: Apache Spark > Find similar products and similar users in MatrixFactorizationModel

[jira] [Updated] (SPARK-5288) Stabilize Spark SQL data type API followup

[ https://issues.apache.org/jira/browse/SPARK-5288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-5288: Assignee: (was: Yin Huai) > Stabilize Spark SQL data type API followup > --

[jira] [Commented] (SPARK-6931) python: struct.pack('!q', value) in write_long(value, stream) in serializers.py require int(but doesn't raise exceptions in common cases)

[ https://issues.apache.org/jira/browse/SPARK-6931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14499036#comment-14499036 ] Josh Rosen commented on SPARK-6931: --- It looks like this was also reported on the mailing

[jira] [Assigned] (SPARK-6418) Add simple per-stage visualization to the UI

[ https://issues.apache.org/jira/browse/SPARK-6418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6418: --- Assignee: Apache Spark (was: Pradyumn Shroff) > Add simple per-stage visualization to the UI

[jira] [Updated] (SPARK-6902) Row() object can be mutated even though it should be immutable

[ https://issues.apache.org/jira/browse/SPARK-6902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-6902: Assignee: Davies Liu > Row() object can be mutated even though it should be immutable >

[jira] [Updated] (SPARK-6929) Alias for more complex expression causes attribute not been able to resolve

[ https://issues.apache.org/jira/browse/SPARK-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-6929: Description: I've extracted the minimal query that don't work with aliases. You can remove

[jira] [Assigned] (SPARK-6972) Add Coalesce to DataFrame

[ https://issues.apache.org/jira/browse/SPARK-6972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6972: --- Assignee: Apache Spark (was: Michael Armbrust) > Add Coalesce to DataFrame > ---

[jira] [Assigned] (SPARK-6972) Add Coalesce to DataFrame

[ https://issues.apache.org/jira/browse/SPARK-6972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6972: --- Assignee: Michael Armbrust (was: Apache Spark) > Add Coalesce to DataFrame > ---

[jira] [Commented] (SPARK-6972) Add Coalesce to DataFrame

[ https://issues.apache.org/jira/browse/SPARK-6972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14498897#comment-14498897 ] Apache Spark commented on SPARK-6972: - User 'marmbrus' has created a pull request for

[jira] [Assigned] (SPARK-6957) groupby

[ https://issues.apache.org/jira/browse/SPARK-6957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6957: --- Assignee: Apache Spark (was: Davies Liu) > groupby > --- > > Key: SPARK-

[jira] [Created] (SPARK-6972) Add Coalesce to DataFrame

Michael Armbrust created SPARK-6972: --- Summary: Add Coalesce to DataFrame Key: SPARK-6972 URL: https://issues.apache.org/jira/browse/SPARK-6972 Project: Spark Issue Type: New Feature

[jira] [Assigned] (SPARK-6958) sort

[ https://issues.apache.org/jira/browse/SPARK-6958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6958: --- Assignee: Apache Spark (was: Davies Liu) > sort > > > Key: SPARK-6958 >

[jira] [Assigned] (SPARK-6958) sort

[ https://issues.apache.org/jira/browse/SPARK-6958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6958: --- Assignee: Davies Liu (was: Apache Spark) > sort > > > Key: SPARK-6958 >

[jira] [Assigned] (SPARK-6957) groupby

[ https://issues.apache.org/jira/browse/SPARK-6957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6957: --- Assignee: Davies Liu (was: Apache Spark) > groupby > --- > > Key: SPARK-

[jira] [Commented] (SPARK-6958) sort

[ https://issues.apache.org/jira/browse/SPARK-6958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14498893#comment-14498893 ] Apache Spark commented on SPARK-6958: - User 'davies' has created a pull request for th

[jira] [Commented] (SPARK-6957) groupby

[ https://issues.apache.org/jira/browse/SPARK-6957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14498892#comment-14498892 ] Apache Spark commented on SPARK-6957: - User 'davies' has created a pull request for th

[jira] [Created] (SPARK-6971) Each Jenkins build should use a distinct Zinc port

Patrick Wendell created SPARK-6971: -- Summary: Each Jenkins build should use a distinct Zinc port Key: SPARK-6971 URL: https://issues.apache.org/jira/browse/SPARK-6971 Project: Spark Issue Ty

[jira] [Commented] (SPARK-6889) Streamline contribution process with update to Contribution wiki, JIRA rules

[ https://issues.apache.org/jira/browse/SPARK-6889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14498848#comment-14498848 ] Joseph K. Bradley commented on SPARK-6889: -- [~srowen] I like the updates in the d

[jira] [Commented] (SPARK-6962) Netty BlockTransferService hangs in the middle of SQL query

[ https://issues.apache.org/jira/browse/SPARK-6962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14498806#comment-14498806 ] Patrick Wendell commented on SPARK-6962: [~adav] One thing that could cause this i

[jira] [Assigned] (SPARK-6966) JDBC datasources use Class.forName to load driver

[ https://issues.apache.org/jira/browse/SPARK-6966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6966: --- Assignee: Apache Spark (was: Michael Armbrust) > JDBC datasources use Class.forName to load

[jira] [Assigned] (SPARK-6966) JDBC datasources use Class.forName to load driver

[ https://issues.apache.org/jira/browse/SPARK-6966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6966: --- Assignee: Michael Armbrust (was: Apache Spark) > JDBC datasources use Class.forName to load

[jira] [Commented] (SPARK-6966) JDBC datasources use Class.forName to load driver

[ https://issues.apache.org/jira/browse/SPARK-6966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14498802#comment-14498802 ] Apache Spark commented on SPARK-6966: - User 'marmbrus' has created a pull request for

[jira] [Commented] (SPARK-6635) DataFrame.withColumn can create columns with identical names

[ https://issues.apache.org/jira/browse/SPARK-6635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14498736#comment-14498736 ] Joseph K. Bradley commented on SPARK-6635: -- Btw, when I say "that seems like a us

[jira] [Commented] (SPARK-6635) DataFrame.withColumn can create columns with identical names

[ https://issues.apache.org/jira/browse/SPARK-6635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14498734#comment-14498734 ] Rakesh Chalasani commented on SPARK-6635: - Join over to two data frames also leads

[jira] [Comment Edited] (SPARK-6635) DataFrame.withColumn can create columns with identical names

[ https://issues.apache.org/jira/browse/SPARK-6635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14498734#comment-14498734 ] Rakesh Chalasani edited comment on SPARK-6635 at 4/16/15 9:09 PM: --

[jira] [Commented] (SPARK-6635) DataFrame.withColumn can create columns with identical names

[ https://issues.apache.org/jira/browse/SPARK-6635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14498725#comment-14498725 ] Reynold Xin commented on SPARK-6635: cc [~marmbrus] to chime in. I think about it mor

[jira] [Created] (SPARK-6970) Document what the options: Map[String, String] does on DataFrame.save and DataFrame.saveAsTable

John Muller created SPARK-6970: -- Summary: Document what the options: Map[String, String] does on DataFrame.save and DataFrame.saveAsTable Key: SPARK-6970 URL: https://issues.apache.org/jira/browse/SPARK-6970

[jira] [Commented] (SPARK-6969) Refresh the cached table when REFRESH TABLE is used

[ https://issues.apache.org/jira/browse/SPARK-6969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14498710#comment-14498710 ] Yin Huai commented on SPARK-6969: - We can lazily recache the table if a user call {{REFRES

[jira] [Created] (SPARK-6969) Refresh the cached table when REFRESH TABLE is used

Yin Huai created SPARK-6969: --- Summary: Refresh the cached table when REFRESH TABLE is used Key: SPARK-6969 URL: https://issues.apache.org/jira/browse/SPARK-6969 Project: Spark Issue Type: Improveme

[jira] [Created] (SPARK-6968) Make maniuplating an underlying RDD of a DataFrame easier

John Muller created SPARK-6968: -- Summary: Make maniuplating an underlying RDD of a DataFrame easier Key: SPARK-6968 URL: https://issues.apache.org/jira/browse/SPARK-6968 Project: Spark Issue Typ

[jira] [Commented] (SPARK-6940) PySpark ML.Tuning Wrappers are missing

[ https://issues.apache.org/jira/browse/SPARK-6940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14498678#comment-14498678 ] Omede Firouz commented on SPARK-6940: - Thanks [~josephkb], I'll look into 5874 and be

[jira] [Commented] (SPARK-6940) PySpark ML.Tuning Wrappers are missing

[ https://issues.apache.org/jira/browse/SPARK-6940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14498663#comment-14498663 ] Joseph K. Bradley commented on SPARK-6940: -- [~omede] Can you please coordinate wi

[jira] [Updated] (SPARK-6947) Make ml.tuning accessible from Python API

[ https://issues.apache.org/jira/browse/SPARK-6947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-6947: - Fix Version/s: (was: SPARK-6940) > Make ml.tuning accessible from Python API > ---

[jira] [Resolved] (SPARK-6947) Make ml.tuning accessible from Python API

[ https://issues.apache.org/jira/browse/SPARK-6947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-6947. -- Resolution: Duplicate Fix Version/s: SPARK-6940 > Make ml.tuning accessible from

[jira] [Commented] (SPARK-6857) Python SQL schema inference should support numpy types

[ https://issues.apache.org/jira/browse/SPARK-6857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14498651#comment-14498651 ] Joseph K. Bradley commented on SPARK-6857: -- Based on past discussions with [~meng

[jira] [Resolved] (SPARK-6855) Set R includes in each file to get right collate order

[ https://issues.apache.org/jira/browse/SPARK-6855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-6855. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 546

[jira] [Commented] (SPARK-6857) Python SQL schema inference should support numpy types

[ https://issues.apache.org/jira/browse/SPARK-6857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14498593#comment-14498593 ] Davies Liu commented on SPARK-6857: --- It's not good that we use array or numpy.array as p

[jira] [Commented] (SPARK-6844) Memory leak occurs when register temp table with cache table on

[ https://issues.apache.org/jira/browse/SPARK-6844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14498587#comment-14498587 ] Michael Armbrust commented on SPARK-6844: - I was not planning to. I do not think

[jira] [Commented] (SPARK-6962) Netty BlockTransferService hangs in the middle of SQL query

[ https://issues.apache.org/jira/browse/SPARK-6962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14498570#comment-14498570 ] Aaron Davidson commented on SPARK-6962: --- Renamed the JIRA to avoid having everyone w

[jira] [Updated] (SPARK-6962) Netty BlockTransferService hangs in the middle of SQL query

[ https://issues.apache.org/jira/browse/SPARK-6962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Davidson updated SPARK-6962: -- Summary: Netty BlockTransferService hangs in the middle of SQL query (was: Spark gets stuck on

[jira] [Comment Edited] (SPARK-6950) Spark master UI believes some applications are in progress when they are actually completed

[ https://issues.apache.org/jira/browse/SPARK-6950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14498555#comment-14498555 ] Matt Cheah edited comment on SPARK-6950 at 4/16/15 7:32 PM: Th

[jira] [Comment Edited] (SPARK-6950) Spark master UI believes some applications are in progress when they are actually completed

[ https://issues.apache.org/jira/browse/SPARK-6950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14498555#comment-14498555 ] Matt Cheah edited comment on SPARK-6950 at 4/16/15 7:31 PM: Th

[jira] [Resolved] (SPARK-6950) Spark master UI believes some applications are in progress when they are actually completed

[ https://issues.apache.org/jira/browse/SPARK-6950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Cheah resolved SPARK-6950. --- Resolution: Cannot Reproduce Fix Version/s: 1.3.1 > Spark master UI believes some applications

[jira] [Commented] (SPARK-6950) Spark master UI believes some applications are in progress when they are actually completed

[ https://issues.apache.org/jira/browse/SPARK-6950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14498555#comment-14498555 ] Matt Cheah commented on SPARK-6950: --- This is no longer an issue on the tip of branch-1.3

[jira] [Updated] (SPARK-5427) Add support for floor function in Spark SQL

[ https://issues.apache.org/jira/browse/SPARK-5427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated SPARK-5427: -- Description: floor() function is supported in Hive SQL. This issue is to add floor() function to Spark SQL. Rel

[jira] [Updated] (SPARK-6857) Python SQL schema inference should support numpy types

[ https://issues.apache.org/jira/browse/SPARK-6857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-6857: - Description: **UPDATE**: Closing this JIRA since a better fix will be better UDT support.

[jira] [Resolved] (SPARK-6857) Python SQL schema inference should support numpy types

[ https://issues.apache.org/jira/browse/SPARK-6857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-6857. -- Resolution: Not A Problem > Python SQL schema inference should support numpy types > ---

[jira] [Updated] (SPARK-6967) Internal DateType not handled correctly in caching

[ https://issues.apache.org/jira/browse/SPARK-6967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-6967: Target Version/s: 1.3.2, 1.4.0 > Internal DateType not handled correctly in caching > --

[jira] [Commented] (SPARK-6857) Python SQL schema inference should support numpy types

[ https://issues.apache.org/jira/browse/SPARK-6857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14498508#comment-14498508 ] Joseph K. Bradley commented on SPARK-6857: -- [~davies] Yes, that OK with me. It's

[jira] [Resolved] (SPARK-6934) Fix the bug that using a wrong configuration for “ask” timeout in RpcEnv

[ https://issues.apache.org/jira/browse/SPARK-6934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-6934. Resolution: Fixed Fix Version/s: 1.4.0 Assignee: Shixiong Zhu > Fix the bug that usi

[jira] [Created] (SPARK-6967) Internal DateType not handled correctly in caching

Michael Armbrust created SPARK-6967: --- Summary: Internal DateType not handled correctly in caching Key: SPARK-6967 URL: https://issues.apache.org/jira/browse/SPARK-6967 Project: Spark Issue

[jira] [Commented] (SPARK-4233) Simplify the Aggregation Function implementation

[ https://issues.apache.org/jira/browse/SPARK-4233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14498497#comment-14498497 ] Apache Spark commented on SPARK-4233: - User 'chenghao-intel' has created a pull reques

[jira] [Updated] (SPARK-6966) JDBC datasources use Class.forName to load driver

[ https://issues.apache.org/jira/browse/SPARK-6966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-6966: Issue Type: Bug (was: New Feature) > JDBC datasources use Class.forName to load driver > --

[jira] [Created] (SPARK-6966) JDBC datasources use Class.forName to load driver

Michael Armbrust created SPARK-6966: --- Summary: JDBC datasources use Class.forName to load driver Key: SPARK-6966 URL: https://issues.apache.org/jira/browse/SPARK-6966 Project: Spark Issue T

[jira] [Commented] (SPARK-6857) Python SQL schema inference should support numpy types

[ https://issues.apache.org/jira/browse/SPARK-6857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14498414#comment-14498414 ] Davies Liu commented on SPARK-6857: --- [~josephkb] Because the serializer do not support n

[jira] [Commented] (SPARK-2734) DROP TABLE should also uncache table

[ https://issues.apache.org/jira/browse/SPARK-2734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14498403#comment-14498403 ] Michael Armbrust commented on SPARK-2734: - How do you know it occurring? What que

[jira] [Comment Edited] (SPARK-6950) Spark master UI believes some applications are in progress when they are actually completed

[ https://issues.apache.org/jira/browse/SPARK-6950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14498390#comment-14498390 ] Matt Cheah edited comment on SPARK-6950 at 4/16/15 5:57 PM: Th

[jira] [Commented] (SPARK-6950) Spark master UI believes some applications are in progress when they are actually completed

[ https://issues.apache.org/jira/browse/SPARK-6950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14498390#comment-14498390 ] Matt Cheah commented on SPARK-6950: --- There's one way I could reproduce this locally, but

[jira] [Updated] (SPARK-6955) Do not let Yarn Shuffle Server retry its server port.

[ https://issues.apache.org/jira/browse/SPARK-6955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-6955: - Target Version/s: 1.4.0 > Do not let Yarn Shuffle Server retry its server port. >

[jira] [Updated] (SPARK-6955) Do not let Yarn Shuffle Server retry its server port.

[ https://issues.apache.org/jira/browse/SPARK-6955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-6955: - Assignee: SaintBacchus > Do not let Yarn Shuffle Server retry its server port. > -

[jira] [Updated] (SPARK-6955) Do not let Yarn Shuffle Server retry its server port.

[ https://issues.apache.org/jira/browse/SPARK-6955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-6955: - Affects Version/s: 1.2.0 > Do not let Yarn Shuffle Server retry its server port. > ---

[jira] [Created] (SPARK-6965) StringIndexer should convert input to Strings

Joseph K. Bradley created SPARK-6965: Summary: StringIndexer should convert input to Strings Key: SPARK-6965 URL: https://issues.apache.org/jira/browse/SPARK-6965 Project: Spark Issue Typ

[jira] [Updated] (SPARK-6964) Support Cancellation in the Thrift Server

[ https://issues.apache.org/jira/browse/SPARK-6964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-6964: Description: There is already a hook in {{ExecuteStatementOperation}}, we just need to conne

[jira] [Updated] (SPARK-6964) Support Cancellation in the Thrift Server

[ https://issues.apache.org/jira/browse/SPARK-6964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-6964: Description: There is already a hook in > Support Cancellation in the Thrift Server > -

[jira] [Updated] (SPARK-6964) Support Cancellation in the Thrift Server

[ https://issues.apache.org/jira/browse/SPARK-6964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-6964: Target Version/s: 1.4.0 > Support Cancellation in the Thrift Server > --

[jira] [Updated] (SPARK-1442) Add Window function support

[ https://issues.apache.org/jira/browse/SPARK-1442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-1442: Priority: Blocker (was: Critical) > Add Window function support > -

[jira] [Created] (SPARK-6964) Support Cancellation in the Thrift Server

Michael Armbrust created SPARK-6964: --- Summary: Support Cancellation in the Thrift Server Key: SPARK-6964 URL: https://issues.apache.org/jira/browse/SPARK-6964 Project: Spark Issue Type: New

[jira] [Updated] (SPARK-6963) Flaky test: o.a.s.ContextCleanerSuite automatically cleanup checkpoint

[ https://issues.apache.org/jira/browse/SPARK-6963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-6963: - Description: Observed on an unrelated streaming PR https://github.com/apache/spark/pull/5428 https://ampla

[jira] [Created] (SPARK-6963) Flaky test: o.a.s.ContextCleanerSuite automatically cleanup checkpoint

Andrew Or created SPARK-6963: Summary: Flaky test: o.a.s.ContextCleanerSuite automatically cleanup checkpoint Key: SPARK-6963 URL: https://issues.apache.org/jira/browse/SPARK-6963 Project: Spark

[jira] [Updated] (SPARK-6963) Flaky test: o.a.s.ContextCleanerSuite automatically cleanup checkpoint

[ https://issues.apache.org/jira/browse/SPARK-6963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-6963: - Labels: flaky-test (was: ) > Flaky test: o.a.s.ContextCleanerSuite automatically cleanup checkpoint > ---

[jira] [Commented] (SPARK-6635) DataFrame.withColumn can create columns with identical names

[ https://issues.apache.org/jira/browse/SPARK-6635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14498329#comment-14498329 ] Joseph K. Bradley commented on SPARK-6635: -- [~rxin] Your select statement does hi

[jira] [Updated] (SPARK-4081) Categorical feature indexing

[ https://issues.apache.org/jira/browse/SPARK-4081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-4081: - Description: **Updated Description** Decision Trees and tree ensembles require that categ

[jira] [Updated] (SPARK-4897) Python 3 support

[ https://issues.apache.org/jira/browse/SPARK-4897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-4897: -- Priority: Blocker (was: Minor) > Python 3 support > > > Key: SPARK-489

[jira] [Commented] (SPARK-6936) SQLContext.sql() caused deadlock in multi-thread env

[ https://issues.apache.org/jira/browse/SPARK-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14498305#comment-14498305 ] Paul Wu commented on SPARK-6936: You are right: I used spark-hive_2.10 instead of spark-hi

[jira] [Issue Comment Deleted] (SPARK-6936) SQLContext.sql() caused deadlock in multi-thread env

[ https://issues.apache.org/jira/browse/SPARK-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Wu updated SPARK-6936: --- Comment: was deleted (was: Not sure about HiveContext. I tried to do the following program and I got exceptio

[jira] [Commented] (SPARK-6923) Get invalid hive table columns after save DataFrame to hive table

[ https://issues.apache.org/jira/browse/SPARK-6923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14498271#comment-14498271 ] Michael Armbrust commented on SPARK-6923: - Only Spark 1.3 has the ability to read

[jira] [Commented] (SPARK-6936) SQLContext.sql() caused deadlock in multi-thread env

[ https://issues.apache.org/jira/browse/SPARK-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14498258#comment-14498258 ] Michael Armbrust commented on SPARK-6936: - NoSuchMethodError almost always means y

[jira] [Commented] (SPARK-6936) SQLContext.sql() caused deadlock in multi-thread env

[ https://issues.apache.org/jira/browse/SPARK-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14498247#comment-14498247 ] Paul Wu commented on SPARK-6936: Not sure about HiveContext. I tried to do the following p

[jira] [Updated] (SPARK-6962) Spark gets stuck on a step, hangs forever - jobs do not complete

[ https://issues.apache.org/jira/browse/SPARK-6962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Chase updated SPARK-6962: - Attachment: jstacks.txt Here are the stack dumps I took when Spark is hanging. > Spark gets stuck on a st

  1   2   >