[jira] [Commented] (SPARK-19103) In web ui,URL's host name should be a specific IP address.

2017-01-06 Thread guoxiaolong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15807033#comment-15807033 ] guoxiaolong commented on SPARK-19103: - Because the browser opens this address 404.We must configure

[jira] [Created] (SPARK-19115) SparkSQL unsupported the command " create external table if not exist new_tbl like old_tbl"

2017-01-06 Thread Xiaochen Ouyang (JIRA)
Xiaochen Ouyang created SPARK-19115: --- Summary: SparkSQL unsupported the command " create external table if not exist new_tbl like old_tbl" Key: SPARK-19115 URL:

[jira] [Commented] (SPARK-18113) Sending AskPermissionToCommitOutput failed, driver enter into task deadloop

2017-01-06 Thread jin xing (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15806897#comment-15806897 ] jin xing commented on SPARK-18113: -- [~xq2005], [~aash] I am seeing this issue in my cluster some times.

[jira] [Closed] (SPARK-18929) Add Tweedie distribution in GLM

2017-01-06 Thread Wayne Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wayne Zhang closed SPARK-18929. --- Resolution: Unresolved > Add Tweedie distribution in GLM > --- > >

[jira] [Commented] (SPARK-19108) Broadcast all shared parts of tasks (to reduce task serialization time)

2017-01-06 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15806774#comment-15806774 ] Shivaram Venkataraman commented on SPARK-19108: --- +1 - This is a good idea. One thing I'd

[jira] [Updated] (SPARK-19110) DistributedLDAModel returns different logPrior for original and loaded model

2017-01-06 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-19110: -- Shepherd: Joseph K. Bradley Target Version/s: 1.6.4, 2.0.3, 2.1.1, 2.2.0 >

[jira] [Updated] (SPARK-19110) DistributedLDAModel returns different logPrior for original and loaded model

2017-01-06 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-19110: -- Affects Version/s: 2.2.0 1.3.1 1.4.1

[jira] [Resolved] (SPARK-18194) Log instrumentation in OneVsRest, CrossValidator, TrainValidationSplit

2017-01-06 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-18194. --- Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 16480

[jira] [Commented] (SPARK-16920) Investigate and fix issues introduced in SPARK-15858

2017-01-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15806575#comment-15806575 ] Apache Spark commented on SPARK-16920: -- User 'mhmoudr' has created a pull request for this issue:

[jira] [Assigned] (SPARK-16920) Investigate and fix issues introduced in SPARK-15858

2017-01-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16920: Assignee: (was: Apache Spark) > Investigate and fix issues introduced in SPARK-15858

[jira] [Assigned] (SPARK-16920) Investigate and fix issues introduced in SPARK-15858

2017-01-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16920: Assignee: Apache Spark > Investigate and fix issues introduced in SPARK-15858 >

[jira] [Created] (SPARK-19114) Backpressure rate is cast from double to long to double

2017-01-06 Thread Tony Novak (JIRA)
Tony Novak created SPARK-19114: -- Summary: Backpressure rate is cast from double to long to double Key: SPARK-19114 URL: https://issues.apache.org/jira/browse/SPARK-19114 Project: Spark Issue

[jira] [Comment Edited] (SPARK-9215) Implement WAL-free Kinesis receiver that give at-least once guarantee

2017-01-06 Thread Gaurav Shah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15804425#comment-15804425 ] Gaurav Shah edited comment on SPARK-9215 at 1/7/17 2:04 AM: [~tdas] I know

[jira] [Updated] (SPARK-18372) .Hive-staging folders created from Spark hiveContext are not getting cleaned up

2017-01-06 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-18372: Assignee: mingjie tang > .Hive-staging folders created from Spark hiveContext are not getting cleaned >

[jira] [Resolved] (SPARK-18372) .Hive-staging folders created from Spark hiveContext are not getting cleaned up

2017-01-06 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-18372. - Resolution: Resolved Fix Version/s: 1.6.4 > .Hive-staging folders created from Spark hiveContext

[jira] [Commented] (SPARK-17975) EMLDAOptimizer fails with ClassCastException on YARN

2017-01-06 Thread Ilya Matiach (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15806368#comment-15806368 ] Ilya Matiach commented on SPARK-17975: -- I was able to reproduce the issue based on your dataset and

[jira] [Assigned] (SPARK-17975) EMLDAOptimizer fails with ClassCastException on YARN

2017-01-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17975: Assignee: (was: Apache Spark) > EMLDAOptimizer fails with ClassCastException on YARN

[jira] [Assigned] (SPARK-17975) EMLDAOptimizer fails with ClassCastException on YARN

2017-01-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17975: Assignee: Apache Spark > EMLDAOptimizer fails with ClassCastException on YARN >

[jira] [Commented] (SPARK-17975) EMLDAOptimizer fails with ClassCastException on YARN

2017-01-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15806365#comment-15806365 ] Apache Spark commented on SPARK-17975: -- User 'imatiach-msft' has created a pull request for this

[jira] [Assigned] (SPARK-19093) Cached tables are not used in SubqueryExpression

2017-01-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19093: Assignee: (was: Apache Spark) > Cached tables are not used in SubqueryExpression >

[jira] [Commented] (SPARK-19093) Cached tables are not used in SubqueryExpression

2017-01-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15806285#comment-15806285 ] Apache Spark commented on SPARK-19093: -- User 'dilipbiswal' has created a pull request for this

[jira] [Assigned] (SPARK-19093) Cached tables are not used in SubqueryExpression

2017-01-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19093: Assignee: Apache Spark > Cached tables are not used in SubqueryExpression >

[jira] [Commented] (SPARK-7768) Make user-defined type (UDT) API public

2017-01-06 Thread Randall Whitman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15806218#comment-15806218 ] Randall Whitman commented on SPARK-7768: Would you mind commenting on how the updated UDT works

[jira] [Assigned] (SPARK-7768) Make user-defined type (UDT) API public

2017-01-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7768: --- Assignee: Apache Spark > Make user-defined type (UDT) API public >

[jira] [Assigned] (SPARK-7768) Make user-defined type (UDT) API public

2017-01-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7768: --- Assignee: (was: Apache Spark) > Make user-defined type (UDT) API public >

[jira] [Commented] (SPARK-7768) Make user-defined type (UDT) API public

2017-01-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15806167#comment-15806167 ] Apache Spark commented on SPARK-7768: - User 'viirya' has created a pull request for this issue:

[jira] [Updated] (SPARK-19111) S3 Mesos history upload fails if too large

2017-01-06 Thread Charles Allen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charles Allen updated SPARK-19111: -- Summary: S3 Mesos history upload fails if too large (was: S3 Mesos history upload fails if

[jira] [Assigned] (SPARK-19113) Fix flaky test: o.a.s.sql.streaming.StreamSuite fatal errors from a source should be sent to the user

2017-01-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19113: Assignee: Apache Spark (was: Shixiong Zhu) > Fix flaky test:

[jira] [Commented] (SPARK-19113) Fix flaky test: o.a.s.sql.streaming.StreamSuite fatal errors from a source should be sent to the user

2017-01-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15806085#comment-15806085 ] Apache Spark commented on SPARK-19113: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19113) Fix flaky test: o.a.s.sql.streaming.StreamSuite fatal errors from a source should be sent to the user

2017-01-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19113: Assignee: Shixiong Zhu (was: Apache Spark) > Fix flaky test:

[jira] [Updated] (SPARK-19111) S3 Mesos history upload fails silently if too large

2017-01-06 Thread Charles Allen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charles Allen updated SPARK-19111: -- Summary: S3 Mesos history upload fails silently if too large (was: S3 Mesos history upload

[jira] [Created] (SPARK-19113) Fix flaky test: o.a.s.sql.streaming.StreamSuite fatal errors from a source should be sent to the user

2017-01-06 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-19113: Summary: Fix flaky test: o.a.s.sql.streaming.StreamSuite fatal errors from a source should be sent to the user Key: SPARK-19113 URL:

[jira] [Created] (SPARK-19112) add codec for ZStandard

2017-01-06 Thread Thomas Graves (JIRA)
Thomas Graves created SPARK-19112: - Summary: add codec for ZStandard Key: SPARK-19112 URL: https://issues.apache.org/jira/browse/SPARK-19112 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-19111) S3 Mesos history upload fails if too large or if distributed datastore is misbehaving

2017-01-06 Thread Charles Allen (JIRA)
Charles Allen created SPARK-19111: - Summary: S3 Mesos history upload fails if too large or if distributed datastore is misbehaving Key: SPARK-19111 URL: https://issues.apache.org/jira/browse/SPARK-19111

[jira] [Closed] (SPARK-18710) Add offset to GeneralizedLinearRegression models

2017-01-06 Thread Wayne Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wayne Zhang closed SPARK-18710. --- Resolution: Unresolved > Add offset to GeneralizedLinearRegression models >

[jira] [Assigned] (SPARK-19110) DistributedLDAModel returns different logPrior for original and loaded model

2017-01-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19110: Assignee: Apache Spark > DistributedLDAModel returns different logPrior for original and

[jira] [Commented] (SPARK-19110) DistributedLDAModel returns different logPrior for original and loaded model

2017-01-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15805688#comment-15805688 ] Apache Spark commented on SPARK-19110: -- User 'wangmiao1981' has created a pull request for this

[jira] [Assigned] (SPARK-19110) DistributedLDAModel returns different logPrior for original and loaded model

2017-01-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19110: Assignee: (was: Apache Spark) > DistributedLDAModel returns different logPrior for

[jira] [Created] (SPARK-19110) DistributedLDAModel returns different logPrior for original and loaded model

2017-01-06 Thread Miao Wang (JIRA)
Miao Wang created SPARK-19110: - Summary: DistributedLDAModel returns different logPrior for original and loaded model Key: SPARK-19110 URL: https://issues.apache.org/jira/browse/SPARK-19110 Project:

[jira] [Updated] (SPARK-19109) ORC metadata section can sometimes exceed protobuf message size limit

2017-01-06 Thread Nic Eggert (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nic Eggert updated SPARK-19109: --- Description: Basically, Spark inherits HIVE-11592 from its Hive dependency. From that issue: If

[jira] [Created] (SPARK-19109) ORC metadata section can sometimes exceed protobuf message size limit

2017-01-06 Thread Nic Eggert (JIRA)
Nic Eggert created SPARK-19109: -- Summary: ORC metadata section can sometimes exceed protobuf message size limit Key: SPARK-19109 URL: https://issues.apache.org/jira/browse/SPARK-19109 Project: Spark

[jira] [Resolved] (SPARK-6099) Stabilize mllib ClassificationModel, RegressionModel APIs

2017-01-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6099. -- Resolution: Done > Stabilize mllib ClassificationModel, RegressionModel APIs >

[jira] [Resolved] (SPARK-6098) Propagate Experimental tag to child classes

2017-01-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6098. -- Resolution: Not A Problem This is defunct now that these aren't even experimental > Propagate

[jira] [Updated] (SPARK-18890) Do all task serialization in CoarseGrainedExecutorBackend thread (rather than TaskSchedulerImpl)

2017-01-06 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout updated SPARK-18890: --- Issue Type: Improvement (was: Bug) > Do all task serialization in

[jira] [Commented] (SPARK-18890) Do all task serialization in CoarseGrainedExecutorBackend thread (rather than TaskSchedulerImpl)

2017-01-06 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15805495#comment-15805495 ] Kay Ousterhout commented on SPARK-18890: I just opened SPARK-19108 for the broadcast issue. In

[jira] [Created] (SPARK-19108) Broadcast all shared parts of tasks (to reduce task serialization time)

2017-01-06 Thread Kay Ousterhout (JIRA)
Kay Ousterhout created SPARK-19108: -- Summary: Broadcast all shared parts of tasks (to reduce task serialization time) Key: SPARK-19108 URL: https://issues.apache.org/jira/browse/SPARK-19108 Project:

[jira] [Resolved] (SPARK-19074) Update Structured Streaming Programming guide for Update Mode

2017-01-06 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-19074. --- Resolution: Fixed Fix Version/s: 2.2.0 2.1.1 Issue resolved by

[jira] [Closed] (SPARK-3937) Unsafe memory access inside of Snappy library

2017-01-06 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout closed SPARK-3937. - > Unsafe memory access inside of Snappy library > - > >

[jira] [Resolved] (SPARK-3937) Unsafe memory access inside of Snappy library

2017-01-06 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout resolved SPARK-3937. --- Resolution: Won't Fix Closing this due to lack of activity / reports of issues on recent

[jira] [Commented] (SPARK-10078) Vector-free L-BFGS

2017-01-06 Thread Seth Hendrickson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15805366#comment-15805366 ] Seth Hendrickson commented on SPARK-10078: -- As a part of

[jira] [Commented] (SPARK-6099) Stabilize mllib ClassificationModel, RegressionModel APIs

2017-01-06 Thread Ilya Matiach (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15805232#comment-15805232 ] Ilya Matiach commented on SPARK-6099: - It doesn't look like the API's are experimental anymore, can

[jira] [Commented] (SPARK-5493) Support proxy users under kerberos

2017-01-06 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15805205#comment-15805205 ] Marcelo Vanzin commented on SPARK-5493: --- > Since keytab will be owned by a "service" account, not by

[jira] [Comment Edited] (SPARK-5493) Support proxy users under kerberos

2017-01-06 Thread Ruslan Dautkhanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15805198#comment-15805198 ] Ruslan Dautkhanov edited comment on SPARK-5493 at 1/6/17 6:19 PM: --

[jira] [Commented] (SPARK-5493) Support proxy users under kerberos

2017-01-06 Thread Ruslan Dautkhanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15805198#comment-15805198 ] Ruslan Dautkhanov commented on SPARK-5493: -- {quote}There might be ways to hack support for that

[jira] [Commented] (SPARK-11968) ALS recommend all methods spend most of time in GC

2017-01-06 Thread Ilya Matiach (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15805169#comment-15805169 ] Ilya Matiach commented on SPARK-11968: -- Can someone with permissions change the status from In

[jira] [Resolved] (SPARK-19083) sbin/start-history-server.sh scripts use of $@ without ""

2017-01-06 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-19083. Resolution: Fixed Assignee: zuotingbing Fix Version/s: 2.2.0

[jira] [Commented] (SPARK-18890) Do all task serialization in CoarseGrainedExecutorBackend thread (rather than TaskSchedulerImpl)

2017-01-06 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15805087#comment-15805087 ] Imran Rashid commented on SPARK-18890: -- [~gq] I think you misunderstood my suggestion about using a

[jira] [Assigned] (SPARK-19106) Styling for the configuration docs is broken

2017-01-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19106: Assignee: Apache Spark > Styling for the configuration docs is broken >

[jira] [Commented] (SPARK-18836) Serialize Task Metrics once per stage

2017-01-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15805042#comment-15805042 ] Apache Spark commented on SPARK-18836: -- User 'squito' has created a pull request for this issue:

[jira] [Commented] (SPARK-19106) Styling for the configuration docs is broken

2017-01-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15805043#comment-15805043 ] Apache Spark commented on SPARK-19106: -- User 'srowen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19106) Styling for the configuration docs is broken

2017-01-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19106: Assignee: (was: Apache Spark) > Styling for the configuration docs is broken >

[jira] [Commented] (SPARK-19106) Styling for the configuration docs is broken

2017-01-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15805037#comment-15805037 ] Sean Owen commented on SPARK-19106: --- Yeah, the section headings aren't rendering as section titles. Not

[jira] [Updated] (SPARK-17931) taskScheduler has some unneeded serialization

2017-01-06 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-17931: - Assignee: Kay Ousterhout > taskScheduler has some unneeded serialization >

[jira] [Resolved] (SPARK-17931) taskScheduler has some unneeded serialization

2017-01-06 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid resolved SPARK-17931. -- Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 16053

[jira] [Comment Edited] (SPARK-10078) Vector-free L-BFGS

2017-01-06 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15804899#comment-15804899 ] Yanbo Liang edited comment on SPARK-10078 at 1/6/17 4:27 PM: - [~debasish83]

[jira] [Commented] (SPARK-10078) Vector-free L-BFGS

2017-01-06 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15804899#comment-15804899 ] Yanbo Liang commented on SPARK-10078: - [~debasish83] We are aim to implement VL-BFGS as an optimizer

[jira] [Resolved] (SPARK-19033) HistoryServer still uses old ACLs even if ACLs are updated

2017-01-06 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-19033. --- Resolution: Fixed Fix Version/s: 2.2.0 2.1.1 > HistoryServer still

[jira] [Updated] (SPARK-19033) HistoryServer still uses old ACLs even if ACLs are updated

2017-01-06 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-19033: -- Assignee: Saisai Shao > HistoryServer still uses old ACLs even if ACLs are updated >

[jira] [Commented] (SPARK-10078) Vector-free L-BFGS

2017-01-06 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15804882#comment-15804882 ] Yanbo Liang commented on SPARK-10078: - [~sethah] The description is a little misleading, it means the

[jira] [Assigned] (SPARK-19107) support creating hive table with DataFrameWriter and Catalog

2017-01-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19107: Assignee: Apache Spark (was: Wenchen Fan) > support creating hive table with

[jira] [Assigned] (SPARK-19107) support creating hive table with DataFrameWriter and Catalog

2017-01-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19107: Assignee: Wenchen Fan (was: Apache Spark) > support creating hive table with

[jira] [Commented] (SPARK-19107) support creating hive table with DataFrameWriter and Catalog

2017-01-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15804809#comment-15804809 ] Apache Spark commented on SPARK-19107: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Created] (SPARK-19107) support creating hive table with DataFrameWriter and Catalog

2017-01-06 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-19107: --- Summary: support creating hive table with DataFrameWriter and Catalog Key: SPARK-19107 URL: https://issues.apache.org/jira/browse/SPARK-19107 Project: Spark

[jira] [Created] (SPARK-19106) Styling for the configuration docs is broken

2017-01-06 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-19106: Summary: Styling for the configuration docs is broken Key: SPARK-19106 URL: https://issues.apache.org/jira/browse/SPARK-19106 Project: Spark Issue

[jira] [Updated] (SPARK-19106) Styling for the configuration docs is broken

2017-01-06 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-19106: - Attachment: Screen Shot 2017-01-06 at 10.20.52 AM.png > Styling for the configuration

[jira] [Closed] (SPARK-19086) Improper scoping of name resolution of columns in HAVING clause

2017-01-06 Thread Nattavut Sutyanyong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nattavut Sutyanyong closed SPARK-19086. --- Resolution: Not A Problem > Improper scoping of name resolution of columns in HAVING

[jira] [Updated] (SPARK-19099) Wrong time display on Spark History Server web UI

2017-01-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-19099: -- Target Version/s: (was: 2.1.0) Labels: (was: none) Fix Version/s: (was:

[jira] [Commented] (SPARK-19105) yarn/Client.scala copyToRemote does not include keytab destination name

2017-01-06 Thread Peter Parente (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15804545#comment-15804545 ] Peter Parente commented on SPARK-19105: --- >From chat in the PR, the keytab on AM does have the

[jira] [Closed] (SPARK-19105) yarn/Client.scala copyToRemote does not include keytab destination name

2017-01-06 Thread Peter Parente (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Parente closed SPARK-19105. - Resolution: Invalid > yarn/Client.scala copyToRemote does not include keytab destination name >

[jira] [Assigned] (SPARK-19105) yarn/Client.scala copyToRemote does not include keytab destination name

2017-01-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19105: Assignee: (was: Apache Spark) > yarn/Client.scala copyToRemote does not include

[jira] [Issue Comment Deleted] (SPARK-19105) yarn/Client.scala copyToRemote does not include keytab destination name

2017-01-06 Thread Peter Parente (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Parente updated SPARK-19105: -- Comment: was deleted (was: Related PR https://github.com/apache/spark/pull/16482) >

[jira] [Commented] (SPARK-19105) yarn/Client.scala copyToRemote does not include keytab destination name

2017-01-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15804475#comment-15804475 ] Apache Spark commented on SPARK-19105: -- User 'parente' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19105) yarn/Client.scala copyToRemote does not include keytab destination name

2017-01-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19105: Assignee: Apache Spark > yarn/Client.scala copyToRemote does not include keytab

[jira] [Commented] (SPARK-19105) yarn/Client.scala copyToRemote does not include keytab destination name

2017-01-06 Thread Peter Parente (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15804476#comment-15804476 ] Peter Parente commented on SPARK-19105: --- Related PR https://github.com/apache/spark/pull/16482 >

[jira] [Updated] (SPARK-19105) yarn/Client.scala copyToRemote does not include keytab destination name

2017-01-06 Thread Peter Parente (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Parente updated SPARK-19105: -- Description: When I specify --principal user@REALM and --keytab /some/path/user.keytab, I see

[jira] [Created] (SPARK-19105) yarn/Client.scala copyToRemote does not include keytab destination name

2017-01-06 Thread Peter Parente (JIRA)
Peter Parente created SPARK-19105: - Summary: yarn/Client.scala copyToRemote does not include keytab destination name Key: SPARK-19105 URL: https://issues.apache.org/jira/browse/SPARK-19105 Project:

[jira] [Commented] (SPARK-9215) Implement WAL-free Kinesis receiver that give at-least once guarantee

2017-01-06 Thread Gaurav Shah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15804425#comment-15804425 ] Gaurav Shah commented on SPARK-9215: [~tdas] I know this is an old pull request but was still

[jira] [Created] (SPARK-19104) CompileException with Map and Case Class in Spark 2.1.0

2017-01-06 Thread Nils Grabbert (JIRA)
Nils Grabbert created SPARK-19104: - Summary: CompileException with Map and Case Class in Spark 2.1.0 Key: SPARK-19104 URL: https://issues.apache.org/jira/browse/SPARK-19104 Project: Spark

[jira] [Commented] (SPARK-18997) Recommended upgrade libthrift to 0.9.3

2017-01-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15804135#comment-15804135 ] Sean Owen commented on SPARK-18997: --- Help with what, opening a PR?

[jira] [Resolved] (SPARK-19097) virtualenv example failed with conda due to ImportError: No module named ruamel.yaml.comments

2017-01-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-19097. --- Resolution: Duplicate I don't see value in opening a bunch of JIRAs on the same theme. These look

[jira] [Updated] (SPARK-19102) Accuracy error of spark SQL results

2017-01-06 Thread XiaodongCui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiaodongCui updated SPARK-19102: Description: the problem is cube6's second column named sumprice is 1 times bigger than the

[jira] [Reopened] (SPARK-19102) Accuracy error of spark SQL results

2017-01-06 Thread XiaodongCui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiaodongCui reopened SPARK-19102: - the data under the path :hdfs://cdh01:8020/sandboxdata_A/test/a in the attach file > Accuracy

[jira] [Commented] (SPARK-19101) Spark Beeline catch a exeception when run command " load data inpath '/data/test/test.csv' overwrite into table db.test partition(area='021')"

2017-01-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15804113#comment-15804113 ] Sean Owen commented on SPARK-19101: --- This is more of a Hive error, and suggests something else went

[jira] [Updated] (SPARK-19102) Accuracy error of spark SQL results

2017-01-06 Thread XiaodongCui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiaodongCui updated SPARK-19102: Description: the problem is cube6's second column named sumprice is 1 times bigger than the

[jira] [Updated] (SPARK-19102) Accuracy error of spark SQL results

2017-01-06 Thread XiaodongCui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiaodongCui updated SPARK-19102: Attachment: a.zip the attach file is my data,the data is parquet format > Accuracy error of spark

[jira] [Resolved] (SPARK-19102) Accuracy error of spark SQL results

2017-01-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-19102. --- Resolution: Invalid This doesn't describe a problem clearly. There's no data, no specifics about

[jira] [Resolved] (SPARK-19095) virtualenv example does not work in yarn cluster mode

2017-01-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-19095. --- Resolution: Duplicate If this isn't something supported yet, it's not a bug, and I'd resolve this as

[jira] [Resolved] (SPARK-19096) Kmeans.py application fails with virtualenv and due to parse error

2017-01-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-19096. --- Resolution: Duplicate > Kmeans.py application fails with virtualenv and due to parse error >

[jira] [Updated] (SPARK-19098) Shuffled data leak/size doubling in ConnectedComponents/Pregel iterations

2017-01-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-19098: -- Priority: Minor (was: Critical) > Shuffled data leak/size doubling in ConnectedComponents/Pregel

[jira] [Resolved] (SPARK-19103) In web ui,URL's host name should be a specific IP address.

2017-01-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-19103. --- Resolution: Invalid I can't understand what this means, but, it's not true that the UI should use

  1   2   >