[jira] [Commented] (SPARK-3146) Improve the flexibility of Spark Streaming Kafka API to offer user the ability to process message before storing into BM

[ https://issues.apache.org/jira/browse/SPARK-3146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14103520#comment-14103520 ] Apache Spark commented on SPARK-3146: - User 'jerryshao' has created a pull request for

[jira] [Updated] (SPARK-3066) Support recommendAll in matrix factorization model

[ https://issues.apache.org/jira/browse/SPARK-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-3066: - Target Version/s: 1.2.0 > Support recommendAll in matrix factorization model > --

[jira] [Updated] (SPARK-3146) Improve the flexibility of Spark Streaming Kafka API to offer user the ability to process message before storing into BM

[ https://issues.apache.org/jira/browse/SPARK-3146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-3146: --- Description: Currently Spark Streaming Kafka API stores the key and value of each message into BM fo

[jira] [Updated] (SPARK-2121) Not fully cached when there is enough memory in ALS

[ https://issues.apache.org/jira/browse/SPARK-2121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-2121: - Summary: Not fully cached when there is enough memory in ALS (was: Not fully cached when there i

[jira] [Created] (SPARK-3147) Implement A/B testing

Xiangrui Meng created SPARK-3147: Summary: Implement A/B testing Key: SPARK-3147 URL: https://issues.apache.org/jira/browse/SPARK-3147 Project: Spark Issue Type: New Feature Compone

[jira] [Commented] (SPARK-3146) Improve the flexibility of Spark Streaming Kafka API to offer user the ability to process message before storing into BM

[ https://issues.apache.org/jira/browse/SPARK-3146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14103486#comment-14103486 ] Saisai Shao commented on SPARK-3146: This issue can actually solve the problem mention

[jira] [Created] (SPARK-3146) Improve the flexibility of Spark Streaming Kafka API to offer user the ability to process message before storing into BM

Saisai Shao created SPARK-3146: -- Summary: Improve the flexibility of Spark Streaming Kafka API to offer user the ability to process message before storing into BM Key: SPARK-3146 URL: https://issues.apache.org/jira/b

[jira] [Resolved] (SPARK-3141) sortByKey() break take()

[ https://issues.apache.org/jira/browse/SPARK-3141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-3141. Resolution: Fixed Fix Version/s: 1.1.0 Issue resolved by pull request 2045 [https://

[jira] [Resolved] (SPARK-2974) Utils.getLocalDir() may return non-existent spark.local.dir directory

[ https://issues.apache.org/jira/browse/SPARK-2974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-2974. Resolution: Fixed Fix Version/s: 1.1.0 Issue resolved by pull request 2002 [https://

[jira] [Created] (SPARK-3145) Hive on Spark dependency umbrella

bc Wong created SPARK-3145: -- Summary: Hive on Spark dependency umbrella Key: SPARK-3145 URL: https://issues.apache.org/jira/browse/SPARK-3145 Project: Spark Issue Type: Epic Components: Bu

[jira] [Resolved] (SPARK-3142) Reduce memory usage in Word2Vec

[ https://issues.apache.org/jira/browse/SPARK-3142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-3142. -- Resolution: Fixed Fix Version/s: 1.1.0 Issue resolved by pull request 2049 [https://gith

[jira] [Resolved] (SPARK-3119) Re-implement TorrentBroadcast

[ https://issues.apache.org/jira/browse/SPARK-3119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-3119. Resolution: Fixed Fix Version/s: 1.1.0 > Re-implement TorrentBroadcast > ---

[jira] [Commented] (SPARK-1267) Add a pip installer for PySpark

[ https://issues.apache.org/jira/browse/SPARK-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14103381#comment-14103381 ] Chandan Kumar commented on SPARK-1267: -- [~adgaudio] I had similar reservations about

[jira] [Resolved] (SPARK-3117) Avoid serialization for TorrentBroadcast blocks

[ https://issues.apache.org/jira/browse/SPARK-3117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-3117. Resolution: Fixed Fix Version/s: 1.1.0 > Avoid serialization for TorrentBroadcast blocks > -

[jira] [Resolved] (SPARK-3130) Should not allow negative values in naive Bayes

[ https://issues.apache.org/jira/browse/SPARK-3130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-3130. -- Resolution: Fixed Fix Version/s: 1.1.0 Issue resolved by pull request 2038 [https://gith

[jira] [Commented] (SPARK-3144) No need to set "spark.local.dir" in ExecutorLauncher

[ https://issues.apache.org/jira/browse/SPARK-3144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14103286#comment-14103286 ] Apache Spark commented on SPARK-3144: - User 'hzw19900416' has created a pull request f

[jira] [Created] (SPARK-3144) No need to set "spark.local.dir" in ExecutorLauncher

hzw created SPARK-3144: -- Summary: No need to set "spark.local.dir" in ExecutorLauncher Key: SPARK-3144 URL: https://issues.apache.org/jira/browse/SPARK-3144 Project: Spark Issue Type: Bug Comp

[jira] [Commented] (SPARK-3120) Local Dirs is not useful in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-3120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14103241#comment-14103241 ] hzw commented on SPARK-3120: I can not understand what you say clearly. Do you mean that there

[jira] [Created] (SPARK-3143) Documentation for TF-IDF

Xiangrui Meng created SPARK-3143: Summary: Documentation for TF-IDF Key: SPARK-3143 URL: https://issues.apache.org/jira/browse/SPARK-3143 Project: Spark Issue Type: Sub-task Compone

[jira] [Resolved] (SPARK-3112) Documentation for Streaming Logistic Regression Streaming

[ https://issues.apache.org/jira/browse/SPARK-3112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-3112. -- Resolution: Fixed Fix Version/s: 1.1.0 Issue resolved by pull request 2047 [https://gith

[jira] [Created] (SPARK-3142) Reduce memory usage in Word2Vec

Xiangrui Meng created SPARK-3142: Summary: Reduce memory usage in Word2Vec Key: SPARK-3142 URL: https://issues.apache.org/jira/browse/SPARK-3142 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-3142) Reduce memory usage in Word2Vec

[ https://issues.apache.org/jira/browse/SPARK-3142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14103180#comment-14103180 ] Apache Spark commented on SPARK-3142: - User 'mengxr' has created a pull request for th

[jira] [Commented] (SPARK-2312) Spark Actors do not handle unknown messages in their receive methods

[ https://issues.apache.org/jira/browse/SPARK-2312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14103152#comment-14103152 ] Apache Spark commented on SPARK-2312: - User 'isaias' has created a pull request for th

[jira] [Commented] (SPARK-2312) Spark Actors do not handle unknown messages in their receive methods

[ https://issues.apache.org/jira/browse/SPARK-2312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14103148#comment-14103148 ] Isaias Barroso commented on SPARK-2312: --- Created a Pull Request https://github.com/a

[jira] [Commented] (SPARK-3112) Documentation for Streaming Logistic Regression Streaming

[ https://issues.apache.org/jira/browse/SPARK-3112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14103142#comment-14103142 ] Apache Spark commented on SPARK-3112: - User 'freeman-lab' has created a pull request f

[jira] [Commented] (SPARK-3141) sortByKey() break take()

[ https://issues.apache.org/jira/browse/SPARK-3141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14103097#comment-14103097 ] Apache Spark commented on SPARK-3141: - User 'davies' has created a pull request for th

[jira] [Created] (SPARK-3141) sortByKey() break take()

Davies Liu created SPARK-3141: - Summary: sortByKey() break take() Key: SPARK-3141 URL: https://issues.apache.org/jira/browse/SPARK-3141 Project: Spark Issue Type: Bug Components: PySpar

[jira] [Resolved] (SPARK-3136) create java-friendly methods in RandomRDDs

[ https://issues.apache.org/jira/browse/SPARK-3136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-3136. -- Resolution: Fixed Fix Version/s: 1.1.0 Issue resolved by pull request 2041 [https://gith

[jira] [Commented] (SPARK-3138) sqlContext.parquetFile should be able to take a single file as parameter

[ https://issues.apache.org/jira/browse/SPARK-3138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14102979#comment-14102979 ] Teng Qiu commented on SPARK-3138: - after this PR, we can pass the full path of a parquet f

[jira] [Commented] (SPARK-3138) sqlContext.parquetFile should be able to take a single file as parameter

[ https://issues.apache.org/jira/browse/SPARK-3138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14102967#comment-14102967 ] Apache Spark commented on SPARK-3138: - User 'chutium' has created a pull request for t

[jira] [Created] (SPARK-3140) PySpark start-up throws confusing exception

Andrew Or created SPARK-3140: Summary: PySpark start-up throws confusing exception Key: SPARK-3140 URL: https://issues.apache.org/jira/browse/SPARK-3140 Project: Spark Issue Type: Bug C

[jira] [Commented] (SPARK-3138) sqlContext.parquetFile should be able to take a single file as parameter

[ https://issues.apache.org/jira/browse/SPARK-3138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14102961#comment-14102961 ] Teng Qiu commented on SPARK-3138: - be careful if someone is working on SPARK-2551, make su

[jira] [Commented] (SPARK-3139) Akka timeouts from ContextCleaner when cleaning shuffles

[ https://issues.apache.org/jira/browse/SPARK-3139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14102960#comment-14102960 ] Josh Rosen commented on SPARK-3139: --- I used pssh + grep to search through the applicatio

[jira] [Commented] (SPARK-3139) Akka timeouts from ContextCleaner when cleaning shuffles

[ https://issues.apache.org/jira/browse/SPARK-3139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14102950#comment-14102950 ] Andrew Or commented on SPARK-3139: -- These are both caused by akka timing out because of c

[jira] [Created] (SPARK-3139) Akka timeouts from ContextCleaner when cleaning shuffles

Josh Rosen created SPARK-3139: - Summary: Akka timeouts from ContextCleaner when cleaning shuffles Key: SPARK-3139 URL: https://issues.apache.org/jira/browse/SPARK-3139 Project: Spark Issue Type:

[jira] [Created] (SPARK-3138) sqlContext.parquetFile should be able to take a single file as parameter

Teng Qiu created SPARK-3138: --- Summary: sqlContext.parquetFile should be able to take a single file as parameter Key: SPARK-3138 URL: https://issues.apache.org/jira/browse/SPARK-3138 Project: Spark

[jira] [Resolved] (SPARK-2790) PySpark zip() doesn't work properly if RDDs have different serializers

[ https://issues.apache.org/jira/browse/SPARK-2790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-2790. --- Resolution: Fixed Fix Version/s: 1.1.0 > PySpark zip() doesn't work properly if RDDs have diff

[jira] [Updated] (SPARK-3112) Documentation for Streaming Logistic Regression Streaming

[ https://issues.apache.org/jira/browse/SPARK-3112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-3112: - Assignee: Jeremy Freeman > Documentation for Streaming Logistic Regression Streaming > --

[jira] [Updated] (SPARK-2839) Documentation for statistical functions

[ https://issues.apache.org/jira/browse/SPARK-2839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-2839: - Assignee: Burak Yavuz > Documentation for statistical functions > ---

[jira] [Resolved] (SPARK-2333) spark_ec2 script should allow option for existing security group

[ https://issues.apache.org/jira/browse/SPARK-2333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-2333. --- Resolution: Fixed > spark_ec2 script should allow option for existing security group > --

[jira] [Updated] (SPARK-2333) spark_ec2 script should allow option for existing security group

[ https://issues.apache.org/jira/browse/SPARK-2333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-2333: -- Issue Type: Improvement (was: Bug) > spark_ec2 script should allow option for existing security group

[jira] [Commented] (SPARK-3136) create java-friendly methods in RandomRDDs

[ https://issues.apache.org/jira/browse/SPARK-3136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14102791#comment-14102791 ] Apache Spark commented on SPARK-3136: - User 'mengxr' has created a pull request for th

[jira] [Updated] (SPARK-3137) Use finer grained locking in TorrentBroadcast.readObject

[ https://issues.apache.org/jira/browse/SPARK-3137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-3137: --- Component/s: Spark Core Target Version/s: 1.2.0 > Use finer grained locking in TorrentBroadc

[jira] [Created] (SPARK-3137) Use finer grained locking in TorrentBroadcast.readObject

Reynold Xin created SPARK-3137: -- Summary: Use finer grained locking in TorrentBroadcast.readObject Key: SPARK-3137 URL: https://issues.apache.org/jira/browse/SPARK-3137 Project: Spark Issue Type

[jira] [Updated] (SPARK-3133) Piggyback get location RPC call to fetch small blocks

[ https://issues.apache.org/jira/browse/SPARK-3133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-3133: --- Description: We should add a new API to the BlockManagerMasterActor to get location or the data bloc

[jira] [Updated] (SPARK-3133) Piggyback get location RPC call to fetch small blocks

[ https://issues.apache.org/jira/browse/SPARK-3133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-3133: --- Description: We should add a new API to the BlockManagerMasterActor to get location or the data bloc

[jira] [Resolved] (SPARK-3128) Use streaming test suite for StreamingLR

[ https://issues.apache.org/jira/browse/SPARK-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-3128. -- Resolution: Fixed Fix Version/s: 1.2.0 1.1.0 > Use streaming test sui

[jira] [Updated] (SPARK-3135) Avoid memory copy in TorrentBroadcast serialization

[ https://issues.apache.org/jira/browse/SPARK-3135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-3135: --- Labels: starter (was: ) > Avoid memory copy in TorrentBroadcast serialization >

[jira] [Created] (SPARK-3136) create java-friendly methods in RandomRDDs

Xiangrui Meng created SPARK-3136: Summary: create java-friendly methods in RandomRDDs Key: SPARK-3136 URL: https://issues.apache.org/jira/browse/SPARK-3136 Project: Spark Issue Type: Improvem

[jira] [Updated] (SPARK-3135) Avoid memory copy in TorrentBroadcast serialization

[ https://issues.apache.org/jira/browse/SPARK-3135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-3135: --- Description: TorrentBroadcast.blockifyObject uses a ByteArrayOutputStream to serialize broadcast obje

[jira] [Created] (SPARK-3135) Avoid memory copy in TorrentBroadcast serialization

Reynold Xin created SPARK-3135: -- Summary: Avoid memory copy in TorrentBroadcast serialization Key: SPARK-3135 URL: https://issues.apache.org/jira/browse/SPARK-3135 Project: Spark Issue Type: Sub

[jira] [Created] (SPARK-3134) Update block locations asynchronously in TorrentBroadcast

Reynold Xin created SPARK-3134: -- Summary: Update block locations asynchronously in TorrentBroadcast Key: SPARK-3134 URL: https://issues.apache.org/jira/browse/SPARK-3134 Project: Spark Issue Typ

[jira] [Created] (SPARK-3133) Piggyback get location RPC call to fetch small blocks

Reynold Xin created SPARK-3133: -- Summary: Piggyback get location RPC call to fetch small blocks Key: SPARK-3133 URL: https://issues.apache.org/jira/browse/SPARK-3133 Project: Spark Issue Type: S

[jira] [Created] (SPARK-3132) Avoid serialization for Array[Byte] in TorrentBroadcast

Reynold Xin created SPARK-3132: -- Summary: Avoid serialization for Array[Byte] in TorrentBroadcast Key: SPARK-3132 URL: https://issues.apache.org/jira/browse/SPARK-3132 Project: Spark Issue Type:

[jira] [Commented] (SPARK-3117) Avoid serialization for TorrentBroadcast blocks

[ https://issues.apache.org/jira/browse/SPARK-3117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14102758#comment-14102758 ] Reynold Xin commented on SPARK-3117: This is going to be fixed by https://github.com/a

[jira] [Commented] (SPARK-3131) Allow user to set parquet compression codec for writing ParquetFile in SQLContext

[ https://issues.apache.org/jira/browse/SPARK-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14102724#comment-14102724 ] Apache Spark commented on SPARK-3131: - User 'chutium' has created a pull request for t

[jira] [Updated] (SPARK-3131) Allow user to set parquet compression codec for writing ParquetFile in SQLContext

[ https://issues.apache.org/jira/browse/SPARK-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Teng Qiu updated SPARK-3131: Description: There are 4 different compression codec available for ParquetOutputFormat in Spark SQL it was

[jira] [Created] (SPARK-3131) Allow user to set parquet compression codec

Teng Qiu created SPARK-3131: --- Summary: Allow user to set parquet compression codec Key: SPARK-3131 URL: https://issues.apache.org/jira/browse/SPARK-3131 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-3131) Allow user to set parquet compression codec for writing ParquetFile in SQLContext

[ https://issues.apache.org/jira/browse/SPARK-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Teng Qiu updated SPARK-3131: Summary: Allow user to set parquet compression codec for writing ParquetFile in SQLContext (was: Allow use

[jira] [Commented] (SPARK-3129) Prevent data loss in Spark Streaming

[ https://issues.apache.org/jira/browse/SPARK-3129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14102579#comment-14102579 ] Hari Shreedharan commented on SPARK-3129: - The way the driver "finds" the executor

[jira] [Commented] (SPARK-3129) Prevent data loss in Spark Streaming

[ https://issues.apache.org/jira/browse/SPARK-3129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14102504#comment-14102504 ] Thomas Graves commented on SPARK-3129: -- A couple of random thoughts on this for yarn.

[jira] [Commented] (SPARK-3129) Prevent data loss in Spark Streaming

[ https://issues.apache.org/jira/browse/SPARK-3129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14102501#comment-14102501 ] Hari Shreedharan commented on SPARK-3129: - This doc is an early list of fixes. I m

[jira] [Commented] (SPARK-3130) Should not allow negative values in naive Bayes

[ https://issues.apache.org/jira/browse/SPARK-3130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14102499#comment-14102499 ] Apache Spark commented on SPARK-3130: - User 'mengxr' has created a pull request for th

[jira] [Resolved] (SPARK-3089) Fix meaningless error message in ConnectionManager

[ https://issues.apache.org/jira/browse/SPARK-3089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-3089. --- Resolution: Fixed Fix Version/s: 1.1.0 Assignee: Kousuke Saruta > Fix meaningless err

[jira] [Commented] (SPARK-3128) Use streaming test suite for StreamingLR

[ https://issues.apache.org/jira/browse/SPARK-3128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14102481#comment-14102481 ] Apache Spark commented on SPARK-3128: - User 'freeman-lab' has created a pull request f

[jira] [Updated] (SPARK-3110) Add a "ha" mode in YARN mode to keep executors in between restarts

[ https://issues.apache.org/jira/browse/SPARK-3110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Shreedharan updated SPARK-3110: Issue Type: Sub-task (was: Bug) Parent: SPARK-3129 > Add a "ha" mode in YARN mode

[jira] [Created] (SPARK-3130) Should not allow negative values in naive Bayes

Xiangrui Meng created SPARK-3130: Summary: Should not allow negative values in naive Bayes Key: SPARK-3130 URL: https://issues.apache.org/jira/browse/SPARK-3130 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-3122) hadoop-yarn dependencies cannot be resolved

[ https://issues.apache.org/jira/browse/SPARK-3122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14102472#comment-14102472 ] Sean Owen commented on SPARK-3122: -- [~gq] You do not need to depend on hadoop-client for

[jira] [Updated] (SPARK-3129) Prevent data loss in Spark Streaming

[ https://issues.apache.org/jira/browse/SPARK-3129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Shreedharan updated SPARK-3129: Attachment: StreamingPreventDataLoss.pdf > Prevent data loss in Spark Streaming > -

[jira] [Updated] (SPARK-3129) Prevent data loss in Spark Streaming

[ https://issues.apache.org/jira/browse/SPARK-3129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Shreedharan updated SPARK-3129: Issue Type: New Feature (was: Bug) > Prevent data loss in Spark Streaming > --

[jira] [Created] (SPARK-3128) Use streaming test suite for StreamingLR

Jeremy Freeman created SPARK-3128: - Summary: Use streaming test suite for StreamingLR Key: SPARK-3128 URL: https://issues.apache.org/jira/browse/SPARK-3128 Project: Spark Issue Type: Improvem

[jira] [Created] (SPARK-3129) Prevent data loss in Spark Streaming

Hari Shreedharan created SPARK-3129: --- Summary: Prevent data loss in Spark Streaming Key: SPARK-3129 URL: https://issues.apache.org/jira/browse/SPARK-3129 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-3120) Local Dirs is not useful in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-3120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-3120. -- Resolution: Invalid > Local Dirs is not useful in yarn-client mode > --

[jira] [Commented] (SPARK-3120) Local Dirs is not useful in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-3120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14102444#comment-14102444 ] Thomas Graves commented on SPARK-3120: -- If you want to change the local-dirs then you

[jira] [Commented] (SPARK-1782) svd for sparse matrix using ARPACK

[ https://issues.apache.org/jira/browse/SPARK-1782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14102427#comment-14102427 ] Xiangrui Meng commented on SPARK-1782: -- The plan is to release v1.1 by the end of the

[jira] [Commented] (SPARK-3125) hive thriftserver test suite failure

[ https://issues.apache.org/jira/browse/SPARK-3125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14102424#comment-14102424 ] wangfei commented on SPARK-3125: for clisuite i print the error info, as follows: log4j:WA

[jira] [Commented] (SPARK-3127) Modifying Spark SQL related scripts should trigger Spark SQL test suites

[ https://issues.apache.org/jira/browse/SPARK-3127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14102373#comment-14102373 ] Apache Spark commented on SPARK-3127: - User 'liancheng' has created a pull request for

[jira] [Updated] (SPARK-2929) Rewrite HiveThriftServer2Suite and CliSuite

[ https://issues.apache.org/jira/browse/SPARK-2929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-2929: -- Priority: Major (was: Blocker) > Rewrite HiveThriftServer2Suite and CliSuite > ---

[jira] [Commented] (SPARK-3126) HiveThriftServer2Suite hangs

[ https://issues.apache.org/jira/browse/SPARK-3126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14102372#comment-14102372 ] Apache Spark commented on SPARK-3126: - User 'liancheng' has created a pull request for

[jira] [Commented] (SPARK-2929) Rewrite HiveThriftServer2Suite and CliSuite

[ https://issues.apache.org/jira/browse/SPARK-2929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14102370#comment-14102370 ] Cheng Lian commented on SPARK-2929: --- Opened SPARK-3126 & SPARK-3127 to track failure of

[jira] [Commented] (SPARK-3124) Jar version conflict in the assembly package

[ https://issues.apache.org/jira/browse/SPARK-3124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14102352#comment-14102352 ] Cheng Hao commented on SPARK-3124: -- Yes, actually I did in the PR. > Jar version conflic

[jira] [Commented] (SPARK-3120) Local Dirs is not useful in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-3120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14102350#comment-14102350 ] hzw commented on SPARK-3120: Do you mean that : If I want to change the local-dirs in Yarn Mod

[jira] [Created] (SPARK-3127) Modifying Spark SQL related scripts should trigger Spark SQL test suites

Cheng Lian created SPARK-3127: - Summary: Modifying Spark SQL related scripts should trigger Spark SQL test suites Key: SPARK-3127 URL: https://issues.apache.org/jira/browse/SPARK-3127 Project: Spark

[jira] [Created] (SPARK-3126) HiveThriftServer2Suite hangs

Cheng Lian created SPARK-3126: - Summary: HiveThriftServer2Suite hangs Key: SPARK-3126 URL: https://issues.apache.org/jira/browse/SPARK-3126 Project: Spark Issue Type: Bug Components: SQ

[jira] [Commented] (SPARK-3124) Jar version conflict in the assembly package

[ https://issues.apache.org/jira/browse/SPARK-3124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14102344#comment-14102344 ] Guoqiang Li commented on SPARK-3124: We should modify the file sql/hive-thriftserver/

[jira] [Created] (SPARK-3125) hive thriftserver test suite failure

wangfei created SPARK-3125: -- Summary: hive thriftserver test suite failure Key: SPARK-3125 URL: https://issues.apache.org/jira/browse/SPARK-3125 Project: Spark Issue Type: Bug Components:

[jira] [Commented] (SPARK-3124) Jar version conflict in the assembly package

[ https://issues.apache.org/jira/browse/SPARK-3124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14102338#comment-14102338 ] Cheng Hao commented on SPARK-3124: -- Can you try "bin/spark-sql" after make distribution?

[jira] [Commented] (SPARK-3124) Jar version conflict in the assembly package

[ https://issues.apache.org/jira/browse/SPARK-3124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14102334#comment-14102334 ] Apache Spark commented on SPARK-3124: - User 'chenghao-intel' has created a pull reques

[jira] [Comment Edited] (SPARK-3124) Jar version conflict in the assembly package

[ https://issues.apache.org/jira/browse/SPARK-3124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14102330#comment-14102330 ] Guoqiang Li edited comment on SPARK-3124 at 8/19/14 3:42 PM: -

[jira] [Comment Edited] (SPARK-3124) Jar version conflict in the assembly package

[ https://issues.apache.org/jira/browse/SPARK-3124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14102330#comment-14102330 ] Guoqiang Li edited comment on SPARK-3124 at 8/19/14 3:42 PM: -

[jira] [Commented] (SPARK-3124) Jar version conflict in the assembly package

[ https://issues.apache.org/jira/browse/SPARK-3124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14102330#comment-14102330 ] Guoqiang Li commented on SPARK-3124: What's your command? > Jar version conflict in t

[jira] [Created] (SPARK-3124) Jar version conflict in the assembly package

Cheng Hao created SPARK-3124: Summary: Jar version conflict in the assembly package Key: SPARK-3124 URL: https://issues.apache.org/jira/browse/SPARK-3124 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-3122) hadoop-yarn dependencies cannot be resolved

[ https://issues.apache.org/jira/browse/SPARK-3122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14102318#comment-14102318 ] Guoqiang Li commented on SPARK-3122: It is not necessary. Only need to these: {code:xm

[jira] [Commented] (SPARK-3118) add "SHOW TBLPROPERTIES tblname;" and "SHOW COLUMNS (FROM|IN) table_name [(FROM|IN) db_name]" support

[ https://issues.apache.org/jira/browse/SPARK-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14102317#comment-14102317 ] Apache Spark commented on SPARK-3118: - User 'u0jing' has created a pull request for th

[jira] [Commented] (SPARK-3123) override the "setName" function to set EdgeRDD's name manually just as VertexRDD does.

[ https://issues.apache.org/jira/browse/SPARK-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14102308#comment-14102308 ] Apache Spark commented on SPARK-3123: - User 'uncleGen' has created a pull request for

[jira] [Created] (SPARK-3123) override the "setName" function to set EdgeRDD's name manually just as VertexRDD does.

uncleGen created SPARK-3123: --- Summary: override the "setName" function to set EdgeRDD's name manually just as VertexRDD does. Key: SPARK-3123 URL: https://issues.apache.org/jira/browse/SPARK-3123 Project: S

[jira] [Commented] (SPARK-3122) hadoop-yarn dependencies cannot be resolved

[ https://issues.apache.org/jira/browse/SPARK-3122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14102300#comment-14102300 ] Ran Levi commented on SPARK-3122: - It was my understanding that it is required to create a

[jira] [Commented] (SPARK-3122) hadoop-yarn dependencies cannot be resolved

[ https://issues.apache.org/jira/browse/SPARK-3122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14102288#comment-14102288 ] Guoqiang Li commented on SPARK-3122: Why add {{spark-yarn_2.10}} dependency? Normall

[jira] [Comment Edited] (SPARK-3098) In some cases, operation zipWithIndex get a wrong results

[ https://issues.apache.org/jira/browse/SPARK-3098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14102279#comment-14102279 ] Guoqiang Li edited comment on SPARK-3098 at 8/19/14 3:02 PM: -

[jira] [Created] (SPARK-3122) hadoop-yarn dependencies cannot be resolved

Ran Levi created SPARK-3122: --- Summary: hadoop-yarn dependencies cannot be resolved Key: SPARK-3122 URL: https://issues.apache.org/jira/browse/SPARK-3122 Project: Spark Issue Type: Bug Com

  1   2   >