Nope --- I cut that last Friday but had an error. I will remove it and cut a new one.
On Mon, Aug 24, 2015 at 2:06 PM, Sandy Ryza <sandy.r...@cloudera.com> wrote: > I see that there's an 1.5.0-rc2 tag in github now. Is that the official > RC2 tag to start trying out? > > -Sandy > > On Mon, Aug 24, 2015 at 7:23 AM, Sean Owen <so...@cloudera.com> wrote: > >> PS Shixiong Zhu is correct that this one has to be fixed: >> https://issues.apache.org/jira/browse/SPARK-10168 >> >> For example you can see assemblies like this are nearly empty: >> >> https://repository.apache.org/content/repositories/orgapachespark-1137/org/apache/spark/spark-streaming-flume-assembly_2.10/1.5.0-rc1/ >> >> Just a publishing glitch but worth a few more eyes on. >> >> On Fri, Aug 21, 2015 at 5:28 PM, Sean Owen <so...@cloudera.com> wrote: >> > Signatures, license, etc. look good. I'm getting some fairly >> > consistent failures using Java 7 + Ubuntu 15 + "-Pyarn -Phive >> > -Phive-thriftserver -Phadoop-2.6" -- does anyone else see these? they >> > are likely just test problems, but worth asking. Stack traces are at >> > the end. >> > >> > There are currently 79 issues targeted for 1.5.0, of which 19 are >> > bugs, of which 1 is a blocker. (1032 have been resolved for 1.5.0.) >> > That's significantly better than at the last release. I presume a lot >> > of what's still targeted is not critical and can now be >> > untargeted/retargeted. >> > >> > It occurs to me that the flurry of planning that took place at the >> > start of the 1.5 QA cycle a few weeks ago was quite helpful, and is >> > the kind of thing that would be even more useful at the start of a >> > release cycle. So would be great to do this for 1.6 in a few weeks. >> > Indeed there are already 267 issues targeted for 1.6.0 -- a decent >> > roadmap already. >> > >> > >> > Test failures: >> > >> > Core >> > >> > - Unpersisting TorrentBroadcast on executors and driver in distributed >> > mode *** FAILED *** >> > java.util.concurrent.TimeoutException: Can't find 2 executors before >> > 10000 milliseconds elapsed >> > at >> org.apache.spark.ui.jobs.JobProgressListener.waitUntilExecutorsUp(JobProgressListener.scala:561) >> > at >> org.apache.spark.broadcast.BroadcastSuite.testUnpersistBroadcast(BroadcastSuite.scala:313) >> > at org.apache.spark.broadcast.BroadcastSuite.org >> $apache$spark$broadcast$BroadcastSuite$$testUnpersistTorrentBroadcast(BroadcastSuite.scala:287) >> > at >> org.apache.spark.broadcast.BroadcastSuite$$anonfun$16.apply$mcV$sp(BroadcastSuite.scala:165) >> > at >> org.apache.spark.broadcast.BroadcastSuite$$anonfun$16.apply(BroadcastSuite.scala:165) >> > at >> org.apache.spark.broadcast.BroadcastSuite$$anonfun$16.apply(BroadcastSuite.scala:165) >> > at >> org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22) >> > at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85) >> > at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104) >> > at org.scalatest.Transformer.apply(Transformer.scala:22) >> > ... >> > >> > Streaming >> > >> > - stop slow receiver gracefully *** FAILED *** >> > 0 was not greater than 0 (StreamingContextSuite.scala:324) >> > >> > Kafka >> > >> > - offset recovery *** FAILED *** >> > The code passed to eventually never returned normally. Attempted 191 >> > times over 10.043196973 seconds. Last failure message: >> > strings.forall({ >> > ((elem: Any) => DirectKafkaStreamSuite.collectedData.contains(elem)) >> > }) was false. (DirectKafkaStreamSuite.scala:249) >> > >> > On Fri, Aug 21, 2015 at 5:37 AM, Reynold Xin <r...@databricks.com> >> wrote: >> >> Please vote on releasing the following candidate as Apache Spark >> version >> >> 1.5.0! >> >> >> >> The vote is open until Monday, Aug 17, 2015 at 20:00 UTC and passes if >> a >> >> majority of at least 3 +1 PMC votes are cast. >> >> >> >> [ ] +1 Release this package as Apache Spark 1.5.0 >> >> [ ] -1 Do not release this package because ... >> >> >> >> To learn more about Apache Spark, please see http://spark.apache.org/ >> >> >> >> >> >> The tag to be voted on is v1.5.0-rc1: >> >> >> https://github.com/apache/spark/tree/4c56ad772637615cc1f4f88d619fac6c372c8552 >> >> >> >> The release files, including signatures, digests, etc. can be found at: >> >> http://people.apache.org/~pwendell/spark-releases/spark-1.5.0-rc1-bin/ >> >> >> >> Release artifacts are signed with the following key: >> >> https://people.apache.org/keys/committer/pwendell.asc >> >> >> >> The staging repository for this release can be found at: >> >> >> https://repository.apache.org/content/repositories/orgapachespark-1137/ >> >> >> >> The documentation corresponding to this release can be found at: >> >> >> http://people.apache.org/~pwendell/spark-releases/spark-1.5.0-rc1-docs/ >> >> >> >> >> >> ======================================= >> >> == How can I help test this release? == >> >> ======================================= >> >> If you are a Spark user, you can help us test this release by taking an >> >> existing Spark workload and running on this release candidate, then >> >> reporting any regressions. >> >> >> >> >> >> ================================================ >> >> == What justifies a -1 vote for this release? == >> >> ================================================ >> >> This vote is happening towards the end of the 1.5 QA period, so -1 >> votes >> >> should only occur for significant regressions from 1.4. Bugs already >> present >> >> in 1.4, minor regressions, or bugs related to new features will not >> block >> >> this release. >> >> >> >> >> >> =============================================================== >> >> == What should happen to JIRA tickets still targeting 1.5.0? == >> >> =============================================================== >> >> 1. It is OK for documentation patches to target 1.5.0 and still go into >> >> branch-1.5, since documentations will be packaged separately from the >> >> release. >> >> 2. New features for non-alpha-modules should target 1.6+. >> >> 3. Non-blocker bug fixes should target 1.5.1 or 1.6.0, or drop the >> target >> >> version. >> >> >> >> >> >> ================================================== >> >> == Major changes to help you focus your testing == >> >> ================================================== >> >> As of today, Spark 1.5 contains more than 1000 commits from 220+ >> >> contributors. I've curated a list of important changes for 1.5. For the >> >> complete list, please refer to Apache JIRA changelog. >> >> >> >> RDD/DataFrame/SQL APIs >> >> >> >> - New UDAF interface >> >> - DataFrame hints for broadcast join >> >> - expr function for turning a SQL expression into DataFrame column >> >> - Improved support for NaN values >> >> - StructType now supports ordering >> >> - TimestampType precision is reduced to 1us >> >> - 100 new built-in expressions, including date/time, string, math >> >> - memory and local disk only checkpointing >> >> >> >> DataFrame/SQL Backend Execution >> >> >> >> - Code generation on by default >> >> - Improved join, aggregation, shuffle, sorting with cache friendly >> >> algorithms and external algorithms >> >> - Improved window function performance >> >> - Better metrics instrumentation and reporting for DF/SQL execution >> plans >> >> >> >> Data Sources, Hive, Hadoop, Mesos and Cluster Management >> >> >> >> - Dynamic allocation support in all resource managers (Mesos, YARN, >> >> Standalone) >> >> - Improved Mesos support (framework authentication, roles, dynamic >> >> allocation, constraints) >> >> - Improved YARN support (dynamic allocation with preferred locations) >> >> - Improved Hive support (metastore partition pruning, metastore >> connectivity >> >> to 0.13 to 1.2, internal Hive upgrade to 1.2) >> >> - Support persisting data in Hive compatible format in metastore >> >> - Support data partitioning for JSON data sources >> >> - Parquet improvements (upgrade to 1.7, predicate pushdown, faster >> metadata >> >> discovery and schema merging, support reading non-standard legacy >> Parquet >> >> files generated by other libraries) >> >> - Faster and more robust dynamic partition insert >> >> - DataSourceRegister interface for external data sources to specify >> short >> >> names >> >> >> >> SparkR >> >> >> >> - YARN cluster mode in R >> >> - GLMs with R formula, binomial/Gaussian families, and elastic-net >> >> regularization >> >> - Improved error messages >> >> - Aliases to make DataFrame functions more R-like >> >> >> >> Streaming >> >> >> >> - Backpressure for handling bursty input streams. >> >> - Improved Python support for streaming sources (Kafka offsets, >> Kinesis, >> >> MQTT, Flume) >> >> - Improved Python streaming machine learning algorithms (K-Means, >> linear >> >> regression, logistic regression) >> >> - Native reliable Kinesis stream support >> >> - Input metadata like Kafka offsets made visible in the batch details >> UI >> >> - Better load balancing and scheduling of receivers across cluster >> >> - Include streaming storage in web UI >> >> >> >> Machine Learning and Advanced Analytics >> >> >> >> - Feature transformers: CountVectorizer, Discrete Cosine >> transformation, >> >> MinMaxScaler, NGram, PCA, RFormula, StopWordsRemover, and VectorSlicer. >> >> - Estimators under pipeline APIs: naive Bayes, k-means, and isotonic >> >> regression. >> >> - Algorithms: multilayer perceptron classifier, PrefixSpan for >> sequential >> >> pattern mining, association rule generation, 1-sample >> Kolmogorov-Smirnov >> >> test. >> >> - Improvements to existing algorithms: LDA, trees/ensembles, GMMs >> >> - More efficient Pregel API implementation for GraphX >> >> - Model summary for linear and logistic regression. >> >> - Python API: distributed matrices, streaming k-means and linear >> models, >> >> LDA, power iteration clustering, etc. >> >> - Tuning and evaluation: train-validation split and multiclass >> >> classification evaluator. >> >> - Documentation: document the release version of public API methods >> >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org >> For additional commands, e-mail: dev-h...@spark.apache.org >> >> >
