Thank you, Shane! :D Bests, Dongjoon
On Tue, Feb 4, 2020 at 13:28 shane knapp ☠ <skn...@berkeley.edu> wrote: > all the 3.0 builds have been created and are currently churning away! > > (the failed builds were to a silly bug in the build scripts sneaking it's > way back in, but that's resolved now) > > shane > > On Sat, Feb 1, 2020 at 6:16 PM Reynold Xin <r...@databricks.com> wrote: > >> Note that branch-3.0 was cut. Please focus on testing, polish, and let's >> get the release out! >> >> >> On Wed, Jan 29, 2020 at 3:41 PM, Reynold Xin <r...@databricks.com> wrote: >> >>> Just a reminder - code freeze is coming this Fri! >>> >>> There can always be exceptions, but those should be exceptions and >>> discussed on a case by case basis rather than becoming the norm. >>> >>> >>> >>> On Tue, Dec 24, 2019 at 4:55 PM, Jungtaek Lim < >>> kabhwan.opensou...@gmail.com> wrote: >>> >>>> Jan 31 sounds good to me. >>>> >>>> Just curious, do we allow some exception on code freeze? One thing came >>>> into my mind is that some feature could have multiple subtasks and part of >>>> subtasks have been merged and other subtask(s) are in reviewing. In this >>>> case do we allow these subtasks to have more days to get reviewed and >>>> merged later? >>>> >>>> Happy Holiday! >>>> >>>> Thanks, >>>> Jungtaek Lim (HeartSaVioR) >>>> >>>> On Wed, Dec 25, 2019 at 8:36 AM Takeshi Yamamuro <linguin....@gmail.com> >>>> wrote: >>>> >>>>> Looks nice, happy holiday, all! >>>>> >>>>> Bests, >>>>> Takeshi >>>>> >>>>> On Wed, Dec 25, 2019 at 3:56 AM Dongjoon Hyun <dongjoon.h...@gmail.com> >>>>> wrote: >>>>> >>>>>> +1 for January 31st. >>>>>> >>>>>> Bests, >>>>>> Dongjoon. >>>>>> >>>>>> On Tue, Dec 24, 2019 at 7:11 AM Xiao Li <lix...@databricks.com> >>>>>> wrote: >>>>>> >>>>>>> Jan 31 is pretty reasonable. Happy Holidays! >>>>>>> >>>>>>> Xiao >>>>>>> >>>>>>> On Tue, Dec 24, 2019 at 5:52 AM Sean Owen <sro...@gmail.com> wrote: >>>>>>> >>>>>>>> Yep, always happens. Is earlier realistic, like Jan 15? it's all >>>>>>>> arbitrary but indeed this has been in progress for a while, and >>>>>>>> there's a >>>>>>>> downside to not releasing it, to making the gap to 3.0 larger. >>>>>>>> On my end I don't know of anything that's holding up a release; is >>>>>>>> it basically DSv2? >>>>>>>> >>>>>>>> BTW these are the items still targeted to 3.0.0, some of which may >>>>>>>> not have been legitimately tagged. It may be worth reviewing what's >>>>>>>> still >>>>>>>> open and necessary, and what should be untargeted. >>>>>>>> >>>>>>>> SPARK-29768 nondeterministic expression fails column pruning >>>>>>>> SPARK-29345 Add an API that allows a user to define and observe >>>>>>>> arbitrary metrics on streaming queries >>>>>>>> SPARK-29348 Add observable metrics >>>>>>>> SPARK-29429 Support Prometheus monitoring natively >>>>>>>> SPARK-29577 Implement p-value simulation and unit tests for chi2 >>>>>>>> test >>>>>>>> SPARK-28900 Test Pyspark, SparkR on JDK 11 with run-tests >>>>>>>> SPARK-28883 Fix a flaky test: ThriftServerQueryTestSuite >>>>>>>> SPARK-28717 Update SQL ALTER TABLE RENAME to use TableCatalog API >>>>>>>> SPARK-28588 Build a SQL reference doc >>>>>>>> SPARK-28629 Capture the missing rules in HiveSessionStateBuilder >>>>>>>> SPARK-28684 Hive module support JDK 11 >>>>>>>> SPARK-28548 explain() shows wrong result for persisted DataFrames >>>>>>>> after some operations >>>>>>>> SPARK-28264 Revisiting Python / pandas UDF >>>>>>>> SPARK-28301 fix the behavior of table name resolution with >>>>>>>> multi-catalog >>>>>>>> SPARK-28155 do not leak SaveMode to file source v2 >>>>>>>> SPARK-28103 Cannot infer filters from union table with empty local >>>>>>>> relation table properly >>>>>>>> SPARK-27986 Support Aggregate Expressions with filter >>>>>>>> SPARK-28024 Incorrect numeric values when out of range >>>>>>>> SPARK-27936 Support local dependency uploading from --py-files >>>>>>>> SPARK-27780 Shuffle server & client should be versioned to enable >>>>>>>> smoother upgrade >>>>>>>> SPARK-27714 Support Join Reorder based on Genetic Algorithm when >>>>>>>> the # of joined tables > 12 >>>>>>>> SPARK-27471 Reorganize public v2 catalog API >>>>>>>> SPARK-27520 Introduce a global config system to replace >>>>>>>> hadoopConfiguration >>>>>>>> SPARK-24625 put all the backward compatible behavior change configs >>>>>>>> under spark.sql.legacy.* >>>>>>>> SPARK-24941 Add RDDBarrier.coalesce() function >>>>>>>> SPARK-25017 Add test suite for ContextBarrierState >>>>>>>> SPARK-25083 remove the type erasure hack in data source scan >>>>>>>> SPARK-25383 Image data source supports sample pushdown >>>>>>>> SPARK-27272 Enable blacklisting of node/executor on fetch failures >>>>>>>> by default >>>>>>>> SPARK-27296 Efficient User Defined Aggregators >>>>>>>> SPARK-25128 multiple simultaneous job submissions against k8s >>>>>>>> backend cause driver pods to hang >>>>>>>> SPARK-26664 Make DecimalType's minimum adjusted scale configurable >>>>>>>> SPARK-21559 Remove Mesos fine-grained mode >>>>>>>> SPARK-24942 Improve cluster resource management with jobs >>>>>>>> containing barrier stage >>>>>>>> SPARK-25914 Separate projection from grouping and aggregate in >>>>>>>> logical Aggregate >>>>>>>> SPARK-20964 Make some keywords reserved along with the ANSI/SQL >>>>>>>> standard >>>>>>>> SPARK-26221 Improve Spark SQL instrumentation and metrics >>>>>>>> SPARK-26425 Add more constraint checks in file streaming source to >>>>>>>> avoid checkpoint corruption >>>>>>>> SPARK-25843 Redesign rangeBetween API >>>>>>>> SPARK-25841 Redesign window function rangeBetween API >>>>>>>> SPARK-25752 Add trait to easily whitelist logical operators that >>>>>>>> produce named output from CleanupAliases >>>>>>>> SPARK-25640 Clarify/Improve EvalType for grouped aggregate and >>>>>>>> window aggregate >>>>>>>> SPARK-25531 new write APIs for data source v2 >>>>>>>> SPARK-25547 Pluggable jdbc connection factory >>>>>>>> SPARK-20845 Support specification of column names in INSERT INTO >>>>>>>> SPARK-24724 Discuss necessary info and access in barrier mode + >>>>>>>> Kubernetes >>>>>>>> SPARK-24725 Discuss necessary info and access in barrier mode + >>>>>>>> Mesos >>>>>>>> SPARK-25074 Implement maxNumConcurrentTasks() in >>>>>>>> MesosFineGrainedSchedulerBackend >>>>>>>> SPARK-23710 Upgrade the built-in Hive to 2.3.5 for hadoop-3.2 >>>>>>>> SPARK-25186 Stabilize Data Source V2 API >>>>>>>> SPARK-25376 Scenarios we should handle but missed in 2.4 for >>>>>>>> barrier execution mode >>>>>>>> SPARK-7768 Make user-defined type (UDT) API public >>>>>>>> SPARK-14922 Alter Table Drop Partition Using Predicate-based >>>>>>>> Partition Spec >>>>>>>> SPARK-15694 Implement ScriptTransformation in sql/core >>>>>>>> SPARK-18134 SQL: MapType in Group BY and Joins not working >>>>>>>> SPARK-19842 Informational Referential Integrity Constraints Support >>>>>>>> in Spark >>>>>>>> SPARK-22231 Support of map, filter, withColumn, dropColumn in >>>>>>>> nested list of structures >>>>>>>> SPARK-22386 Data Source V2 improvements >>>>>>>> SPARK-24723 Discuss necessary info and access in barrier mode + YARN >>>>>>>> >>>>>>>> >>>>>>>> On Mon, Dec 23, 2019 at 5:48 PM Reynold Xin <r...@databricks.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> We've pushed out 3.0 multiple times. The latest release window >>>>>>>>> documented on the website >>>>>>>>> <http://spark.apache.org/versioning-policy.html> says we'd code >>>>>>>>> freeze and cut branch-3.0 early Dec. It looks like we are suffering a >>>>>>>>> bit >>>>>>>>> from the tragedy of the commons, that nobody is pushing for getting >>>>>>>>> the >>>>>>>>> release out. I understand the natural tendency for each individual is >>>>>>>>> to >>>>>>>>> finish or extend the feature/bug that the person has been working on. >>>>>>>>> At >>>>>>>>> some point we need to say "this is it" and get the release out. I'm >>>>>>>>> happy >>>>>>>>> to help drive this process. >>>>>>>>> >>>>>>>>> To be realistic, I don't think we should just code freeze *today*. >>>>>>>>> Although we have updated the website, contributors have all been >>>>>>>>> operating >>>>>>>>> under the assumption that all active developments are still going on. >>>>>>>>> I >>>>>>>>> propose we *cut the branch on **Jan 31**, and code freeze and >>>>>>>>> switch over to bug squashing mode, and try to get the 3.0 official >>>>>>>>> release >>>>>>>>> out in Q1*. That is, by default no new features can go into the >>>>>>>>> branch starting Jan 31. >>>>>>>>> >>>>>>>>> What do you think? >>>>>>>>> >>>>>>>>> And happy holidays everybody. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> [image: Databricks Summit - Watch the talks] >>>>>>> <https://databricks.com/sparkaisummit/north-america> >>>>>>> >>>>>> >>>>> >>>>> -- >>>>> --- >>>>> Takeshi Yamamuro >>>>> >>>> >> > > -- > Shane Knapp > Computer Guy / Voice of Reason > UC Berkeley EECS Research / RISELab Staff Technical Lead > https://rise.cs.berkeley.edu > > --------------------------------------------------------------------- > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
