Re: Kicking off the process around Spark 2.2.1
Ok I think we are there, after getting rounds of fixes through in the last few weeks. I'm going to kick off a separate thread on this to be absolutely clear. From: holden.ka...@gmail.com on behalf of Holden Karau Sent: Thursday, November 2, 2017 12:47:13 PM To: Reynold Xin Cc: Felix Cheung; Sean Owen; dev@spark.apache.org Subject: Re: Kicking off the process around Spark 2.2.1 I agree, except in this case we probably want some of the fixes that are going into the maintenance release to be present in the new feature release (like the CRAN issue). On Thu, Nov 2, 2017 at 12:12 PM, Reynold Xin mailto:r...@databricks.com>> wrote: Why tie a maintenance release to a feature release? They are supposed to be independent and we should be able to make a lot of maintenance releases as needed. On Thu, Nov 2, 2017 at 7:13 PM Sean Owen mailto:so...@cloudera.com>> wrote: The feature freeze is "mid November" : http://spark.apache.org/versioning-policy.html Let's say... Nov 15? any body have a better date? Although it'd be nice to get 2.2.1 out sooner than later in all events, and kind of makes sense to get out first, they need not go in order. It just might be distracting to deal with 2 at once. (BTW there was still one outstanding issue from the last release: https://issues.apache.org/jira/browse/SPARK-22401 ) On Thu, Nov 2, 2017 at 6:06 PM Felix Cheung mailto:felixcheun...@hotmail.com>> wrote: I think it will be great to set a feature freeze date for 2.3.0 first, as a minor release. There are a few new stuff that would be good to have and then we will likely need time to stabilize, before cutting RCs. -- Twitter: https://twitter.com/holdenkarau
Re: Kicking off the process around Spark 2.2.1
I agree, except in this case we probably want some of the fixes that are going into the maintenance release to be present in the new feature release (like the CRAN issue). On Thu, Nov 2, 2017 at 12:12 PM, Reynold Xin wrote: > Why tie a maintenance release to a feature release? They are supposed to > be independent and we should be able to make a lot of maintenance releases > as needed. > > On Thu, Nov 2, 2017 at 7:13 PM Sean Owen wrote: > >> The feature freeze is "mid November" : http://spark.apache.org/ >> versioning-policy.html >> Let's say... Nov 15? any body have a better date? >> >> Although it'd be nice to get 2.2.1 out sooner than later in all events, >> and kind of makes sense to get out first, they need not go in order. It >> just might be distracting to deal with 2 at once. >> >> (BTW there was still one outstanding issue from the last release: >> https://issues.apache.org/jira/browse/SPARK-22401 ) >> >> On Thu, Nov 2, 2017 at 6:06 PM Felix Cheung >> wrote: >> >>> I think it will be great to set a feature freeze date for 2.3.0 first, >>> as a minor release. There are a few new stuff that would be good to have >>> and then we will likely need time to stabilize, before cutting RCs. >>> >>> -- Twitter: https://twitter.com/holdenkarau
Re: Kicking off the process around Spark 2.2.1
Why tie a maintenance release to a feature release? They are supposed to be independent and we should be able to make a lot of maintenance releases as needed. On Thu, Nov 2, 2017 at 7:13 PM Sean Owen wrote: > The feature freeze is "mid November" : > http://spark.apache.org/versioning-policy.html > Let's say... Nov 15? any body have a better date? > > Although it'd be nice to get 2.2.1 out sooner than later in all events, > and kind of makes sense to get out first, they need not go in order. It > just might be distracting to deal with 2 at once. > > (BTW there was still one outstanding issue from the last release: > https://issues.apache.org/jira/browse/SPARK-22401 ) > > On Thu, Nov 2, 2017 at 6:06 PM Felix Cheung > wrote: > >> I think it will be great to set a feature freeze date for 2.3.0 first, as >> a minor release. There are a few new stuff that would be good to have and >> then we will likely need time to stabilize, before cutting RCs. >> >>
Re: Kicking off the process around Spark 2.2.1
The feature freeze is "mid November" : http://spark.apache.org/versioning-policy.html Let's say... Nov 15? any body have a better date? Although it'd be nice to get 2.2.1 out sooner than later in all events, and kind of makes sense to get out first, they need not go in order. It just might be distracting to deal with 2 at once. (BTW there was still one outstanding issue from the last release: https://issues.apache.org/jira/browse/SPARK-22401 ) On Thu, Nov 2, 2017 at 6:06 PM Felix Cheung wrote: > I think it will be great to set a feature freeze date for 2.3.0 first, as > a minor release. There are a few new stuff that would be good to have and > then we will likely need time to stabilize, before cutting RCs. > >
Re: Kicking off the process around Spark 2.2.1
I’m fine with picking a feature freeze, although then we should branch close to that point. Is there interest in still seeing 2.3 try and go out around the nominal schedule? Personally, from a release stand point, I’d rather see 2.2.1 go out first so we don’t end up with 2.3 potentially going out with missing fixes from 2.2.1 (especially with the CRAN stuff - it would be unfortunate to have 2.2.1 available in CRAN but not be able to provide 2.3 in the same manner). On Thu, Nov 2, 2017 at 11:06 AM Felix Cheung wrote: > I think it will be great to set a feature freeze date for 2.3.0 first, as > a minor release. There are a few new stuff that would be good to have and > then we will likely need time to stabilize, before cutting RCs. > > -- > *From:* Holden Karau > *Sent:* Thursday, November 2, 2017 10:38:48 AM > *To:* Felix Cheung; Kevin Grealish; Sean Owen > > *Cc:* dev@spark.apache.org > *Subject:* Re: Kicking off the process around Spark 2.2.1 > If it’s desired I’d be happy to start on 2.3 once 2.2.1 is finished. > > On Thu, Nov 2, 2017 at 10:24 AM Felix Cheung > wrote: > >> For the 2.2.1, we are still working through a few bugs. Hopefully it >> won't be long. >> >> >> -- >> *From:* Kevin Grealish >> *Sent:* Thursday, November 2, 2017 9:51:56 AM >> *To:* Felix Cheung; Sean Owen; Holden Karau >> *Cc:* dev@spark.apache.org >> *Subject:* RE: Kicking off the process around Spark 2.2.1 >> >> >> Any update on expected 2.2.1 (or 2.3.0) release process? >> >> >> >> *From:* Felix Cheung [mailto:felixcheun...@hotmail.com] >> *Sent:* Thursday, October 26, 2017 10:04 AM >> *To:* Sean Owen ; Holden Karau >> *Cc:* dev@spark.apache.org >> *Subject:* Re: Kicking off the process around Spark 2.2.1 >> >> >> >> Yes! I can take on RM for 2.2.1. >> >> >> >> We are still working out what to do with temp files created by Hive and >> Java that cause the policy issue with CRAN and will report back shortly, >> hopefully. >> >> >> -- >> >> *From:* Sean Owen >> *Sent:* Wednesday, October 25, 2017 4:39:15 AM >> *To:* Holden Karau >> *Cc:* Felix Cheung; dev@spark.apache.org >> *Subject:* Re: Kicking off the process around Spark 2.2.1 >> >> >> >> It would be reasonably consistent with the timing of other x.y.1 >> releases, and more release managers sounds useful, yeah. >> >> >> >> Note also that in theory the code freeze for 2.3.0 starts in about 2 >> weeks. >> >> >> >> On Wed, Oct 25, 2017 at 12:29 PM Holden Karau >> wrote: >> >> Now that Spark 2.1.2 is out it seems like now is a good time to get >> started on the Spark 2.2.1 release. There are some streaming fixes I’m >> aware of that would be good to get into a release, is there anything else >> people are working on for 2.2.1 we should be tracking? >> >> >> >> To switch it up I’d like to suggest Felix to be the RM for this since >> there are also likely some R packaging changes to be included in the >> release. This also gives us a chance to see if my updated release >> documentation if enough for a new RM to get started from. >> >> >> >> What do folks think? >> >> -- >> >> Twitter: https://twitter.com/holdenkarau >> <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftwitter.com%2Fholdenkarau&data=02%7C01%7Ckevingre%40microsoft.com%7C4112dae09f8c48f6b1f908d51c939572%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636446342556242224&sdata=sy9vsBZmlSWj0OYa2c9aNNlWeS0BJNUMMboSINtIQCY%3D&reserved=0> >> >> -- > Twitter: https://twitter.com/holdenkarau > -- Twitter: https://twitter.com/holdenkarau
Re: Kicking off the process around Spark 2.2.1
I think it will be great to set a feature freeze date for 2.3.0 first, as a minor release. There are a few new stuff that would be good to have and then we will likely need time to stabilize, before cutting RCs. From: Holden Karau Sent: Thursday, November 2, 2017 10:38:48 AM To: Felix Cheung; Kevin Grealish; Sean Owen Cc: dev@spark.apache.org Subject: Re: Kicking off the process around Spark 2.2.1 If it’s desired I’d be happy to start on 2.3 once 2.2.1 is finished. On Thu, Nov 2, 2017 at 10:24 AM Felix Cheung mailto:felixcheun...@hotmail.com>> wrote: For the 2.2.1, we are still working through a few bugs. Hopefully it won't be long. From: Kevin Grealish mailto:kevin...@microsoft.com>> Sent: Thursday, November 2, 2017 9:51:56 AM To: Felix Cheung; Sean Owen; Holden Karau Cc: dev@spark.apache.org<mailto:dev@spark.apache.org> Subject: RE: Kicking off the process around Spark 2.2.1 Any update on expected 2.2.1 (or 2.3.0) release process? From: Felix Cheung [mailto:felixcheun...@hotmail.com<mailto:felixcheun...@hotmail.com>] Sent: Thursday, October 26, 2017 10:04 AM To: Sean Owen mailto:so...@cloudera.com>>; Holden Karau mailto:hol...@pigscanfly.ca>> Cc: dev@spark.apache.org<mailto:dev@spark.apache.org> Subject: Re: Kicking off the process around Spark 2.2.1 Yes! I can take on RM for 2.2.1. We are still working out what to do with temp files created by Hive and Java that cause the policy issue with CRAN and will report back shortly, hopefully. From: Sean Owen mailto:so...@cloudera.com>> Sent: Wednesday, October 25, 2017 4:39:15 AM To: Holden Karau Cc: Felix Cheung; dev@spark.apache.org<mailto:dev@spark.apache.org> Subject: Re: Kicking off the process around Spark 2.2.1 It would be reasonably consistent with the timing of other x.y.1 releases, and more release managers sounds useful, yeah. Note also that in theory the code freeze for 2.3.0 starts in about 2 weeks. On Wed, Oct 25, 2017 at 12:29 PM Holden Karau mailto:hol...@pigscanfly.ca>> wrote: Now that Spark 2.1.2 is out it seems like now is a good time to get started on the Spark 2.2.1 release. There are some streaming fixes I’m aware of that would be good to get into a release, is there anything else people are working on for 2.2.1 we should be tracking? To switch it up I’d like to suggest Felix to be the RM for this since there are also likely some R packaging changes to be included in the release. This also gives us a chance to see if my updated release documentation if enough for a new RM to get started from. What do folks think? -- Twitter: https://twitter.com/holdenkarau<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftwitter.com%2Fholdenkarau&data=02%7C01%7Ckevingre%40microsoft.com%7C4112dae09f8c48f6b1f908d51c939572%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636446342556242224&sdata=sy9vsBZmlSWj0OYa2c9aNNlWeS0BJNUMMboSINtIQCY%3D&reserved=0> -- Twitter: https://twitter.com/holdenkarau
Re: Kicking off the process around Spark 2.2.1
If it’s desired I’d be happy to start on 2.3 once 2.2.1 is finished. On Thu, Nov 2, 2017 at 10:24 AM Felix Cheung wrote: > For the 2.2.1, we are still working through a few bugs. Hopefully it won't > be long. > > > -- > *From:* Kevin Grealish > *Sent:* Thursday, November 2, 2017 9:51:56 AM > *To:* Felix Cheung; Sean Owen; Holden Karau > *Cc:* dev@spark.apache.org > *Subject:* RE: Kicking off the process around Spark 2.2.1 > > > Any update on expected 2.2.1 (or 2.3.0) release process? > > > > *From:* Felix Cheung [mailto:felixcheun...@hotmail.com] > *Sent:* Thursday, October 26, 2017 10:04 AM > *To:* Sean Owen ; Holden Karau > *Cc:* dev@spark.apache.org > *Subject:* Re: Kicking off the process around Spark 2.2.1 > > > > Yes! I can take on RM for 2.2.1. > > > > We are still working out what to do with temp files created by Hive and > Java that cause the policy issue with CRAN and will report back shortly, > hopefully. > > > -- > > *From:* Sean Owen > *Sent:* Wednesday, October 25, 2017 4:39:15 AM > *To:* Holden Karau > *Cc:* Felix Cheung; dev@spark.apache.org > *Subject:* Re: Kicking off the process around Spark 2.2.1 > > > > It would be reasonably consistent with the timing of other x.y.1 releases, > and more release managers sounds useful, yeah. > > > > Note also that in theory the code freeze for 2.3.0 starts in about 2 weeks. > > > > On Wed, Oct 25, 2017 at 12:29 PM Holden Karau > wrote: > > Now that Spark 2.1.2 is out it seems like now is a good time to get > started on the Spark 2.2.1 release. There are some streaming fixes I’m > aware of that would be good to get into a release, is there anything else > people are working on for 2.2.1 we should be tracking? > > > > To switch it up I’d like to suggest Felix to be the RM for this since > there are also likely some R packaging changes to be included in the > release. This also gives us a chance to see if my updated release > documentation if enough for a new RM to get started from. > > > > What do folks think? > > -- > > Twitter: https://twitter.com/holdenkarau > <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftwitter.com%2Fholdenkarau&data=02%7C01%7Ckevingre%40microsoft.com%7C4112dae09f8c48f6b1f908d51c939572%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636446342556242224&sdata=sy9vsBZmlSWj0OYa2c9aNNlWeS0BJNUMMboSINtIQCY%3D&reserved=0> > > -- Twitter: https://twitter.com/holdenkarau
Re: Kicking off the process around Spark 2.2.1
For the 2.2.1, we are still working through a few bugs. Hopefully it won't be long. From: Kevin Grealish Sent: Thursday, November 2, 2017 9:51:56 AM To: Felix Cheung; Sean Owen; Holden Karau Cc: dev@spark.apache.org Subject: RE: Kicking off the process around Spark 2.2.1 Any update on expected 2.2.1 (or 2.3.0) release process? From: Felix Cheung [mailto:felixcheun...@hotmail.com] Sent: Thursday, October 26, 2017 10:04 AM To: Sean Owen ; Holden Karau Cc: dev@spark.apache.org Subject: Re: Kicking off the process around Spark 2.2.1 Yes! I can take on RM for 2.2.1. We are still working out what to do with temp files created by Hive and Java that cause the policy issue with CRAN and will report back shortly, hopefully. From: Sean Owen mailto:so...@cloudera.com>> Sent: Wednesday, October 25, 2017 4:39:15 AM To: Holden Karau Cc: Felix Cheung; dev@spark.apache.org<mailto:dev@spark.apache.org> Subject: Re: Kicking off the process around Spark 2.2.1 It would be reasonably consistent with the timing of other x.y.1 releases, and more release managers sounds useful, yeah. Note also that in theory the code freeze for 2.3.0 starts in about 2 weeks. On Wed, Oct 25, 2017 at 12:29 PM Holden Karau mailto:hol...@pigscanfly.ca>> wrote: Now that Spark 2.1.2 is out it seems like now is a good time to get started on the Spark 2.2.1 release. There are some streaming fixes I’m aware of that would be good to get into a release, is there anything else people are working on for 2.2.1 we should be tracking? To switch it up I’d like to suggest Felix to be the RM for this since there are also likely some R packaging changes to be included in the release. This also gives us a chance to see if my updated release documentation if enough for a new RM to get started from. What do folks think? -- Twitter: https://twitter.com/holdenkarau<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftwitter.com%2Fholdenkarau&data=02%7C01%7Ckevingre%40microsoft.com%7C4112dae09f8c48f6b1f908d51c939572%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636446342556242224&sdata=sy9vsBZmlSWj0OYa2c9aNNlWeS0BJNUMMboSINtIQCY%3D&reserved=0>
RE: Kicking off the process around Spark 2.2.1
Any update on expected 2.2.1 (or 2.3.0) release process? From: Felix Cheung [mailto:felixcheun...@hotmail.com] Sent: Thursday, October 26, 2017 10:04 AM To: Sean Owen ; Holden Karau Cc: dev@spark.apache.org Subject: Re: Kicking off the process around Spark 2.2.1 Yes! I can take on RM for 2.2.1. We are still working out what to do with temp files created by Hive and Java that cause the policy issue with CRAN and will report back shortly, hopefully. From: Sean Owen mailto:so...@cloudera.com>> Sent: Wednesday, October 25, 2017 4:39:15 AM To: Holden Karau Cc: Felix Cheung; dev@spark.apache.org<mailto:dev@spark.apache.org> Subject: Re: Kicking off the process around Spark 2.2.1 It would be reasonably consistent with the timing of other x.y.1 releases, and more release managers sounds useful, yeah. Note also that in theory the code freeze for 2.3.0 starts in about 2 weeks. On Wed, Oct 25, 2017 at 12:29 PM Holden Karau mailto:hol...@pigscanfly.ca>> wrote: Now that Spark 2.1.2 is out it seems like now is a good time to get started on the Spark 2.2.1 release. There are some streaming fixes I'm aware of that would be good to get into a release, is there anything else people are working on for 2.2.1 we should be tracking? To switch it up I'd like to suggest Felix to be the RM for this since there are also likely some R packaging changes to be included in the release. This also gives us a chance to see if my updated release documentation if enough for a new RM to get started from. What do folks think? -- Twitter: https://twitter.com/holdenkarau<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftwitter.com%2Fholdenkarau&data=02%7C01%7Ckevingre%40microsoft.com%7C4112dae09f8c48f6b1f908d51c939572%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636446342556242224&sdata=sy9vsBZmlSWj0OYa2c9aNNlWeS0BJNUMMboSINtIQCY%3D&reserved=0>
Re: Kicking off the process around Spark 2.2.1
Yes! I can take on RM for 2.2.1. We are still working out what to do with temp files created by Hive and Java that cause the policy issue with CRAN and will report back shortly, hopefully. From: Sean Owen Sent: Wednesday, October 25, 2017 4:39:15 AM To: Holden Karau Cc: Felix Cheung; dev@spark.apache.org Subject: Re: Kicking off the process around Spark 2.2.1 It would be reasonably consistent with the timing of other x.y.1 releases, and more release managers sounds useful, yeah. Note also that in theory the code freeze for 2.3.0 starts in about 2 weeks. On Wed, Oct 25, 2017 at 12:29 PM Holden Karau mailto:hol...@pigscanfly.ca>> wrote: Now that Spark 2.1.2 is out it seems like now is a good time to get started on the Spark 2.2.1 release. There are some streaming fixes I’m aware of that would be good to get into a release, is there anything else people are working on for 2.2.1 we should be tracking? To switch it up I’d like to suggest Felix to be the RM for this since there are also likely some R packaging changes to be included in the release. This also gives us a chance to see if my updated release documentation if enough for a new RM to get started from. What do folks think? -- Twitter: https://twitter.com/holdenkarau
Re: Kicking off the process around Spark 2.2.1
It would be reasonably consistent with the timing of other x.y.1 releases, and more release managers sounds useful, yeah. Note also that in theory the code freeze for 2.3.0 starts in about 2 weeks. On Wed, Oct 25, 2017 at 12:29 PM Holden Karau wrote: > Now that Spark 2.1.2 is out it seems like now is a good time to get > started on the Spark 2.2.1 release. There are some streaming fixes I’m > aware of that would be good to get into a release, is there anything else > people are working on for 2.2.1 we should be tracking? > > To switch it up I’d like to suggest Felix to be the RM for this since > there are also likely some R packaging changes to be included in the > release. This also gives us a chance to see if my updated release > documentation if enough for a new RM to get started from. > > What do folks think? > -- > Twitter: https://twitter.com/holdenkarau >
