Re: auto-generating changes.txt was: migrating private branches to the new git repo
Fixed the subtasks. Hacked on my changes generator a bit more. Source is in my github repo if anyone wants to play with it. Here’s a (merged) 3.x changes.txt file from the current output built off of JIRA. (The unmerged versions are also created, but look pretty much identical.) http://pastebin.com/fEUKSDG4 A couple of things stick out: - Had to fix quite a few ‘assigned’ JIRAs. Still missed some. - Some of these should probably be moved to ‘New Feature’ type rather than ‘Improvement’ - Formatting is still a little off, esp Doug’s requirement to be able to read on his monitor. ;) - Sub-tasks aren’t handled particularly well, getting merged in with the rest. I’m not particularly sure it matters except when a major branch merge happened.
Re: auto-generating changes.txt was: migrating private branches to the new git repo
On Thu, Sep 4, 2014 at 8:37 AM, Allen Wittenauer a...@altiscale.com wrote: I hacked on it some more and yes, it’s very easy to detect: * type of jira (improvement, bug, new feature, wish, task, sub-task) * incompatible or not (regardless of type) * reviewed or not (regardless of type) A key question is what to do about tasks, sub-tasks, and wishes. I haven’t tried yet, but I’m fairly confident that sub-tasks list what the parent is, so those can probably be handled specially. We either need to list tasks and wishes as what they are or, in the script, turn them into something else. Also added some sorting to it based upon number. I’m not sure what order we’re getting from JIRA now. Last updated time is going to be wonky with all the changes I did. I didn’t look to see if there was a ‘resolve date’. It should be noted that the release note script already created different files: a master one and one for each project. That functionality was kept but I just shared the master one because it was easier. ;) https://github.com/aw-altiscale/hadoop/blob/trunk/dev-support/changes.py is the current version if someone wants to look at it. We do need to have a talk about 3.x though. Looking over the list, it would appear that a lot of (what became) early 2.x JIRAs were marked as Fixed against 3.x. Probably 2/3’rd of the JIRAs showing up! I think it may be safe to just say “everything before date X should be set to 2.x”, but I’m not sure of what that date would be. Doing that would chop the 3.x change list down to a much more realistic size, esp when compared to the manual changes file. Would it help to look at the git commits for a release, and fetch the corresponding details from JIRA for those commits that have a JIRA #? On Sep 4, 2014, at 4:38 AM, Steve Loughran ste...@hortonworks.com wrote: is there any way of isolating compatible/incompatible changes, new features? I know that any change is potentially incompatible —but it is still good to highlight the things we know are likely to cause trouble On 4 September 2014 02:51, Allen Wittenauer a...@altiscale.com wrote: Nothing official or clean or whatever, but just to give people an idea of what an auto generated CHANGES.txt file might look like, here are some sample runs of the hacky thing I built, based upon the fixVersion information. It doesn't break it down by improvement, etc. Also, the name on the end is the person who the JIRA is assigned. If it is unassigned, then it comes out blank. It's interesting to note that in the 2.5.1 notes, it does appear to have caught a commit missing from the changes.txt…. 2.5.1: http://pastebin.com/jXfz5wXz 2.6.0: http://pastebin.com/5nkSsU18 3.0.0: http://pastebin.com/3Ek4tP8d One thing I didn't do was append the previous versions onto these files, which is what I'd expect to happen. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: auto-generating changes.txt was: migrating private branches to the new git repo
On Sep 5, 2014, at 9:19 AM, Karthik Kambatla ka...@cloudera.com wrote: On Thu, Sep 4, 2014 at 8:37 AM, Allen Wittenauer a...@altiscale.com wrote: We do need to have a talk about 3.x though. Looking over the list, it would appear that a lot of (what became) early 2.x JIRAs were marked as Fixed against 3.x. Probably 2/3’rd of the JIRAs showing up! I think it may be safe to just say “everything before date X should be set to 2.x”, but I’m not sure of what that date would be. Doing that would chop the 3.x change list down to a much more realistic size, esp when compared to the manual changes file. Would it help to look at the git commits for a release, and fetch the corresponding details from JIRA for those commits that have a JIRA #? Another thought I had was to see how many of these actually show up in the current changes file and set them appropriately. Then there is stuff like YARN-154…. In either case, it’s likely going to be a manual process going through these. :( *rolls up sleeves*
Re: auto-generating changes.txt was: migrating private branches to the new git repo
is there any way of isolating compatible/incompatible changes, new features? I know that any change is potentially incompatible —but it is still good to highlight the things we know are likely to cause trouble On 4 September 2014 02:51, Allen Wittenauer a...@altiscale.com wrote: Nothing official or clean or whatever, but just to give people an idea of what an auto generated CHANGES.txt file might look like, here are some sample runs of the hacky thing I built, based upon the fixVersion information. It doesn't break it down by improvement, etc. Also, the name on the end is the person who the JIRA is assigned. If it is unassigned, then it comes out blank. It's interesting to note that in the 2.5.1 notes, it does appear to have caught a commit missing from the changes.txt…. 2.5.1: http://pastebin.com/jXfz5wXz 2.6.0: http://pastebin.com/5nkSsU18 3.0.0: http://pastebin.com/3Ek4tP8d One thing I didn't do was append the previous versions onto these files, which is what I'd expect to happen. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: auto-generating changes.txt was: migrating private branches to the new git repo
I hacked on it some more and yes, it’s very easy to detect: * type of jira (improvement, bug, new feature, wish, task, sub-task) * incompatible or not (regardless of type) * reviewed or not (regardless of type) A key question is what to do about tasks, sub-tasks, and wishes. I haven’t tried yet, but I’m fairly confident that sub-tasks list what the parent is, so those can probably be handled specially. We either need to list tasks and wishes as what they are or, in the script, turn them into something else. Also added some sorting to it based upon number. I’m not sure what order we’re getting from JIRA now. Last updated time is going to be wonky with all the changes I did. I didn’t look to see if there was a ‘resolve date’. It should be noted that the release note script already created different files: a master one and one for each project. That functionality was kept but I just shared the master one because it was easier. ;) https://github.com/aw-altiscale/hadoop/blob/trunk/dev-support/changes.py is the current version if someone wants to look at it. We do need to have a talk about 3.x though. Looking over the list, it would appear that a lot of (what became) early 2.x JIRAs were marked as Fixed against 3.x. Probably 2/3’rd of the JIRAs showing up! I think it may be safe to just say “everything before date X should be set to 2.x”, but I’m not sure of what that date would be. Doing that would chop the 3.x change list down to a much more realistic size, esp when compared to the manual changes file. On Sep 4, 2014, at 4:38 AM, Steve Loughran ste...@hortonworks.com wrote: is there any way of isolating compatible/incompatible changes, new features? I know that any change is potentially incompatible —but it is still good to highlight the things we know are likely to cause trouble On 4 September 2014 02:51, Allen Wittenauer a...@altiscale.com wrote: Nothing official or clean or whatever, but just to give people an idea of what an auto generated CHANGES.txt file might look like, here are some sample runs of the hacky thing I built, based upon the fixVersion information. It doesn't break it down by improvement, etc. Also, the name on the end is the person who the JIRA is assigned. If it is unassigned, then it comes out blank. It's interesting to note that in the 2.5.1 notes, it does appear to have caught a commit missing from the changes.txt…. 2.5.1: http://pastebin.com/jXfz5wXz 2.6.0: http://pastebin.com/5nkSsU18 3.0.0: http://pastebin.com/3Ek4tP8d One thing I didn't do was append the previous versions onto these files, which is what I'd expect to happen. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
auto-generating changes.txt was: migrating private branches to the new git repo
Nothing official or clean or whatever, but just to give people an idea of what an auto generated CHANGES.txt file might look like, here are some sample runs of the hacky thing I built, based upon the fixVersion information. It doesn't break it down by improvement, etc. Also, the name on the end is the person who the JIRA is assigned. If it is unassigned, then it comes out blank. It's interesting to note that in the 2.5.1 notes, it does appear to have caught a commit missing from the changes.txt…. 2.5.1: http://pastebin.com/jXfz5wXz 2.6.0: http://pastebin.com/5nkSsU18 3.0.0: http://pastebin.com/3Ek4tP8d One thing I didn't do was append the previous versions onto these files, which is what I'd expect to happen.
Re: auto-generating changes.txt was: migrating private branches to the new git repo
2.5.1 - I believe all the commits here are in CHANGES.txt of 2.5.1. Which one is missing? On Wed, Sep 3, 2014 at 6:51 PM, Allen Wittenauer a...@altiscale.com wrote: Nothing official or clean or whatever, but just to give people an idea of what an auto generated CHANGES.txt file might look like, here are some sample runs of the hacky thing I built, based upon the fixVersion information. It doesn't break it down by improvement, etc. Also, the name on the end is the person who the JIRA is assigned. If it is unassigned, then it comes out blank. It's interesting to note that in the 2.5.1 notes, it does appear to have caught a commit missing from the changes.txt…. 2.5.1: http://pastebin.com/jXfz5wXz 2.6.0: http://pastebin.com/5nkSsU18 3.0.0: http://pastebin.com/3Ek4tP8d One thing I didn't do was append the previous versions onto these files, which is what I'd expect to happen.
Re: auto-generating changes.txt was: migrating private branches to the new git repo
I don't see HADOOP-10957 in hadoop-common-project/hadoop-cmmon/CHANGES.txt on github in the 2.5.1 branch. On Sep 3, 2014, at 7:00 PM, Karthik Kambatla ka...@cloudera.com wrote: 2.5.1 - I believe all the commits here are in CHANGES.txt of 2.5.1. Which one is missing? On Wed, Sep 3, 2014 at 6:51 PM, Allen Wittenauer a...@altiscale.com wrote: Nothing official or clean or whatever, but just to give people an idea of what an auto generated CHANGES.txt file might look like, here are some sample runs of the hacky thing I built, based upon the fixVersion information. It doesn't break it down by improvement, etc. Also, the name on the end is the person who the JIRA is assigned. If it is unassigned, then it comes out blank. It's interesting to note that in the 2.5.1 notes, it does appear to have caught a commit missing from the changes.txt…. 2.5.1: http://pastebin.com/jXfz5wXz 2.6.0: http://pastebin.com/5nkSsU18 3.0.0: http://pastebin.com/3Ek4tP8d One thing I didn't do was append the previous versions onto these files, which is what I'd expect to happen.
Re: auto-generating changes.txt was: migrating private branches to the new git repo
Oh, it's in hdfs. Sneaky. On Sep 3, 2014, at 7:10 PM, Allen Wittenauer a...@altiscale.com wrote: I don't see HADOOP-10957 in hadoop-common-project/hadoop-cmmon/CHANGES.txt on github in the 2.5.1 branch. On Sep 3, 2014, at 7:00 PM, Karthik Kambatla ka...@cloudera.com wrote: 2.5.1 - I believe all the commits here are in CHANGES.txt of 2.5.1. Which one is missing? On Wed, Sep 3, 2014 at 6:51 PM, Allen Wittenauer a...@altiscale.com wrote: Nothing official or clean or whatever, but just to give people an idea of what an auto generated CHANGES.txt file might look like, here are some sample runs of the hacky thing I built, based upon the fixVersion information. It doesn't break it down by improvement, etc. Also, the name on the end is the person who the JIRA is assigned. If it is unassigned, then it comes out blank. It's interesting to note that in the 2.5.1 notes, it does appear to have caught a commit missing from the changes.txt…. 2.5.1: http://pastebin.com/jXfz5wXz 2.6.0: http://pastebin.com/5nkSsU18 3.0.0: http://pastebin.com/3Ek4tP8d One thing I didn't do was append the previous versions onto these files, which is what I'd expect to happen.
