[Distutils] GSoC 2017 - Request for Comments on Proposal
Hello Everyone! I had previously sent a mail on this list, stating that I would like to work on pip's dependency resolution for my GSoC 2017. I now have drafted a proposal for the same; with help from my mentors - Donald Stufft and Justin Cappos. I'd also take this opportunity to thank them for agreeing to be my mentors for this GSoC. I would like to request for comments on my proposal for GSoC - it is hosted at https://gist.github.com/pradyunsg/5cf4a35b81f08b6432f280aba6f511eb. Please find trailing a MarkDown version of the proposal. Thanks, Pradyun Gedam - # Adding Proper Dependency Resolution to pip - **Name:** Pradyun S. Gedam - **Email:** [pradyu...@gmail.com][mailto-email] - **Github:** [pradyunsg][github-profile] - **University:** [VIT University, Vellore, India][vit-homepage] - **Course:** Bachelor of Technology in Computer Science and Engineering - **Course Term:** 2016/17 - 2019/20 (4 Year) - **Timezone:** IST (GMT +5:30) - **GSoC Blog RSS Feed URL:** < https://pradyunsg.github.io/gsoc-2017/feed.xml> [github-profile]: http://github.com/pradyunsg/ [vit-homepage]: http://vit.ac.in/ [mailto-email]: mailto:pradyu...@gmail.com ## About Me I was introduced to Python about five years ago, through "Core Python Programming" by Weasley J Chun. After the initial struggle with Python 2/3, the ball was set rolling and hasn't stopped since. I have fiddled around with Game Programming (PyGame), Computer Vision (OpenCV), Data Analytics (Pandas, SciPy, NumPy), transcompilers (Py2C) and more. As a high school student, I did internship at Enthought in 2013 and 2014. The two summers that I spent at Enthought were a great learning experience and I thoroughly enjoyed the environment there. Other than Python, I have also used C, C++, Web Technologies (JavaScript, HTML, CSS) and preprocessors (Pug, TypeScript, LESS/SASS/SCSS), Java and Bash/Zsh for some other projects. Curious to understand how pip works, I began looking into pip's source code. I started contributing to pip in May 2016. I am now fairly familiar with the design of pip and have a fair understanding of how it works, due to the various contributions I have made to pip in the past year. ## Mentors - Donald Stufft (@dstufft on GitHub) - Justin Cappos (@JustinCappos on GitHub) Communication with the mentors will be done mostly on issues and pull requests on pip's GitHub repository. If at any point in time, a real time discussion is to be done with the mentors, it would be done over IRC or Skype. Email can also be used if needed. ## Proposal This project is regarding improving dependency resolution performed within pip by implementing a dependency resolver within it. ### Abstract Currently, pip does not resolve dependencies correctly when there are conflicting requirements. The lack of dependency resolution has caused hard-to-debug bugs/failures due to the installation of incompatible packages. The lack of a dependency resolver is also a blocker for various other features - adding an upgrade-all functionality to pip and properly determining build-time dependencies for packages are two such features. ### Deliverables At the end of this project, pip will have the ability to: - detect requirement conflicts - resolve conflicting dependency requirements where possible ### Implementation The implementation language will be Python. The code will maintain the compatibility requirements of pip - the same source code will support the multiple Python implementations and version, including but not limited to, CPython 2.7, CPython 3.3+, PyPy 2, PyPy3. New Tests for the functionality introduced will be added to pip's current test suite. User documentation would not be a major part of this project. The only changes would be to mention pip can now resolve dependencies properly. There are certain sections that might need updating. Overview The project will be composed of the following stages: 1. Refactor the dependency resolution logic of pip into a separate module. 1. Implement dependency information caching in pip. 1. Implement a backtracking dependency resolver to resolve the dependencies. Every stage depends on the previous ones being completed. This step-wise approach would make incremental improvements so that there is a constant feedback on the work being done as well as making it easy to change course without losing existing work; if needed for some unforeseen reason. Discussion There is a class in pip - `RequirementSet`, which is currently a God class. It is responsible for the following: 1. Holding a list of things to be installed 1. Downloading Files 1. Dependency Resolution 1. Unpacking downloaded files (preparation for installation) 1. Uninstalling packages 1. Installing packages This is clearly a bad situation since this is most of the heavy lifting involved in pip. These responsibilities need to be separated and moved out into their independent modules/classes, to allow for simplification of `Requiremen
Re: [Distutils] GSoC 2017 - Request for Comments on Proposal
On 25 March 2017 at 15:44, Pradyun Gedam wrote: > Hello Everyone! > > I had previously sent a mail on this list, stating that I would like to work > on pip's dependency resolution for my GSoC 2017. I now have drafted a > proposal for the same; with help from my mentors - Donald Stufft and Justin > Cappos. I'd also take this opportunity to thank them for agreeing to be my > mentors for this GSoC. > > I would like to request for comments on my proposal for GSoC - it is hosted > at https://gist.github.com/pradyunsg/5cf4a35b81f08b6432f280aba6f511eb. > > Please find trailing a MarkDown version of the proposal. Hi Pradyun, Your proposal looks pretty impressive - well structured and thought out. I've looked through it and the plans seem reasonable - I've looked more at what you're proposing than at the timescales, but your staged approach seems sensible - if you do hit issues with time, it looks like you'll be able to deliver real improvements even if you don't complete everything, which is fantastic. Best of luck - assuming you complete all the work you've planned, it will be a significant benefit to pip. If I can be of any help, with PR reviews or anything similar, feel free to ping me. Paul ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] GSoC 2017 - Request for Comments on Proposal
Tiny change: I believe its Wesley, not Weasley :) On Sat, Mar 25, 2017, 12:33 PM Paul Moore wrote: > On 25 March 2017 at 15:44, Pradyun Gedam wrote: > > Hello Everyone! > > > > I had previously sent a mail on this list, stating that I would like to > work > > on pip's dependency resolution for my GSoC 2017. I now have drafted a > > proposal for the same; with help from my mentors - Donald Stufft and > Justin > > Cappos. I'd also take this opportunity to thank them for agreeing to be > my > > mentors for this GSoC. > > > > I would like to request for comments on my proposal for GSoC - it is > hosted > > at https://gist.github.com/pradyunsg/5cf4a35b81f08b6432f280aba6f511eb. > > > > Please find trailing a MarkDown version of the proposal. > > Hi Pradyun, > Your proposal looks pretty impressive - well structured and thought > out. I've looked through it and the plans seem reasonable - I've > looked more at what you're proposing than at the timescales, but your > staged approach seems sensible - if you do hit issues with time, it > looks like you'll be able to deliver real improvements even if you > don't complete everything, which is fantastic. > > Best of luck - assuming you complete all the work you've planned, it > will be a significant benefit to pip. If I can be of any help, with PR > reviews or anything similar, feel free to ping me. > > Paul > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] GSoC 2017 - Request for Comments on Proposal
On Sat, Mar 25, 2017, 23:02 Paul Moore wrote: > On 25 March 2017 at 15:44, Pradyun Gedam wrote: > > Hello Everyone! > > > > I had previously sent a mail on this list, stating that I would like to > work > > on pip's dependency resolution for my GSoC 2017. I now have drafted a > > proposal for the same; with help from my mentors - Donald Stufft and > Justin > > Cappos. I'd also take this opportunity to thank them for agreeing to be > my > > mentors for this GSoC. > > > > I would like to request for comments on my proposal for GSoC - it is > hosted > > at https://gist.github.com/pradyunsg/5cf4a35b81f08b6432f280aba6f511eb. > > > > Please find trailing a MarkDown version of the proposal. > > Hi Pradyun, > Your proposal looks pretty impressive - well structured and thought > out. I've looked through it and the plans seem reasonable - I've > looked more at what you're proposing than at the timescales, but your > staged approach seems sensible - if you do hit issues with time, it > looks like you'll be able to deliver real improvements even if you > don't complete everything, which is fantastic. > > Best of luck - assuming you complete all the work you've planned, it > will be a significant benefit to pip. If I can be of any help, with PR > reviews or anything similar, feel free to ping me. > > Paul > Thank You Paul! Hopefully, I'll be able to overcome any issue that come up. I definitely will ping you. :) ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] GSoC 2017 - Request for Comments on Proposal
On Sun, Mar 26, 2017, 08:01 Wayne Werner wrote: > Tiny change: I believe its Wesley, not Weasley :) > Indeed. My bad. I'll fix it. Thank you! > On Sat, Mar 25, 2017, 12:33 PM Paul Moore wrote: > > On 25 March 2017 at 15:44, Pradyun Gedam wrote: > > Hello Everyone! > > > > I had previously sent a mail on this list, stating that I would like to > work > > on pip's dependency resolution for my GSoC 2017. I now have drafted a > > proposal for the same; with help from my mentors - Donald Stufft and > Justin > > Cappos. I'd also take this opportunity to thank them for agreeing to be > my > > mentors for this GSoC. > > > > I would like to request for comments on my proposal for GSoC - it is > hosted > > at https://gist.github.com/pradyunsg/5cf4a35b81f08b6432f280aba6f511eb. > > > > Please find trailing a MarkDown version of the proposal. > > Hi Pradyun, > Your proposal looks pretty impressive - well structured and thought > out. I've looked through it and the plans seem reasonable - I've > looked more at what you're proposing than at the timescales, but your > staged approach seems sensible - if you do hit issues with time, it > looks like you'll be able to deliver real improvements even if you > don't complete everything, which is fantastic. > > Best of luck - assuming you complete all the work you've planned, it > will be a significant benefit to pip. If I can be of any help, with PR > reviews or anything similar, feel free to ping me. > > Paul > > ___ > Distutils-SIG maillist - Distutils-SIG@python.org > https://mail.python.org/mailman/listinfo/distutils-sig > > ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
