Hi Svyatoslav, Contribution is not must, but it is very nice to have :)
Before we can start with Scrapy porting we need 6-7 twisted modules ported (I tried to outline it in a comment to http://twistedmatrix.com/trac/ticket/5965<http://www.google.com/url?q=http%3A%2F%2Ftwistedmatrix.com%2Ftrac%2Fticket%2F5965&sa=D&sntz=1&usg=AFQjCNEFA7ah9ug8g9CYicAt8qGgGHEHfg>). Also, https://github.com/mitmproxy/mitmproxy which we use for tests is not ported (and its https://github.com/mitmproxy/netlib and https://github.com/mitmproxy/pathod dependencies), but I think it is not critical and can wait (unlike the twisted part). I think that porting of some of the smaller twisted modules is a good way to start. Don't expect it to be an easy task, pay attention to details, and make sure you've read and followed their https://twistedmatrix.com/trac/wiki/Plan/Python3. You can also build a plan for the Python 3 porting proposal - identify what is needed to make Scrapy tutorial work, what is needed to make relevant tests run and pass, and on which twisted features Scrapy itself and Scrapy tests in particular depend; based on your findings try to estimate what should be done first and what should be done next, and how much would it take. We don't aim for full Scrapy Python 3 compatibility for this GSoC project; covering tutorial-level features would be a great outcome. Such plan plus a demonstrated ability to port a Python module (and do it well enough to get the patch merged) will form a very-very solid proposal. Feel free to contact me via Hangouts or email if you need some assistance! I'm not a heavy IM user, but I think we can schedule some chats. среда, 19 февраля 2014 г., 3:06:21 UTC+6 пользователь Svyatoslav Sydorenko написал: > > Hi Mikhail, > > Thank you for your feedback. > I'll try setting up environment and will get familiar > with twisted.trial.runner. Don't you mind texting via some IM (say > Hangouts) in person if I meet some difficulties? > P.S. You wrote contribution is a must. Is there any issue which I could > resolve in scope of such requirement? > > Понеділок, 17 лютого 2014 р. 13:37:07 UTC+2 користувач Mikhail Korobov > написав: >> >> Hey Svyatoslav, >> >> That's great you're interested in adding py3k support for Scrapy, and >> that you've started thinking about the GSoC application early! The >> preferred way to discuss the details is to use this mailing list. Feel free >> to ask any questions here. >> >> For Py3k project I suggest anyone to get familiar with Scrapy testing >> setup first. The first goal is to make Scrapy tests run in Python 3.3 via >> tox. One thing that blocks it is that the test runner we use >> (twisted.trial.runner) is not ported yet - see >> http://twistedmatrix.com/trac/ticket/5965. Twisted guys have pretty >> strict policies regarding Python 3 support patches: basically, if a patch >> is not minimal, doesn't follow some rules or is not covered with tests, >> most likely it won't get merged. >> >> It'll be great to see some steps in this direction before GSoC actually >> starts :) I have ported a lot of code to Python 3.x myself, and usually >> this is more about proper testing than about making changes to the actual >> code. I may be asking too much, and this is not necessary, but we'll >> definitely prefer applications from student who have a track record of >> contributing to Scrapy and/or Twisted, because such students are more >> likely to succeed - and we all want GSoC to be successful. >> >> воскресенье, 16 февраля 2014 г., 19:52:59 UTC+6 пользователь Svyatoslav >> Sydorenko написал: >>> >>> Hi, >>> >>> My name is Svyatoslav Sydorenko. I'm postgraduate at NTUU >>> 'KPI'<http://kpi.ua/en>(Kyiv, Ukraine), Bachelor in Computer Science. >>> I'd like to work on mentioned >>> task<https://github.com/scrapy/scrapy/wiki/GSoC-2014-Ideas#wiki-python-3-support>in >>> scope of this year's GSOC. >>> >>> I've started using scrapy about 6 months ago (crawled real estate >>> web-sites and some more complex resources with POST-request only >>> navigation), and being a big fan of Python 3 I'd like to implement py3k >>> support for Scrapy. >>> Should I contact mentor in order to discuss some details? What is a >>> preferable way of communication? >>> >>> >>> Kind regards, >>> Svyatoslav. >>> >> -- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/groups/opt_out.
