Hi everybody, I know it is pretty late now, but I just saw this event and really interested in participating. I am currently a graduate student in computer engineering, and have used python for a long time. I have used different open source projects but haven't made concrete contributions. I think this could be the opportunity for me to get involved.
My proposal will be on the topic of "HTTP API for Scrapy spiders". To be honest I haven't used Scrapy at all until recent, but I believe my experience in API should help me solve the problem. APIs I worked with are mostly fixed, they provide detailed functions to obtain each kind of data they have. However, I think in this project, users have to firstly define what kind of data they want, and then use fixed APIs we provide. Or better, if we could build a system that let users generate their own APIs depends on the data. Before I get down to the proposal, I have some questions though: 1. I see one requirement is the reusability of the same spider. Do you have any other explicit requirements? 2. I know Scrapy is built upon Twisted, but these APIs should be a wrapper of a spider, why would the API involved Twisted? I see Twisted as a lower level of tool Scrapy use. 3. When I have a very rough draft of the proposal, is it possible to discuss with the mentor? If so, where should I find him, here, IRC, or other places? Do we discuss details publicly or separately? 4. I understand contribution to the sub-org is a must. I already worked on an issue <https://github.com/scrapy/scrapy/issues/573> I found from Scrapy's repository. However the issuer has not replied yet. Is there any other places I could look to contribute? Lots of questions, sorry for the length. Any guidance is appreciated! Yulong -- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/d/optout.
