So how to deal with legitimate uses that require many requests? Is it better to not serve them at all? On Jan 16, 2016 19:50, "John" <phoenixoverr...@gmail.com> wrote:
> In cases of excessive resource usage we have several options. Contact the > source, throttle them, or flat out disable access depending on what each > case calls for. > > I have seen the dev team to this liberally in the past when needed. If any > one person or group is exploiting us by using unproportionate amounts of > resources thats one thing, if we are just trying to make money by selling > access to what we already have thats another. Limiting abusive sources > shouldnt be an issue, but as soon as we start selling access we loose sight > of our mission. > > On Sat, Jan 16, 2016 at 9:11 PM, Denny Vrandecic <dvrande...@wikimedia.org > > > wrote: > > > I find it rather surprising, but I very much find myself in agreement > with > > most what Andreas Kolbe said on this thread. > > > > To give a bit more thoughts: I am not terribly worried about current > > crawlers. But currently, and more in the future, I expect us to provide > > more complex and this expensive APIs: a SPARQL endpoint, parsing APIs, > etc. > > These will be simply expensive to operate. Not for infrequent users - > say, > > to the benefit of us 70,000 editors - but for use cases that involve tens > > or millions of requests per day. These have the potential of burning a > lot > > of funds to basically support the operations of commercial companies > whose > > mission might or might not be aligned with our. > > > > Is monetizing such use cases really entirely unthinkable? Even under > > restrictions like the ones suggested by Andreas, or other such > restrictions > > we should discuss? > > On Jan 16, 2016 3:49 PM, "Risker" <risker...@gmail.com> wrote: > > > > > Hmm. The majority of those crawlers are from search engines - the very > > > search engines that keep us in the top 10 of their results (and often > in > > > the top 3), thus leading to the usage and donations that we need to > > > survive. If they have to pay, then they might prefer to change their > > > algorithm, or reduce the frequency of scraping (thus also failing to > > catch > > > updates to articles including removal of vandalism in the lead > > paragraphs, > > > which is historically one of the key reasons for frequently crawling > the > > > same articles). Those crawlers are what attracts people to our sites, > to > > > read, to make donations, to possibly edit. Of course there are lesser > > > crawlers, but they're not really big players. > > > > > > I'm at a loss to understand why the Wikimedia Foundation should take on > > the > > > costs and indemnities associated with hiring staff to create a for-pay > > API > > > that would have to meet the expectations of a customer (or more than > one > > > customer) that hasn't even agreed to pay for access. If they want a > > > specialized API (and we've been given no evidence that they do), let > THEM > > > hire the staff, pay them, write the code in an appropriately > open-source > > > way, and donate it to the WMF with the understanding that it could be > > > modified as required, and that it will be accessible to everyone. > > > > > > It is good that the WMF has studied the usage patterns. Could a link > be > > > given to the report, please? It's public, correct? This is exactly > the > > > point of transparency. If only the WMF has the information, then it > > gives > > > an excuse for the community's comments to be ignored "because they > don't > > > know the facts". So let's lay out all the facts on the table, please. > > > > > > Risker/Anne > > > > > > > > > > > > On 16 January 2016 at 15:06, Vituzzu <vituzzu.w...@gmail.com> wrote: > > > > > > > Thank you for sharing this but, above all, to focus on digging real > > data. > > > > > > > > IMHO we shouldn't forget our mission, so licenses must be as free as > > > > possible. Turning into something "more closed" would definitely > deplete > > > one > > > > of the most valuable source (the open source world) of volunteering > we > > > have. > > > > > > > > Crawlers' owner should definitely share our increasing expenses but > any > > > > kind of agreement with them should include ways to improve our > > userbase. > > > > I'm wondering about an agreement with Google (or any other player) to > > add > > > > an "edit" button to knowledge graph. Sort of a "knowledge vs. users" > > > > agreement. > > > > > > > > So, we definitely need a long term strategy which the Foundation will > > > > pursue in *negotiating* with anyone who wants a big scale access to > > *our > > > > resources* (while access to our knowledge will have no limits, as > > usual). > > > > > > > > Vito > > > > > > > > > > > > Il 16/01/2016 19:21, Lila Tretikov ha scritto: > > > > > > > >> To share some context of the discussion the board had around this > -- I > > > >> don't think the minutes give enough detail. APIs -- as we are freely > > and > > > >> rapidly creating them today are important, but are not quite at the > > core > > > >> of > > > >> the issue we are facing. > > > >> > > > >> Today Wikimedia is the largest internet channel for open free > > knowledge > > > in > > > >> the world. But the trends are against us. We have to face them > > together. > > > >> We > > > >> have to have the answers on how. The strategic discussion next week > > will > > > >> help guide us. > > > >> > > > >> Over the last year we looked at the trends in Wikimedia traffic, > > > internet > > > >> as a whole and user behaviors. It took a lot of research. When we > > > started > > > >> the process we have not had solid internal data about unique > visitors > > or > > > >> human vs. crawler usage on the site. For a top 10 website this is a > > big > > > >> issue; it hurts our ability to make smart decisions. We've learned a > > > lot. > > > >> > > > >> We found data that supports Leigh's point -- our permissive license > > > >> supports our core value, we are (I know I am) here for free > knowledge. > > > Yet > > > >> it allows others to use the content in ways that truncates, > simplifies > > > and > > > >> reduces it. More importantly this type of reuse separates our > readers > > > from > > > >> our site, disconnecting readers from our contributors (no edit > > buttons) > > > >> and > > > >> ultimately reduces traffic. Is this a problem? I'd like to hear if > > > people > > > >> on this list see it as such. And how we sustain contributions over > > time. > > > >> > > > >> Meanwhile estimated half of our hosting is used to support crawlers > > that > > > >> scan our content. This has an associated cost in infrastructure, > > power, > > > >> servers, employees to support some well-funded organizations. The > > > content > > > >> is used for a variety of commercial purposes, sometimes having > nothing > > > to > > > >> do with putting our contributor's work in front of more readers. > > Still, > > > we > > > >> can say this is tangentially supportive of our mission. > > > >> > > > >> As these two trends increase without our intervention, our traffic > > > decline > > > >> will accelerate, our ability to grow editors, content and cover > costs > > > will > > > >> decline as well. > > > >> > > > >> The first question on the upcoming consultation next week will be > > > squarely > > > >> on this. Please help us. API conversation is a consequence of this > > > >> challenge. If we were to build more for reuse: APIs are a good way > to > > do > > > >> so. If we are to somehow incentivize users of SIri to come back to > > > >> Wikipedia, what would we need to do? Should we improve our site so > > more > > > >> people come to us directly as the first stop? How do we bring people > > > into > > > >> our world vs. the world of commercial knowledge out there? How do we > > > fund > > > >> this if the people moved to access our content through other > > interfaces > > > (a > > > >> trend that has been accelerating)? > > > >> > > > >> Those are the core questions we need to face. We will have to have > > some > > > >> uncomfortable, honest discussions as we test our hypothesis this > year. > > > The > > > >> conversation next week is a good start to prioritize those. Please > > join > > > >> it. > > > >> > > > >> Lila > > > >> > > > >> > > > >> > > > >> On Sat, Jan 16, 2016 at 6:11 AM, Leigh Thelmadatter < > > > osama...@hotmail.com > > > >> > > > > >> wrote: > > > >> > > > >> If we are concerned about Google taking unfair advantage of > Wikipedia, > > > one > > > >>> simple solution is to allow content donations with a non-commercial > > > >>> restriction. Right now, the concept of "free" include commercial > use. > > > An > > > >>> added bonus to this is that we would get a lot more institutional > > > >>> donations > > > >>> of content if we allowed an non-commercial option. > > > >>> My problem with allowing for paying for "premium access" is that we > > are > > > >>> allowing Google to have a priviledged position. There is no way > > around > > > >>> that. > > > >>> What is the impetus behind this proposal? Its not like we are > lacking > > > >>> money. And limiting growth of the Foundation is not a bad thing... > > at > > > >>> least not to the community. > > > >>> > > > >>> > > > >>> To: wikimedia-l@lists.wikimedia.org > > > >>>> From: ricordisa...@openmailbox.org > > > >>>> Date: Sat, 16 Jan 2016 14:13:06 +0100 > > > >>>> Subject: Re: [Wikimedia-l] Monetizing Wikimedia APIs > > > >>>> > > > >>>> "Imagine a world in which every single human being can freemiumly > > > share > > > >>>> in the sum of all knowledge." XD > > > >>>> > > > >>>> Il 16/01/2016 10:23, Pete Forsyth ha scritto: > > > >>>> > > > >>>>> I'm interested to hear some perspectives on the following line of > > > >>>>> > > > >>>> thinking: > > > >>> > > > >>>> Lisa presented some alternative strategies for revenue needs for > the > > > >>>>> Foundation, including the possibility of charging for premium > > access > > > >>>>> > > > >>>> to the > > > >>> > > > >>>> services and APIs, expanding major donor and foundation > fundraising, > > > >>>>> providing specific services for a fee, or limiting the Wikimedia > > > >>>>> Foundation's growth. The Board emphasized the importance of > keeping > > > >>>>> > > > >>>> free > > > >>> > > > >>>> access to the existing APIs and services, keeping operational > growth > > > in > > > >>>>> line with the organization's effectiveness, providing room for > > > >>>>> > > > >>>> innovation > > > >>> > > > >>>> in the Foundation's activities, and other potential fundraising > > > >>>>> > > > >>>> strategies. > > > >>> > > > >>>> The Board asked Lila to analyze and develop some of these > potential > > > >>>>> strategies for further discussion at a Board meeting in 2016. > > > >>>>> Source: https://wikimediafoundation.org/wiki/Minutes/2015-11-07 > > > >>>>> -Pete[[User:Peteforsyth]] > > > >>>>> _______________________________________________ > > > >>>>> Wikimedia-l mailing list, guidelines at: > > > >>>>> > > > >>>> https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines > > > >>> > > > >>>> New messages to: Wikimedia-l@lists.wikimedia.org > > > >>>>> Unsubscribe: > > > https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, > > > >>>>> > > > >>>> <mailto:wikimedia-l-requ...@lists.wikimedia.org > > ?subject=unsubscribe> > > > >>> > > > >>>> > > > >>>> _______________________________________________ > > > >>>> Wikimedia-l mailing list, guidelines at: > > > >>>> > > > >>> https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines > > > >>> > > > >>>> New messages to: Wikimedia-l@lists.wikimedia.org > > > >>>> Unsubscribe: > > https://lists.wikimedia.org/mailman/listinfo/wikimedia-l > > > , > > > >>>> > > > >>> <mailto:wikimedia-l-requ...@lists.wikimedia.org > ?subject=unsubscribe> > > > >>> > > > >>> _______________________________________________ > > > >>> Wikimedia-l mailing list, guidelines at: > > > >>> https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines > > > >>> New messages to: Wikimedia-l@lists.wikimedia.org > > > >>> Unsubscribe: > > https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, > > > >>> <mailto:wikimedia-l-requ...@lists.wikimedia.org > ?subject=unsubscribe> > > > >>> > > > >>> > > > >> > > > >> > > > > > > > > _______________________________________________ > > > > Wikimedia-l mailing list, guidelines at: > > > > https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines > > > > New messages to: Wikimedia-l@lists.wikimedia.org > > > > Unsubscribe: > https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, > > > > <mailto:wikimedia-l-requ...@lists.wikimedia.org?subject=unsubscribe> > > > > > > > _______________________________________________ > > > Wikimedia-l mailing list, guidelines at: > > > https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines > > > New messages to: Wikimedia-l@lists.wikimedia.org > > > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, > > > <mailto:wikimedia-l-requ...@lists.wikimedia.org?subject=unsubscribe> > > _______________________________________________ > > Wikimedia-l mailing list, guidelines at: > > https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines > > New messages to: Wikimedia-l@lists.wikimedia.org > > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, > > <mailto:wikimedia-l-requ...@lists.wikimedia.org?subject=unsubscribe> > > > _______________________________________________ > Wikimedia-l mailing list, guidelines at: > https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines > New messages to: Wikimedia-l@lists.wikimedia.org > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, > <mailto:wikimedia-l-requ...@lists.wikimedia.org?subject=unsubscribe> _______________________________________________ Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines New messages to: Wikimedia-l@lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, <mailto:wikimedia-l-requ...@lists.wikimedia.org?subject=unsubscribe>
