Re: [Wikitech-l] changes coming to large dumps

2018-04-10 Thread Derk-Jan Hartman
It seems that the pagecounts-ez sets disappeared from dumps.wikimedia.org starting this date. Is that a coincidence ? Is it https://phabricator.wikimedia.org/T189283 perhaps ? DJ On Thu, Mar 29, 2018 at 2:42 PM, Ariel Glenn WMF wrote: > Here it comes: > > For the April 1st run and all following

Re: [Wikitech-l] changes coming to large dumps

2018-03-29 Thread Ariel Glenn WMF
Here it comes: For the April 1st run and all following runs, the Wikidata dumps of pages-meta-current.bz2 will be produced only as separate downloadable files, no recombined single file will be produced. No other dump jobs will be impacted. A reminder that each of the single downloadable pieces

Re: [Wikitech-l] changes coming to large dumps

2018-03-19 Thread Ariel Glenn WMF
A reprieve! Code's not ready and I need to do some timing tests, so the March 20th run will do the standard recombining. For updates, don't forget to check the Phab ticket! https://phabricator.wikimedia.org/T179059 On Mon, Mar 5, 2018 at 1:10 PM, Ariel Glenn WMF wrote: > Please forward whereve

Re: [Wikitech-l] changes coming to large dumps

2018-03-05 Thread Bináris
That's really big. :-) 2018-03-05 13:10 GMT+01:00 Ariel Glenn WMF : > We'll probably start at 20GB, which means that WIkidata will be the only > wiki affected for now. > > Ariel > > On Mon, Mar 5, 2018 at 1:40 PM, Bináris wrote: > > > Could you please translate "too large" to megabytes? > > > >

Re: [Wikitech-l] changes coming to large dumps

2018-03-05 Thread Ariel Glenn WMF
We'll probably start at 20GB, which means that WIkidata will be the only wiki affected for now. Ariel On Mon, Mar 5, 2018 at 1:40 PM, Bináris wrote: > Could you please translate "too large" to megabytes? > > 2018-03-05 12:10 GMT+01:00 Ariel Glenn WMF : > > > Please forward wherever you think ap

Re: [Wikitech-l] changes coming to large dumps

2018-03-05 Thread Bináris
Could you please translate "too large" to megabytes? 2018-03-05 12:10 GMT+01:00 Ariel Glenn WMF : > Please forward wherever you think appropriate. > > For some time we have provided multiple numbered pages-articles bz2 file > for large wikis, as well as a single file with all of the contents comb

[Wikitech-l] changes coming to large dumps

2018-03-05 Thread Ariel Glenn WMF
Please forward wherever you think appropriate. For some time we have provided multiple numbered pages-articles bz2 file for large wikis, as well as a single file with all of the contents combined into one. This is consuming enough time for Wikidata that it is no longer sustainable. For wikis whe