Re: [Pywikipedia-l] Tox jobs are only triggered for whitelisted person

2015-01-15 Thread Merlijn van Deen
On 15 January 2015 at 21:42, Ricordisamoa wrote: > This may sound stupid, but... why can't jobs be run by patch submitters? > Because they would effectively get full (user-level) access on the unit test servers. It's basically a check to prevent non-trusted users (after all, registration is comp

Re: [Pywikipedia-l] Tox jobs are only triggered for whitelisted person

2015-01-15 Thread Merlijn van Deen
On 15 January 2015 at 13:02, Fabian Neundorf wrote: > And just for clarification, when someone uploads a patch, not the > e-mail in the git commit are important but the account which you used > to push it to the repository? Yes, the account used to submit the Gerrit changeset. > Couldn't you aut

Re: [Pywikipedia-l] Status of the python3 branch

2015-01-08 Thread Merlijn van Deen
Done. On 8 January 2015 at 16:20, Ricordisamoa wrote: > pywikibot/core has a 'python3' branch which has been last updated in October > 2013. > Since the 'master' branch supports py3k fairly well now, it looks like the > old branch should be deleted. > Thoughts? > > ___

Re: [Pywikipedia-l] Running pywikibot on PyPy

2015-01-02 Thread Merlijn van Deen
On 2 January 2015 at 12:10, Ricordisamoa wrote: > Given the importance that MediaWiki people have recently given to HHVM, what > do you think we could/should do on the performance side of the framework? Our main bottleneck is a social one: the limit on the number of changes per minute is not dict

[Pywikipedia-l] Pywikibot git hosting RFC

2014-11-15 Thread Merlijn van Deen
Hello pywikibot community, cc wider Wikimedia community, Given the discussion on pywikipedia-l on the bugzilla migration, I thought it might be a good idea to have a more formal discussion beforehand on what we, as pywikibot community, want to do once the rest of the community moves from Gerrit to

Re: [Pywikipedia-l] Pywikobot and phabricator

2014-11-14 Thread Merlijn van Deen
Filed that as https://phabricator.wikimedia.org/T1282 with blocker for https://phabricator.wikimedia.org/T15. On 14 November 2014 22:49, John Mark Vandenberg wrote: > On Sat, Nov 15, 2014 at 2:12 AM, Andre Klapper wrote: >> Hi, >> >> On Fri, 2014-11-14 at 11:39 +0330, Amir Ladsgroup wrote: >>> I

Re: [Pywikipedia-l] [Pywikipedia-bugs] [Bug 72680] 'user-config.py' not found if runs with cron

2014-11-09 Thread Merlijn van Deen
On 9 Nov 2014 16:27, "Merlijn van Deen" wrote: > As a workaround, chdir to the pwb.py path before running, or explicitly > define the user-config path with -dir:/path/where/pwb.py/is > >> >> OK, that did not quite work as expected. I have now set -bugs to not

Re: [Pywikipedia-l] [Pywikipedia-bugs] [Bug 72680] 'user-config.py' not found if runs with cron

2014-11-09 Thread Merlijn van Deen
As a workaround, chdir to the pwb.py path before running, or explicitly define the user-config path with -dir:/path/where/pwb.py/is On 9 Nov 2014 15:10, wrote: > https://bugzilla.wikimedia.org/show_bug.cgi?id=72680 > > Steinsplitter changed: > >What|Removed |A

Re: [Pywikipedia-l] running bot on Labs

2014-11-02 Thread Merlijn van Deen
On 2 November 2014 09:46, rubin.happy wrote: > 1) how can I define which file should be used as user-config.py in the > script's running parameters? For example, I want to run "redirect.py br > -delete -always" with config of sysop bot and "redirect.py do -always" with > usual bot account, that's

Re: [Pywikipedia-l] welcome.py and -break

2014-09-01 Thread Merlijn van Deen
On 10 August 2014 12:08, Amir Ladsgroup wrote: > locale.Error: unsupported locale setting > i.e. you're using a locale that's not supported by the host you're running on (tools-dev). What locale did you set (i.e. what's the output of the 'locale' command)? Check locale -a to see what locales ar

Re: [Pywikipedia-l] Password

2014-08-30 Thread Merlijn van Deen
Hello Konstantinos, Is this with compat or core? On tool labs, using the shared framework, I can run login.py twice, and I'm logged in the second time. In any case, you can prevent any questions for passwords by using a password_file: # By default you are asked for a password on the terminal. #

Re: [Pywikipedia-l] Standard argument parsing

2014-08-25 Thread Merlijn van Deen
On 25 August 2014 13:50, Amir Ladsgroup wrote: > No, zip and rar > > both use standard system (note that -r is standard, - > *recurse is not) * > No, they don't (but

Re: [Pywikipedia-l] Standard argument parsing

2014-08-25 Thread Merlijn van Deen
On 25 August 2014 13:00, Amir Ladsgroup wrote: > Hello, > pywikibot doesn't use standard arguments and argument parsing > It doesn't do what /unix/ considers standard. However, we use the same system as e.g. rar and zip. What's the advantage of switching to a new system that will force every us

Re: [Pywikipedia-l] archivebot-older-than

2014-08-06 Thread Merlijn van Deen
Siebrand confirms: we should change it in code, and tw.net will pick it up. Merlijn On 5 August 2014 20:56, Merlijn van Deen wrote: > That has never been quite clear to me -- I *think* we are supposed to edit > 'en' and 'qqq' in code, and then tw.net <http://twn.n

Re: [Pywikipedia-l] archivebot-older-than

2014-08-05 Thread Merlijn van Deen
That has never been quite clear to me -- I *think* we are supposed to edit 'en' and 'qqq' in code, and then tw.net will pick it up. I will check with TWN'ers tomorrow during the hackathon to check! Merlijn On 5 August 2014 19:50, Ricordisamoa wrote: > The format should be chan

Re: [Pywikipedia-l] Manual:Pywikibot/i18n_conversion on MediaWiki.org

2014-08-05 Thread Merlijn van Deen
Great improvement - Thanks! On 5 August 2014 20:14, Ricordisamoa wrote: > I've cleaned it up > . Please > feel free to make additional improvements. > > ___ > Pywikipedia-l mailing list

Re: [Pywikipedia-l] best place for general feature requests?

2014-07-28 Thread Merlijn van Deen
Hi Amir, On 27 July 2014 07:14, Amir Ladsgroup wrote: > Another thing: pywikipedia-l isn't a hospitable place? Today, I discussed this with Frances and Sumana a bit, and these are my thoughts on it. Basically, I think there are several things we could improve on. It's not so much that pywikipe

Re: [Pywikipedia-l] Retrieve content of multiple pages at once

2014-07-25 Thread Merlijn van Deen
wikipedia.getAll was an internal function; I think it has now been superseded by site.preloadpages? On 25 July 2014 00:07, Sorawee Porncharoenwase wrote: > I remember that in compat we have a function named "getAll" or something > like that which will retrieve content of pages at once. Although

Re: [Pywikipedia-l] SourceForge bug tracker

2014-07-23 Thread Merlijn van Deen
On 23 July 2014 11:13, Ricordisamoa wrote: > A notice about the move to Bugzilla has been added, but many bugs still > show up as 'open'. > Is this expected/wanted? > I remember keeping bugs open to reduce bugspam, but I'm not completely sure if that was the case here. Is it a huge issue? It's i

Re: [Pywikipedia-l] Compat: pywikibot.output() forces everything to original stdout and stderr

2014-06-24 Thread Merlijn van Deen
t I'll look into giving > it another try within the next day or two when I have some time. > > > On Tue, Jun 24, 2014 at 3:17 AM, Merlijn van Deen > wrote: > >> On 24 June 2014 06:17, Jonathan Goble wrote: >> >> I've given up trying to solve a bug tha

Re: [Pywikipedia-l] Compat: pywikibot.output() forces everything to original stdout and stderr

2014-06-24 Thread Merlijn van Deen
On 24 June 2014 06:17, Jonathan Goble wrote: > I've given up trying to solve a bug that popped up in my scripts a couple > days ago. I run a bot for Wookieepedia, over at Wikia, and run three simple > scripts on a daily basis. They are set up to run automatically through > Windows Task Scheduler.

[Pywikipedia-l] i18n configuration

2014-06-17 Thread Merlijn van Deen
The following might be a bit unclear, as it's a bit of a brain dump. It's mainly meant as a response to https://gerrit.wikimedia.org/r/#/c/137904/2/tests/l10n_tests.py and https://gerrit.wikimedia.org/r/#/c/137924/ and as 'food for thought'. Basically, the question is how we can let the i18n not d

Re: [Pywikipedia-l] Page delete and protect

2014-06-16 Thread Merlijn van Deen
Then it should be in the default 'bot' class, not in the Page class. On 16 June 2014 16:33, John wrote: > Avoiding the headache of re-inventing the wheel across dozens of scripts > that use that code? > > > On Mon, Jun 16, 2014 at 10:19 AM, John Mark Vandenberg > wrote: > >> In https://gerrit.

Re: [Pywikipedia-l] Secret management

2014-06-15 Thread Merlijn van Deen
On 15 June 2014 09:19, John Mark Vandenberg wrote: > We have four sets of cleartext passwords (http & proxy & db), and secrets > such as various API keys and mw cookies and edit tokens. > > The passwords are stored in two files in clear text (user-config.py and. > passwd). Other secrets are in ca

Re: [Pywikipedia-l] drop support for python 2.6

2014-06-15 Thread Merlijn van Deen
On 14 June 2014 21:44, Ricordisamoa wrote: > Some tests fail only in py2.6: > https://travis-ci.org/wikimedia/pywikibot-core/jobs/27558463 ( > https://gerrit.wikimedia.org/r/139591/ is an attempted fix). > But I think we may instead drop support for py2.6 entirely, focusing > ourselves on py2.7

Re: [Pywikipedia-l] Interested in hacking on pywikibot

2014-06-08 Thread Merlijn van Deen
Hi Travis, On 6 June 2014 08:34, Travis Briggs wrote: > All this to say, I'd like to see if I can help out on the pywikibot > project itself, though I don't have a specific wiki that I work on or a > specific bot that I'd like to write. > Amir already gave some excellent pointers; there are two

Re: [Pywikipedia-l] mwparserfromhell

2014-06-08 Thread Merlijn van Deen
On 1 June 2014 01:57, Ricordisamoa wrote: > Since gerrit:131263 , it seems > to me that the excellent mwpfh is going to be used more and more > extensively within our framework. > Am I right? For example, the DuplicateReferences detection and fix in > ref

Re: [Pywikipedia-l] lists vs tuples

2014-06-06 Thread Merlijn van Deen
On 6 June 2014 23:49, Ricordisamoa wrote: > Should we always use tuples when instantiating them only once? E.g. > *if* abc *in* ('e', 'f'): > instead of > *if* abc *in* ['e', 'f']: > Performance-wise, I don't think it matters much; after all, most time is spent inside the loop, not in instantia

Re: [Pywikipedia-l] Core

2014-06-06 Thread Merlijn van Deen
Hi John, Great to see you contributing to core. I'm looking forward to your patches. Merlijn On 7 June 2014 00:02, John wrote: > Ive been waiting for core to get usable, and Im tired of waiting. In the > next month or two Ill be taking a good look and probably end up re-writing > half the cod

Re: [Pywikipedia-l] proxy issues and httplib2 update

2014-06-04 Thread Merlijn van Deen
Hi John, On 4 June 2014 10:00, John Mark Vandenberg wrote: > As I was fiddling with httplib2, I also tried to take bug 65189 which > upgrades our externals to httplib2 v0.9. however, when I initiate a > git review after having done a lovely merge, I see the following scary > message. > > (...)

Re: [Pywikipedia-l] Test needing fix

2014-06-04 Thread Merlijn van Deen
I have reverted the commit that caused this. Thanks for the heads-up. Merlijn On 4 June 2014 16:42, Ricordisamoa wrote: > https://travis-ci.org/wikimedia/pywikibot-core/jobs/26754246 > > ___ > Pywikipedia-l mailing list > Pywikipedia-l@lists.wikimedi

Re: [Pywikipedia-l] Deprecation of compat

2014-06-01 Thread Merlijn van Deen
On 1 June 2014 21:09, John wrote: > Until such time as core has moved all the features over and has proven > stable that shouldnt happen. Given the number of problems and missing > features that should be a minimum of 6 months away. > > Given that compat is already de facto deprecated -- bugs tak

Re: [Pywikipedia-l] Deprecation of compat

2014-06-01 Thread Merlijn van Deen
On 1 June 2014 21:06, Ricordisamoa wrote: > Does anyone know how to print a deprecation warning every time someone > runs 'compat'? > I have tried putting it in wikipedia.py, but with no results. > This was the trick used for the survey we had two years ago, on deprecating old python versions:

Re: [Pywikipedia-l] Mails from bugzilla?

2014-05-27 Thread Merlijn van Deen
For some reason, bugzilla is no longer sending e-mails to the mailing list (it explicitly excludes the list...) See https://bugzilla.wikimedia.org/show_bug.cgi?id=65828 for some more details. Merlijn On 27 May 2014 20:23, Mpaa wrote: > Hi. > > I am not receiving mails from bugzilla since May,

Re: [Pywikipedia-l] Moving Manual:Pywikibot to Pywikibot

2014-05-26 Thread Merlijn van Deen
Hi Amir, On 24 May 2014 03:52, "Amir Ladsgroup" wrote: > Based on several discussions here and there I want to move > [[Manual:Pywikibot]] (and all of the > subpages which are about 200 pages) to [[Pywikibot]] like other products > > Another reason that motiv

[Pywikipedia-l] Pywikibot documentation [was: Guidelines for codes of PWB]

2014-05-20 Thread Merlijn van Deen
On 19 May 2014 22:59, Pyfisch wrote: > Sphinx can generate really good docs from the source code and you only > need to change source code, and then we could generate and host the docs on > the labs server. If we require to update mediawiki.org, we wont ever > reach sync between code and document

Re: [Pywikipedia-l] Unlinking non-existent files

2014-05-09 Thread Merlijn van Deen
Hi Rubin, On 9 May 2014 21:09, rubin.happy wrote: > So, are there any ready methods/scripts to do this task? Ru.wiki has about > 4000+ files with red links to files and such a bot would be really > helpful... > Do you have a list of the files that should be unlinked, or should all non-existing i

[Pywikipedia-l] Login broken on WMF sites

2014-05-04 Thread Merlijn van Deen
Dear Pywikibot user, The Wikimedia Foundation recently [1] pushed MediaWiki 1.24wmf2 to all sites. This version contains a change [2] which causes two problems: - For compat, bots will edit as anonymous user; - For core, bots will ask for a password at each script invocation. This bug is

Re: [Pywikipedia-l] Solving fatal pywikibot bug in Windows

2014-03-29 Thread Merlijn van Deen
On 29 March 2014 19:52, Amir Ladsgroup wrote: > I looked for your patch but I couldn't find it, what was your technique? > https://gerrit.wikimedia.org/r/#/c/113185/ ___ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimed

Re: [Pywikipedia-l] Solving fatal pywikibot bug in Windows

2014-03-29 Thread Merlijn van Deen
Hi Amir, Have you looked at the patch I committed for core? The same technique might be applicable to compat. Otherwise, you could revert the patch that broke it (which is probably one of the last ones in https://github.com/wikimedia/pywikibot-compat/commits/master/pywikibot/version.py) Merlijn

Re: [Pywikipedia-l] Any volunteers around?

2014-02-25 Thread Merlijn van Deen
On 25 February 2014 20:40, Dr. Trigon wrote: > Would be cool! Just contact me when you start in order that we can > discuss about the details! > If possible, provide as many details as possible on the bug report. Then it's in a central location, and available for anyone interested in contributin

Re: [Pywikipedia-l] Ignore Changes not staged for commit

2014-01-31 Thread Merlijn van Deen
7 [submodule "externals/httplib2"] > 8 path = ex/httplib2ternals/httplib2 > 9 url = > https://gerrit.wikimedia.org/r/p/pywikibot/externals/httplib2.git > 10 branch = . > 11 ignore = dirty > > Looks like ignore = dirty is already there ... > > &

Re: [Pywikipedia-l] Ignore Changes not staged for commit

2014-01-31 Thread Merlijn van Deen
https://stackoverflow.com/questions/3240881/git-can-i-suppress-listing-of-modified-content-dirty-submodule-entries-in-sta So the trick is to adapt .gitmodules, not .gitignore. Merlijn On 31 January 2014 09:56, Mpaa wrote: > Hi. > > Is there a way to avoid having these to file to always show u

Re: [Pywikipedia-l] Propose to create another function to login

2014-01-24 Thread Merlijn van Deen
Hi Sorawee, On 24 January 2014 02:09, Sorawee Porncharoenwase wrote: > I think that we should have a function which tries to login and return the > status whether that logging in is successful. What we currently do (for > example in APISite.deletepage()) is that we "try: site.login(sysop=True) >

[Pywikipedia-l] mitmproxy: sniffing pywikibot traffic to debug network issues

2014-01-16 Thread Merlijn van Deen
Hello all, mainly co-developers, mitmproxy is a tool that allows you to sniff network traffic -- which is incredibly useful for debugging pywikibot network issues. Of course, there are other tools to do this (e.g. ethereal), but those are not usable anymore now that the WMF switched to HTTPS. In a

Re: [Pywikipedia-l] Version numbers

2014-01-15 Thread Merlijn van Deen
On 15 January 2014 13:47, Bináris wrote: > I am afraid I don't really understand you. > Was there any real problem with SVN version numbers? > What kind of local commit? There was one system with one series of > numbers. Do you mean your own computer by local? That is not interesting > for public

Re: [Pywikipedia-l] Version numbers

2014-01-15 Thread Merlijn van Deen
A version number is an ill-defined concept for a distributed version control system (if we both commit something locally after commit 1234, which one is really 1235?), but the system used by version.py is git rev-list HEAD | wc -l but this is *not *the same number that would be returned by an SVN

Re: [Pywikipedia-l] Unknown command git

2014-01-12 Thread Merlijn van Deen
On 12 January 2014 13:12, Wieland Hoffmann wrote: > I'm not sure why you expect that checkout to work: [0] clearly says > > > git clone --recursive https://gerrit.wikimedia.org/r/pywikibot/core.git > > (note the "--recursive" part). Your checkout will obviously not work if > you don't check the "

Re: [Pywikipedia-l] Github SVN broken

2014-01-12 Thread Merlijn van Deen
On 12 January 2014 05:03, Shantanoo wrote: > Am not a git advance user. Can git submodules > (http://www.git-scm.com/book/en/Git-Tools-Submodules) be used for such > scenario? > Yes -- that's what we are doing. 'git clone --recursive http://github.com/wikimedia/pywikibot-compat' would do the rig

Re: [Pywikipedia-l] Unknown command git

2014-01-11 Thread Merlijn van Deen
On 11 January 2014 23:28, John wrote: > Given the support and feedback that I have received going back to the > horribly broken addition by Trigon of a logging system Filling bugs is > useless. Here is a quote from IRC earlier today: > (...) > Per valhallasw's request I did some profiling, They

Re: [Pywikipedia-l] Bugzilla threads

2014-01-11 Thread Merlijn van Deen
Should be fixed when Bugzilla is upgraded to 4.4, which is soon(TM). I haven't found a Mailman option to fix this on the mailing list level. See https://bugzilla.wikimedia.org/show_bug.cgi?id=49597 for details. Merlijn On 12 January 2014 00:55, Bináris wrote: > Hi, > > Bugzilla sends e-mails

Re: [Pywikipedia-l] Unknown command git

2014-01-11 Thread Merlijn van Deen
On 11 January 2014 11:39, Bináris wrote: > First problem: whatever script I try to run DOS (Windows command) writes > me the thing in subject. > I think I have nothing to do with this stuff called git at the moment and > didn't say to Windows to run anything by this name. Why I am damned with > t

Re: [Pywikipedia-l] Github SVN broken

2014-01-11 Thread Merlijn van Deen
First of all, GitHub's SVN can be useful, but it's not something we have control over. It is a service provided by GitHub -- not by us. The GitHub SVN checkout at no point contained i18n. You'll have to checkout i18n explicitly from https://github.com/wikimedia/pywikibot-i18n -- GitHub provides no

[Pywikipedia-l] +2 for Mpaa and Pyfisch

2014-01-04 Thread Merlijn van Deen
Hello all, Because of their contributions to the project, we have offered +2 voting access to Mpaa and Pyfisch. Mpaa and Pyfisch, welcome to the team, and we hope you stay for a long time! :-) Merlijn ___ Pywikipedia-l mailing list Pywikipedia-l@lists.

Re: [Pywikipedia-l] Bug 55583: Calling generate_user_files.py in setup.py

2013-12-15 Thread Merlijn van Deen
On 15 December 2013 02:40, legoktm wrote: > Currently in setup.py, we call generate_user_files.py, which requires > interactive input. This is bad for package managers like the bug > report mentions, and also for pypi/pip. > > Removing it is a little more difficult for users, they need to now run

Re: [Pywikipedia-l] NotImplementedError in site.py

2013-12-09 Thread Merlijn van Deen
Hi Strainu, The _address functions are not used in core -- those were the 'pages to be screen scraped' in compat. However, adding missing (useful) functionality would be great. I'm not sure if a list exists, though. One example is the work by Allen in https://gerrit.wikimedia.org/r/#/c/100363/, b

Re: [Pywikipedia-l] Py3 compatibility

2013-12-07 Thread Merlijn van Deen
Hi xqt, On 7 December 2013 10:49, wrote: > I tried to review your commits relating to py3 compatibility. I am unsure > about "PY3" os environment. Is this set by the python3 installation or must > be set by hand? > This must be set by hand. Python 3 support is not functional yet, so I have kept

Re: [Pywikipedia-l] Commonscat.py

2013-11-22 Thread Merlijn van Deen
Hi Bojan, Based on those strings, it's a bit hard to determine -- the typical method to get more information is to search through all files for the message you get, e.g. (venv)valhallasw@lisilwen:~/src/pwb/pywikibot-core$ grep -R * -i -e 'disambiguationspage does not exist' but this returns noth

[Pywikipedia-l] Gerrit +2 voting rights for Strainu

2013-11-21 Thread Merlijn van Deen
Hello all, Because Strainu is coordinating Google Code-In for pywikibot, I have given him +2 voting rights on our repositories. Merlijn ___ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikip

Re: [Pywikipedia-l] gerrit patch uploader

2013-11-20 Thread Merlijn van Deen
Hi xqt, Yes, but it's not as easy as I would like it to be. The trick is to add the Change-Id line from the previous patchset commit message to the new commit message. So if you want to add a new patch to https://gerrit.wikimedia.org/r/#/c/96693/, fill in the patch uploader as usual, but add Chan

Re: [Pywikipedia-l] Google Code-In and a little problem

2013-11-11 Thread Merlijn van Deen
Hi Strainu, On 11 November 2013 13:30, Strainu wrote: > I've logged bugs and tasks for all the compat->core porting. This should > keep the students busy for a couple of days :) > Thanks for organising this! > I've also took a look at the bug report for wikidata. I would like to > propose the

Re: [Pywikipedia-l] python styling (pep8/pyflakes)

2013-11-04 Thread Merlijn van Deen
ine if it is really wrong. > ---------- > Von: Merlijn van Deen > Gesendet: 03.11.2013 21:50 > An: Pywikipedia discussion list > > Betreff: Re: [Pywikipedia-l] python styling (pep8/pyflakes) > > Some of them (I think mainly the line breaking ones) can be ignored by > adding

Re: [Pywikipedia-l] python styling (pep8/pyflakes)

2013-11-03 Thread Merlijn van Deen
eff: Re: [Pywikipedia-l] python styling (pep8/pyflakes) > > > Le 29/07/13 21:19, Merlijn van Deen a écrit : > > > Hm, indeed. # noqa is only supported in v1.4.1+; # nopep8 was > introduced > > > in v1.4+, and thus is also not available in 1.3. Would it be possible &g

Re: [Pywikipedia-l] Survey 2012 (Message still there)

2013-10-26 Thread Merlijn van Deen
On 26 October 2013 18:32, Dr. Trigon wrote: > Ok! Thanks for pointing this out! So I have to modify my question: > > "Why do we not have a user-config.py variable (like 'suppresssurvey') > in order to suppress this new message? Or even better introduce a > general one for future usage also?" > B

Re: [Pywikipedia-l] Survey 2012 (Message still there)

2013-10-26 Thread Merlijn van Deen
On 26 October 2013 10:47, Dr. Trigon wrote: > Can somebody explain to me why the user-config.py variable > "suppresssurvey = True" was removed but not the message? Now I get > dozens of such messages... > What is the output of version.py? Are you running core or compat? The config option *and*

Re: [Pywikipedia-l] Snapshots Broken

2013-10-23 Thread Merlijn van Deen
Should be fixed now -- the repository on the server was in a broken state. I'm not sure why that happened, but I've adapted the scripts to reset --hard origin/master before updating. Sorry for the inconvenience, and thanks for reporting. Merlijn On 23 October 2013 13:03, John wrote: > It look

Re: [Pywikipedia-l] Damned git

2013-10-08 Thread Merlijn van Deen
On 6 October 2013 20:22, Strainu wrote: > Gerrit has a bad reputation, and seemingly well deserved. Just > yesterday, Andrew Klapper was asking in a bug a submitted if the > pywikibot would not prefer getting patches in gerrit directly and I > was like "oh God, plese don't let that happen" :) >

Re: [Pywikipedia-l] Damned git

2013-10-08 Thread Merlijn van Deen
On 1 October 2013 20:34, Jan Dudík wrote: > now there was critical bug with interwiki.py, which happened about 15th > september. In these days was old sourceforge tracker moved to bugzilla, so > report was lost somewhere. After ten days I reported this bug again[1]. > Three days later there was p

Re: [Pywikipedia-l] Version numbers

2013-10-08 Thread Merlijn van Deen
On 6 October 2013 10:52, Dr. Trigon wrote: > As we have introduced version numbers into the code with [1] and [2], > namely '1.0b1' and '2.0b1' I want to mention that we should agree on a > version numbering philosophy. I assume the 'b1' stands for 'beta > relase 1' is that correct? Why beta? > B

Re: [Pywikipedia-l] Mailing lists

2013-09-23 Thread Merlijn van Deen
> > On Mon, Sep 23, 2013 at 1:43 PM, Pyfisch wrote: > >> at the moment Pywikibot has four mailing lists. >> * pywikipedia-announce – Only two messages this year (one spam, the other >> one to say sorry.) no last year >> * pywikipedia-l – This list, very active ;-) >> * pywikipedia-svn – Not needed

Re: [Pywikipedia-l] GIT stashing issue

2013-09-21 Thread Merlijn van Deen
On 21 September 2013 10:03, Dr. Trigon wrote: > -BEGIN PGP SIGNED MESSAGE- > I get following error while using git: > > > error: Your local changes to the following files would be > overwritten > by merge: > > maintenance/make_i18n_dict.py Please, commit your changes or stash > > them bef

Re: [Pywikipedia-l] wbsearchentities()

2013-09-12 Thread Merlijn van Deen
On 11 September 2013 20:31, Chinmay Naik wrote: > Can i retreive more than 100 items using this? I notice the > 'search-continue' returned by the search result disappears after 50 items. > for ex > https://www.wikidata.org/wiki/Special:ApiSandbox#action=wbsearchentities&format=json&search=abc&lan

Re: [Pywikipedia-l] BS and externals (again ;)

2013-09-11 Thread Merlijn van Deen
On 11 September 2013 19:17, Dr. Trigon wrote: > Just a thought about that; might it be possible that we have a > philosophy mismatch between win (xqt, ...) and linux (me, ...) users? > Might it be possible that win users want to have "everything" included > whereas linux useres are used to have i

Re: [Pywikipedia-l] Guide for migration from SVN to Git update

2013-09-09 Thread Merlijn van Deen
On 9 September 2013 12:31, wrote: > +1 > There is a problem with changes on the working copy which are not > committed or stashed. Pulling the repo often fails with the following: > > C:\pwb\compat>git pull "origin" master -v --progress > From https://gerrit.wikimedia.org/r/pywikibot/compat > *

Re: [Pywikipedia-l] Missing namespaces in Wikisource family?

2013-09-08 Thread Merlijn van Deen
Hi Mpaa, On 25 August 2013 00:11, Mpaa wrote: > I have some python knowledge (but I'm not a professional developer) and > would be nice to contribute to this project in my spare time. Any advice on > where/how one could help? > Help is always very welcome. Amir already replied with the more tec

Re: [Pywikipedia-l] httplib2 in setup.pys install_requires

2013-09-08 Thread Merlijn van Deen
Hi Wieland, On 4 September 2013 16:29, Wieland Hoffmann wrote: > with the recent switch to HTTPS by default [0] I'm wondering if it still > makes sense to keep httplib2 (the upstream version, not the one > maintained at [1]) as a dependency in setup.py? > Yes, you are completely right. It makes

Re: [Pywikipedia-l] [Errored] wikimedia/pywikibot-core#114 (master - 5105954)

2013-09-08 Thread Merlijn van Deen
On 3 September 2013 12:53, wrote: > I see that diff on github but it is not places at core. Maybe the reason > was that the push was hanging during transmission. > It seems the commit was pushed without going to code-review - this was possible because of a misconfiguration. Now direct pushes are

Re: [Pywikipedia-l] Switch to HTTPS, implications for Pywikibot?

2013-09-03 Thread Merlijn van Deen
On 3 September 2013 12:23, Cheng Yu Tin wrote: > It seems that pwb.py cannot import httplib2 from externals directory. > Possibly your httplib2 still refers to the old repository, which does not include fixes for this. Run git submodule sync git submodule update to re-sync to the new repositor

Re: [Pywikipedia-l] [Errored] wikimedia/pywikibot-core#114 (master - 5105954)

2013-09-03 Thread Merlijn van Deen
Hi xqt, On 3 September 2013 01:36, wrote: > Could anyone explain this message please. I tried to push some changes on > family files to core but it hanged until I aborted the pushing. I did not > found these changes on Gerrit. How to proceed in that matter? > > This message is unrelated to the f

[Pywikipedia-l] httplib2 subrepository (was: problem with git pullall)

2013-09-02 Thread Merlijn van Deen
On 2 September 2013 13:39, Dr. Trigon wrote: > > Sorry for the inconvenience - but at least we're not going to > > switch repositories again ;-) > > Oooho - are you a oracle? Or are you comming from future? ;))) > I'm fairly certain we are not switching the externals (other than maybe adding one)

Re: [Pywikipedia-l] problem with git pullall

2013-09-01 Thread Merlijn van Deen
Hey Sk!d, I think this is related to our source switch for httplib2 (from a github clone to our own gerrit repo). I can think of two things to try; 1) check .git/config to see if externals/httplib2 correctly has https://gerrit.wikimedia.org/r/pywikibot/externals/httplib2.git as URL git submodule s

Re: [Pywikipedia-l] Pywikibot, GIT and TS

2013-09-01 Thread Merlijn van Deen
On 30 August 2013 11:11, Dr. Trigon wrote: > Could you please explain how this works?? > > git gc --aggressive, as described earlier. I created a bz bug to ask to run this for -compat. The other repositories do not see much of a difference in re-packing. __

Re: [Pywikipedia-l] Switch to HTTPS, implications for Pywikibot?

2013-08-29 Thread Merlijn van Deen
We have switched to HTTPS again! If you get the following error message: On 20 August 2013 23:46, Merlijn van Deen wrote: > ERROR: Traceback (most recent call last): > File "X\core\pywikibot\data\api.py", line 286, in submit > body=paramstring) > File "X\core\pyw

Re: [Pywikipedia-l] Switch to HTTPS, implications for Pywikibot?

2013-08-24 Thread Merlijn van Deen
On 20 August 2013 23:46, Merlijn van Deen wrote: > SSLHandshakeError: [Errno 1] _ssl.c:504: error:14090086:SSL > routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify > failed > > Small update on this: - compat doesn't do certificate checking at all, so

Re: [Pywikipedia-l] Cruft in compat

2013-08-24 Thread Merlijn van Deen
Hi John, On 23 August 2013 00:36, John wrote: > The final straw was when I just converted to Git and discovered that he is > trying to run executables. > I think most of us agree using patch.exe was not the best decision. However, creating a way to install dependencies locally, which also works

[Pywikipedia-l] pywikibot-compat is now a small clone (was: Pywikibot, GIT and TS)

2013-08-22 Thread Merlijn van Deen
On 10 August 2013 11:26, Dr. Trigon wrote: > Pywikibot finished GIT migration. Now a question has anybody ever > tryed to clone and use the e.g. the pywikibot/compat repo on the TS? > If I do, I get and .git folder of about 200MB which nearly fills my > home quota. (this is the plain clone - noth

Re: [Pywikipedia-l] Switch to HTTPS, implications for Pywikibot?

2013-08-22 Thread Merlijn van Deen
On 22 August 2013 16:06, Antoine Musso wrote: > You might want to pipeline requests to avoid having to renegotiate the > certificate each time you want to do an action. > Very good point. core keeps connections alive (and hangs when I kill the connection... that's not supposed to happen [1]). Co

Re: [Pywikipedia-l] Switch to HTTPS, implications for Pywikibot?

2013-08-22 Thread Merlijn van Deen
I have submitted two revert patches, as the WMF is not switching until next week. This gives us some time to fix the certificate issue. Please check and +2. https://gerrit.wikimedia.org/r/80350 https://gerrit.wikimedia.org/r/80351 Merlijn On 20 August 2013 23:46, Merlijn van Deen wrote

Re: [Pywikipedia-l] ascii encode error in upload.py

2013-08-21 Thread Merlijn van Deen
It was a bug on our end - we were incorrectly handling urls, which caused python to try and interpret the image data as text. Not surprisingly, this results in errors. https://gerrit.wikimedia.org/r/80228 and https://gerrit.wikimedia.org/r/80229 both fix this (but in different and complementary wa

Re: [Pywikipedia-l] Switch to HTTPS, implications for Pywikibot?

2013-08-20 Thread Merlijn van Deen
On 20 August 2013 15:40, Amir Ladsgroup wrote: > after that bot operators just need to update > > OK, this seems to be giving some problems. If you get ERROR: Traceback (most recent call last): File "X\core\pywikibot\data\api.py", line 286, in submit body=paramstring) File "X\core\pywikibot

Re: [Pywikipedia-l] BeautifulSoup

2013-08-20 Thread Merlijn van Deen
On 20 August 2013 22:18, William H. Magill wrote: > Please insert URL to wiki: http//www.lotro-wiki.com > (...snip...) > ValueError: unknown url type: http//www.lotro-wiki.com Try using http://www.lotro-wiki.com (note the colon) instead. Merlijn

Re: [Pywikipedia-l] A basic pywikipedia version

2013-08-20 Thread Merlijn van Deen
Hi Alex, I'm not sure what you are proposing - what do you want to prune off, exactly, and how is python-wikitools [1] related to this? We already have 'a real "pywikipedia core" containing nothing but really needed modules to run basics with personal scripts.' - it's called pywikibot-core ;-). W

Re: [Pywikipedia-l] Time lag between wikidata writes

2013-08-20 Thread Merlijn van Deen
The sleep time as implemented in pywikipedia is influenced by three factors: - the minimum time between requests, minthrottle=1 (by default) in user-config.py - the minimum time between page saves, put_throttle=10 - the database lag on the DB servers - pages will not be saved unless the lag

Re: [Pywikipedia-l] Pywikibot, GIT and TS

2013-08-15 Thread Merlijn van Deen
On 15 August 2013 12:27, Amir Ladsgroup wrote: > I asked about a solution of the 200M everywhere. The best option is > cloning shallowly > >> >> No, it's not. After running git gc, I can re-clone the existing repository by only transferring 8MB: valhallasw@lisilwen:~/src/pywikibot-compat$ git da

Re: [Pywikipedia-l] Making installation of PWB a little bit more user-freindly

2013-08-15 Thread Merlijn van Deen
On 15 August 2013 10:01, Amir Ladsgroup wrote: > The biggest issue was we couldn't get reach to user-cofnig.py (sometimes > people try to install several clones). > I'm not sure what you mean by that. > I really prefer the old school in this field. We can store user-config in > the clone (core

Re: [Pywikipedia-l] Making installation of PWB a little bit more user-freindly

2013-08-14 Thread Merlijn van Deen
Hi Amir, On 14 August 2013 16:22, Amir Ladsgroup wrote: > As you probably know We had a workshop > http://wikimania2013.wikimedia.org/wiki/DevCamp/Schedule on Wikimania's > DevCamp about using PWB for people who are interested. > Cool! > but It was a very big problem that installation of PWB i

Re: [Pywikipedia-l] Code review in gerrit

2013-08-11 Thread Merlijn van Deen
On 11 August 2013 12:48, Merlijn van Deen wrote: > I'm not really sure though how you can do this under windows easily, as > tortoisegit doesn't seem to have any git-review capabilities... > > http://wiki.typo3.org/Contribution_Walkthrough_with_TortoiseGit seems to be a g

Re: [Pywikipedia-l] Code review in gerrit

2013-08-11 Thread Merlijn van Deen
On 11 August 2013 12:26, wrote: > Yes I aggree with pre-commit code review. But there also could be > remaining bugs after the code was pushed and identified to that code. Yes, that is true. I think it would be best to have them as bugs, especially after we migrate to bugzilla, which would make

Re: [Pywikipedia-l] Pywikibot, GIT and TS

2013-08-10 Thread Merlijn van Deen
On 10 August 2013 11:26, Dr. Trigon wrote: > Pywikibot finished GIT migration. Now a question has anybody ever > tryed to clone and use the e.g. the pywikibot/compat repo on the TS? > If I do, I get and .git folder of about 200MB which nearly fills my > home quota. (this is the plain clone - noth

Re: [Pywikipedia-l] SVN2GIT

2013-08-10 Thread Merlijn van Deen
On 10 August 2013 13:07, Dr. Trigon wrote: > I heard something about your experiments with a SVN2GIT tool or > solution? May I ask what the status there? What about that? > We used KDE's svn2git to convert core and compat from svn to git. The code & configuration is available here: https://githu

  1   2   3   4   >