[Pywikipedia-bugs] [Maniphest] [Commented On] T266117: Frequent chunk-too-small errors

[AntiCompositeNumber]

AntiCompositeNumber added a comment.


  Could not reproduce. Please provide at least the following information:
  
  - Operating system
  - Python environment and version (`import sys; print(sys.version)`)
  - Pywikibot version (`import pywikibot; print(pywikibot.__version__)`)
  - Relevant code or command used, including the chunk size configuration
  - Complete logs of upload attempt (VERBOSE-level or lower preferred)
  - Hash of the file that could not be uploaded
  
  It would also be very useful if you could provide the request and response 
headers for the failed chunk upload, including the exact size of the chunk. 
Information about if the file appears in Special:UploadStash would also be 
helpful.

TASK DETAIL
  https://phabricator.wikimedia.org/T266117

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: AntiCompositeNumber
Cc: AntiCompositeNumber, Mpaa, zhuyifei1999, Aklapper, pywikibot-bugs-list, 
Fae, JohnsonLee01, SHEKH, Dijkstra, CBogen, Khutuck, Zkhalido, Viztor, Wenyi, 
Ramsey-WMF, Tbscho, MayS, Mdupont, JJMC89, Dvorapa, Poyekhali, Altostratus, 
Taiwania_Justo, Avicennasis, Ixocactus, Wong128hk, mys_721tx, El_Grafo, 
Dinoguy1000, jayvdb, Masti, Alchimista, Steinsplitter, Rxy, Keegan
___
pywikibot-bugs mailing list
pywikibot-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs

Re: [pywikibot] Getting started with pywikibot

[masti]

Hi Larry,
ad 2.
yes. use -cat:categoryName or -catr:categoryName as generator.

look in pagegenarators.py script. It gives all generators you can use 
when starting script.


masti

On 19.08.2020 01:39, Larry Martell wrote:

On Mon, Aug 17, 2020 at 11:43 PM  wrote:

Hi Larry,

there are two way to use Pywikibot as a side package:
1. add a user-config.py in your /variant folder. A minimal sample can be found 
at 
https://gerrit.wikimedia.org/r/plugins/gitiles/pywikibot/core/+/88f22977fd186d888c4d9e8cbac11d76e87bd607/user-config.py.sample
2. or set a PYWIKIBOT_NO_USER_CONFIG environment variable to 1 or 2 to ignore 
user-config settings

The pwb.py script wrapper (as well as the generate_... scripts are available in 
directory mode only. You can get it including predefined scripts from nightly 
https://pywikibot.toolforge.org/ or as git repository from gerrit or from 
github clone https://github.com/wikimedia/pywikibot

Adding pwb.py and generate_user_files.py/generate_family_file.py to side 
oackage is still under development 
https://gerrit.wikimedia.org/r/c/pywikibot/core/+/560057

Seem our documentation is insufficient, sorry for that. If you have further 
questions do not hesitate to ask.


Thanks for the reply - this was very helpful. A couple more questions:

1. Is there any way to login with MFA? Our wiki requires either
choosing a number from an authenticator app or using a userid and
password and then entering a code which was texted.

2. Is there a way to get all pages within a category?

Thanks!
Larry


Am 17.08.2020 um 21:41 schrieb Larry Martell :

We have an internal wiki and I want to use pywikibot to get some data
from it. Installed it from pip but when I import it I get:

import pywikibot

Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/local/lib/python3.7/site-packages/pywikibot/__init__.py",
line 25, in 
from pywikibot.bot import (
  File "/usr/local/lib/python3.7/site-packages/pywikibot/bot.py", line
105, in 
from pywikibot import config2 as config
  File "/usr/local/lib/python3.7/site-packages/pywikibot/config2.py",
line 373, in 
base_dir = get_base_dir()
  File "/usr/local/lib/python3.7/site-packages/pywikibot/config2.py",
line 367, in get_base_dir
raise RuntimeError(exc_text)
RuntimeError: No user-config.py found in directory '/vagrant'.

Looking here 
https://www.mediawiki.org/wiki/Manual:Pywikibot/Installation#Configure_Pywikibot

I see a reference to pwb.py, but that does not exist in
/usr/local/lib/python3.7/site-packages/pywikibot.

What am I missing here?

Thanks!

___
pywikibot mailing list
pywikibot@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot

___
pywikibot mailing list
pywikibot@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot

___
pywikibot mailing list
pywikibot@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot



___
pywikibot mailing list
pywikibot@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot

[Pywikipedia-bugs] [Maniphest] [Commented On] T266117: Frequent chunk-too-small errors

[Fae]

Fae added a comment.


  As my 'personal' work-around, just for UK legislation PDFs that the API flags 
with chunk-too-small and fails on a second upload, the pdf is trimmed of the 
final byte and re-attempted. In my view this is a terrible hack rather than a 
fix.
  
  However, this initially appears to be working with the files both uploading 
and displaying successfully, though it may later cause unpredictable errors as 
it's hardly an intelligent fix. Ref to this category 

 for examples.
  
  Code snippet:
  
rec = uptry(local, fn, dd, comment, False)
if rec in ['chunk-too-small']:
print "Chunk-too-small, so trying trimming off 1 byte"
with open(local, 'rb+') as filehandle:
filehandle.seek(-1, os.SEEK_END)
filehandle.truncate()
rec = uptry(local, fn, dd + "\n[[Category:Work around of byte trimmed 
for chunk-too-small API error]]", comment, False)

TASK DETAIL
  https://phabricator.wikimedia.org/T266117

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Fae
Cc: Mpaa, zhuyifei1999, Aklapper, pywikibot-bugs-list, Fae, JohnsonLee01, 
SHEKH, Dijkstra, CBogen, Khutuck, Zkhalido, Viztor, Wenyi, Ramsey-WMF, Tbscho, 
MayS, Mdupont, JJMC89, Dvorapa, Poyekhali, Altostratus, Taiwania_Justo, 
Avicennasis, Ixocactus, Wong128hk, mys_721tx, El_Grafo, Dinoguy1000, jayvdb, 
Masti, Alchimista, Steinsplitter, Rxy, Keegan
___
pywikibot-bugs mailing list
pywikibot-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs

Re: [pywikibot] checking links against SpamFilter

[masti]

  
  
thanks,
that's how I've done it. I tought there is maybe an ready class.

masti

On 24.10.2020 13:44, John wrote:


  
  https://en.wikipedia.org/w/api.php?action="">
  
  
  
On Sat, Oct 24, 2020 at 2:29
  AM masti  wrote:

Hi,
  is there a way to check if the link will be catched by
  SpamFilter 
  without saving the page?
  
  I am doing some link statistics and cannot save the results
  page due to 
  global SpamFilter.
  
  masti
  
  ___
  pywikibot mailing list
  pywikibot@lists.wikimedia.org
  https://lists.wikimedia.org/mailman/listinfo/pywikibot

  
  
  
  ___
pywikibot mailing list
pywikibot@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot



  


___
pywikibot mailing list
pywikibot@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot

Re: [pywikibot] checking links against SpamFilter

[John]

https://en.wikipedia.org/w/api.php?action=help=spamblacklist

On Sat, Oct 24, 2020 at 2:29 AM masti  wrote:

> Hi,
> is there a way to check if the link will be catched by SpamFilter
> without saving the page?
>
> I am doing some link statistics and cannot save the results page due to
> global SpamFilter.
>
> masti
>
> ___
> pywikibot mailing list
> pywikibot@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/pywikibot
>
___
pywikibot mailing list
pywikibot@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot

[Pywikibot-commits] [Gerrit] ...core[master]: [IMPR] make showStatus function a bot method and rename it to show_st...

[jenkins-bot (Code Review)]

jenkins-bot has submitted this change. ( 
https://gerrit.wikimedia.org/r/c/pywikibot/core/+/559492 )

Change subject: [IMPR] make showStatus function a bot method and rename it to 
show_status
..

[IMPR] make showStatus function a bot method and rename it to show_status

Patch detached from Iff53f3818

Change-Id: I9ae0a1a5e47cf9ff940c90b9e32501c7d9346d16
---
M scripts/welcome.py
1 file changed, 26 insertions(+), 25 deletions(-)

Approvals:
  Xqt: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/scripts/welcome.py b/scripts/welcome.py
index e9b0271..355f2c2 100755
--- a/scripts/welcome.py
+++ b/scripts/welcome.py
@@ -557,7 +557,7 @@
  .format(self.site))
 list_loaded = load_word_function(badword_page.get())
 else:
-showStatus(4)
+self.show_status(4)
 pywikibot.output("The bad word page doesn't exist!")
 self._blacklist = elenco + elenco_others + list_loaded
 del elenco, elenco_others, list_loaded
@@ -574,10 +574,10 @@
  .format(self.site))
 list_white = load_word_function(whitelist_page.get())
 else:
-showStatus(4)
+self.show_status(4)
 pywikibot.output("The whitelist's page doesn't exist!")
 else:
-showStatus(4)
+self.show_status(4)
 pywikibot.warning("The whitelist hasn't been set!")

 # Join the whitelist words.
@@ -614,7 +614,7 @@
 answer = 'y'

 if answer.lower() in ['yes', 'y'] or not globalvar.confirm:
-showStatus()
+self.show_status()
 pywikibot.output(
 '{} is possibly an unwanted username. It will be reported.'
 .format(name))
@@ -656,7 +656,7 @@
 if rep_text != '':
 rep_page.put(text_get + rep_text, summary=com, force=True,
  minor=True)
-showStatus(5)
+self.show_status(5)
 pywikibot.output('Reported')
 self.BAQueue = []

@@ -677,7 +677,7 @@
 text = log_page.get()
 else:
 # make new log page
-showStatus()
+self.show_status()
 pywikibot.output(
 'Log page is not exist, getting information for page creation')
 text = i18n.translate(self.site, logpage_header,
@@ -735,7 +735,7 @@
 if not globalvar.signFileName:
 sign_page_name = i18n.translate(self.site, random_sign)
 if not sign_page_name:
-showStatus(4)
+self.show_status(4)
 pywikibot.output(
 "{} doesn't allow random signature, force disable."
 .format(self.site))
@@ -770,25 +770,25 @@
 def skip_page(self, user) -> bool:
 """Check whether the user is to be skipped."""
 if user.isBlocked():
-showStatus(3)
+self.show_status(3)
 pywikibot.output('{} has been blocked!'.format(user.username))

 elif 'bot' in user.groups():
-showStatus(3)
+self.show_status(3)
 pywikibot.output('{} is a bot!'.format(user.username))

 elif 'bot' in user.username.lower():
-showStatus(3)
+self.show_status(3)
 pywikibot.output('{} might be a global bot!'
  .format(user.username))

 elif user.editCount() < globalvar.attachEditCount:
 if not user.editCount() == 0:
-showStatus(1)
+self.show_status(1)
 pywikibot.output('{0} has only {1} contributions.'
  .format(user.username, user.editCount()))
 elif not globalvar.quiet:
-showStatus(1)
+self.show_status(1)
 pywikibot.output('{} has no contributions.'
  .format(user.username))
 else:
@@ -804,12 +804,12 @@
 if self.skip_page(user):
 continue

-showStatus(2)
+self.show_status(2)
 pywikibot.output('{} has enough edits to be welcomed.'
  .format(user.username))
 ustp = user.getUserTalkPage()
 if ustp.exists():
-showStatus(3)
+self.show_status(3)
 pywikibot.output('{} has been already welcomed.'
  .format(user.username))
 continue
@@ -838,7 +838,7 @@
 # append welcomed, welcome_count++
 ustp.put(welcome_text, 

[pywikibot] checking links against SpamFilter

[masti]

Hi,
is there a way to check if the link will be catched by SpamFilter 
without saving the page?


I am doing some link statistics and cannot save the results page due to 
global SpamFilter.


masti

___
pywikibot mailing list
pywikibot@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot