Re: [Pywikipedia-l] Cruft in compat
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 01.09.2013 01:56, Mpaa wrote: Hi. My 2 cents ... on compat regarding the message about logging handlers. When no log is required, log.log(_level, text, extra=context, **kwargs) in wikipedia.logoutput() actually has no handlers and no parents (root) with handlers. Then, from here the statement that no handlers are defined for pywiki. If logs are required by param settings, wikipedia.setLogfileStatus() defines a handler for logger 'root', which will become parent of the above log in wikipedia.logoutput(). log still has no handlers but the logging module will look for parents of log. now finding them. Maybe it is too late here ... for this to be accurate ... The implementation is very convoluted, with globals variables and local variables, several call to init functions, confusing names (e.g. log is a method of a logger object, a function, etc.) If you share this opinion, a refactoring would be appropriate? What you see there is the result of a refactoring of the very ancient mode trunk/compat used; it just dumped to file without timestamp and else. Since core on the other hand had/has a very clean implementation using the logging module, I started a transition from just dump to file to using the logging module as core does in order to finally converge with compat against core. So you are charging an open door - that was the reason why I asked for can you give me more details please e.g. a description and a traceback of an actual crash situation? That would give me a point to start on and fix this concrete bug as well as improve the situation overall. Of course you can also implement your own changes - I would be happy to review them - but please keep in mind you have to be very careful not to break things here. E.g. a message that no logger is defined is annoying indeed, but does not break the code to run... Greetings DrTrigon -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.14 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlIjE3sACgkQAXWvBxzBrDDmXgCfeIIR7uWq9kzPZKDYyrtEwIHa 5gcAoMzc9nVnpfF1+X4L/HLk3OSDuKPq =kqcg -END PGP SIGNATURE- ___ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l
Re: [Pywikipedia-l] Cruft in compat
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 I did not find any crash situation or bug to fix. I just tried to understand what was the root cause of the missing logger message, someone was complaining about. Which I described above (always assuming it is the correct one ...). Yes that's exactly the point it is a situation that needs to be IMPROVED but there is actually NO BUG to fix... ;)) (and it's not main priority at the moment) Greetings and thanks! DrTrigon -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.14 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlIjfSMACgkQAXWvBxzBrDBYlgCg4cyieNNoQCcja/h5AY2Yb1fi 1eAAoNGz4sFsEfRKghV6MDO4sPdMxo2W =6X8W -END PGP SIGNATURE- ___ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l
Re: [Pywikipedia-l] Cruft in compat
2013/8/24 John phoenixoverr...@gmail.com Another point that is really frustrating is the logging system that complains about not having a logger defined if you run any custom scripts. So this is the reason! I didn't understand why I suddenly got those strange messages. I have a dream: someday we get back to schöne alte Zeiten when things just worked and there was no need to dissipate our energies for continous maintenance and following changes. And we can just USE the bot again. ___ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l
Re: [Pywikipedia-l] Cruft in compat
logging should activated for those scripts listed in config.log list only. Isn't it? xqt - Original Nachricht Von: Bináris wikipo...@gmail.com An: Pywikipedia discussion list pywikipedia-l@lists.wikimedia.org Datum: 31.08.2013 18:08 Betreff: Re: [Pywikipedia-l] Cruft in compat 2013/8/24 John phoenixoverr...@gmail.com Another point that is really frustrating is the logging system that complains about not having a logger defined if you run any custom scripts. So this is the reason! I didn't understand why I suddenly got those strange messages. I have a dream: someday we get back to schöne alte Zeiten when things just worked and there was no need to dissipate our energies for continous maintenance and following changes. And we can just USE the bot again. ___ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l ___ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l
Re: [Pywikipedia-l] Cruft in compat
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Compat/trunk never worked like this... I always (at least 5 years now) checked out/cloned the current version - tested it a few days with my bots, commited bug fixes if needed and then ran this code for about 6 months. After that period the same procedure was repeated. I now there are communities that retrieve the most current code once a day by crontab job but I consider this as very dangerous. May be for nightlies there is some unittesting before deployment...? Now using gerrit and code review should give use the ability just to merge in code that was tested and works (despite some complex bugs) and thus we should be able to have working code in the repo only... But that would require a high level of discipline and very serious testing of commited changes by all developers involved... so I am afraid that it will stay somehow suboptimal as it is because we are all just volunteers only... ;)) ...this is my assumption... ;)) But to be honest we have a quite cool and quite stable bunch of code here in my oppinion - at least for the tasks I am working on... ;) Greetings DrTrigon On 31.08.2013 18:08, Bináris wrote: 2013/8/24 John phoenixoverr...@gmail.com mailto:phoenixoverr...@gmail.com Another point that is really frustrating is the logging system that complains about not having a logger defined if you run any custom scripts. So this is the reason! I didn't understand why I suddenly got those strange messages. I have a dream: someday we get back to schöne alte Zeiten when things just worked and there was no need to dissipate our energies for continous maintenance and following changes. And we can just USE the bot again. ___ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.14 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlIiXqoACgkQAXWvBxzBrDAepACgyA0tA8GhCIaQ0iCXX/2txuyS HPoAn07vn9+55DlRHvt4N9BCmwWL7bCE =9xEj -END PGP SIGNATURE- ___ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l
Re: [Pywikipedia-l] Cruft in compat
Hi. My 2 cents ... on compat regarding the message about logging handlers. When no log is required, log.log(_level, text, extra=context, **kwargs) in wikipedia.logoutput() actually has no handlers and no parents (root) with handlers. Then, from here the statement that no handlers are defined for pywiki. If logs are required by param settings, wikipedia.setLogfileStatus() defines a handler for logger 'root', which will become parent of the above log in wikipedia.logoutput(). log still has no handlers but the logging module will look for parents of log. now finding them. Maybe it is too late here ... for this to be accurate ... The implementation is very convoluted, with globals variables and local variables, several call to init functions, confusing names (e.g. log is a method of a logger object, a function, etc.) If you share this opinion, a refactoring would be appropriate? Bye Mpaa On Sat, Aug 31, 2013 at 11:23 PM, Dr. Trigon dr.tri...@surfeu.ch wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Good question - otherwise we have a bug here... On 31.08.2013 19:38, i...@gno.de wrote: logging should activated for those scripts listed in config.log list only. Isn't it? xqt - Original Nachricht Von: Bináris wikipo...@gmail.com An: Pywikipedia discussion list pywikipedia-l@lists.wikimedia.org Datum: 31.08.2013 18:08 Betreff: Re: [Pywikipedia-l] Cruft in compat 2013/8/24 John phoenixoverr...@gmail.com Another point that is really frustrating is the logging system that complains about not having a logger defined if you run any custom scripts. So this is the reason! I didn't understand why I suddenly got those strange messages. I have a dream: someday we get back to schöne alte Zeiten when things just worked and there was no need to dissipate our energies for continous maintenance and following changes. And we can just USE the bot again. ___ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l ___ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.14 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlIiXtcACgkQAXWvBxzBrDAY7gCgqqoftkrg1v0gHOR2ZOgmYmTY sEEAnRZBYmt2BXbOKm5/wFtM/khlYtUQ =5ZbM -END PGP SIGNATURE- ___ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l ___ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l
Re: [Pywikipedia-l] Cruft in compat
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 I fully agree with Merlijn and Amir! Though John is pointing us to some things we might have to pay attention to: (1) It shoudnt be querying git on every script invocation Indeed you are right - this might be due to version.py imported in wikipedia.py... Can you give me more details here in order I can reproduce this behaviour - but I do not see this as a big problem. (2) I shoudnt have to set wikipedia.setLogfileStatus(True) in all my scripts Again true - and again can you give me more details please - if this is the case, this is clearly a bug that has to be solved. (3) Again, we have to improve the docs, any help and feedback is very welcome here. Greetings DrTrigon On 25.08.2013 13:34, Amir Ladsgroup wrote: We have done it recently: you can read more in http://www.mediawiki.org/wiki/Manual:Pywikipediabot/Installation On 8/25/13, swuensch swuen...@gmail.com wrote: Wouldn't it be possible to implement some kind of plugin-strucutred addons for those special cases where you have to download more, install more, do more by yourself? On Sun, Aug 25, 2013 at 11:34 AM, John phoenixoverr...@gmail.com wrote: I shoudnt have to set wikipedia.setLogfileStatus(True) in all my scripts just because someone cant be bothered to implement their logging code correctly. Moving to pre-configred git sub modules would do the same thing, without running exicutibles on windows computers. Just because someone haphazardly threw stuff together without thinking it through doesnt mean its acceptable On Sun, Aug 25, 2013 at 5:23 AM, Amir Ladsgroup ladsgr...@gmail.comwrote: See this: http://sourceforge.net/p/pywikipediabot/bugs/1633/ The problem is we can't set all of patches for all of users. So many people are not interested in running IRC bots or image handling bots, so They don't need to download almost 100M for that Best On 8/24/13, John phoenixoverr...@gmail.com wrote: Another point that is really frustrating is the logging system that complains about not having a logger defined if you run any custom scripts. The only way that I have found to shut this up is to enable logging for all scripts, however I end up with thousands of log files. (each pid creates its own file. and if you use muti-processing that ends up with thousands or hundreds of thousands of log files) Also It shoudnt be querying git on every script invocation or at all unless version.py is manually called. that kind of overhead is just bloat (with perhaps the exception of interwiki.py). if externals are needed and they need to be patched the best thing to to would include them pre-patched in our own sub-module, that way you can avoid the whole can of worms On Sat, Aug 24, 2013 at 10:14 AM, Merlijn van Deen valhall...@arctus.nlwrote: Hi John, On 23 August 2013 00:36, John phoenixoverr...@gmail.com wrote: The final straw was when I just converted to Git and discovered that he is trying to run executables. I think most of us agree using patch.exe was not the best decision. However, creating a way to install dependencies locally, which also works well for windows users - unfortunately, virtualenv and others don't work too well for some windows workflows, I think was a sensible idea. This is bloatware plain and simple I have seen the number of externals go from 2 to 17. Most of these are probably for one or two pet programs that should never have been added to the project as they are niche programs. I don't agree on this - the number of externals that /can be installed/ has increased, but the only one required is BeautifulSoup. This is not different from how things were before. Thanks to the new packaging system, you are only asked to install them when required. Yes, they are typically for one or two scripts, but calling them 'pet programs' is just nonsense - the scripts DrTrigon has added (e.g. sum_disc/discussion summaries, subster/dynamic page updating from an external source, catimages/smart(er) image categorization based on content) are useful tools, even though Dr Trigon might be the only one running them at the moment. I would really like to see things streamlined and all of the cruft removed and get back to the nice library that we had. Apart from the patch.exe issue, I', not really sure what there is to streamline - the extra externals are only installed on demand and the new scripts are not in the way of using other scripts. Merlijn ___ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l - -- Amir ___ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l ___
Re: [Pywikipedia-l] Cruft in compat
See this: http://sourceforge.net/p/pywikipediabot/bugs/1633/ The problem is we can't set all of patches for all of users. So many people are not interested in running IRC bots or image handling bots, so They don't need to download almost 100M for that Best On 8/24/13, John phoenixoverr...@gmail.com wrote: Another point that is really frustrating is the logging system that complains about not having a logger defined if you run any custom scripts. The only way that I have found to shut this up is to enable logging for all scripts, however I end up with thousands of log files. (each pid creates its own file. and if you use muti-processing that ends up with thousands or hundreds of thousands of log files) Also It shoudnt be querying git on every script invocation or at all unless version.py is manually called. that kind of overhead is just bloat (with perhaps the exception of interwiki.py). if externals are needed and they need to be patched the best thing to to would include them pre-patched in our own sub-module, that way you can avoid the whole can of worms On Sat, Aug 24, 2013 at 10:14 AM, Merlijn van Deen valhall...@arctus.nlwrote: Hi John, On 23 August 2013 00:36, John phoenixoverr...@gmail.com wrote: The final straw was when I just converted to Git and discovered that he is trying to run executables. I think most of us agree using patch.exe was not the best decision. However, creating a way to install dependencies locally, which also works well for windows users - unfortunately, virtualenv and others don't work too well for some windows workflows, I think was a sensible idea. This is bloatware plain and simple I have seen the number of externals go from 2 to 17. Most of these are probably for one or two pet programs that should never have been added to the project as they are niche programs. I don't agree on this - the number of externals that /can be installed/ has increased, but the only one required is BeautifulSoup. This is not different from how things were before. Thanks to the new packaging system, you are only asked to install them when required. Yes, they are typically for one or two scripts, but calling them 'pet programs' is just nonsense - the scripts DrTrigon has added (e.g. sum_disc/discussion summaries, subster/dynamic page updating from an external source, catimages/smart(er) image categorization based on content) are useful tools, even though Dr Trigon might be the only one running them at the moment. I would really like to see things streamlined and all of the cruft removed and get back to the nice library that we had. Apart from the patch.exe issue, I', not really sure what there is to streamline - the extra externals are only installed on demand and the new scripts are not in the way of using other scripts. Merlijn ___ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l -- Amir ___ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l
Re: [Pywikipedia-l] Cruft in compat
I shoudnt have to set wikipedia.setLogfileStatus(True) in all my scripts just because someone cant be bothered to implement their logging code correctly. Moving to pre-configred git sub modules would do the same thing, without running exicutibles on windows computers. Just because someone haphazardly threw stuff together without thinking it through doesnt mean its acceptable On Sun, Aug 25, 2013 at 5:23 AM, Amir Ladsgroup ladsgr...@gmail.com wrote: See this: http://sourceforge.net/p/pywikipediabot/bugs/1633/ The problem is we can't set all of patches for all of users. So many people are not interested in running IRC bots or image handling bots, so They don't need to download almost 100M for that Best On 8/24/13, John phoenixoverr...@gmail.com wrote: Another point that is really frustrating is the logging system that complains about not having a logger defined if you run any custom scripts. The only way that I have found to shut this up is to enable logging for all scripts, however I end up with thousands of log files. (each pid creates its own file. and if you use muti-processing that ends up with thousands or hundreds of thousands of log files) Also It shoudnt be querying git on every script invocation or at all unless version.py is manually called. that kind of overhead is just bloat (with perhaps the exception of interwiki.py). if externals are needed and they need to be patched the best thing to to would include them pre-patched in our own sub-module, that way you can avoid the whole can of worms On Sat, Aug 24, 2013 at 10:14 AM, Merlijn van Deen valhall...@arctus.nlwrote: Hi John, On 23 August 2013 00:36, John phoenixoverr...@gmail.com wrote: The final straw was when I just converted to Git and discovered that he is trying to run executables. I think most of us agree using patch.exe was not the best decision. However, creating a way to install dependencies locally, which also works well for windows users - unfortunately, virtualenv and others don't work too well for some windows workflows, I think was a sensible idea. This is bloatware plain and simple I have seen the number of externals go from 2 to 17. Most of these are probably for one or two pet programs that should never have been added to the project as they are niche programs. I don't agree on this - the number of externals that /can be installed/ has increased, but the only one required is BeautifulSoup. This is not different from how things were before. Thanks to the new packaging system, you are only asked to install them when required. Yes, they are typically for one or two scripts, but calling them 'pet programs' is just nonsense - the scripts DrTrigon has added (e.g. sum_disc/discussion summaries, subster/dynamic page updating from an external source, catimages/smart(er) image categorization based on content) are useful tools, even though Dr Trigon might be the only one running them at the moment. I would really like to see things streamlined and all of the cruft removed and get back to the nice library that we had. Apart from the patch.exe issue, I', not really sure what there is to streamline - the extra externals are only installed on demand and the new scripts are not in the way of using other scripts. Merlijn ___ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l -- Amir ___ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l ___ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l
Re: [Pywikipedia-l] Cruft in compat
Wouldn't it be possible to implement some kind of plugin-strucutred addons for those special cases where you have to download more, install more, do more by yourself? On Sun, Aug 25, 2013 at 11:34 AM, John phoenixoverr...@gmail.com wrote: I shoudnt have to set wikipedia.setLogfileStatus(True) in all my scripts just because someone cant be bothered to implement their logging code correctly. Moving to pre-configred git sub modules would do the same thing, without running exicutibles on windows computers. Just because someone haphazardly threw stuff together without thinking it through doesnt mean its acceptable On Sun, Aug 25, 2013 at 5:23 AM, Amir Ladsgroup ladsgr...@gmail.comwrote: See this: http://sourceforge.net/p/pywikipediabot/bugs/1633/ The problem is we can't set all of patches for all of users. So many people are not interested in running IRC bots or image handling bots, so They don't need to download almost 100M for that Best On 8/24/13, John phoenixoverr...@gmail.com wrote: Another point that is really frustrating is the logging system that complains about not having a logger defined if you run any custom scripts. The only way that I have found to shut this up is to enable logging for all scripts, however I end up with thousands of log files. (each pid creates its own file. and if you use muti-processing that ends up with thousands or hundreds of thousands of log files) Also It shoudnt be querying git on every script invocation or at all unless version.py is manually called. that kind of overhead is just bloat (with perhaps the exception of interwiki.py). if externals are needed and they need to be patched the best thing to to would include them pre-patched in our own sub-module, that way you can avoid the whole can of worms On Sat, Aug 24, 2013 at 10:14 AM, Merlijn van Deen valhall...@arctus.nlwrote: Hi John, On 23 August 2013 00:36, John phoenixoverr...@gmail.com wrote: The final straw was when I just converted to Git and discovered that he is trying to run executables. I think most of us agree using patch.exe was not the best decision. However, creating a way to install dependencies locally, which also works well for windows users - unfortunately, virtualenv and others don't work too well for some windows workflows, I think was a sensible idea. This is bloatware plain and simple I have seen the number of externals go from 2 to 17. Most of these are probably for one or two pet programs that should never have been added to the project as they are niche programs. I don't agree on this - the number of externals that /can be installed/ has increased, but the only one required is BeautifulSoup. This is not different from how things were before. Thanks to the new packaging system, you are only asked to install them when required. Yes, they are typically for one or two scripts, but calling them 'pet programs' is just nonsense - the scripts DrTrigon has added (e.g. sum_disc/discussion summaries, subster/dynamic page updating from an external source, catimages/smart(er) image categorization based on content) are useful tools, even though Dr Trigon might be the only one running them at the moment. I would really like to see things streamlined and all of the cruft removed and get back to the nice library that we had. Apart from the patch.exe issue, I', not really sure what there is to streamline - the extra externals are only installed on demand and the new scripts are not in the way of using other scripts. Merlijn ___ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l -- Amir ___ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l ___ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l ___ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l
Re: [Pywikipedia-l] Cruft in compat
We have done it recently: you can read more in http://www.mediawiki.org/wiki/Manual:Pywikipediabot/Installation On 8/25/13, swuensch swuen...@gmail.com wrote: Wouldn't it be possible to implement some kind of plugin-strucutred addons for those special cases where you have to download more, install more, do more by yourself? On Sun, Aug 25, 2013 at 11:34 AM, John phoenixoverr...@gmail.com wrote: I shoudnt have to set wikipedia.setLogfileStatus(True) in all my scripts just because someone cant be bothered to implement their logging code correctly. Moving to pre-configred git sub modules would do the same thing, without running exicutibles on windows computers. Just because someone haphazardly threw stuff together without thinking it through doesnt mean its acceptable On Sun, Aug 25, 2013 at 5:23 AM, Amir Ladsgroup ladsgr...@gmail.comwrote: See this: http://sourceforge.net/p/pywikipediabot/bugs/1633/ The problem is we can't set all of patches for all of users. So many people are not interested in running IRC bots or image handling bots, so They don't need to download almost 100M for that Best On 8/24/13, John phoenixoverr...@gmail.com wrote: Another point that is really frustrating is the logging system that complains about not having a logger defined if you run any custom scripts. The only way that I have found to shut this up is to enable logging for all scripts, however I end up with thousands of log files. (each pid creates its own file. and if you use muti-processing that ends up with thousands or hundreds of thousands of log files) Also It shoudnt be querying git on every script invocation or at all unless version.py is manually called. that kind of overhead is just bloat (with perhaps the exception of interwiki.py). if externals are needed and they need to be patched the best thing to to would include them pre-patched in our own sub-module, that way you can avoid the whole can of worms On Sat, Aug 24, 2013 at 10:14 AM, Merlijn van Deen valhall...@arctus.nlwrote: Hi John, On 23 August 2013 00:36, John phoenixoverr...@gmail.com wrote: The final straw was when I just converted to Git and discovered that he is trying to run executables. I think most of us agree using patch.exe was not the best decision. However, creating a way to install dependencies locally, which also works well for windows users - unfortunately, virtualenv and others don't work too well for some windows workflows, I think was a sensible idea. This is bloatware plain and simple I have seen the number of externals go from 2 to 17. Most of these are probably for one or two pet programs that should never have been added to the project as they are niche programs. I don't agree on this - the number of externals that /can be installed/ has increased, but the only one required is BeautifulSoup. This is not different from how things were before. Thanks to the new packaging system, you are only asked to install them when required. Yes, they are typically for one or two scripts, but calling them 'pet programs' is just nonsense - the scripts DrTrigon has added (e.g. sum_disc/discussion summaries, subster/dynamic page updating from an external source, catimages/smart(er) image categorization based on content) are useful tools, even though Dr Trigon might be the only one running them at the moment. I would really like to see things streamlined and all of the cruft removed and get back to the nice library that we had. Apart from the patch.exe issue, I', not really sure what there is to streamline - the extra externals are only installed on demand and the new scripts are not in the way of using other scripts. Merlijn ___ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l -- Amir ___ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l ___ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l -- Amir ___ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l
Re: [Pywikipedia-l] Cruft in compat
Hi John, On 23 August 2013 00:36, John phoenixoverr...@gmail.com wrote: The final straw was when I just converted to Git and discovered that he is trying to run executables. I think most of us agree using patch.exe was not the best decision. However, creating a way to install dependencies locally, which also works well for windows users - unfortunately, virtualenv and others don't work too well for some windows workflows, I think was a sensible idea. This is bloatware plain and simple I have seen the number of externals go from 2 to 17. Most of these are probably for one or two pet programs that should never have been added to the project as they are niche programs. I don't agree on this - the number of externals that /can be installed/ has increased, but the only one required is BeautifulSoup. This is not different from how things were before. Thanks to the new packaging system, you are only asked to install them when required. Yes, they are typically for one or two scripts, but calling them 'pet programs' is just nonsense - the scripts DrTrigon has added (e.g. sum_disc/discussion summaries, subster/dynamic page updating from an external source, catimages/smart(er) image categorization based on content) are useful tools, even though Dr Trigon might be the only one running them at the moment. I would really like to see things streamlined and all of the cruft removed and get back to the nice library that we had. Apart from the patch.exe issue, I', not really sure what there is to streamline - the extra externals are only installed on demand and the new scripts are not in the way of using other scripts. Merlijn ___ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l
Re: [Pywikipedia-l] Cruft in compat
Another point that is really frustrating is the logging system that complains about not having a logger defined if you run any custom scripts. The only way that I have found to shut this up is to enable logging for all scripts, however I end up with thousands of log files. (each pid creates its own file. and if you use muti-processing that ends up with thousands or hundreds of thousands of log files) Also It shoudnt be querying git on every script invocation or at all unless version.py is manually called. that kind of overhead is just bloat (with perhaps the exception of interwiki.py). if externals are needed and they need to be patched the best thing to to would include them pre-patched in our own sub-module, that way you can avoid the whole can of worms On Sat, Aug 24, 2013 at 10:14 AM, Merlijn van Deen valhall...@arctus.nlwrote: Hi John, On 23 August 2013 00:36, John phoenixoverr...@gmail.com wrote: The final straw was when I just converted to Git and discovered that he is trying to run executables. I think most of us agree using patch.exe was not the best decision. However, creating a way to install dependencies locally, which also works well for windows users - unfortunately, virtualenv and others don't work too well for some windows workflows, I think was a sensible idea. This is bloatware plain and simple I have seen the number of externals go from 2 to 17. Most of these are probably for one or two pet programs that should never have been added to the project as they are niche programs. I don't agree on this - the number of externals that /can be installed/ has increased, but the only one required is BeautifulSoup. This is not different from how things were before. Thanks to the new packaging system, you are only asked to install them when required. Yes, they are typically for one or two scripts, but calling them 'pet programs' is just nonsense - the scripts DrTrigon has added (e.g. sum_disc/discussion summaries, subster/dynamic page updating from an external source, catimages/smart(er) image categorization based on content) are useful tools, even though Dr Trigon might be the only one running them at the moment. I would really like to see things streamlined and all of the cruft removed and get back to the nice library that we had. Apart from the patch.exe issue, I', not really sure what there is to streamline - the extra externals are only installed on demand and the new scripts are not in the way of using other scripts. Merlijn ___ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l ___ Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l
