[Wikitech-l] wikitech migration still ongoing
Seems my previous updates on this may not have made it to this list, so here's another update: The wikitech dump was rather large and had a lot of revisions (60k revisions), even though it has a fairly small number of pages (1,600 pages). The dump was about 6.1GB. The vast majority of these revisions are from the Server Admin log. Importing this is taking quite a bit longer than anticipated. Until the import is complete, the wikitech documentation is still available at http://wikitech-old.wikimedia.org. When the import finishes, I'll be re-logging any !log messages from the channel that were missed. Until this happens, the !log messages can be found in the IRC channel logs: http://bots.wmflabs.org/~wm-bot/logs/%23wikimedia-operations/ - Ryan ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[Wikitech-l] switching to something better than irc.wikimedia.org
Hi, I think that irc feed of recent changes is working great, but there is still a lot of space for improvement. As Ryan Lane suggested once, we could probably use system of queues instead of irc which would be even more advanced. My suggestion is to create some kind of feed that would be in machine parseable format, like XML This feed would be distributed by some kind of dispatcher living on some server, like feed.wikimedia.org and offering not just recent changes but also a recent history (for example last 5000 changes per project) In case that service which is parsing this feed would be down for a moment, it could retrieve a backlog of changes. The current feed irc.wikimedia.org should stay, but we could change it so that the current bot is retrieving the data from new xml feed instead of directly from apaches. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] switching to something better than irc.wikimedia.org
Hey, It sounds like an interesting idea. Actually, AWS (I've been working with it recently for Extension:AWS) has a similar architecture, where you establish a push notification service using their Simple Notification Service and have it send messages to a queue using their Simple Queue Service. The difficulty in replacing IRC would be that, first of all, it would almost definitely have to be a push-based service, where the wiki would publish the message and the notification server would send out the recent change to all the subscribed clients. This begs the question of whether there's an existing piece of software that does this or whether this would require implementing a daemon in the form of a maintenance script that handle the job. *--* *Tyler Romeo* Stevens Institute of Technology, Class of 2015 Major in Computer Science www.whizkidztech.com | tylerro...@gmail.com On Fri, Mar 1, 2013 at 3:55 AM, Petr Bena benap...@gmail.com wrote: Hi, I think that irc feed of recent changes is working great, but there is still a lot of space for improvement. As Ryan Lane suggested once, we could probably use system of queues instead of irc which would be even more advanced. My suggestion is to create some kind of feed that would be in machine parseable format, like XML This feed would be distributed by some kind of dispatcher living on some server, like feed.wikimedia.org and offering not just recent changes but also a recent history (for example last 5000 changes per project) In case that service which is parsing this feed would be down for a moment, it could retrieve a backlog of changes. The current feed irc.wikimedia.org should stay, but we could change it so that the current bot is retrieving the data from new xml feed instead of directly from apaches. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] switching to something better than irc.wikimedia.org
I believe it would require to create a new daemon (preferably written in c++) which I am willing to write, that could do similar what the ircd does right now. And that is delivery of new change to all connected clients. There would be preferably set of processes that are working together on this system. Some kind of cache daemon that would contain the history for all projects, dispatcher that would handle requests of clients and retrieve the data from cache daemon and listener which would retrieve the UDP traffic from wiki's and forward them to cache daemon. We could of course create a multithreaded single process daemon as well, but that would make it little bit less stable, given that crash of any thread would bring down whole system. On Fri, Mar 1, 2013 at 1:26 PM, Tyler Romeo tylerro...@gmail.com wrote: Hey, It sounds like an interesting idea. Actually, AWS (I've been working with it recently for Extension:AWS) has a similar architecture, where you establish a push notification service using their Simple Notification Service and have it send messages to a queue using their Simple Queue Service. The difficulty in replacing IRC would be that, first of all, it would almost definitely have to be a push-based service, where the wiki would publish the message and the notification server would send out the recent change to all the subscribed clients. This begs the question of whether there's an existing piece of software that does this or whether this would require implementing a daemon in the form of a maintenance script that handle the job. *--* *Tyler Romeo* Stevens Institute of Technology, Class of 2015 Major in Computer Science www.whizkidztech.com | tylerro...@gmail.com On Fri, Mar 1, 2013 at 3:55 AM, Petr Bena benap...@gmail.com wrote: Hi, I think that irc feed of recent changes is working great, but there is still a lot of space for improvement. As Ryan Lane suggested once, we could probably use system of queues instead of irc which would be even more advanced. My suggestion is to create some kind of feed that would be in machine parseable format, like XML This feed would be distributed by some kind of dispatcher living on some server, like feed.wikimedia.org and offering not just recent changes but also a recent history (for example last 5000 changes per project) In case that service which is parsing this feed would be down for a moment, it could retrieve a backlog of changes. The current feed irc.wikimedia.org should stay, but we could change it so that the current bot is retrieving the data from new xml feed instead of directly from apaches. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] switching to something better than irc.wikimedia.org
We actually have an open RFC on this topic: https://www.mediawiki.org/wiki/Requests_for_comment/Structured_data_push_notification_support_for_recent_changes -- ~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://danielfriesen.name/] ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] switching to something better than irc.wikimedia.org
I see that the RFC is considering multiple formats, why not support all of them? We could make the client request the format they like, either XML or JSON, that would be a matter of dispatcher how it produce the output data. On Fri, Mar 1, 2013 at 2:35 PM, Daniel Friesen dan...@nadir-seen-fire.com wrote: We actually have an open RFC on this topic: https://www.mediawiki.org/wiki/Requests_for_comment/Structured_data_push_notification_support_for_recent_changes -- ~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://danielfriesen.name/] ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] switching to something better than irc.wikimedia.org
Because we made that mistake with the API, and now we're stuck with a bunch of deadweight formats that do nothing other than increase maintenance costs. If your first preference as a client developer is for JSON, it's really not that hard for you to go get a library to receive it in XML instead, or vice versa. That's the whole point of a standardised format. --HM On 1 March 2013 13:48, Petr Bena benap...@gmail.com wrote: I see that the RFC is considering multiple formats, why not support all of them? We could make the client request the format they like, either XML or JSON, that would be a matter of dispatcher how it produce the output data. On Fri, Mar 1, 2013 at 2:35 PM, Daniel Friesen dan...@nadir-seen-fire.com wrote: We actually have an open RFC on this topic: https://www.mediawiki.org/wiki/Requests_for_comment/Structured_data_push_notification_support_for_recent_changes -- ~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://danielfriesen.name/] ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] switching to something better than irc.wikimedia.org
The RFC doesn't seem to have gotten much interest (only a burst of edits from Krinkle in August and then it died). But interesting nonetheless. The one thing I do know is that if this were to be implemented, it would probably be pretty complex. It would have to support at least a couple of different push methods (I'd imagine IRC would be one of them for backwards compatibility) and would have to be efficient enough to handle the load of receiving and sending Wikipedia's recent changes. Like Petr said, the client would probably be in C++ or some other language that supports true multithreading and is efficient enough. At that point, it might as well be its own product, i.e., a generic push notification server similar to Amazon's SNS. I feel like such a project would take an insane amount of resources to develop. Between design, coding, and testing, we'd be lucky to see it implemented by MW 1.25 ;) Nonetheless, it sounds like a fun project, and if some developers would be interested in putting together a generic C++ push notification server, I'd be happy to help out. *--* *Tyler Romeo* Stevens Institute of Technology, Class of 2015 Major in Computer Science www.whizkidztech.com | tylerro...@gmail.com ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] switching to something better than irc.wikimedia.org
The problem is that while XML is widely accepted standard supported on all platforms and languages, JSON, even if it might be better, is not so well supported in this moment. For this reason I think it would be cool to be able to offer multiple outputs. In the end, as you said, it's not that hard to get a library which converts it from one to other, so why we can't use such a library on side of dispatcher instead of forcing developers of clients to seek this library for their language so that they can convert it On Fri, Mar 1, 2013 at 2:56 PM, Happy Melon happy.melon.w...@gmail.com wrote: Because we made that mistake with the API, and now we're stuck with a bunch of deadweight formats that do nothing other than increase maintenance costs. If your first preference as a client developer is for JSON, it's really not that hard for you to go get a library to receive it in XML instead, or vice versa. That's the whole point of a standardised format. --HM On 1 March 2013 13:48, Petr Bena benap...@gmail.com wrote: I see that the RFC is considering multiple formats, why not support all of them? We could make the client request the format they like, either XML or JSON, that would be a matter of dispatcher how it produce the output data. On Fri, Mar 1, 2013 at 2:35 PM, Daniel Friesen dan...@nadir-seen-fire.com wrote: We actually have an open RFC on this topic: https://www.mediawiki.org/wiki/Requests_for_comment/Structured_data_push_notification_support_for_recent_changes -- ~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://danielfriesen.name/] ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] switching to something better than irc.wikimedia.org
OK Inserted this to hackaton topics as well... On Fri, Mar 1, 2013 at 3:02 PM, Tyler Romeo tylerro...@gmail.com wrote: The RFC doesn't seem to have gotten much interest (only a burst of edits from Krinkle in August and then it died). But interesting nonetheless. The one thing I do know is that if this were to be implemented, it would probably be pretty complex. It would have to support at least a couple of different push methods (I'd imagine IRC would be one of them for backwards compatibility) and would have to be efficient enough to handle the load of receiving and sending Wikipedia's recent changes. Like Petr said, the client would probably be in C++ or some other language that supports true multithreading and is efficient enough. At that point, it might as well be its own product, i.e., a generic push notification server similar to Amazon's SNS. I feel like such a project would take an insane amount of resources to develop. Between design, coding, and testing, we'd be lucky to see it implemented by MW 1.25 ;) Nonetheless, it sounds like a fun project, and if some developers would be interested in putting together a generic C++ push notification server, I'd be happy to help out. *--* *Tyler Romeo* Stevens Institute of Technology, Class of 2015 Major in Computer Science www.whizkidztech.com | tylerro...@gmail.com ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] switching to something better than irc.wikimedia.org
I have not yet found a good and stable library for JSON parsing in c#, should you know some let me know :) However, I disagree with I feel like such a project would take an insane amount of resources to develop. If we wouldn't make it insanely complicated, it won't take insane amount of time ;). The cache daemon could be memcached which is already written and stable. Listener is a simple daemon that just listen in UDP, parse the data from mediawiki and store them in memcached in some universal format, and dispatcher is just process that takes the data from cache, convert them to specified format and send them to client. Sounds easy ;) On Fri, Mar 1, 2013 at 3:10 PM, Tyler Romeo tylerro...@gmail.com wrote: On Fri, Mar 1, 2013 at 9:04 AM, Petr Bena benap...@gmail.com wrote: The problem is that while XML is widely accepted standard supported on all platforms and languages, JSON, even if it might be better, is not so well supported in this moment. For this reason I think it would be cool to be able to offer multiple outputs. JSON is *very* widely supported in almost every language. It would definitely not be a problem if we only supported JSON. Furthermore, JSON represents data using only native types, all of which are existent in PHP. In other words, a PHP/C++/etc. variable can be directly serialized into JSON, whereas in XML this is very much not the case due to attributes and the ability to have child elements of different types, making it much more difficult to implement. *--* *Tyler Romeo* Stevens Institute of Technology, Class of 2015 Major in Computer Science www.whizkidztech.com | tylerro...@gmail.com ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] switching to something better than irc.wikimedia.org
I still don't see it as too much complex. Matter of month(s) for volunteers with limited time. However I quite don't see what is so complicated on last 2 points. Given the frequency of updates it's most simple to have the client (user / bot / service that need to read the feed) open the persistent connection to server (dispatcher) which fork itself just as sshd does and the new process handle all requests from this client. The client somehow specify what kind of feed they want to have (that's the registration part) and forked dispatcher keeps it updated with information from cache. Nothing hard. And what's the problem with multithreading huh? :) BTW I don't really think there is a need for multithreading at all, but even if there was, it shouldn't be so hard. On Fri, Mar 1, 2013 at 3:47 PM, Tyler Romeo tylerro...@gmail.com wrote: On Fri, Mar 1, 2013 at 9:16 AM, Petr Bena benap...@gmail.com wrote: I have not yet found a good and stable library for JSON parsing in c#, should you know some let me know :) Take a look at http://www.json.org/. They have a list of implementations for different languages. However, I disagree with I feel like such a project would take an insane amount of resources to develop. If we wouldn't make it insanely complicated, it won't take insane amount of time ;). The cache daemon could be memcached which is already written and stable. Listener is a simple daemon that just listen in UDP, parse the data from mediawiki and store them in memcached in some universal format, and dispatcher is just process that takes the data from cache, convert them to specified format and send them to client. Here's a quick list of things that are basic requirements we'd have to implement: - Multi-threading, which is in and of itself a pain in the a**. - Some sort of queue for messages, rather than hoping the daemon can send out every message in realtime. - Ability for clients to register with the daemon (and a place to store a client list) - Multiple methods of notification (IRC would be one, XMPP might be a candidate, and a simple HTTP endpoint would be a must). Just those basics isn't an easy task, especially considering unless WMF allocates resources to it the project would be run solely by those who have enough free time. Also, I wouldn't use memcached as a caching daemon, primarily because I'm not sure such an application even needs a caching daemon. All it does is relay messages. *--* *Tyler Romeo* Stevens Institute of Technology, Class of 2015 Major in Computer Science www.whizkidztech.com | tylerro...@gmail.com ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] switching to something better than irc.wikimedia.org
0mq? RabbitMQ? Seem to fit the use case pretty well / closely. -- Yuvi Panda T http://yuvi.in/blog ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] switching to something better than irc.wikimedia.org
Closely, but seems a bit overcomplicated to me. What I proposed is as simple as you could just use telnet to retrieve the last changes. In rabbitMQ for example you need to use 3rd libraries for client so that you can connect to server and obtain some data... But I don't have a problem with using anything that already works and is fast and stable. Just please let's make it better than what we have now (making it worse would be no fun :P) On Fri, Mar 1, 2013 at 5:36 PM, Yuvi Panda yuvipa...@gmail.com wrote: 0mq? RabbitMQ? Seem to fit the use case pretty well / closely. -- Yuvi Panda T http://yuvi.in/blog ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] switching to something better than irc.wikimedia.org
On Fri, Mar 1, 2013 at 12:55 AM, Petr Bena benap...@gmail.com wrote: Hi, I think that irc feed of recent changes is working great, but there is still a lot of space for improvement. As Ryan Lane suggested once, we could probably use system of queues instead of irc which would be even more advanced. My suggestion is to create some kind of feed that would be in machine parseable format, like XML This feed would be distributed by some kind of dispatcher living on some server, like feed.wikimedia.org and offering not just recent changes but also a recent history (for example last 5000 changes per project) In case that service which is parsing this feed would be down for a moment, it could retrieve a backlog of changes. The current feed irc.wikimedia.org should stay, but we could change it so that the current bot is retrieving the data from new xml feed instead of directly from apaches. There's been a request for years to provide this data with XMPP. https://bugzilla.wikimedia.org/17450 https://bugzilla.wikimedia.org/30555 also seems related to the RFC. -Chad ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] switching to something better than irc.wikimedia.org
I don't think a custom daemon would actually be needed. http://redis.io/topics/pubsub While I was at flickr, we implemented a pubsub based system to push notifications of all photo uploads and metadata changes to google using redis as the backend. The rate of uploads and edits at flickr in 2010 was orders of magnitude greater than the rate of edits across all wmf projects. Publishing to a redis pubsub channel does grow in cost as the number of subscribers increases but I don't see a problem at our scale. If so, there are ways around it. We are planning on migrating the wiki job queues from mysql to redis in the next few weeks, so it's already a growing piece of our infrastructure. I think the bulk of the work here would actually just be in building a frontend webservice that supports websockets / long polling, provides a clean api, and preferably uses oauth or some form of registration to ward off abuse and allow us to limit the growth of subscribers as we scale. On Friday, March 1, 2013, Petr Bena wrote: I still don't see it as too much complex. Matter of month(s) for volunteers with limited time. However I quite don't see what is so complicated on last 2 points. Given the frequency of updates it's most simple to have the client (user / bot / service that need to read the feed) open the persistent connection to server (dispatcher) which fork itself just as sshd does and the new process handle all requests from this client. The client somehow specify what kind of feed they want to have (that's the registration part) and forked dispatcher keeps it updated with information from cache. Nothing hard. And what's the problem with multithreading huh? :) BTW I don't really think there is a need for multithreading at all, but even if there was, it shouldn't be so hard. On Fri, Mar 1, 2013 at 3:47 PM, Tyler Romeo tylerro...@gmail.comjavascript:; wrote: On Fri, Mar 1, 2013 at 9:16 AM, Petr Bena benap...@gmail.comjavascript:; wrote: I have not yet found a good and stable library for JSON parsing in c#, should you know some let me know :) Take a look at http://www.json.org/. They have a list of implementations for different languages. However, I disagree with I feel like such a project would take an insane amount of resources to develop. If we wouldn't make it insanely complicated, it won't take insane amount of time ;). The cache daemon could be memcached which is already written and stable. Listener is a simple daemon that just listen in UDP, parse the data from mediawiki and store them in memcached in some universal format, and dispatcher is just process that takes the data from cache, convert them to specified format and send them to client. Here's a quick list of things that are basic requirements we'd have to implement: - Multi-threading, which is in and of itself a pain in the a**. - Some sort of queue for messages, rather than hoping the daemon can send out every message in realtime. - Ability for clients to register with the daemon (and a place to store a client list) - Multiple methods of notification (IRC would be one, XMPP might be a candidate, and a simple HTTP endpoint would be a must). Just those basics isn't an easy task, especially considering unless WMF allocates resources to it the project would be run solely by those who have enough free time. Also, I wouldn't use memcached as a caching daemon, primarily because I'm not sure such an application even needs a caching daemon. All it does is relay messages. *--* *Tyler Romeo* Stevens Institute of Technology, Class of 2015 Major in Computer Science www.whizkidztech.com | tylerro...@gmail.com javascript:; ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org javascript:; https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org javascript:; https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] switching to something better than irc.wikimedia.org
web frontend you say? if you compare the raw data of irc protocol (1 rc feed message) and raw data of a http request and response for one page consisting only of that 1 rc feed message, you will see a huge difference in size and performance. Also all kinds of authentication required doesn't seem like an improvement to me. It will only complicate what is simple now. Have there been many attempts to abuse irc.wikimedia.org so far? there is no authentication at all. On Fri, Mar 1, 2013 at 5:46 PM, Asher Feldman afeld...@wikimedia.org wrote: I don't think a custom daemon would actually be needed. http://redis.io/topics/pubsub While I was at flickr, we implemented a pubsub based system to push notifications of all photo uploads and metadata changes to google using redis as the backend. The rate of uploads and edits at flickr in 2010 was orders of magnitude greater than the rate of edits across all wmf projects. Publishing to a redis pubsub channel does grow in cost as the number of subscribers increases but I don't see a problem at our scale. If so, there are ways around it. We are planning on migrating the wiki job queues from mysql to redis in the next few weeks, so it's already a growing piece of our infrastructure. I think the bulk of the work here would actually just be in building a frontend webservice that supports websockets / long polling, provides a clean api, and preferably uses oauth or some form of registration to ward off abuse and allow us to limit the growth of subscribers as we scale. On Friday, March 1, 2013, Petr Bena wrote: I still don't see it as too much complex. Matter of month(s) for volunteers with limited time. However I quite don't see what is so complicated on last 2 points. Given the frequency of updates it's most simple to have the client (user / bot / service that need to read the feed) open the persistent connection to server (dispatcher) which fork itself just as sshd does and the new process handle all requests from this client. The client somehow specify what kind of feed they want to have (that's the registration part) and forked dispatcher keeps it updated with information from cache. Nothing hard. And what's the problem with multithreading huh? :) BTW I don't really think there is a need for multithreading at all, but even if there was, it shouldn't be so hard. On Fri, Mar 1, 2013 at 3:47 PM, Tyler Romeo tylerro...@gmail.comjavascript:; wrote: On Fri, Mar 1, 2013 at 9:16 AM, Petr Bena benap...@gmail.comjavascript:; wrote: I have not yet found a good and stable library for JSON parsing in c#, should you know some let me know :) Take a look at http://www.json.org/. They have a list of implementations for different languages. However, I disagree with I feel like such a project would take an insane amount of resources to develop. If we wouldn't make it insanely complicated, it won't take insane amount of time ;). The cache daemon could be memcached which is already written and stable. Listener is a simple daemon that just listen in UDP, parse the data from mediawiki and store them in memcached in some universal format, and dispatcher is just process that takes the data from cache, convert them to specified format and send them to client. Here's a quick list of things that are basic requirements we'd have to implement: - Multi-threading, which is in and of itself a pain in the a**. - Some sort of queue for messages, rather than hoping the daemon can send out every message in realtime. - Ability for clients to register with the daemon (and a place to store a client list) - Multiple methods of notification (IRC would be one, XMPP might be a candidate, and a simple HTTP endpoint would be a must). Just those basics isn't an easy task, especially considering unless WMF allocates resources to it the project would be run solely by those who have enough free time. Also, I wouldn't use memcached as a caching daemon, primarily because I'm not sure such an application even needs a caching daemon. All it does is relay messages. *--* *Tyler Romeo* Stevens Institute of Technology, Class of 2015 Major in Computer Science www.whizkidztech.com | tylerro...@gmail.com javascript:; ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org javascript:; https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org javascript:; https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org
Re: [Wikitech-l] wikitech migration still ongoing
Migration is done and morebots is logging again. I still need to re-log past actions, but I'll do that soon. Let me know if you see any migration issues. On Fri, Mar 1, 2013 at 12:08 AM, Ryan Lane rlan...@gmail.com wrote: Seems my previous updates on this may not have made it to this list, so here's another update: The wikitech dump was rather large and had a lot of revisions (60k revisions), even though it has a fairly small number of pages (1,600 pages). The dump was about 6.1GB. The vast majority of these revisions are from the Server Admin log. Importing this is taking quite a bit longer than anticipated. Until the import is complete, the wikitech documentation is still available at http://wikitech-old.wikimedia.org. When the import finishes, I'll be re-logging any !log messages from the channel that were missed. Until this happens, the !log messages can be found in the IRC channel logs: http://bots.wmflabs.org/~wm-bot/logs/%23wikimedia-operations/ - Ryan ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] switching to something better than irc.wikimedia.org
On Fri, Mar 1, 2013 at 11:46 AM, Asher Feldman afeld...@wikimedia.orgwrote: don't think a custom daemon would actually be needed. http://redis.io/topics/pubsub While I was at flickr, we implemented a pubsub based system to push notifications of all photo uploads and metadata changes to google using redis as the backend. The rate of uploads and edits at flickr in 2010 was orders of magnitude greater than the rate of edits across all wmf projects. Publishing to a redis pubsub channel does grow in cost as the number of subscribers increases but I don't see a problem at our scale. If so, there are ways around it. We are planning on migrating the wiki job queues from mysql to redis in the next few weeks, so it's already a growing piece of our infrastructure. I think the bulk of the work here would actually just be in building a frontend webservice that supports websockets / long polling, provides a clean api, and preferably uses oauth or some form of registration to ward off abuse and allow us to limit the growth of subscribers as we scale. Interesting. Didn't know Redis had something like this. I'm not too knowledgeable about Redis, but would clients be able to subscribe directly to Redis queues? Or would that be a security issue (like allowing people to access Memcached would be) and we would have to implement our own notification service anyway? 0mq? RabbitMQ? Seem to fit the use case pretty well / closely. Hmm, I've always only thought of RabbitMQ as a messaging service between linked applications, but I guess it could be used as a type of push notification service as well. *--* *Tyler Romeo* Stevens Institute of Technology, Class of 2015 Major in Computer Science www.whizkidztech.com | tylerro...@gmail.com ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] switching to something better than irc.wikimedia.org
On Friday, March 1, 2013, Petr Bena wrote: web frontend you say? if you compare the raw data of irc protocol (1 rc feed message) and raw data of a http request and response for one page consisting only of that 1 rc feed message, you will see a huge difference in size and performance. I was sugesting it for websockets or a long poll, the above comparison isn't relevant. Connection is established, with its protocol overhead. It stays open and messages are continually pushed from the server. Not a web request for a page containing one rc message. Also all kinds of authentication required doesn't seem like an improvement to me. It will only complicate what is simple now. Have there been many attempts to abuse irc.wikimedia.org so far? there is no authentication at all. Maybe none is needed but I don't think the irc feed interests anyone outside of a very small community. Doing something a little more modern might attract different uses. It might not, but I have no idea. On Fri, Mar 1, 2013 at 5:46 PM, Asher Feldman afeld...@wikimedia.orgjavascript:; wrote: I don't think a custom daemon would actually be needed. http://redis.io/topics/pubsub While I was at flickr, we implemented a pubsub based system to push notifications of all photo uploads and metadata changes to google using redis as the backend. The rate of uploads and edits at flickr in 2010 was orders of magnitude greater than the rate of edits across all wmf projects. Publishing to a redis pubsub channel does grow in cost as the number of subscribers increases but I don't see a problem at our scale. If so, there are ways around it. We are planning on migrating the wiki job queues from mysql to redis in the next few weeks, so it's already a growing piece of our infrastructure. I think the bulk of the work here would actually just be in building a frontend webservice that supports websockets / long polling, provides a clean api, and preferably uses oauth or some form of registration to ward off abuse and allow us to limit the growth of subscribers as we scale. On Friday, March 1, 2013, Petr Bena wrote: I still don't see it as too much complex. Matter of month(s) for volunteers with limited time. However I quite don't see what is so complicated on last 2 points. Given the frequency of updates it's most simple to have the client (user / bot / service that need to read the feed) open the persistent connection to server (dispatcher) which fork itself just as sshd does and the new process handle all requests from this client. The client somehow specify what kind of feed they want to have (that's the registration part) and forked dispatcher keeps it updated with information from cache. Nothing hard. And what's the problem with multithreading huh? :) BTW I don't really think there is a need for multithreading at all, but even if there was, it shouldn't be so hard. On Fri, Mar 1, 2013 at 3:47 PM, Tyler Romeo tylerro...@gmail.comjavascript:; javascript:; wrote: On Fri, Mar 1, 2013 at 9:16 AM, Petr Bena benap...@gmail.comjavascript:; javascript:; wrote: I have not yet found a good and stable library for JSON parsing in c#, should you know some let me know :) Take a look at http://www.json.org/. They have a list of implementations for different languages. However, I disagree with I feel like such a project would take an insane amount of resources to develop. If we wouldn't make it insanely complicated, it won't take insane amount of time ;). The cache daemon could be memcached which is already written and stable. Listener is a simple daemon that just listen in UDP, parse the data from mediawiki and store them in memcached in some universal format, and dispatcher is just process that takes the data from cache, convert them to specified format and send them to client. Here's a quick list of things that are basic requirements we'd have to implement: - Multi-threading, which is in and of itself a pain in the a**. - Some sort of queue for messages, rather than hoping the daemon can send out every message in realtime. - Ability for clients to register with the daemon (and a place to store a client list) - Multiple methods of notification (IRC would be one, XMPP might be a candidate, and a simple HTTP endpoint would be a must). Just those basics isn't an easy task, especially considering unless WMF allocates resources to it the project would be run solely by those who have enough free time. Also, I wouldn't use memcached as a caching daemon, primarily because I'm not sure such an application even needs a caching daemon. All it does is relay messages. *--* *Tyler Romeo* Stevens Institute of Technology, Class of 2015 Major in Computer Science www.whizkidztech.com | tylerro...@gmail.com
Re: [Wikitech-l] switching to something better than irc.wikimedia.org
On Friday, March 1, 2013, Tyler Romeo wrote: On Fri, Mar 1, 2013 at 11:46 AM, Asher Feldman afeld...@wikimedia.orgjavascript:; wrote: don't think a custom daemon would actually be needed. http://redis.io/topics/pubsub While I was at flickr, we implemented a pubsub based system to push notifications of all photo uploads and metadata changes to google using redis as the backend. The rate of uploads and edits at flickr in 2010 was orders of magnitude greater than the rate of edits across all wmf projects. Publishing to a redis pubsub channel does grow in cost as the number of subscribers increases but I don't see a problem at our scale. If so, there are ways around it. We are planning on migrating the wiki job queues from mysql to redis in the next few weeks, so it's already a growing piece of our infrastructure. I think the bulk of the work here would actually just be in building a frontend webservice that supports websockets / long polling, provides a clean api, and preferably uses oauth or some form of registration to ward off abuse and allow us to limit the growth of subscribers as we scale. Interesting. Didn't know Redis had something like this. I'm not too knowledgeable about Redis, but would clients be able to subscribe directly to Redis queues? Or would that be a security issue (like allowing people to access Memcached would be) and we would have to implement our own notification service anyway? I think a very light weight proxy that only passes subscribe commands to redis would work. A read only redis slave could be provided but I don't think it includes a way to limit what commands clients can run, including administrative ones. I think we'd want a thin proxy layer in front anyways, to track and if necessary, selectively limit access. It could be very simple though. 0mq? RabbitMQ? Seem to fit the use case pretty well / closely. Hmm, I've always only thought of RabbitMQ as a messaging service between linked applications, but I guess it could be used as a type of push notification service as well. *--* *Tyler Romeo* Stevens Institute of Technology, Class of 2015 Major in Computer Science www.whizkidztech.com | tylerro...@gmail.com javascript:; ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org javascript:; https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[Wikitech-l] Updated Gerrit
Hi, We updated Gerrit this morning to 2.5.2-1506-g278aa9a. This is a minor update to fix a couple of issues we spotted in the previous upgrade. Downtime was less than 5 minutes. As always, please let me know or file a bug if you're having any problems. -Chad ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] switching to something better than irc.wikimedia.org
On 01/03/13 05:22, Petr Bena wrote: I believe it would require to create a new daemon (preferably written in c++) which I am willing to write, that could do similar what the ircd does right now. And that is delivery of new change to all connected clients. When this has been proposed in the past, XMPP has been a popular solution, and could be done without writing new daemons. Arbitrary XML can easily be embedded in it, like Google Wave. -- Tim Starling ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] switching to something better than irc.wikimedia.org
- Original Message - From: Tyler Romeo tylerro...@gmail.com I think a very light weight proxy that only passes subscribe commands to redis would work. A read only redis slave could be provided but I don't think it includes a way to limit what commands clients can run, including administrative ones. I think we'd want a thin proxy layer in front anyways, to track and if necessary, selectively limit access. It could be very simple though. Mhm, in that case this might be a viable solution. Dumb question: is the work ESR recently did on irker on-topic for this conversation, and did anyone know it existed? :-) Cheers, -- jra -- Jay R. Ashworth Baylink j...@baylink.com Designer The Things I Think RFC 2100 Ashworth Associates http://baylink.pitas.com 2000 Land Rover DII St Petersburg FL USA #natog +1 727 647 1274 ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] switching to something better than irc.wikimedia.org
On 01/03/13 06:04, Petr Bena wrote: The problem is that while XML is widely accepted standard supported on all platforms and languages, JSON, even if it might be better, is not so well supported in this moment. For this reason I think it would be cool to be able to offer multiple outputs. XMPP is XML. If you embed JSON inside it, you are requiring two parsers instead of one, and unnecessarily increasing the complexity. -- Tim Starling ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] switching to something better than irc.wikimedia.org
On 03/01/2013 03:55 AM, Petr Bena wrote: As Ryan Lane suggested once, we could probably use system of queues instead of irc which would be even more advanced. My suggestion is to create some kind of feed that would be in machine parseable format, like XML Whatever you do, please, don't use XML. Whenever you use XML, a unicorn dies a painful death. Think of the unicorns. Whatever we end up with, I believe it should use JSON. Or at least YAML, though it's not as widely supported as JSON. Also, we will need a WebSocket interface in addition to whatever other protocol we will have. -- Victor. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] HTTPS Login
On Thu, Feb 28, 2013 at 8:28 PM, Tyler Romeo tylerro...@gmail.com wrote: Is there anything else that is holding up enabling wgSecureLogin on WMF wikis? It'd be spectacular if we could actually enable it and take another step in the right direction for security. If there's anything on the software end that needs to be fixed before it can be turned on, I'd be happy to help resolve it as soon as possible. Ryan and I have discussed this a few times, and we're both eager to flip the switch. Ops (and my own platform team) have been swamped this week with big meetings, but ever since the last sites were upgraded to wmf10 on Wed, we're ready to go from my perspective! ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[Wikitech-l] Reminder about the best way to link to bugs in commits
Hi, This is a friendly reminder to everyone about the preferred way to link to bugs in your commit messages. When you include them as part of the footer, they are indexed and are thus searchable. For example: Fixing some weird bug More explanation Blah blah blah. Bug: 1234 Change-Id: Ia90. So when you do this, you're able to search for bug:1234 via Gerrit. By doing this, you're also removing it from the first line (which was our old habit, mostly from SVN days), providing you more space to be descriptive in that first line. -Chad ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Reminder about the best way to link to bugs in commits
I actually prefer bug numbers in the header. When looking at the gerrit dashboard it's useful to see from the commit message whether it is a bug or enhancement to determine what code I should review first (bugs always win). I also tend to write very verbose commit messages so I like to put the bug in the header so the message itself can be ignored if necessary. Apologies in advanced if I have started another bike shed conversation.. :D On Fri, Mar 1, 2013 at 1:46 PM, Chad innocentkil...@gmail.com wrote: Hi, This is a friendly reminder to everyone about the preferred way to link to bugs in your commit messages. When you include them as part of the footer, they are indexed and are thus searchable. For example: Fixing some weird bug More explanation Blah blah blah. Bug: 1234 Change-Id: Ia90. So when you do this, you're able to search for bug:1234 via Gerrit. By doing this, you're also removing it from the first line (which was our old habit, mostly from SVN days), providing you more space to be descriptive in that first line. -Chad ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l -- Jon Robson http://jonrobson.me.uk @rakugojon ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[Wikitech-l] Deployment Highlights - 2013-03-01
Hello, This is your weekly preview of higher-risk or general you should be aware of items for the slew of deployments coming in the near term. == Next week == * Monday March 4: first deploy of -wmf11 to production WMF cluster ** General MediaWiki deploy schedule for reference: https://www.mediawiki.org/wiki/MediaWiki_1.21/Roadmap#Schedule_for_the_deployments * Wednesday March 6th: Wikidata Clients to all Wikipedias == Near Term (Planned, may change) == * March 13th: Scribunto (Lua) to all Wikis * March 27th: Wikidata phase II to Hungarian, Hebrew, Italian Wikipedias Best, Greg -- | Greg GrossmeierGPG: B2FA 27B1 F7EB D327 6B8E | | identi.ca: @gregA18D 1138 8E47 FAC8 1C7D | ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Reminder about the best way to link to bugs in commits
On Fri, Mar 1, 2013 at 2:20 PM, Jon Robson jdlrob...@gmail.com wrote: I actually prefer bug numbers in the header. +1, also useful for release notes. Could the footer line be auto-generated for indexing purposes? Yay for bikeshed topics ;-) -- Erik Möller VP of Engineering and Product Development, Wikimedia Foundation Support Free Knowledge: https://wikimediafoundation.org/wiki/Donate ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Reminder about the best way to link to bugs in commits
Well, that's true if you're only building release notes by copy+pasting the first line. If it's scripted, it's trivial to pull the bug # from the footer as well. And no, commit messages cannot be auto-generated by Gerrit, as that changes the sha1. -Chad On Fri, Mar 1, 2013 at 2:23 PM, Erik Moeller e...@wikimedia.org wrote: On Fri, Mar 1, 2013 at 2:20 PM, Jon Robson jdlrob...@gmail.com wrote: I actually prefer bug numbers in the header. +1, also useful for release notes. Could the footer line be auto-generated for indexing purposes? Yay for bikeshed topics ;-) -- Erik Möller VP of Engineering and Product Development, Wikimedia Foundation Support Free Knowledge: https://wikimediafoundation.org/wiki/Donate ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Reminder about the best way to link to bugs in commits
The proposalhttp://www.mediawiki.org/wiki/Requests_for_comment/RELEASE-NOTES_botis for a bot to parse commit message for special commands to add some text to specific sections of the release-notes file. When bot detects a master merge, it will pull the latest release-notes, change it, and merge it to master right away, avoiding any conflicts. If the bot messes up, or if a more complex file edit is needed, we can do it through the regular git/gerrit process. On Fri, Mar 1, 2013 at 5:28 PM, Chad innocentkil...@gmail.com wrote: Well, that's true if you're only building release notes by copy+pasting the first line. If it's scripted, it's trivial to pull the bug # from the footer as well. And no, commit messages cannot be auto-generated by Gerrit, as that changes the sha1. -Chad On Fri, Mar 1, 2013 at 2:23 PM, Erik Moeller e...@wikimedia.org wrote: On Fri, Mar 1, 2013 at 2:20 PM, Jon Robson jdlrob...@gmail.com wrote: I actually prefer bug numbers in the header. +1, also useful for release notes. Could the footer line be auto-generated for indexing purposes? Yay for bikeshed topics ;-) -- Erik Möller VP of Engineering and Product Development, Wikimedia Foundation Support Free Knowledge: https://wikimediafoundation.org/wiki/Donate ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Reminder about the best way to link to bugs in commits
On Sat, Mar 2, 2013 at 3:16 AM, Chad innocentkil...@gmail.com wrote: Hi, This is a friendly reminder to everyone about the preferred way to link to bugs in your commit messages. When you include them as part of the footer, they are indexed and are thus searchable. For example: Fixing some weird bug More explanation Blah blah blah. Bug: 1234 Change-Id: Ia90. So when you do this, you're able to search for bug:1234 via Gerrit. By doing this, you're also removing it from the first line (which was our old habit, mostly from SVN days), providing you more space to be descriptive in that first line. I also prefer it in the header. The bug report is the best description :) Is it not possible for Gerrit to search if its in the header? or make it so -Chad ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l -- Cheers, Nischay Nahata nischayn22.in ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Reminder about the best way to link to bugs in commits
On Fri, 01 Mar 2013 14:45:14 -0800, Nischay Nahata nischay...@gmail.com wrote: On Sat, Mar 2, 2013 at 3:16 AM, Chad innocentkil...@gmail.com wrote: Hi, This is a friendly reminder to everyone about the preferred way to link to bugs in your commit messages. When you include them as part of the footer, they are indexed and are thus searchable. For example: Fixing some weird bug More explanation Blah blah blah. Bug: 1234 Change-Id: Ia90. So when you do this, you're able to search for bug:1234 via Gerrit. By doing this, you're also removing it from the first line (which was our old habit, mostly from SVN days), providing you more space to be descriptive in that first line. I also prefer it in the header. The bug report is the best description :) Is it not possible for Gerrit to search if its in the header? or make it so +1 Tools should be coded around people. Not the other way around. -- ~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://danielfriesen.name/] ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[Wikitech-l] OpenID as a provider project page
I wrote up some quick documentation on OpenID as a provider. Feel free to modify it, especially for inaccurately used terminology. It's also likely a good time to start bikeshed discussions on the urls, as I think it'll end up taking a while to lock that down. https://www.mediawiki.org/wiki/OpenID_Provider - Ryan ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Reminder about the best way to link to bugs in commits
On Fri, Mar 1, 2013 at 2:59 PM, Daniel Friesen dan...@nadir-seen-fire.com wrote: On Fri, 01 Mar 2013 14:45:14 -0800, Nischay Nahata nischay...@gmail.com wrote: On Sat, Mar 2, 2013 at 3:16 AM, Chad innocentkil...@gmail.com wrote: Hi, This is a friendly reminder to everyone about the preferred way to link to bugs in your commit messages. When you include them as part of the footer, they are indexed and are thus searchable. For example: Fixing some weird bug More explanation Blah blah blah. Bug: 1234 Change-Id: Ia90. So when you do this, you're able to search for bug:1234 via Gerrit. By doing this, you're also removing it from the first line (which was our old habit, mostly from SVN days), providing you more space to be descriptive in that first line. I also prefer it in the header. The bug report is the best description :) Is it not possible for Gerrit to search if its in the header? or make it so +1 Tools should be coded around people. Not the other way around. No, Gerrit cannot detect these in the header. Also, this is pretty standard Git-fu to include this sort of metadata in the footer of the commit. -Chad ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Reminder about the best way to link to bugs in commits
Tools should be coded around people. Not the other way around. Very nicely said Daniel :) ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Reminder about the best way to link to bugs in commits
On 03/01/2013 01:46 PM, Chad wrote: Hi, This is a friendly reminder to everyone about the preferred way to link to bugs in your commit messages. As specified at https://www.mediawiki.org/wiki/Gerrit/Commit_message_guidelines Eyes on this page edits (if needed) are welcome. When you include them as part of the footer, they are indexed and are thus searchable. For example: Fixing some weird bug More explanation Blah blah blah. Bug: 1234 Change-Id: Ia90. So when you do this, you're able to search for bug:1234 via Gerrit. By doing this, you're also removing it from the first line (which was our old habit, mostly from SVN days), providing you more space to be descriptive in that first line. -- Quim Gil Technical Contributor Coordinator @ Wikimedia Foundation http://www.mediawiki.org/wiki/User:Qgil ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Reminder about the best way to link to bugs in commits
Daniel Friesen dan...@nadir-seen-fire.com wrote: This is a friendly reminder to everyone about the preferred way to link to bugs in your commit messages. When you include them as part of the footer, they are indexed and are thus searchable. For example: Fixing some weird bug More explanation Blah blah blah. Bug: 1234 Change-Id: Ia90. So when you do this, you're able to search for bug:1234 via Gerrit. By doing this, you're also removing it from the first line (which was our old habit, mostly from SVN days), providing you more space to be descriptive in that first line. I also prefer it in the header. The bug report is the best description :) Is it not possible for Gerrit to search if its in the header? or make it so +1 Tools should be coded around people. Not the other way around. *Argl* :-) May I repeat my question from http://article.gmane.org/gmane.science.linguistics.wikipedia.technical/66432: | Is there another software project that uses the summary line | in a similar way to MediaWiki? Tim ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Subclassing HTMLForm Question
Tyler left out a critical piece of information: This new subclass is for constructing a matrix of checkbox options in a form. You feed it a list of row labels and a list of column labels and it constructs a table of checkbox options: Category 1 Category 2 Category 3 Option 1 [ ] [ ] [ ] Option 2 [ ] [ ] [ ] Option 3 [ ] [ ] [ ] The suggestion that this should be constructed with divs (or no formatting at all) rather than a table is silly. The setDisplayFormat function in HTMLForm is for setting the formatting of the form itself, not dictating what HTML can and can't be use in the individual elements of the form. The description of the function reads: Set format in which to display the form. If the display format is set to div, the new subclass put the element in a div. If the display format is set to table, it puts it in a table. If the display format is set to raw, it doesn't put the element in anything. This is identical to the functionality of every existing subclass in HTMLForm. The only difference is that the form element itself happens to be a table rather than an input. Blocking this feature simply because it uses a table doesn't make sense to me. The table doesn't cause any problems in any actual output formats. I've tried it in MobileFrontend (with the form set to raw mode), for example, and it looks fine. I think the purpose of the setDisplayFormat function is being misunderstood in this case. If anyone is familiar with HTMLForm, please feel free to weight in at https://gerrit.wikimedia.org/r/#/c/48995/. Ryan Kaldari On 2/21/13 2:41 PM, Jeroen De Dauw wrote: Hey, The new subclass will always output a table for the getInputHTML(), even if the parent form is set to div/raw output. That very much sounds like a violation of the Liskov substitution pattern. In other words, it sounds like code using an instance of HTMLForm might run into problems when this instance happens to be of the type defined by the new subclass. Cheers -- Jeroen De Dauw http://www.bn2vs.com Don't panic. Don't be evil. -- ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
