Re: [SMW-devel] SMW roadmap: how to get new items on it
In reply to the original email and the suggestions that followed: (1) I agree that it would be good if users could easily specify feature requests, and if there was some way of finding out how much each of them is wanted. (2) This could also inspire new extension projects. Users may not know what fits into SMW or into any other extension, so it would be good to have a central mechanism for collecting ideas. (3) I agree that Bugzilla-like organization of information makes sense, but that Bugzilla is not fully suitable. Using some templates and forms, one could easily copy the Bugzilla structure without its restrictions. This has been done, e.g. by Ryan Lane in an SMW-based trouble ticket system. semantic-mediawiki.org would be a good place for this. (4) Such a setup would connect to our recent discussion about collecting interesting implementation tasks. Features are always more fun to implement than bug fixes, and a well-organised list of features could be a starting point for new contributors. (5) It is never easy to measure how important a feature is. We cannot expect such broad interest that we really get representative opinions for our whole community. But a wiki system could have links to user pages to record support. (6) The Roadmap is not immediately related to this. It is a list of things that someone wants to do, not of things that someone wants to be done. It is organised by extension (cf. (2)). The process of editing it is to talk to the maintainer of the respective extension. So my proposal is to set up a form-based feature requesting system and to see how it goes. One or two people are needed to create and maintain this system; please step forward. It does not need to be perfect initially. We may need only a few form fields: title, description, related extension (or unknown/new), maybe a list of related bug reports on Bugzilla, status of the request. Comments could be implemented with a subform (list of items; easy to delete accidentally) or on extra pages (pulled in with a query; cooler, but needs another add comment form that autogenerates a page name and links to the original request). The description should be collaboratively edited (i.e. not owned by the user who reported it), the comments should not. Could/should LiquidThreads be used for having comments below a page instead of on the talk page? To me it seems that this could be an exciting approach for many, especially smaller software projects, so we might actually set standards doing this. We can also directly link such facilities from SMWAdmin to get more users involved. Markus On 24/02/2011 19:29, Jeroen De Dauw wrote: Hey, Following the recent discussions about the roadmap, I'm asking myself if there is no need for a process to get new items on it. I have several idea's that I think would be nice to implement, but not maybe not everyone agrees they should be. I think ideally there should be some place to request features with something that allows the community to categorize these by demand. And then a mechanism to check if the features with high demand make sense to put on the roadmap, and find out where to best put them. Some people might think this is overkill since there is very little community involvement with these things right now, but I think this is in part caused by the current way of doing things itself. Right now I can either create a page somewhere with a list of stuff I want to have implemented, that no one will ever seriously look at, or just place things directly onto the roadmap. I'm not sure what would be a good approach here, but it's probably a good idea to have a look at how other communities are managing this. I'd be very interested in other peoples thoughts (and suggestions) on this. Cheers -- Jeroen De Dauw http://www.bn2vs.com Don't panic. Don't be evil. -- -- Free Software Download: Index, Search Analyze Logs and other IT data in Real-Time with Splunk. Collect, index and harness all the fast moving IT data generated by your applications, servers and devices whether physical, virtual or in the cloud. Deliver compliance at lower cost and gain new business insights. http://p.sf.net/sfu/splunk-dev2dev ___ Semediawiki-devel mailing list Semediawiki-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/semediawiki-devel -- Free Software Download: Index, Search Analyze Logs and other IT data in Real-Time with Splunk. Collect, index and harness all the fast moving IT data generated by your applications, servers and devices whether physical, virtual or in the cloud. Deliver compliance at lower cost and gain new business insights. http://p.sf.net/sfu/splunk-dev2dev
Re: [SMW-devel] SMW roadmap: how to get new items on it
Hi, I'd volunteer to help set up a SMW based trouble ticket system. I cannot promise to put in effort to constantly maintain it afterwards, but I can help set it up in the next one or two weeks. I already have an account on semantic-mediawiki.org, so I could actually get going pretty soon. Can you maybe install SemanticFormsInputs as it might be handy. Would it be possible for me to geht admin rights so I can delete or move a page if I need to? regards, Bernhard -- Mag. Bernhard Krabina KDZ - Zentrum für Verwaltungsforschung Centre for Public Administration Research Guglgasse 13, 1110 Wien Telefon: +43 1 8923492-27, Fax: +43 1 8923492-20 Mobil: +43 676 849579-27 mailto:krab...@kdz.or.at - http://www.kdz.or.at - Plattform Verwaltungskooperation: http://www.verwaltungskooperation.at - Open Government Data Konferenz 2011: http://www.ogd2011.at KDZ News: RISER erhält Datenschutz-Gütesiegel EuroPriSe http://bit.ly/h8m0pQ - Ursprüngliche Mail - In reply to the original email and the suggestions that followed: (1) I agree that it would be good if users could easily specify feature requests, and if there was some way of finding out how much each of them is wanted. (2) This could also inspire new extension projects. Users may not know what fits into SMW or into any other extension, so it would be good to have a central mechanism for collecting ideas. (3) I agree that Bugzilla-like organization of information makes sense, but that Bugzilla is not fully suitable. Using some templates and forms, one could easily copy the Bugzilla structure without its restrictions. This has been done, e.g. by Ryan Lane in an SMW-based trouble ticket system. semantic-mediawiki.org would be a good place for this. (4) Such a setup would connect to our recent discussion about collecting interesting implementation tasks. Features are always more fun to implement than bug fixes, and a well-organised list of features could be a starting point for new contributors. (5) It is never easy to measure how important a feature is. We cannot expect such broad interest that we really get representative opinions for our whole community. But a wiki system could have links to user pages to record support. (6) The Roadmap is not immediately related to this. It is a list of things that someone wants to do, not of things that someone wants to be done. It is organised by extension (cf. (2)). The process of editing it is to talk to the maintainer of the respective extension. So my proposal is to set up a form-based feature requesting system and to see how it goes. One or two people are needed to create and maintain this system; please step forward. It does not need to be perfect initially. We may need only a few form fields: title, description, related extension (or unknown/new), maybe a list of related bug reports on Bugzilla, status of the request. Comments could be implemented with a subform (list of items; easy to delete accidentally) or on extra pages (pulled in with a query; cooler, but needs another add comment form that autogenerates a page name and links to the original request). The description should be collaboratively edited (i.e. not owned by the user who reported it), the comments should not. Could/should LiquidThreads be used for having comments below a page instead of on the talk page? To me it seems that this could be an exciting approach for many, especially smaller software projects, so we might actually set standards doing this. We can also directly link such facilities from SMWAdmin to get more users involved. Markus On 24/02/2011 19:29, Jeroen De Dauw wrote: Hey, Following the recent discussions about the roadmap, I'm asking myself if there is no need for a process to get new items on it. I have several idea's that I think would be nice to implement, but not maybe not everyone agrees they should be. I think ideally there should be some place to request features with something that allows the community to categorize these by demand. And then a mechanism to check if the features with high demand make sense to put on the roadmap, and find out where to best put them. Some people might think this is overkill since there is very little community involvement with these things right now, but I think this is in part caused by the current way of doing things itself. Right now I can either create a page somewhere with a list of stuff I want to have implemented, that no one will ever seriously look at, or just place things directly onto the roadmap. I'm not sure what would be a good approach here, but it's probably a good idea to have a look at how other communities are managing this. I'd be very interested in other peoples thoughts (and suggestions) on this. Cheers -- Jeroen De Dauw http://www.bn2vs.com Don't panic. Don't be evil. --
Re: [SMW-devel] documentation, entry points, interface classes
Thank you very much, Markus! Architecture guide provides much help even being incomplete! I'll do my best to improve it after becoming more proficient with SMW development. Speaking about examples, I think that the best documentation in the world is Qt Assistant in the Qt framework [1] - and because of this the library is so much popular among C++ and Python GUI developers. Each class of the library has the examples of its usage; they also provide rich (but comprehensive) example applications where one can see the way many Qt-classes interact. The SMW community can add useful examples of solving typical tasks on the official SMW wiki - I'm sure it improves the documentation very much. [1] http://doc.qt.nokia.com/4.6/index.html On Thu, Feb 24, 2011 at 1:40 PM, Markus Krötzsch mar...@semantic-mediawiki.org wrote: On 24/02/2011 09:41, Yury Katkov wrote: Hello everyone! I'm trying to make Semantic MediaWiki extension and observing the documentation. I checked the Programmer Guide [1] and found that only the Doxygen documentation is currently available. Here are some questions: 1) Are there any very small example extensions available, whether or not some tutorials exist? 2) What are the main entry points to the code? For example, how can I add a numeric property Age with value 27 to the page User:Paul? Should I use SMWDataValueFactory? Or another example: what classes are to be used when I want to make an #ask query in my PHP code? 3) What classes are interfaces for extension developers and what classes belongs to the SMW core and should not be used by us? Hopefully after figuring out the answers to these questions I will able to improve the current documentation and it will make the life of other newbie extension developers easier. Hi Yury, by a happy coincidence I have just realized that we are lacking a good starting point for understanding SMW's architecture, and I have started to write a suitable documentation [1]. It is still incomplete, but I hope I can add further information within the next week. I agree that this is important to have, not only for helping people to get started but also to reduce the effort we have in aligning contributions with what we intend SMW to be. PHP is not very good for communicating architecture through code alone ;-) I also second your call for examples but there are so many ways in which to extend SMW that it is hard to come up with example code that is of general interest. Maybe we could build a collection of LocalSettings.php code snippets that show how some functions can be extended (e.g. how to register a new datatype, how to add a sidebar item that shows a property value for the given page, etc.) Regards Markus [1] http://semantic-mediawiki.org/wiki/Architecture_guide# -- Yury V. Katkov Laboratory of intelligent systems of the Saint-Petersburg National University of Information Technologies, Mechanics and Optics, Russia http://ailab.ifmo.ru -- Free Software Download: Index, Search Analyze Logs and other IT data in Real-Time with Splunk. Collect, index and harness all the fast moving IT data generated by your applications, servers and devices whether physical, virtual or in the cloud. Deliver compliance at lower cost and gain new business insights. http://p.sf.net/sfu/splunk-dev2dev ___ Semediawiki-devel mailing list Semediawiki-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
Re: [SMW-devel] SMW on large sites [Was: Roadmaps and getting and keeping devs]
On 23.02.2011 15:16, Markus Krötzsch wrote:
[Making this into a new thread]
Hi Krzysztof,
I was already wondering when I would hear from Wikia ...
As you have noticed, running SMW and extensions on large sites (large in
terms of content, or in terms of users) has special requirements.
Typically, we suggest to use more conservative settings for querying, so
that long and difficult queries do not occur. Similarly, some SMW
extensions have not been developed for large sites, and can be
problematic in their own right. But your users obviously want to keep
the features that they already have, so we need to find better ways of
addressing your problem.
But first we need to separate concerns a little bit. You mention the
following distinct problems:
(1) Too many DB writes (about 60% in total)
(2) Too many slow queries (about 90% from SMW)
Moreover, your problem is not caused by SMW alone but by a number of
SMW-related extensions. So there will be multiple issues that need
addressing to fix this, and maybe even in multiple extensions.
Let us first see how big the impact of the extensions you mention could
be. Semantic Forms mainly leads to some additional reads (apparently no
problem for you); the total number could possibly be reduced. It may
also have some effect on query activity if certain autocompletion
features are used. But otherwise I think it is unlikely to be the root
of the problem. Semantic Drilldown might be more of a problem regarding
complex queries. But it uses its own SQL queries, so it should be
possible to find out how much of (2) comes from this extension. Semantic
Drilldown should not contribute to (1).
Are there any other extensions that use SMW on your site?
We use:
- SemanticMediawiki
- SemanticDrilldown
- SemanticForms
- SemanticGallery
- SemanticResultFormats
- SemanticMaps
not all extensions are enabled to all wikis with SMW, most common
configuration is SemanticMediawiki + SemanticForms + SemanticDrilldown.
Regarding SMW, I have some concrete ideas on what could be done for (1)
and (2) but this will need more careful consideration first. I am
grateful if you can help to track down the cause of the problem, but I
am afraid that the changes in SMW core will still need to be done or at
least reviewed carefully by myself -- which makes me kind of a
bottleneck for the SMW part of your problem. I need to think about the
required work a little further before I can promise anything.
Of course.
My short term solution is to separate smw tables from 'regular' wiki
database.
There is only one condition to achieve that, smw tables can't join with
regular
tables. So far I didn't find any joins in current sources. It of course use
additional database connection but it's not problem for us. There are
some changes
but not sure if applicable for wider audience. To have separation for
database I use
wfGetDB( DB_MASTER|DB_SLAVE, 'smw' ) for semantic tables and wfGetDB(
DB_MASTER|DB_SLAVE) for local tables (like page or category).
Later we have our implementation of LBFactory_Multi which switch
connections based on groups parameter in wfGetDB. It would be nice if
SMWSQLStore2 class would have two static methods (or one parametrized),
in stock version they would be something like
public static getSMWDB( $type ) {
return wfGetDB( $type, 'smw' );
}
public static getLocalDB( $type ) {
return wfGetDB( $type );
}
then it would be easier to us merge our changes with upstream changes.
Regards,
eloy
Regards,
Markus
On 22/02/2011 22:38, Krzysztof Krzyżaniak wrote:
I think it's would be right place to jump in.
Hello, my name is Krzysztof Krzyżaniak a.k.a. eloy and I work for Wikia
Inc as backend team leader. We are probably (correct me if I am wrong)
on of the biggest user of Semantic Mediawiki suite. We currently have
enabled it on about 100 wikis for example on familypedia.wikia.com or
yugioh.wikia.com or www.wowwiki.com (but also on wikis which you
probably don't suspect for SMW interest like glee.wikia.com or
madmen.wikia.com). We would like to expand existence of SMW on Wikia
(for example lyrics would love it) but currently we cannot afford it
because of performance reasons. For example, our first cluster contains
about 30.000 wikis, mostly biggest ones. About 60% of writes in
databases came from SMW extensions (SemanticMediawiki,
SemanticDrilldown, SemanticForms), also about 90% queries from slow logs
are from SMW.
I am here to find a way for scaling SMW on our wikis. But also I think
that it will be benefit for every SMW user because we want to help
improve SMW.
What you can expect:
- real world cases, actually lot of them :)
- bugs :) (filled in bugzilla of course)
- bug fixes and patches (either as diff or direct svn commits if you
prefer that way)
- questions
We can offer engineering
Re: [SMW-devel] SMW roadmap: how to get new items on it
All this talk about using SMW and SF reminded me that smw.referata.comalready has such a thing, here: http://smw.referata.com/wiki/Category:Feature_requests http://smw.referata.com/wiki/Category:Feature_requestsThe data structure could probably be improved, though (one of the requests is Improve the feature request template on this site :) ), and it probably makes more sense on semantic-mediawiki.org. But I wanted to note it. (The same holds true for the Tips category, which I've been meaning to move over for probably more than a year now, but still haven't gotten around to... but that's another story.) Bernhard - you don't need administrator rights to move a page. You need them to delete a page, but until you get that privilege you can blank pages instead, which is functionally the same thing. -Yaron On Fri, Feb 25, 2011 at 4:15 AM, Krabina Bernhard krab...@kdz.or.at wrote: Hi, I'd volunteer to help set up a SMW based trouble ticket system. I cannot promise to put in effort to constantly maintain it afterwards, but I can help set it up in the next one or two weeks. I already have an account on semantic-mediawiki.org, so I could actually get going pretty soon. Can you maybe install SemanticFormsInputs as it might be handy. Would it be possible for me to geht admin rights so I can delete or move a page if I need to? regards, Bernhard -- Mag. Bernhard Krabina KDZ - Zentrum für Verwaltungsforschung Centre for Public Administration Research Guglgasse 13, 1110 Wien Telefon: +43 1 8923492-27, Fax: +43 1 8923492-20 Mobil: +43 676 849579-27 mailto:krab...@kdz.or.at - http://www.kdz.or.at - Plattform Verwaltungskooperation: http://www.verwaltungskooperation.at - Open Government Data Konferenz 2011: http://www.ogd2011.at KDZ News: RISER erhält Datenschutz-Gütesiegel EuroPriSe http://bit.ly/h8m0pQ - Ursprüngliche Mail - In reply to the original email and the suggestions that followed: (1) I agree that it would be good if users could easily specify feature requests, and if there was some way of finding out how much each of them is wanted. (2) This could also inspire new extension projects. Users may not know what fits into SMW or into any other extension, so it would be good to have a central mechanism for collecting ideas. (3) I agree that Bugzilla-like organization of information makes sense, but that Bugzilla is not fully suitable. Using some templates and forms, one could easily copy the Bugzilla structure without its restrictions. This has been done, e.g. by Ryan Lane in an SMW-based trouble ticket system. semantic-mediawiki.org would be a good place for this. (4) Such a setup would connect to our recent discussion about collecting interesting implementation tasks. Features are always more fun to implement than bug fixes, and a well-organised list of features could be a starting point for new contributors. (5) It is never easy to measure how important a feature is. We cannot expect such broad interest that we really get representative opinions for our whole community. But a wiki system could have links to user pages to record support. (6) The Roadmap is not immediately related to this. It is a list of things that someone wants to do, not of things that someone wants to be done. It is organised by extension (cf. (2)). The process of editing it is to talk to the maintainer of the respective extension. So my proposal is to set up a form-based feature requesting system and to see how it goes. One or two people are needed to create and maintain this system; please step forward. It does not need to be perfect initially. We may need only a few form fields: title, description, related extension (or unknown/new), maybe a list of related bug reports on Bugzilla, status of the request. Comments could be implemented with a subform (list of items; easy to delete accidentally) or on extra pages (pulled in with a query; cooler, but needs another add comment form that autogenerates a page name and links to the original request). The description should be collaboratively edited (i.e. not owned by the user who reported it), the comments should not. Could/should LiquidThreads be used for having comments below a page instead of on the talk page? To me it seems that this could be an exciting approach for many, especially smaller software projects, so we might actually set standards doing this. We can also directly link such facilities from SMWAdmin to get more users involved. Markus On 24/02/2011 19:29, Jeroen De Dauw wrote: Hey, Following the recent discussions about the roadmap, I'm asking myself if there is no need for a process to get new items on it. I have several idea's that I think would be nice to implement, but not maybe not everyone agrees they should be. I think ideally there should be some place to request
[SMW-devel] SMW 1.5.6 upgrade - performance issues
Hi I just tried to upgrade to 1.5.6 from 1.5.4. I copied over the files, applied the database admin special page (no error there) and started using it. I immediately noticed a huge performance decrease when saving pages. Timeouts actually. I had to roll back to 1.5.4 for the moment as I am not in a position to troubleshoot it right now. I am not even sure how to go about it anyway. I didn't see any error in my PHP error log. Any suggestion of what I could look for ? -- - Laurent Alquier http://www.linfa.net -- Free Software Download: Index, Search Analyze Logs and other IT data in Real-Time with Splunk. Collect, index and harness all the fast moving IT data generated by your applications, servers and devices whether physical, virtual or in the cloud. Deliver compliance at lower cost and gain new business insights. http://p.sf.net/sfu/splunk-dev2dev ___ Semediawiki-devel mailing list Semediawiki-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
Re: [SMW-devel] SMW 1.5.6 upgrade - performance issues
I will try that when I get a chance. Probably late next week. That would be a shame since this hook is the reason why I wanted to upgrade in the first place :) - Laurent On Feb 25, 2011, at 6:00 PM, Markus Krötzsch wrote: On 25/02/2011 21:38, Laurent Alquier wrote: Hi I just tried to upgrade to 1.5.6 from 1.5.4. I copied over the files, applied the database admin special page (no error there) and started using it. I immediately noticed a huge performance decrease when saving pages. Timeouts actually. I had to roll back to 1.5.4 for the moment as I am not in a position to troubleshoot it right now. I am not even sure how to go about it anyway. I didn't see any error in my PHP error log. Any suggestion of what I could look for ? We have added a hook that invalidates the cache of a page right after it is stored to prevent queries on a page from showing outdated results related to that very page. Maybe this causes the problem. You can try to set $smwgAutoRefreshSubject = false; in LocalSettings.php to turn this off. I do not see other changes that could have an effect on performance when saving pages. Regards, Markus -- Free Software Download: Index, Search Analyze Logs and other IT data in Real-Time with Splunk. Collect, index and harness all the fast moving IT data generated by your applications, servers and devices whether physical, virtual or in the cloud. Deliver compliance at lower cost and gain new business insights. http://p.sf.net/sfu/splunk-dev2dev ___ Semediawiki-devel mailing list Semediawiki-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
