Hi Carmmello, I would like to stress that I am only speaking from my own views on the way the project has been moving over the last year and a half or so but I would like to add the following points to address you quite obvious concerns
There has been a lot of correspondence on closely linked topics over the past wee while, I think developers understand that there is a small step up to address the requirements of Nutch 1.3, and there is also always inherent problems when you individuals and communities are faced with change. I would like to say with confidence that the current version of Nutch is a well refined tool which is adapting very accurately to provide the best crawling functionality of a very dynamically developing web consisting of various complex graph structures. I would like to make it clear that it is extremely important to have a stable and well designed crawling implementation such as Nutch 1.3 as if you look on the Dev lists you will see the barrage of tasks, existing in an array of complexity, functionality and accuracy which keep Nutch running in parallel with < daily changes to the dynamic web. If Nutch is not focussing on crawling, then no matter whether we have a web application interface or a Lucene index the quality of data fetched will simply not be up to scratch. I hope you can appreciated the burden which this imposes on the directional decisions made within the Project Management Committee in the last year or so... Developers across many of the ASF projects understand that being user friendly is an excellent attribute to have in any open source Apache implementation. However projects develop and in the case of the Apache Software foundation, some of these project spawn sub-projects which graduate to become thier own top level independant projects. As I'm sure you are aware, this was the case with Nutch, therefore it means that as a community we should be able to make decisions independently in the best interests of the project. We have talk about not reinventing the wheel, well this is also going on across the ASF board of projects. One thing to consider is that many developers, contributors and PMCs do not belong to one project, they give up time, effort and resources to sometimes several projects, therefore it is very important that as a project Nutch reserves the priority of removing duplication across the board. Addressing you point regarding the real objectives established at the beginning of the project, there has been significant progress made within Nutch and excellent sub projects which have since graduated to top level projects (I'm sure there is no reason to name them) with their own bustling communities. Allowing Nutch to stagnate and claim to be a one-size-fits-all search engine would have jeopardised the viability of all of these successful projects and would have therefore prevented the very innovation that earns open source implementation under the ASF the reputation and widespread use that projects are renowned for. For example if we take the latest Nutch 1.3 release, we have two options to deploy, in local mode (running on one machine) or in deploy mode (harnessing the strength of parallel processing jobs for different kinds of Nutch users). The development has been driven purely by variances seen across the community usage of Nutch. We draw upon progress made in other delegated areas for the benefit of the project, not to isolate non-programmers from using newer versions of the code base. I would also like to add that there are many questions asked about Solr due to a number of criteria, namely: >Various developers/committers/PMC members of Nutch are also members of various Solr groups. >Developers and users are kind enough to take the time and effort to answer Solr related questions as it is commonly recognised now that Solr is the widespread indexing mechanism (which also has an easily configurable GUI). >It is not very often that users on the Nutch user@ list are ignored or thier queries unanswered, however if this is the case there is good reason behind it. In general, and in my opinion, when I started using Nutch I found the help on user@ not only extremely beneficial but also a confidence boost to get me working on Sold and other project lists. I suppose that there are always 2 sides to every story and it is very uncomforting to hear that you are really not happy with the latest release, there was a lot of hard work put into its development. Amongst bug fixes and other potential barriers mentioned above and previously on this list I would like to think that as the project matures it users can also recognise the dynamism which needs to exist in a project of Nutch's nature i order to present users with a stable and robust software choice. Instead of becoming handicapped we have a clear vision for Nutch 2.0, Nutch branches e.g. 1.4 and many new fixes on the way. I suppose it depends from which side of the table you are on when you mention that it is becoming a handicapped project but my honest opinion is one from the other end of the spectrum. To conclude I would quite like to say that the option is always there to work with Nutch 1.2 and gradually make yourself aware of the great features and benefits your general audience could benefit from when using a nicely tunes Nutch Solr search engine. As a simple example, everyone uses ebay right? Well with some careful and well thought out design decisions the work to get something similar with probably better functionality failored for your specific users up and running is really not a hundred miles away if you were comfortable using < Nutch 1.3. As I said it is not good to hear that you are unhappy with the current release, but I hope I have put across some points in such a way that they provide justification as well as substance to not only the design decisions but also to the roadmap for the Nutch project moving forward. Thank you On Fri, Jul 8, 2011 at 10:59 PM, <carmme...@qualidade.info> wrote: > Relating to softwares, I think there are two types of people: on one side > there are the users , that want some funcionalities from them and support > those softwares by using them and, sometimes, with some comments or > sugestions. On the other side we have the programmers, many of them very > happy in developping things that can be useful, or not quite so, for > some main tasks, defined as the basis of project. > > In the case o Nutch, it seems to me that in the very begining of the > project there was a clear vision of the mission: to have an open source, > powerful search engine. > > But with open source projects, sometimes there is on the developers side, > a lack, or a deviation of the real focus and the people, even if they > follow some pattern, sometimes, don't go strictly after the real > objectives, established at the start of the project. > > Today, with all those Tika, Solr, etc, it all became quite complicate for > me and other non-programmer people and it seems that the original aim > was lost, and Nutch is being used as a basis or as an excuse for a lot > of programming ventures, no matter how worthwhile they can be. But I > think that the real pedestrian users, those who use local instalations of > Nutch, vertical search engines, with about some hundred of thousands of > indexed pages, the people who really support the efforts to make Nutch > run, were forgotten on the road followed nowadays. > I think that Nutch should have been, allways, a primary project and other > projects should have been considered as add-ons to some particular, > special needs. Today it seems quite the oposite. Just look at the > mailing list and see how many questions are asked about Solr. > > I am, really, very sad with the latest version, where Nutch was delegated > to just crawling functions. > I can't find an alternative to set up a real production search engine, > without any suitable web search interface that can be of any use to the > general audience. I really miss the searching function of Nutch with all > those nice features and easy of customization and it seems, to me, that > the original reason for building Nutch was defeated. For my purposes - a > real web search engine- the situation as it is in this version 1.3, led > me to think that Nutch is becoming a very handicapped project. > > I was proud to use the logo “Powered by Nutch”. Now, I am afraid that I > will end up with a Google Custom Engine. > > > > > > -- *Lewis*
