Thank you for your sincere reply, Mr. McCandless. When I posted an email in a mailing-list, I was afraid for not getting a considerable reply, but I’m now so glad I might find a way.
I agree that a new search engine in Go would be competitive. I think we all need a next generation search engine core redesigned from the start. And, I understand Lucene’s limitations you mentioned. They are good points to get started. I have been used a search engine for first 4 years and developed a search engines for last 6 years from the bottom, and I got feedback “It’s faster than Solr in indexing and searching”. (http://ddakker.tistory.com/248 <http://ddakker.tistory.com/248>) ===Result=== Data size : 529,188 Fastcat indexing time : 1m 26s Solr3.5 indexing time : 5m 30s Fastcat searching time : 48ms Solr3.5 searching time : 73ms It applied to Korea’s greatest shopping service(http://danawa.com/ <http://danawa.com/>) a month ago to my delight. But my goal has been making a globally-used open source search engine. As you suggested, now I want to make a whole-new search engine in Go. I have made my first search engine alone, but I would not make a new search engine alone. I want to make it with global developers together. If you plan to make a new search engine in Go, or know someone around you, could you help me gathering members for a new search engine, and guide us technically(feature requirement, efficient design)? Or if there is already a new search engine project in Go, could you let me know? In Korea, no one develops a search engine except people who work at a search engine solution company, and even they are very few and do not spend time to an open source project. In my case, I found a tiny venture company for making time to develop an open source search engine 4 year ago. I want to be involved in a next-generation search engine project. I would be happy to make a new search engine itself. Your little help could be great for me. Thank you. Sang Song > 2014. 11. 15., 오후 8:22, Michael McCandless <[email protected]> 작성: > > Actually I think competing projects is very healthy for open source > development. > > There are many things you could explore to "contrast" with Lucene, > e.g. write your new search engine in Go not Java: Java has many > problems, maybe Go fixes them. Go also has a low-latency garbage > collector in development ... and Java's GC options still can't scale > to the heap sizes that are practical now. > > Lucene has many limitations, so your competing engine could focus on > them. E.g. the "schemalessness" of Lucene has become a big problem, > and near impossible to fix at this point, and prevents new important > features like LUCENE-5879 from being possible, so you could give your > engine a "gentle" schema from the start. > > The Lucene Filter/Query situation is a mess: one should extend the other. > > Lucene has weak support for proximity queries (SpanQuery is slow and > does not get much attention). > > Lucene is showing its age, missing some compelling features like a > builtin transaction log, "core" support for numerics (they are sort of > hacked on top), optimistic concurrency support (sequence ids, > versions, something), distributed support (near real time replication, > etc.), multi-tenancy, an example server implementation, so the search > servers on top of Lucene have had to fill these gaps. Maybe you could > make your engine distributed from the start (Go is a great match for > that, from what little I know). > > All 3 highlighter options have problems. > > The analysis chain (attributes) is overly complex. > > In your competing engine you can borrow/copy/steal from Lucene's good > parts to get started... > > > Mike McCandless > > http://blog.mikemccandless.com > > > On Fri, Nov 14, 2014 at 8:43 PM, swsong_dev <[email protected]> wrote: >> I’m developing search engine, Fastcatsearch. http://github >> <hthttp://githubtp//github>.com/fastcatsearch/fastcatsearch >> >> Lucene is widely known and famous project and I cannot beat Lucene for now. >> >> But is there any chance to beat Lucene? >> >> Anything like features, performance. >> >> Please, let me know what to do to make better product than Lucene. >> >> Thank you.
