I don't know what I think about an engine in Go. :-(
-----Original Message----- From: sangwook DEV [mailto:[email protected]] Sent: Sunday, November 16, 2014 2:52 AM To: [email protected] Subject: Re: How can I make better project than Lucene? Do you mean you want to make a new search engine with me in Go? I don't understand what you mean exactly.. 2014년 11월 16일 일요일, Will Martin<[email protected]>님이 작성한 메시지: > Please let me know if you go the GO route, so I can choose the language I > like best for my work. Wmartinusa at google online mail. > > -----Original Message----- > From: swsong_dev [mailto:[email protected] <javascript:;>] > Sent: Saturday, November 15, 2014 6:23 PM > To: [email protected] <javascript:;> > Subject: Re: How can I make better project than Lucene? > > Thank you for your sincere reply, Mr. McCandless. > > When I posted an email in a mailing-list, I was afraid for not getting > a considerable reply, but I’m now so glad I might find a way. > > I agree that a new search engine in Go would be competitive. I think > we all need a next generation search engine core redesigned from the start. > > And, I understand Lucene’s limitations you mentioned. They are good > points to get started. > > I have been used a search engine for first 4 years and developed a > search engines for last 6 years from the bottom, and I got feedback > “It’s faster than Solr in indexing and searching”. > (http://ddakker.tistory.com/248 > <http://ddakker.tistory.com/248>) > > ===Result=== > Data size : 529,188 > Fastcat indexing time : 1m 26s > Solr3.5 indexing time : 5m 30s > Fastcat searching time : 48ms > Solr3.5 searching time : 73ms > > It applied to Korea’s greatest shopping service(http://danawa.com/ > <http://danawa.com/>) a month ago to my delight. > > But my goal has been making a globally-used open source search engine. > > As you suggested, now I want to make a whole-new search engine in Go. > > I have made my first search engine alone, but I would not make a new > search engine alone. I want to make it with global developers together. > > If you plan to make a new search engine in Go, or know someone around > you, could you help me gathering members for a new search engine, and > guide us technically(feature requirement, efficient design)? > > Or if there is already a new search engine project in Go, could you > let me know? > > In Korea, no one develops a search engine except people who work at a > search engine solution company, and even they are very few and do not > spend time to an open source project. > > In my case, I found a tiny venture company for making time to develop > an open source search engine 4 year ago. > > I want to be involved in a next-generation search engine project. I > would be happy to make a new search engine itself. > > Your little help could be great for me. > > Thank you. > > Sang Song > > > > 2014. 11. 15., 오후 8:22, Michael McCandless > > <[email protected] > <javascript:;>> > 작성: > > > > Actually I think competing projects is very healthy for open source > development. > > > > There are many things you could explore to "contrast" with Lucene, > > e.g. write your new search engine in Go not Java: Java has many > > problems, maybe Go fixes them. Go also has a low-latency garbage > > collector in development ... and Java's GC options still can't scale > > to the heap sizes that are practical now. > > > > Lucene has many limitations, so your competing engine could focus on > > them. E.g. the "schemalessness" of Lucene has become a big problem, > > and near impossible to fix at this point, and prevents new important > > features like LUCENE-5879 from being possible, so you could give > > your engine a "gentle" schema from the start. > > > > The Lucene Filter/Query situation is a mess: one should extend the other. > > > > Lucene has weak support for proximity queries (SpanQuery is slow and > > does not get much attention). > > > > Lucene is showing its age, missing some compelling features like a > > builtin transaction log, "core" support for numerics (they are sort > > of hacked on top), optimistic concurrency support (sequence ids, > > versions, something), distributed support (near real time > > replication, etc.), multi-tenancy, an example server implementation, > > so the search servers on top of Lucene have had to fill these gaps. > > Maybe you could make your engine distributed from the start (Go is a > > great match for that, from what little I know). > > > > All 3 highlighter options have problems. > > > > The analysis chain (attributes) is overly complex. > > > > In your competing engine you can borrow/copy/steal from Lucene's > > good parts to get started... > > > > > > Mike McCandless > > > > http://blog.mikemccandless.com > > > > > > On Fri, Nov 14, 2014 at 8:43 PM, swsong_dev <[email protected] > <javascript:;>> > wrote: > >> I’m developing search engine, Fastcatsearch. http://github > >> <hthttp://githubtp//github>.com/fastcatsearch/fastcatsearch > >> > >> Lucene is widely known and famous project and I cannot beat Lucene > >> for > now. > >> > >> But is there any chance to beat Lucene? > >> > >> Anything like features, performance. > >> > >> Please, let me know what to do to make better product than Lucene. > >> > >> Thank you. > > >
