[appengine-java] Re: Best way to perform search on DataStore - Design Question
Haha, like many of us you probably thought that GOOGLE app engine would have decent text search capabilities. It looks like your doing all you can do by creating the inverse table, you may just want to star http://code.google.com/p/googleappengine/issues/detail?id=217 On Mar 13, 8:59 pm, niraj wrote: > My case: > I am building a website that has several searchable fields from > various entities (example Artist names from artist entity , Album > names from album entity). To have an efficient search capability I > have defined another Entity - SearchType which carries the Searchable > string and the Foreign key to the Entity. Instead of querying all the > Entities one my one - I query SearchType. > > My preliminary tests indicate that the query performance on > SearchType is not great (the names are indexed) . I need google > suggest like quick results in a drop down. What is the best way to > design this. > > I have considered Memcache , but I dont think I can run queries on > Memcache . i.e I am running a startsWith() query on JDO today. > > Any best practices . -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Best way to perform search on DataStore - Design Question
I think there are a lot of posts here that conclude Compass is not viable. Has that changed? On Mar 15, 12:23 am, yjun hu wrote: > you can try compass to make you project searchable. a simple demo > herehttp://hapeblog.appspot.com/blog.shtml?id=7002 > > On Sun, Mar 14, 2010 at 8:36 PM, John Patterson wrote: > > > > > Interesting to see the existence protected Query.setFullTextSearch(String) > > method when you open the Query class in Eclipse. I suppose it won't be too > > far away. I can't wait to see if they just give us a take-it-or-leave-it > > solution or also the tools required to roll your own. > > > On 14 Mar 2010, at 15:05, Robert Lancer wrote: > > > Haha, like many of us you probably thought that GOOGLE app engine > >> would have decent text search capabilities. > > >> It looks like your doing all you can do by creating the inverse table, > >> you may just want to star > >>http://code.google.com/p/googleappengine/issues/detail?id=217 > > >> On Mar 13, 8:59 pm, niraj wrote: > > >>> My case: > >>> I am building a website that has several searchable fields from > >>> various entities (example Artist names from artist entity , Album > >>> names from album entity). To have an efficient search capability I > >>> have defined another Entity - SearchType which carries the Searchable > >>> string and the Foreign key to the Entity. Instead of querying all the > >>> Entities one my one - I query SearchType. > > >>> My preliminary tests indicate that the query performance on > >>> SearchType is not great (the names are indexed) . I need google > >>> suggest like quick results in a drop down. What is the best way to > >>> design this. > > >>> I have considered Memcache , but I dont think I can run queries on > >>> Memcache . i.e I am running a startsWith() query on JDO today. > > >>> Any best practices . > > >> -- > >> You received this message because you are subscribed to the Google Groups > >> "Google App Engine for Java" group. > >> To post to this group, send email to > >> google-appengine-j...@googlegroups.com. > >> To unsubscribe from this group, send email to > >> google-appengine-java+unsubscr...@googlegroups.com > >> . > >> For more options, visit this group at > >>http://groups.google.com/group/google-appengine-java?hl=en. > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine for Java" group. > > To post to this group, send email to > > google-appengine-j...@googlegroups.com. > > To unsubscribe from this group, send email to > > google-appengine-java+unsubscr...@googlegroups.com > > . > > For more options, visit this group at > >http://groups.google.com/group/google-appengine-java?hl=en. > > -- > dream or truth -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Best way to perform search on DataStore - Design Question
I am using compass as well and it works fine. The indexing part is costly and you might need to do task queue if you reindex large amount of data. Starting the search manager is slow and it happens a lot due to the suspend policy. You can also take a look at this project: http://code.google.com/p/gaelucene/ I hope there will be a native solution by GAE one day (is there an issue to vote for?) On Mar 16, 10:21 am, yjun hu wrote: > haha, up to now,i use compass okay! > > > > > > On Tue, Mar 16, 2010 at 8:53 AM, objectuser wrote: > > I think there are a lot of posts here that conclude Compass is not > > viable. Has that changed? > > > On Mar 15, 12:23 am, yjun hu wrote: > > > you can try compass to make you project searchable. a simple demo > > herehttp://hapeblog.appspot.com/blog.shtml?id=7002 > > > > On Sun, Mar 14, 2010 at 8:36 PM, John Patterson > >wrote: > > > > > Interesting to see the existence protected > > Query.setFullTextSearch(String) > > > > method when you open the Query class in Eclipse. I suppose it won't be > > too > > > > far away. I can't wait to see if they just give us a > > take-it-or-leave-it > > > > solution or also the tools required to roll your own. > > > > > On 14 Mar 2010, at 15:05, Robert Lancer wrote: > > > > > Haha, like many of us you probably thought that GOOGLE app engine > > > >> would have decent text search capabilities. > > > > >> It looks like your doing all you can do by creating the inverse table, > > > >> you may just want to star > > > >>http://code.google.com/p/googleappengine/issues/detail?id=217 > > > > >> On Mar 13, 8:59 pm, niraj wrote: > > > > >>> My case: > > > >>> I am building a website that has several searchable fields from > > > >>> various entities (example Artist names from artist entity , Album > > > >>> names from album entity). To have an efficient search capability I > > > >>> have defined another Entity - SearchType which carries the Searchable > > > >>> string and the Foreign key to the Entity. Instead of querying all the > > > >>> Entities one my one - I query SearchType. > > > > >>> My preliminary tests indicate that the query performance on > > > >>> SearchType is not great (the names are indexed) . I need google > > > >>> suggest like quick results in a drop down. What is the best way to > > > >>> design this. > > > > >>> I have considered Memcache , but I dont think I can run queries on > > > >>> Memcache . i.e I am running a startsWith() query on JDO today. > > > > >>> Any best practices . > > > > >> -- > > > >> You received this message because you are subscribed to the Google > > Groups > > > >> "Google App Engine for Java" group. > > > >> To post to this group, send email to > > > >> google-appengine-j...@googlegroups.com. > > > >> To unsubscribe from this group, send email to > > > >> google-appengine-java+unsubscr...@googlegroups.com > > >> unsubscr...@googlegroups.com> > > > %252bunsubscr...@googlegroups.com> > > > > >> . > > > >> For more options, visit this group at > > > >>http://groups.google.com/group/google-appengine-java?hl=en. > > > > > -- > > > > You received this message because you are subscribed to the Google > > Groups > > > > "Google App Engine for Java" group. > > > > To post to this group, send email to > > > > google-appengine-j...@googlegroups.com. > > > > To unsubscribe from this group, send email to > > > > google-appengine-java+unsubscr...@googlegroups.com > > > unsubscr...@googlegroups.com> > > > %252bunsubscr...@googlegroups.com> > > > > > . > > > > For more options, visit this group at > > > >http://groups.google.com/group/google-appengine-java?hl=en. > > > > -- > > > dream or truth > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine for Java" group. > > To post to this group, send email to > > google-appengine-j...@googlegroups.com. > > To unsubscribe from this group, send email to > > google-appengine-java+unsubscr...@googlegroups.com > unsubscr...@googlegroups.com> > > . > > For more options, visit this group at > >http://groups.google.com/group/google-appengine-java?hl=en. > > -- > dream or truth -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Best way to perform search on DataStore - Design Question
I am debating if I should use Compass or wait for Google to release something. I can wait for 3-4 months as my website is going to take that amount of time to complete. Niraj On Mar 16, 3:16 am, Toby wrote: > I am using compass as well and it works fine. The indexing part is > costly and you might need to do task queue if you reindex large amount > of data. Starting the search manager is slow and it happens a lot due > to the suspend policy. > > You can also take a look at this project:http://code.google.com/p/gaelucene/ > > I hope there will be a native solution by GAE one day (is there an > issue to vote for?) > > On Mar 16, 10:21 am, yjun hu wrote: > > > > > haha, up to now,i use compass okay! > > > On Tue, Mar 16, 2010 at 8:53 AM, objectuser wrote: > > > I think there are a lot of posts here that conclude Compass is not > > > viable. Has that changed? > > > > On Mar 15, 12:23 am, yjun hu wrote: > > > > you can try compass to make you project searchable. a simple demo > > > herehttp://hapeblog.appspot.com/blog.shtml?id=7002 > > > > > On Sun, Mar 14, 2010 at 8:36 PM, John Patterson > > >wrote: > > > > > > Interesting to see the existence protected > > > Query.setFullTextSearch(String) > > > > > method when you open the Query class in Eclipse. I suppose it won't > > > > > be > > > too > > > > > far away. I can't wait to see if they just give us a > > > take-it-or-leave-it > > > > > solution or also the tools required to roll your own. > > > > > > On 14 Mar 2010, at 15:05, Robert Lancer wrote: > > > > > > Haha, like many of us you probably thought that GOOGLE app engine > > > > >> would have decent text search capabilities. > > > > > >> It looks like your doing all you can do by creating the inverse > > > > >> table, > > > > >> you may just want to star > > > > >>http://code.google.com/p/googleappengine/issues/detail?id=217 > > > > > >> On Mar 13, 8:59 pm, niraj wrote: > > > > > >>> My case: > > > > >>> I am building a website that has several searchable fields from > > > > >>> various entities (example Artist names from artist entity , Album > > > > >>> names from album entity). To have an efficient search capability I > > > > >>> have defined another Entity - SearchType which carries the > > > > >>> Searchable > > > > >>> string and the Foreign key to the Entity. Instead of querying all > > > > >>> the > > > > >>> Entities one my one - I query SearchType. > > > > > >>> My preliminary tests indicate that the query performance on > > > > >>> SearchType is not great (the names are indexed) . I need google > > > > >>> suggest like quick results in a drop down. What is the best way to > > > > >>> design this. > > > > > >>> I have considered Memcache , but I dont think I can run queries on > > > > >>> Memcache . i.e I am running a startsWith() query on JDO today. > > > > > >>> Any best practices . > > > > > >> -- > > > > >> You received this message because you are subscribed to the Google > > > Groups > > > > >> "Google App Engine for Java" group. > > > > >> To post to this group, send email to > > > > >> google-appengine-j...@googlegroups.com. > > > > >> To unsubscribe from this group, send email to > > > > >> google-appengine-java+unsubscr...@googlegroups.com > > > >> unsubscr...@googlegroups.com> > > > > > %252bunsubscr...@googlegroups.com> > > > > > >> . > > > > >> For more options, visit this group at > > > > >>http://groups.google.com/group/google-appengine-java?hl=en. > > > > > > -- > > > > > You received this message because you are subscribed to the Google > > > Groups > > > > > "Google App Engine for Java" group. > > > > > To post to this group, send email to > > > > > google-appengine-j...@googlegroups.com. > > > > > To unsubscribe from this group, send email to > > > > > google-appengine-java+unsubscr...@googlegroups.com > > > > unsubscr...@googlegroups.com> > > > > > %252bunsubscr...@googlegroups.com> > > > > > > . > > > > > For more options, visit this group at > > > > >http://groups.google.com/group/google-appengine-java?hl=en. > > > > > -- > > > > dream or truth > > > > -- > > > You received this message because you are subscribed to the Google Groups > > > "Google App Engine for Java" group. > > > To post to this group, send email to > > > google-appengine-j...@googlegroups.com. > > > To unsubscribe from this group, send email to > > > google-appengine-java+unsubscr...@googlegroups.com > > unsubscr...@googlegroups.com> > > > . > > > For more options, visit this group at > > >http://groups.google.com/group/google-appengine-java?hl=en. > > > -- > > dream or truth -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Best way to perform search on DataStore - Design Question
another way is to implement a full-text search yourself... implementing this would work: http://www.miislita.com/term-vector/term-vector-3.html (use tri-grams instead of full words as tokens) i would recommend low-level datastore api for that, also, your index term entities will blow up beyond 1MB, i built mindash-datastore to handle that :) or you can wait for a few months for Google On Mar 25, 3:05 pm, niraj wrote: > I am debating if I should use Compass or wait for Google to release > something. > > I can wait for 3-4 months as my website is going to take that amount > of time to complete. > > Niraj > > On Mar 16, 3:16 am, Toby wrote: > > > > > I am using compass as well and it works fine. The indexing part is > > costly and you might need to do task queue if you reindex large amount > > of data. Starting the search manager is slow and it happens a lot due > > to the suspend policy. > > > You can also take a look at this project:http://code.google.com/p/gaelucene/ > > > I hope there will be a native solution by GAE one day (is there an > > issue to vote for?) > > > On Mar 16, 10:21 am, yjun hu wrote: > > > > haha, up to now,i use compass okay! > > > > On Tue, Mar 16, 2010 at 8:53 AM, objectuser > > > wrote: > > > > I think there are a lot of posts here that conclude Compass is not > > > > viable. Has that changed? > > > > > On Mar 15, 12:23 am, yjun hu wrote: > > > > > you can try compass to make you project searchable. a simple demo > > > > herehttp://hapeblog.appspot.com/blog.shtml?id=7002 > > > > > > On Sun, Mar 14, 2010 at 8:36 PM, John Patterson > > > >wrote: > > > > > > > Interesting to see the existence protected > > > > Query.setFullTextSearch(String) > > > > > > method when you open the Query class in Eclipse. I suppose it > > > > > > won't be > > > > too > > > > > > far away. I can't wait to see if they just give us a > > > > take-it-or-leave-it > > > > > > solution or also the tools required to roll your own. > > > > > > > On 14 Mar 2010, at 15:05, Robert Lancer wrote: > > > > > > > Haha, like many of us you probably thought that GOOGLE app engine > > > > > >> would have decent text search capabilities. > > > > > > >> It looks like your doing all you can do by creating the inverse > > > > > >> table, > > > > > >> you may just want to star > > > > > >>http://code.google.com/p/googleappengine/issues/detail?id=217 > > > > > > >> On Mar 13, 8:59 pm, niraj wrote: > > > > > > >>> My case: > > > > > >>> I am building a website that has several searchable fields from > > > > > >>> various entities (example Artist names from artist entity , Album > > > > > >>> names from album entity). To have an efficient search capability I > > > > > >>> have defined another Entity - SearchType which carries the > > > > > >>> Searchable > > > > > >>> string and the Foreign key to the Entity. Instead of querying all > > > > > >>> the > > > > > >>> Entities one my one - I query SearchType. > > > > > > >>> My preliminary tests indicate that the query performance on > > > > > >>> SearchType is not great (the names are indexed) . I need google > > > > > >>> suggest like quick results in a drop down. What is the best way to > > > > > >>> design this. > > > > > > >>> I have considered Memcache , but I dont think I can run queries on > > > > > >>> Memcache . i.e I am running a startsWith() query on JDO today. > > > > > > >>> Any best practices . > > > > > > >> -- > > > > > >> You received this message because you are subscribed to the Google > > > > Groups > > > > > >> "Google App Engine for Java" group. > > > > > >> To post to this group, send email to > > > > > >> google-appengine-j...@googlegroups.com. > > > > > >> To unsubscribe from this group, send email to > > > > > >> google-appengine-java+unsubscr...@googlegroups.com > > > > >> unsubscr...@googlegroups.com> > > > > > > > %252bunsubscr...@googlegroups.com> > > > > > > >> . > > > > > >> For more options, visit this group at > > > > > >>http://groups.google.com/group/google-appengine-java?hl=en. > > > > > > > -- > > > > > > You received this message because you are subscribed to the Google > > > > Groups > > > > > > "Google App Engine for Java" group. > > > > > > To post to this group, send email to > > > > > > google-appengine-j...@googlegroups.com. > > > > > > To unsubscribe from this group, send email to > > > > > > google-appengine-java+unsubscr...@googlegroups.com > > > > > unsubscr...@googlegroups.com> > > > > > > > %252bunsubscr...@googlegroups.com> > > > > > > > . > > > > > > For more options, visit this group at > > > > > >http://groups.google.com/group/google-appengine-java?hl=en. > > > > > > -- > > > > > dream or truth > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups > > > > "Google App Engine for Java" group. > > > > To post to this group, send email to > > > > google-appengine-j...@googlegroups.com. > > > > To unsubscribe from this group, send email to > > > > google-appeng
[appengine-java] Re: Best way to perform search on DataStore - Design Question
Hello, I am wondering if text search is actually on the official road map and if so when it will be available. If not, would it make sense to start a google code project to do our own implementation? I am thinking of something simple and light without the need of lots of jar. I think it is quite some work and it would be worthless if GAE will have this feature in the near future. Also I guess it can be implemented more efficiently on google side reusing all the infrastructure that they have already in place. Cheers, Toby On Mar 29, 6:32 am, Tristan wrote: > another way is to implement a full-textsearchyourself... > implementing this would work: > > http://www.miislita.com/term-vector/term-vector-3.html (use tri-grams > instead of full words as tokens) > > i would recommend low-level datastore api for that, also, your index > term entities will blow up beyond 1MB, i built mindash-datastore to > handle that > > :) or you can wait for a few months for Google > > On Mar 25, 3:05 pm, niraj wrote: > > > > > I am debating if I should use Compass or wait for Google to release > > something. > > > I can wait for 3-4 months as my website is going to take that amount > > of time to complete. > > > Niraj > > > On Mar 16, 3:16 am, Toby wrote: > > > > I am using compass as well and it works fine. The indexing part is > > > costly and you might need to do task queue if you reindex large amount > > > of data. Starting thesearchmanager is slow and it happens a lot due > > > to the suspend policy. > > > > You can also take a look at this > > > project:http://code.google.com/p/gaelucene/ > > > > I hope there will be a native solution by GAE one day (is there an > > > issue to vote for?) > > > > On Mar 16, 10:21 am, yjun hu wrote: > > > > > haha, up to now,i use compass okay! > > > > > On Tue, Mar 16, 2010 at 8:53 AM, objectuser > > > > wrote: > > > > > I think there are a lot of posts here that conclude Compass is not > > > > > viable. Has that changed? > > > > > > On Mar 15, 12:23 am, yjun hu wrote: > > > > > > you can try compass to make you project searchable. a simple demo > > > > > herehttp://hapeblog.appspot.com/blog.shtml?id=7002 > > > > > > > On Sun, Mar 14, 2010 at 8:36 PM, John Patterson > > > > > > > > > > >wrote: > > > > > > > > Interesting to see the existence protected > > > > > Query.setFullTextSearch(String) > > > > > > > method when you open the Query class in Eclipse. I suppose it > > > > > > > won't be > > > > > too > > > > > > > far away. I can't wait to see if they just give us a > > > > > take-it-or-leave-it > > > > > > > solution or also the tools required to roll your own. > > > > > > > > On 14 Mar 2010, at 15:05, Robert Lancer wrote: > > > > > > > > Haha, like many of us you probably thought that GOOGLE app engine > > > > > > >> would have decenttextsearchcapabilities. > > > > > > > >> It looks like your doing all you can do by creating the inverse > > > > > > >> table, > > > > > > >> you may just want to star > > > > > > >>http://code.google.com/p/googleappengine/issues/detail?id=217 > > > > > > > >> On Mar 13, 8:59 pm, niraj wrote: > > > > > > > >>> My case: > > > > > > >>> I am building a website that has several searchable fields from > > > > > > >>> various entities (example Artist names from artist entity , > > > > > > >>> Album > > > > > > >>> names from album entity). To have an efficientsearchcapability I > > > > > > >>> have defined another Entity - SearchType which carries the > > > > > > >>> Searchable > > > > > > >>> string and the Foreign key to the Entity. Instead of querying > > > > > > >>> all the > > > > > > >>> Entities one my one - I query SearchType. > > > > > > > >>> My preliminary tests indicate that the query performance on > > > > > > >>> SearchType is not great (the names are indexed) . I need google > > > > > > >>> suggest like quick results in a drop down. What is the best way > > > > > > >>> to > > > > > > >>> design this. > > > > > > > >>> I have considered Memcache , but I dont think I can run queries > > > > > > >>> on > > > > > > >>> Memcache . i.e I am running a startsWith() query on JDO today. > > > > > > > >>> Any best practices . > > > > > > > >> -- > > > > > > >> You received this message because you are subscribed to the > > > > > > >> Google > > > > > Groups > > > > > > >> "Google App Engine for Java" group. > > > > > > >> To post to this group, send email to > > > > > > >> google-appengine-j...@googlegroups.com. > > > > > > >> To unsubscribe from this group, send email to > > > > > > >> google-appengine-java+unsubscr...@googlegroups.com > > > > > >> unsubscr...@googlegroups.com> > > > > > > > > > %252bunsubscr...@googlegroups.com> > > > > > > > >> . > > > > > > >> For more options, visit this group at > > > > > > >>http://groups.google.com/group/google-appengine-java?hl=en. > > > > > > > > -- > > > > > > > You received this message because you are subscribed to the Google > > > > > Groups > > > > > > > "Google App Engine for Java
[appengine-java] Re: Best way to perform search on DataStore - Design Question
Here is workaround to the problem. http://practicingengineer.blogspot.com/2010/05/another-approach-to-full-text-search-on.html Summary: 1. Utilize the search capability on Amazon simpleDB. 2. I use heroku free account to look up on amazon simple db On Apr 6, 2:55 am, John Patterson wrote: > Yes fulltextsearchis on the road map. It can't be far away because > some of the implementation has already slipped into the current > release. See the protected method Query.setFullTextSearch() > > On 6 Apr 2010, at 16:04, Toby wrote: > > > > > Hello, > > > I am wondering iftextsearchis actually on the official road map and > > if so when it will be available. If not, would it make sense to start > > a google code project to do our own implementation? I am thinking of > > something simple and light without the need of lots of jar. I think it > > is quite some work and it would be worthless if GAE will have this > > feature in the near future. Also I guess it can be implemented more > > efficiently on google side reusing all the infrastructure that they > > have already in place. > > > Cheers, > > Toby > > > On Mar 29, 6:32 am, Tristan wrote: > >> another way is to implement a full-textsearchyourself... > >> implementing this would work: > > >>http://www.miislita.com/term-vector/term-vector-3.html(use tri-grams > >> instead of full words as tokens) > > >> i would recommend low-level datastore api for that, also, your index > >> term entities will blow up beyond 1MB, i built mindash-datastore to > >> handle that > > >> :) or you can wait for a few months for Google > > >> On Mar 25, 3:05 pm,niraj wrote: > > >>> I am debating if I should use Compass or wait for Google to release > >>> something. > > >>> I can wait for 3-4 months as my website is going to take that amount > >>> of time to complete. > > >>>Niraj > > >>> On Mar 16, 3:16 am, Toby wrote: > > I am using compass as well and it works fine. The indexing part is > costly and you might need to do task queue if you reindex large > amount > of data. Starting thesearchmanager is slow and it happens a lot due > to the suspend policy. > > You can also take a look at this > project:http://code.google.com/p/gaelucene/ > > I hope there will be a native solution by GAE one day (is there an > issue to vote for?) > > On Mar 16, 10:21 am, yjun hu wrote: > > > haha, up to now,i use compass okay! > > > On Tue, Mar 16, 2010 at 8:53 AM, objectuser > > wrote: > >> I think there are a lot of posts here that conclude Compass is > >> not > >> viable. Has that changed? > > >> On Mar 15, 12:23 am, yjun hu wrote: > >>> you can try compass to make you project searchable. a simple > >>> demo > >> herehttp://hapeblog.appspot.com/blog.shtml?id=7002 > > >>> On Sun, Mar 14, 2010 at 8:36 PM, John Patterson > >>> >>> wrote: > > Interesting to see the existence protected > >> Query.setFullTextSearch(String) > method when you open the Query class in Eclipse. I suppose > it won't be > >> too > far away. I can't wait to see if they just give us a > >> take-it-or-leave-it > solution or also the tools required to roll your own. > > On 14 Mar 2010, at 15:05, Robert Lancer wrote: > > Haha, like many of us you probably thought that GOOGLE app > engine > > would have decenttextsearchcapabilities. > > > It looks like your doing all you can do by creating the > > inverse table, > > you may just want to star > >http://code.google.com/p/googleappengine/issues/detail?id=217 > > > On Mar 13, 8:59 pm,niraj wrote: > > >> My case: > >> I am building a website that has several searchable fields > >> from > >> various entities (example Artist names from artist entity , > >> Album > >> names from album entity). To have an > >> efficientsearchcapability I > >> have defined another Entity - SearchType which carries the > >> Searchable > >> string and the Foreign key to the Entity. Instead of > >> querying all the > >> Entities one my one - I query SearchType. > > >> My preliminary tests indicate that the query performance on > >> SearchType is not great (the names are indexed) . I need > >> google > >> suggest like quick results in a drop down. What is the best > >> way to > >> design this. > > >> I have considered Memcache , but I dont think I can run > >> queries on > >> Memcache . i.e I am running a startsWith() query on JDO > >> today. > > >> Any best practices . > > > -- > > You received this message because you are subscribed to the > > Google > >> Groups > > "Google App Engine for Java" group. > > To
Re: [appengine-java] Re: Best way to perform search on DataStore - Design Question
Interesting to see the existence protected Query.setFullTextSearch(String) method when you open the Query class in Eclipse. I suppose it won't be too far away. I can't wait to see if they just give us a take-it-or-leave-it solution or also the tools required to roll your own. On 14 Mar 2010, at 15:05, Robert Lancer wrote: Haha, like many of us you probably thought that GOOGLE app engine would have decent text search capabilities. It looks like your doing all you can do by creating the inverse table, you may just want to star http://code.google.com/p/googleappengine/issues/detail?id=217 On Mar 13, 8:59 pm, niraj wrote: My case: I am building a website that has several searchable fields from various entities (example Artist names from artist entity , Album names from album entity). To have an efficient search capability I have defined another Entity - SearchType which carries the Searchable string and the Foreign key to the Entity. Instead of querying all the Entities one my one - I query SearchType. My preliminary tests indicate that the query performance on SearchType is not great (the names are indexed) . I need google suggest like quick results in a drop down. What is the best way to design this. I have considered Memcache , but I dont think I can run queries on Memcache . i.e I am running a startsWith() query on JDO today. Any best practices . -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com . To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en . -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Best way to perform search on DataStore - Design Question
you can try compass to make you project searchable. a simple demo here http://hapeblog.appspot.com/blog.shtml?id=7002 On Sun, Mar 14, 2010 at 8:36 PM, John Patterson wrote: > Interesting to see the existence protected Query.setFullTextSearch(String) > method when you open the Query class in Eclipse. I suppose it won't be too > far away. I can't wait to see if they just give us a take-it-or-leave-it > solution or also the tools required to roll your own. > > > On 14 Mar 2010, at 15:05, Robert Lancer wrote: > > Haha, like many of us you probably thought that GOOGLE app engine >> would have decent text search capabilities. >> >> It looks like your doing all you can do by creating the inverse table, >> you may just want to star >> http://code.google.com/p/googleappengine/issues/detail?id=217 >> >> >> On Mar 13, 8:59 pm, niraj wrote: >> >>> My case: >>> I am building a website that has several searchable fields from >>> various entities (example Artist names from artist entity , Album >>> names from album entity). To have an efficient search capability I >>> have defined another Entity - SearchType which carries the Searchable >>> string and the Foreign key to the Entity. Instead of querying all the >>> Entities one my one - I query SearchType. >>> >>> My preliminary tests indicate that the query performance on >>> SearchType is not great (the names are indexed) . I need google >>> suggest like quick results in a drop down. What is the best way to >>> design this. >>> >>> I have considered Memcache , but I dont think I can run queries on >>> Memcache . i.e I am running a startsWith() query on JDO today. >>> >>> Any best practices . >>> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine for Java" group. >> To post to this group, send email to >> google-appengine-j...@googlegroups.com. >> To unsubscribe from this group, send email to >> google-appengine-java+unsubscr...@googlegroups.com >> . >> For more options, visit this group at >> http://groups.google.com/group/google-appengine-java?hl=en. >> >> > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine for Java" group. > To post to this group, send email to > google-appengine-j...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine-java+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/google-appengine-java?hl=en. > > -- dream or truth -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Best way to perform search on DataStore - Design Question
haha, up to now,i use compass okay! On Tue, Mar 16, 2010 at 8:53 AM, objectuser wrote: > I think there are a lot of posts here that conclude Compass is not > viable. Has that changed? > > On Mar 15, 12:23 am, yjun hu wrote: > > you can try compass to make you project searchable. a simple demo > herehttp://hapeblog.appspot.com/blog.shtml?id=7002 > > > > On Sun, Mar 14, 2010 at 8:36 PM, John Patterson >wrote: > > > > > > > > > Interesting to see the existence protected > Query.setFullTextSearch(String) > > > method when you open the Query class in Eclipse. I suppose it won't be > too > > > far away. I can't wait to see if they just give us a > take-it-or-leave-it > > > solution or also the tools required to roll your own. > > > > > On 14 Mar 2010, at 15:05, Robert Lancer wrote: > > > > > Haha, like many of us you probably thought that GOOGLE app engine > > >> would have decent text search capabilities. > > > > >> It looks like your doing all you can do by creating the inverse table, > > >> you may just want to star > > >>http://code.google.com/p/googleappengine/issues/detail?id=217 > > > > >> On Mar 13, 8:59 pm, niraj wrote: > > > > >>> My case: > > >>> I am building a website that has several searchable fields from > > >>> various entities (example Artist names from artist entity , Album > > >>> names from album entity). To have an efficient search capability I > > >>> have defined another Entity - SearchType which carries the Searchable > > >>> string and the Foreign key to the Entity. Instead of querying all the > > >>> Entities one my one - I query SearchType. > > > > >>> My preliminary tests indicate that the query performance on > > >>> SearchType is not great (the names are indexed) . I need google > > >>> suggest like quick results in a drop down. What is the best way to > > >>> design this. > > > > >>> I have considered Memcache , but I dont think I can run queries on > > >>> Memcache . i.e I am running a startsWith() query on JDO today. > > > > >>> Any best practices . > > > > >> -- > > >> You received this message because you are subscribed to the Google > Groups > > >> "Google App Engine for Java" group. > > >> To post to this group, send email to > > >> google-appengine-j...@googlegroups.com. > > >> To unsubscribe from this group, send email to > > >> google-appengine-java+unsubscr...@googlegroups.com > > > > > >> . > > >> For more options, visit this group at > > >>http://groups.google.com/group/google-appengine-java?hl=en. > > > > > -- > > > You received this message because you are subscribed to the Google > Groups > > > "Google App Engine for Java" group. > > > To post to this group, send email to > > > google-appengine-j...@googlegroups.com. > > > To unsubscribe from this group, send email to > > > google-appengine-java+unsubscr...@googlegroups.com > > > > > > . > > > For more options, visit this group at > > >http://groups.google.com/group/google-appengine-java?hl=en. > > > > -- > > dream or truth > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine for Java" group. > To post to this group, send email to > google-appengine-j...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine-java+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/google-appengine-java?hl=en. > > -- dream or truth -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Best way to perform search on DataStore - Design Question
Yes full text search is on the road map. It can't be far away because some of the implementation has already slipped into the current release. See the protected method Query.setFullTextSearch() On 6 Apr 2010, at 16:04, Toby wrote: Hello, I am wondering if text search is actually on the official road map and if so when it will be available. If not, would it make sense to start a google code project to do our own implementation? I am thinking of something simple and light without the need of lots of jar. I think it is quite some work and it would be worthless if GAE will have this feature in the near future. Also I guess it can be implemented more efficiently on google side reusing all the infrastructure that they have already in place. Cheers, Toby On Mar 29, 6:32 am, Tristan wrote: another way is to implement a full-textsearchyourself... implementing this would work: http://www.miislita.com/term-vector/term-vector-3.html (use tri-grams instead of full words as tokens) i would recommend low-level datastore api for that, also, your index term entities will blow up beyond 1MB, i built mindash-datastore to handle that :) or you can wait for a few months for Google On Mar 25, 3:05 pm, niraj wrote: I am debating if I should use Compass or wait for Google to release something. I can wait for 3-4 months as my website is going to take that amount of time to complete. Niraj On Mar 16, 3:16 am, Toby wrote: I am using compass as well and it works fine. The indexing part is costly and you might need to do task queue if you reindex large amount of data. Starting thesearchmanager is slow and it happens a lot due to the suspend policy. You can also take a look at this project:http://code.google.com/p/gaelucene/ I hope there will be a native solution by GAE one day (is there an issue to vote for?) On Mar 16, 10:21 am, yjun hu wrote: haha, up to now,i use compass okay! On Tue, Mar 16, 2010 at 8:53 AM, objectuser wrote: I think there are a lot of posts here that conclude Compass is not viable. Has that changed? On Mar 15, 12:23 am, yjun hu wrote: you can try compass to make you project searchable. a simple demo herehttp://hapeblog.appspot.com/blog.shtml?id=7002 On Sun, Mar 14, 2010 at 8:36 PM, John Patterson wrote: Interesting to see the existence protected Query.setFullTextSearch(String) method when you open the Query class in Eclipse. I suppose it won't be too far away. I can't wait to see if they just give us a take-it-or-leave-it solution or also the tools required to roll your own. On 14 Mar 2010, at 15:05, Robert Lancer wrote: Haha, like many of us you probably thought that GOOGLE app engine would have decenttextsearchcapabilities. It looks like your doing all you can do by creating the inverse table, you may just want to star http://code.google.com/p/googleappengine/issues/detail?id=217 On Mar 13, 8:59 pm, niraj wrote: My case: I am building a website that has several searchable fields from various entities (example Artist names from artist entity , Album names from album entity). To have an efficientsearchcapability I have defined another Entity - SearchType which carries the Searchable string and the Foreign key to the Entity. Instead of querying all the Entities one my one - I query SearchType. My preliminary tests indicate that the query performance on SearchType is not great (the names are indexed) . I need google suggest like quick results in a drop down. What is the best way to design this. I have considered Memcache , but I dont think I can run queries on Memcache . i.e I am running a startsWith() query on JDO today. Any best practices . -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comappengine-java%2B unsubscr...@googlegroups.com> appengine-java %252bunsubscr...@googlegroups.com> . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comappengine-java%2B unsubscr...@googlegroups.com> appengine-java %252bunsubscr...@googlegroups.com> . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- dream or truth -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appe
Re: [appengine-java] Re: Best way to perform search on DataStore - Design Question
if you use objectify, you can use this: http://code.google.com/p/fulltext-search-in-objectify/ On Mon, May 17, 2010 at 3:51 PM, niraj wrote: > Here is workaround to the problem. > > > http://practicingengineer.blogspot.com/2010/05/another-approach-to-full-text-search-on.html > > Summary: > 1. Utilize the search capability on Amazon simpleDB. > 2. I use heroku free account to look up on amazon simple db > > > On Apr 6, 2:55 am, John Patterson wrote: > > Yes fulltextsearchis on the road map. It can't be far away because > > some of the implementation has already slipped into the current > > release. See the protected method Query.setFullTextSearch() > > > > On 6 Apr 2010, at 16:04, Toby wrote: > > > > > > > > > Hello, > > > > > I am wondering iftextsearchis actually on the official road map and > > > if so when it will be available. If not, would it make sense to start > > > a google code project to do our own implementation? I am thinking of > > > something simple and light without the need of lots of jar. I think it > > > is quite some work and it would be worthless if GAE will have this > > > feature in the near future. Also I guess it can be implemented more > > > efficiently on google side reusing all the infrastructure that they > > > have already in place. > > > > > Cheers, > > > Toby > > > > > On Mar 29, 6:32 am, Tristan wrote: > > >> another way is to implement a full-textsearchyourself... > > >> implementing this would work: > > > > >>http://www.miislita.com/term-vector/term-vector-3.html(use tri-grams > > >> instead of full words as tokens) > > > > >> i would recommend low-level datastore api for that, also, your index > > >> term entities will blow up beyond 1MB, i built mindash-datastore to > > >> handle that > > > > >> :) or you can wait for a few months for Google > > > > >> On Mar 25, 3:05 pm,niraj wrote: > > > > >>> I am debating if I should use Compass or wait for Google to release > > >>> something. > > > > >>> I can wait for 3-4 months as my website is going to take that amount > > >>> of time to complete. > > > > >>>Niraj > > > > >>> On Mar 16, 3:16 am, Toby wrote: > > > > I am using compass as well and it works fine. The indexing part is > > costly and you might need to do task queue if you reindex large > > amount > > of data. Starting thesearchmanager is slow and it happens a lot due > > to the suspend policy. > > > > You can also take a look at this project: > http://code.google.com/p/gaelucene/ > > > > I hope there will be a native solution by GAE one day (is there an > > issue to vote for?) > > > > On Mar 16, 10:21 am, yjun hu wrote: > > > > > haha, up to now,i use compass okay! > > > > > On Tue, Mar 16, 2010 at 8:53 AM, objectuser > > > wrote: > > >> I think there are a lot of posts here that conclude Compass is > > >> not > > >> viable. Has that changed? > > > > >> On Mar 15, 12:23 am, yjun hu wrote: > > >>> you can try compass to make you project searchable. a simple > > >>> demo > > >> herehttp://hapeblog.appspot.com/blog.shtml?id=7002 > > > > >>> On Sun, Mar 14, 2010 at 8:36 PM, John Patterson > > >>> > >>> wrote: > > > > Interesting to see the existence protected > > >> Query.setFullTextSearch(String) > > method when you open the Query class in Eclipse. I suppose > > it won't be > > >> too > > far away. I can't wait to see if they just give us a > > >> take-it-or-leave-it > > solution or also the tools required to roll your own. > > > > On 14 Mar 2010, at 15:05, Robert Lancer wrote: > > > > Haha, like many of us you probably thought that GOOGLE app > > engine > > > would have decenttextsearchcapabilities. > > > > > It looks like your doing all you can do by creating the > > > inverse table, > > > you may just want to star > > >http://code.google.com/p/googleappengine/issues/detail?id=217 > > > > > On Mar 13, 8:59 pm,niraj wrote: > > > > >> My case: > > >> I am building a website that has several searchable fields > > >> from > > >> various entities (example Artist names from artist entity , > > >> Album > > >> names from album entity). To have an > > >> efficientsearchcapability I > > >> have defined another Entity - SearchType which carries the > > >> Searchable > > >> string and the Foreign key to the Entity. Instead of > > >> querying all the > > >> Entities one my one - I query SearchType. > > > > >> My preliminary tests indicate that the query performance on > > >> SearchType is not great (the names are indexed) . I need > > >> google > > >> suggest like quick results in a drop down. What is the best > > >> way to > > >> design this. > > > > >> I have considered Memcache , but I
