full-text search
Hi all I'm trying to implement a search on my site, find this[1] solution interesting because it is not necessary to use any external code (or as xapian sphinx). Unfortunately this solution only works in older versions of django. I tried to fix backend.quote_name, but failed. Someone has used a similar solution for versions after 1.0? [1]http://www.mercurytide.co.uk/news/article/django-full-text-search/ thanks --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Full Text Search
Is there any way to make a FULL TEXT search not in BOOLEAN MODE? I've found only http://docs.djangoproject.com/en/1.1/ref/models/querysets/#search It says it uses BOOLEAN MODE as a default, but doesn't tell how to avoid default... -- Alessandro Ronchi http://www.soasi.com Hobby & Giochi http://hobbygiochi.com http://www.facebook.com/hobbygiochi -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-us...@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Re: full-text search
Hello Petry, Is there something about http://docs.djangoproject.com/en/dev/ref/models/querysets/#search that isn't working for you in 1.0.x? On May 28, 7:10 am, Petry wrote: > I'm trying to implement a search on my site, find this[1] solution > interesting because it is not necessary to use any external code (or > as xapian sphinx). Unfortunately this solution only works in older > versions of django. I tried to fix backend.quote_name, but failed. > > Someone has used a similar solution for versions after 1.0? > > [1]http://www.mercurytide.co.uk/news/article/django-full-text-search/ Toodle-lo... creecode --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: full-text search
On Thu, May 28, 2009 at 12:33 PM, creecode wrote: > > Hello Petry, > > Is there something about > http://docs.djangoproject.com/en/dev/ref/models/querysets/#search > that isn't working for you in 1.0.x? > > On May 28, 7:10 am, Petry wrote: > > > I'm trying to implement a search on my site, find this[1] solution > > interesting because it is not necessary to use any external code (or > > as xapian sphinx). Unfortunately this solution only works in older > > versions of django. I tried to fix backend.quote_name, but failed. > > > > Someone has used a similar solution for versions after 1.0? > > > > [1]http://www.mercurytide.co.uk/news/article/django-full-text-search/ > > Toodle-lo... > creecode > > > Well for one thing that is mysql only, which is hangup for a lot of people. In the search department your best bet is to look at haystack: http://haystacksearch.org/ Alex -- "I disapprove of what you say, but I will defend to the death your right to say it." --Voltaire "The people's good is the highest law."--Cicero --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: Full Text Search
On Thu, Sep 30, 2010 at 8:34 AM, Alessandro Ronchi < alessandro.ron...@soasi.com> wrote: > Is there any way to make a FULL TEXT search not in BOOLEAN MODE? > I've found only > http://docs.djangoproject.com/en/1.1/ref/models/querysets/#search > > It says it uses BOOLEAN MODE as a default, but doesn't tell how to > avoid default.. > Um, I hate to do this, but for the moment, I'm going to tell you "yes" - but I can't remember how right now. Been awake for 30 hours and waiting for a plane to take me home! Did a demo today for a big client, involving Django/MySQL full text and parametric search and they loved it... so it's on my mind. If you don't get a real answer by tomorrow afternoon, I may be conscious again then. You have to use one of the alternative approaches to search, as I recall. Brain is mush, though. Nick -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-us...@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Re: Full Text Search
On 9/30/2010 8:26 PM, Nick Arnett wrote: > Brain is mush, though. Obviously, otherwise you'd have left it in the hope that someone else would reply with an answer before you recovered consciousness. Great heavens, this community is *so* anxious to help. Kudos for trying with a frazzled brain! regards Steve -- DjangoCon US 2010 September 7-9 http://djangocon.us/ -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-us...@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Re: Full Text Search
On Fri, Oct 1, 2010 at 3:35 AM, Steve Holden wrote: > On 9/30/2010 8:26 PM, Nick Arnett wrote: > > Brain is mush, though. > > Thank you. I'm waiting with hope :) -- Alessandro Ronchi http://www.soasi.com -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-us...@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Re: Full Text Search
On Fri, Oct 1, 2010 at 3:52 AM, Alessandro Ronchi < alessandro.ron...@soasi.com> wrote: > > > On Fri, Oct 1, 2010 at 3:35 AM, Steve Holden wrote: > >> On 9/30/2010 8:26 PM, Nick Arnett wrote: >> > Brain is mush, though. >> >> > Thank you. I'm waiting with hope :) > Brain less mushy now... Sometimes just knowing something is possible helps, which was why I posted last night. I used this idea: http://www.mercurytide.co.uk/news/article/django-full-text-search/ and added a parameter to turn boolean on and off. However, I'll add that I didn't much care for the results when not using boolean. Nick -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-us...@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Re: Full Text Search
Check this out: http://www.julienphalip.com/blog/2008/08/16/adding-search-django-site-snap/ On Oct 1, 6:52 am, Alessandro Ronchi wrote: > On Fri, Oct 1, 2010 at 3:35 AM, Steve Holden wrote: > > On 9/30/2010 8:26 PM, Nick Arnett wrote: > > > Brain is mush, though. > > Thank you. I'm waiting with hope :) > > -- > Alessandro Ronchihttp://www.soasi.com -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-us...@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Re: Full Text Search
Hello!
I use PostgreSQL, and this is how I accomplished a full text search:
def search(request):
# how to set up postgresql for full text searching
#alter table quote_quote add column body_tsv tsvector;
#CREATE TRIGGER quote_quote_tsvectorupdate BEFORE INSERT OR UPDATE
ON quote_quote FOR EACH ROW EXECUTE PROCEDURE
tsvector_update_trigger(quote_tsv, 'pg_catalog.english', quote);
#CREATE INDEX quote_quote_tsv ON quote_quote USING gin(quote_tsv);
#UPDATE quote_quote SET quote_tsv=to_tsvector(quote);
q = request.GET['q']
quotes = Quote.objects.extra(
select={
'snippet': "ts_headline(quote, plainto_tsquery(%s))",
'rank': "ts_rank_cd(quote_tsv, plainto_tsquery(%s), 32)",
},
where=["quote_tsv @@ plainto_tsquery(%s)"],
params=[q],
select_params=[q, q],
order_by=('-rank',)
)
paginator = Paginator(quotes, 50)
try:
page = int(request.GET.get('page', '1'))
except ValueError:
page = 1
try:
quotes = paginator.page(page)
except (EmptyPage, InvalidPage):
quotes = paginator.page(paginator.num_pages)
my_data_dictionary = {
'quotes': quotes,
'q' : q,
}
return render_to_response('results.html',
my_data_dictionary,
context_instance=RequestContext(request))
urlpatterns = patterns('',
url(r'^search/$', search),
)
On Fri, Oct 1, 2010 at 7:26 PM, Andrej wrote:
> Check this out:
>
> http://www.julienphalip.com/blog/2008/08/16/adding-search-django-site-snap/
>
> On Oct 1, 6:52 am, Alessandro Ronchi
> wrote:
>> On Fri, Oct 1, 2010 at 3:35 AM, Steve Holden wrote:
>> > On 9/30/2010 8:26 PM, Nick Arnett wrote:
>> > > Brain is mush, though.
>>
>> Thank you. I'm waiting with hope :)
>>
>> --
>> Alessandro Ronchihttp://www.soasi.com
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To post to this group, send email to django-us...@googlegroups.com.
> To unsubscribe from this group, send email to
> django-users+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-users?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-users?hl=en.
Re: Full Text Search
maybe i misunderstood your question, but maybe it will help
On Fri, Oct 1, 2010 at 8:19 PM, m1chael wrote:
> Hello!
>
> I use PostgreSQL, and this is how I accomplished a full text search:
>
> def search(request):
> # how to set up postgresql for full text searching
> #alter table quote_quote add column body_tsv tsvector;
> #CREATE TRIGGER quote_quote_tsvectorupdate BEFORE INSERT OR UPDATE
> ON quote_quote FOR EACH ROW EXECUTE PROCEDURE
> tsvector_update_trigger(quote_tsv, 'pg_catalog.english', quote);
> #CREATE INDEX quote_quote_tsv ON quote_quote USING gin(quote_tsv);
> #UPDATE quote_quote SET quote_tsv=to_tsvector(quote);
>
> q = request.GET['q']
>
> quotes = Quote.objects.extra(
> select={
> 'snippet': "ts_headline(quote, plainto_tsquery(%s))",
> 'rank': "ts_rank_cd(quote_tsv, plainto_tsquery(%s), 32)",
> },
> where=["quote_tsv @@ plainto_tsquery(%s)"],
> params=[q],
> select_params=[q, q],
> order_by=('-rank',)
> )
>
> paginator = Paginator(quotes, 50)
>
> try:
> page = int(request.GET.get('page', '1'))
> except ValueError:
> page = 1
>
> try:
> quotes = paginator.page(page)
> except (EmptyPage, InvalidPage):
> quotes = paginator.page(paginator.num_pages)
>
> my_data_dictionary = {
> 'quotes': quotes,
> 'q' : q,
> }
>
> return render_to_response('results.html',
> my_data_dictionary,
> context_instance=RequestContext(request))
>
> urlpatterns = patterns('',
> url(r'^search/$', search),
> )
>
> On Fri, Oct 1, 2010 at 7:26 PM, Andrej wrote:
>> Check this out:
>>
>> http://www.julienphalip.com/blog/2008/08/16/adding-search-django-site-snap/
>>
>> On Oct 1, 6:52 am, Alessandro Ronchi
>> wrote:
>>> On Fri, Oct 1, 2010 at 3:35 AM, Steve Holden wrote:
>>> > On 9/30/2010 8:26 PM, Nick Arnett wrote:
>>> > > Brain is mush, though.
>>>
> >> Thank you. I'm waiting with hope :)
>>>
>>> --
>>> Alessandro Ronchihttp://www.soasi.com
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django users" group.
>> To post to this group, send email to django-us...@googlegroups.com.
>> To unsubscribe from this group, send email to
>> django-users+unsubscr...@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/django-users?hl=en.
>>
>>
>
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-users?hl=en.
Sphinx Full-Text Search
I know some of you out there use my full-text layer relying on Sphinx in Django. I released a new version yesterday, which on my initial tests is fully backwards compatible. So the point, what I'm looking for, is has anyone put the new version to use yet? If not, can you try it on your working codebases? http://www.davidcramer.net/code/117/announcing-django-sphinx-200.html --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Django full text search best solution
Hi, I'm looking at all the available solutions to implement a full text index for django models and I found various projects at different stages of maturity but none of them seems to have a strong community support ... so here is the list along with my comments (and last commit time) ... it any of you has some advice or feedback on any of those (or can propose an alternative solution), I'll be glad to here from you! * django-search-lucene (http://code.google.com/p/django-search-lucene) no issues ... wiki not updated since django 1.0 last commit : aug 08 * djapian (http://code.google.com/p/djapian) seems fairly active (not much issues added / resolved though) last commit : oct 08 * django-sphinx (http://code.google.com/p/django-sphinx) sphinx looks like it's the most difficult backend to setup last commit : oct 08 * using database full text index postgresql (in code since release 8.3) and mysql have full text indexes * djangosearch (http://code.google.com/p/djangosearch) multi backend (Solr being the preferred). Wiki says to be used in production but "API still very unstable" last commit : oct 08 * django-search (http://code.google.com/p/django-search/) does not seems to be very active last commit : may 08 My understanding is that djangosearch and djapian are the most interesting projects at the moment ... --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Full-text search: what to use
Hello, I'm working on a huge django-based application that heavily utilizes full-text search. Up until now I've been using external search service built using SOLR that is deployed separately. The problem is with such an approach I need to maintain 2 different environments - one for python/django and one for java/solr. It's not a big issue but I started to look for an alternative that will allow me to build search functionality in the django application. I've found several solutions so far: 1. pylucene 2. sphinx 3. whoosh Now the problem is that I have no real experience with any of those and would like to have some advise from those who tried any of the options (or may be there is anything else available?). PyLucene seems to be using java lucene library which may unnecessary affect the application performance as it is "embeds a Java VM with Lucene into a Python process". I found some links in the internet saying that sphinx can not handle non-ascii character sets well which is critical for me. Whoosh seems to be a little amateur - did not see any links on real world usage. Did you try one of these options? Or do you know any other solutions for the problem? Alex. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Full text search available on PostgreSQL?
Hi, Just saw that django.contrib.admin.ModelAdmin.search_fields <https://docs.djangoproject.com/en/dev/ref/contrib/admin/#django.contrib.admin.ModelAdmin.search_fields> full text search only support MySQL. Are there plans on any progress towards getting this functionality on PostgreSQL? Thanks -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/f0071b4f-a368-4512-a6f7-79d921e32141%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Full Text Search w/ MySQL
Nice work John, good stuff. Do any of the core devs see this as a candidate for inclusion in contrib? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users -~--~~~~--~~--~--~---
Re: Full Text Search w/ MySQL
Adam Endicott wrote: > > Do any of the core devs see this as a candidate for inclusion in > contrib? Speaking as the author of the piece -- I don't. This code isn't developed enough to be particularly practical as-is: to be more useful it should probably support both natural language and boolean mode queries; it could perhaps also support multiple full text queries on one table. I don't know. At the very least it should support stemming. Bear in mind that I kept it as simple as possible so that it illustrated the essentials more clearly -- subclassing Manager and QuerySet, and building the extra parts of the SQL statement -- so the lack of features is intentional. But as the introduction states, the code in the paper is released into the public domain, so there's nothing stopping any interested person from taking it and developing it into something more suited for general searching. Andrew --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users -~--~~~~--~~--~--~---
Re: Django full text search best solution
alain D. schrieb: > Hi, > > I'm looking at all the available solutions to implement a full text > index for django models and I found various projects at different > stages of maturity but none of them seems to have a strong community > support ... so here is the list along with my comments (and last > commit time) ... it any of you has some advice or feedback on any of > those (or can propose an alternative solution), I'll be glad to here > from you! > Since all external full text indexing (sphinx, lucene) won't be part of the database transaction, I choose the full text index of postgres. I wrote an own to_tsvector method in python, instead of customizing the to_tsvector method of postgres. Thomas -- Thomas Guettler, http://www.thomas-guettler.de/ E-Mail: guettli (*) thomas-guettler + de --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: Django full text search best solution
Being inside database transactions is a good reason to use database's full text index ... but the sake of simplicity, a full text index created by python code inside the models declaration (easily maintainable) would be nice ... > Since all external full text indexing (sphinx, lucene) won't be part of > the database transaction, > I choose the full text index of postgres. > > I wrote an own to_tsvector method in python, instead of customizing > the to_tsvector method of postgres. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: Django full text search best solution
alain D. schrieb: > Being inside database transactions is a good reason to use database's > full text index ... but the sake of simplicity, a full text index > created by python code inside the models declaration (easily > maintainable) would be nice ... > > > This is inside the model declaration: class MyModel(models.Model): ... def save(self, text_update=True): models.Model.save(self) if text_update: text=utils.mymodel_to_text(self) tsvector=tsearchutils.to_tsvector(text) cursor=connection.cursor() cursor.execute(u'''UPDATE myapp_mymodel SET content=%s::tsvector where id=%s''', [tsvector, self.id]) The column "content" of type tsvector is created by a SQL snippet named mymodel.sql. HTH, Thomas -- Thomas Guettler, http://www.thomas-guettler.de/ E-Mail: guettli (*) thomas-guettler + de --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Postgres Full-text search -- is it possible?
Hi -- I'm wanting to improve searches by using full-text searching. The documentation says: > > > search¶ > <http://docs.djangoproject.com/en/dev/ref/models/querysets/#search> > > A boolean full-text search, taking advantage of full-text indexing. > This is like contains but is significantly faster due to full-text > indexing. > > Note this is only available in MySQL and requires direct manipulation > of the database to add the full-text index. > What options are available for Postgres if any? Postgres has this capability, I believe. Can Django not use it? Liam --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: Full-text search: what to use
On Mon, Jul 27, 2009 at 11:14 PM, alex finn wrote: > > Hello, > > I'm working on a huge django-based application that heavily utilizes > full-text search. Up until now I've been using external search service > built using SOLR that is deployed separately. > The problem is with such an approach I need to maintain 2 different > environments - one for python/django and one for java/solr. It's not a > big issue but I started to look for an alternative that will allow me > to build search functionality in the django application. I've found > several solutions so far: > 1. pylucene > 2. sphinx > 3. whoosh > > Now the problem is that I have no real experience with any of those > and would like to have some advise from those who tried any of the > options (or may be there is anything else available?). > PyLucene seems to be using java lucene library which may unnecessary > affect the application performance as it is "embeds a Java VM with > Lucene into a Python process". > I found some links in the internet saying that sphinx can not handle > non-ascii character sets well which is critical for me. > > Whoosh seems to be a little amateur - did not see any links on real > world usage. > > Did you try one of these options? Or do you know any other solutions > for the problem? As one data point - I use Sphinx fairly extensively at work. During development, I looked at Lucene as well, in the form of Solr (which is a nice wrapper around Lucene). Unfortunately, my experience has been that it's not as easy as saying "which one is better". Both have their advantages, and both have limitations. It depends upon the feature set that you are looking for in your searches. In particular, issues like: - Frequency with which your corpus will update - How much you need to update items in your index - The type of queries you want to issue (simple words? documents between two dates?) - The type of aggregate queries you want to issue (e.g., number of documents with a given author) Regarding non-ascii character sets - Sphinx can handle them fine, as long as it is correctly configured. There is documentation, but it adequate, not fantastic. A little experimentation will probably be required before you can be confident you've got things right. My recommendation in short is that Sphinx is probably worth a try. It isn't perfect, and it won't suit every user, but what it does, it does quite well, and it's a lot less heavyweight than a J2EE server. Yours, Russ Magee %-) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: Full-text search: what to use
Hallöchen! Russell Keith-Magee writes: > [...] > > As one data point - I use Sphinx fairly extensively at work. During > development, I looked at Lucene as well, in the form of Solr (which is > a nice wrapper around Lucene). I program a bibliography manager with Django. Users are able to upload the papers as PDFs, and I'd like to add full-text search. So far, I planned to user Solr Cell for it, but can one also do that with Sphinx? Tschö, Torsten. -- Torsten Bronger, aquisgrana, europa vetus Jabber ID: torsten.bron...@jabber.rwth-aachen.de or http://bronger-jmp.appspot.com --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: Full-text search: what to use
On Tue, Jul 28, 2009 at 1:58 PM, Torsten Bronger wrote: > > Hallöchen! > > Russell Keith-Magee writes: > >> [...] >> >> As one data point - I use Sphinx fairly extensively at work. During >> development, I looked at Lucene as well, in the form of Solr (which is >> a nice wrapper around Lucene). > > I program a bibliography manager with Django. Users are able to > upload the papers as PDFs, and I'd like to add full-text search. So > far, I planned to user Solr Cell for it, but can one also do that > with Sphinx? Sphinx doesn't provide any native utilities for handling PDF (that I am aware of), so you'll need to find a way to extract the text from the PDF and put it into your database ready for indexing. There are plenty of tools available to do this sort of processing. Their effectiveness depends very much on the quality of the PDF going in. PDFs that have been digitally produced (i.e. PDF printer) generally work pretty well; PDFs containing bitmaps of scanned pages are a minefield. Yours, Russ Magee %-) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: Full-text search: what to use
Thanks for your replies, it is useful. Has anyone used whoosh in production so far? Any ideas about it's performance? Alex. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: Full-text search: what to use
I've recently adopted the use of xapian for full text search, via the djapian django app which works like a charm. Xapian is written in C++ with python bindings, and with the djapian app u can stay within the django realm; e.g. updating the index can be done with manage.py index.. When using django, djapian seems to be the best option to integrate full text search (i'm using it with a postgresql db). I'm interested in what you think, as i haven't seen anyone mention xapian/djapian so far. I looked a bit into sphinx as well, but it seems djapian is far more easily set up, but i have no idea performancewise. Anyone any thoughts/experience with xapian/djapian? thanks, Domenic On Jul 28, 6:10 pm, alex finn wrote: > Thanks for your replies, it is useful. > > Has anyone used whoosh in production so far? Any ideas about it's > performance? > > Alex. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: Full-text search: what to use
Any thoughts on using PostgreSQL built-in full text search ? On Jul 29, 5:49 pm, "dlvosse...@gmail.com" wrote: > I've recently adopted the use of xapian for full text search, via the > djapian django app which works like a charm. Xapian is written in C++ > with python bindings, and with the djapian app u can stay within the > django realm; e.g. updating the index can be done with manage.py > index.. > > When using django, djapian seems to be the best option to integrate > full text search (i'm using it with a postgresql db). > > I'm interested in what you think, as i haven't seen anyone mention > xapian/djapian so far. I looked a bit into sphinx as well, but it > seems djapian is far more easily set up, but i have no idea > performancewise. > > Anyone any thoughts/experience with xapian/djapian? > > thanks, > Domenic > > On Jul 28, 6:10 pm, alex finn wrote: > > > > > Thanks for your replies, it is useful. > > > Has anyone used whoosh in production so far? Any ideas about it's > > performance? > > > Alex. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Full text search with PostgreSQL and Django
Hi!! I need to implement a search engine with full text search, I'm using postgreSQL and Django, I have searched in google ( http://barryp.org/blog/entries/postgresql-full-text-search-django/ ) but isn't there a more simple way ?? Or just another way? -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-us...@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Re: Full text search available on PostgreSQL?
On Saturday, June 28, 2014 7:15:05 PM UTC+3, Bastian Kuberek wrote: > > Hi, > > Just saw that django.contrib.admin.ModelAdmin.search_fields > <https://docs.djangoproject.com/en/dev/ref/contrib/admin/#django.contrib.admin.ModelAdmin.search_fields> > full > text search only support MySQL. > > Are there plans on any progress towards getting this functionality on > PostgreSQL? > Yes, see https://www.kickstarter.com/projects/mjtamlyn/improved-postgresql-support-in-django. - Anssi -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/2972b581-4fcc-4c3d-bf78-67a44e3f1278%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Full text search available on PostgreSQL?
you could have a look at djorm-ext-pgfulltex. bg, Johannes On 28.06.2014 18:15, Bastian Kuberek wrote: Hi, Just saw that django.contrib.admin.ModelAdmin.search_fields <https://docs.djangoproject.com/en/dev/ref/contrib/admin/#django.contrib.admin.ModelAdmin.search_fields> full text search only support MySQL. Are there plans on any progress towards getting this functionality on PostgreSQL? Thanks -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com <mailto:django-users+unsubscr...@googlegroups.com>. To post to this group, send email to django-users@googlegroups.com <mailto:django-users@googlegroups.com>. Visit this group at http://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/f0071b4f-a368-4512-a6f7-79d921e32141%40googlegroups.com <https://groups.google.com/d/msgid/django-users/f0071b4f-a368-4512-a6f7-79d921e32141%40googlegroups.com?utm_medium=email&utm_source=footer>. For more options, visit https://groups.google.com/d/optout. -- Johannes Schneider Webentwicklung johannes.schnei...@galileo-press.de Tel.: +49.228.42150.xxx Galileo Press GmbH Rheinwerkallee 4 - 53227 Bonn - Germany Tel.: +49.228.42.150.0 (Zentrale) .77 (Fax) http://www.galileo-press.de/ Geschäftsführer: Tomas Wehren, Ralf Kaulisch, Rainer Kaltenecker HRB 8363 Amtsgericht Bonn -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/53BB9D0F.6050403%40galileo-press.de. For more options, visit https://groups.google.com/d/optout.
Re: Full text search available on PostgreSQL?
This looks promising. Thanks -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CANRHea%3DMXR_2z4CSq21b1YoMtJad_GzjXG-RN%3DNV9h3eyjtg2A%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: Full text search available on PostgreSQL?
Also, not directly tied to ModelAdmin, but you might want to take a look at django-watson. HTH Jirka -Original Message- From: Johannes Schneider Sender: django-users@googlegroups.com Date: Tue, 08 Jul 2014 09:26:07 To: Reply-To: django-users@googlegroups.com Subject: Re: Full text search available on PostgreSQL? you could have a look at djorm-ext-pgfulltex. bg, Johannes On 28.06.2014 18:15, Bastian Kuberek wrote: > Hi, > > Just saw that django.contrib.admin.ModelAdmin.search_fields > <https://docs.djangoproject.com/en/dev/ref/contrib/admin/#django.contrib.admin.ModelAdmin.search_fields> > full > text search only support MySQL. > > Are there plans on any progress towards getting this functionality on > PostgreSQL? > > Thanks > > -- > You received this message because you are subscribed to the Google > Groups "Django users" group. > To unsubscribe from this group and stop receiving emails from it, send > an email to django-users+unsubscr...@googlegroups.com > <mailto:django-users+unsubscr...@googlegroups.com>. > To post to this group, send email to django-users@googlegroups.com > <mailto:django-users@googlegroups.com>. > Visit this group at http://groups.google.com/group/django-users. > To view this discussion on the web visit > https://groups.google.com/d/msgid/django-users/f0071b4f-a368-4512-a6f7-79d921e32141%40googlegroups.com > <https://groups.google.com/d/msgid/django-users/f0071b4f-a368-4512-a6f7-79d921e32141%40googlegroups.com?utm_medium=email&utm_source=footer>. > For more options, visit https://groups.google.com/d/optout. -- Johannes Schneider Webentwicklung johannes.schnei...@galileo-press.de Tel.: +49.228.42150.xxx Galileo Press GmbH Rheinwerkallee 4 - 53227 Bonn - Germany Tel.: +49.228.42.150.0 (Zentrale) .77 (Fax) http://www.galileo-press.de/ Geschäftsführer: Tomas Wehren, Ralf Kaulisch, Rainer Kaltenecker HRB 8363 Amtsgericht Bonn -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/53BB9D0F.6050403%40galileo-press.de. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/1509172023-1404852641-cardhu_decombobulator_blackberry.rim.net-509912240-%40b4.c3.bise7.blackberry. For more options, visit https://groups.google.com/d/optout.
Re: Full text search available on PostgreSQL?
Did you find a solution that worked with ModelAdmin.search_fields & Postgres? I'm running into the same question. thanx, -- Ben On Sat, Jun 28, 2014 at 11:15 PM, Bastian Kuberek wrote: > Hi, > > Just saw that django.contrib.admin.ModelAdmin.search_fields > <https://docs.djangoproject.com/en/dev/ref/contrib/admin/#django.contrib.admin.ModelAdmin.search_fields> > full > text search only support MySQL. > > Are there plans on any progress towards getting this functionality on > PostgreSQL? > > Thanks > > -- > You received this message because you are subscribed to the Google Groups > "Django users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to django-users+unsubscr...@googlegroups.com. > To post to this group, send email to django-users@googlegroups.com. > Visit this group at http://groups.google.com/group/django-users. > To view this discussion on the web visit > https://groups.google.com/d/msgid/django-users/f0071b4f-a368-4512-a6f7-79d921e32141%40googlegroups.com > <https://groups.google.com/d/msgid/django-users/f0071b4f-a368-4512-a6f7-79d921e32141%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- Chief Systems Architect Proteus Technologies <http://proteus-tech.com> Chief Fan Biggest Fan Productions <http://biggestfan.net> Personal blog where I am not your demographic <http://notyourdemographic.com>. This email intended solely for those who have received it. If you have received this email by accident - well lucky you!! -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAHN%3D9D7g09po2usFO7G0RAKD3nYmTdO%2B9B3xNuc6X_bfjeJZMQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
full-text search and mediawiki markup apps
I've taken the time to upload both our SphinxSearch layer, and our MediaWiki markup util to Google code today. For more info see my blog post: http://www.davidcramer.net/code/54/mediawiki-markup-and-sphinxsearch-... I haven't used Google code before, so if theres any problems just let me know. Enjoy. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
[White-paper] Full Text Search w/ MySQL
Hi all, I just wanted to let you know about a white-paper written by one of the guys in my office. "Extending Django's database API to include full-text search" <http://www.mercurytide.com/knowledge/white-papers/django-full-text-search> We spent a little bit of time coming up with some generic search code that we could use on our most recent work hijobs.co.uk and thought the community would benefit from our work. Enjoy, and keep your eyes peeled for more of the same in coming months. Cheers, John -- [EMAIL PROTECTED] --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users -~--~~~~--~~--~--~---
Pg Full Text search in the ORM
Hi, does anyone know how to translate the following query to the ORM?
select * from company where to_tsvector(name) @@ to_tsquery('gold:*')
So far I have the following, but cannot seem to integrate the [:*] piece.
Company.objects.annotate(search = SearchVector('name')).filter(search =
SearchQuery('gold'))
TIA
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-users/CAMZO7wK99QAXeoEQCxCcyFc9RMvOQ_YPpmfn4m2o7rFT6UNvhg%40mail.gmail.com.
Full text search in multi language columns
I am trying to implement a full text search in django 1.11, in my model product_name and foreign_name can have values in English and Spanish, in pastebin <https://pastebin.com/RQS1AHtN> I have published my model with a manually created index which works very well with the query, I would like to transpose that query to a queryset using both languages english and spanish, can anybody help me? -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at https://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/ad68da0b-5bc8-436f-9cce-7c5e0c4fe173%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Postgres Full-text search -- is it possible?
It can be done using extra as shown here: http://barryp.org/blog/entries/postgresql-full-text-search-django/ there isn't a way built into django to do it however, there are also a few external search projects to create a nicer search api in django. This one: http://code.google.com/p/djangosearch/ specifically was worked on as a part of the google summer of code and perhaps you could speak with them about a pg backend if they don't have one already. Alex On Dec 15, 2:35 pm, Info Cascade wrote: > Hi -- > > I'm wanting to improve searches by using full-text searching. The > documentation says: > > > search¶ > > <http://docs.djangoproject.com/en/dev/ref/models/querysets/#search> > > > A boolean full-text search, taking advantage of full-text indexing. > > This is like contains but is significantly faster due to full-text > > indexing. > > > Note this is only available in MySQL and requires direct manipulation > > of the database to add the full-text index. > > What options are available for Postgres if any? Postgres has this > capability, I believe. Can Django not use it? > > Liam --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: Full text search with PostgreSQL and Django
http://haystacksearch.org/ 2010/12/17 marcoarreguin > Hi!! > > I need to implement a search engine with full text search, I'm using > postgreSQL and Django, I have searched in google ( > http://barryp.org/blog/entries/postgresql-full-text-search-django/ ) > but isn't there a more simple way ?? > > Or just another way? > > -- > You received this message because you are subscribed to the Google Groups > "Django users" group. > To post to this group, send email to django-us...@googlegroups.com. > To unsubscribe from this group, send email to > django-users+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/django-users?hl=en. > > -- Benito Jorge Bastida jo...@thecodefarm.com thecodefarm SL Av. Gasteiz 21, 1º Derecha 01008 Vitoria-Gasteiz http://thecodefarm.com Tel: (+34) 945 06 55 09 -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-us...@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Re: full-text search and mediawiki markup apps
Just yesterday, I was thinking how useful it would be to be able to use mediawiki markup syntax for a new project... and voila -- my prayers are answered! Thank you for putting this together! One problem... I'm getting a syntax error on import (using python 2.5): >>> import wikimarkup Traceback (most recent call last): File "", line 1, in File "C:\django\django\Python25\Lib\site-packages\wikimarkup \__init__.py", lin e 28 _attributePat = re.compile(ur'(?:^|\s)([A-Za-z0-9]+)(?:\s*= \s*(?:"([^<"]*)"| '([^<']*)'|([a-zA-Z0-9!#$%&()*,\-./:;<>[EMAIL PROTECTED]|}~]+)|#([0-9a-fA-F] +)))', re.UN ICODE) ^ SyntaxError: invalid syntax Am I doing something wrong? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: full-text search and mediawiki markup apps
Hello friends, I recently happened to see the below information ... just thought that it could be useful to you people ... Thanks. -- *Introducing for the first time in **India**, a Product that could be required for all types of Industries and Consultants in **India** and across the World.* *We have Quality Databases of all types of Doctors, Engineers and Graduates in **India**.* *Engineers*: Computer, Software, Electronics, Civil, Mechanical, Production, Chemical, Pharmacy, PhDs, etc. *Medical*: Doctors, Surgeons, Physicians, PhDs, etc. *Graduates*: Physics, Chemistry, Biology, Agriculture, etc. The database is in the form of Excel Sheets and Word Documents. The word format document contains the resume of the concerned person. The database is divided into the following catagories: 1. *By Functional Area*: Basically technical people working in projects, production, maintenance, software, quality control, research & development, etc. The number of resumes in each category, are around 30,000. 2. *By Industrial Segment*: Pharmaceutical, Drugs, Chemical, Marine, Electronics, Medical, Shipping, Construction, Automotives, Hotels, Computers, Rural Areas, and Government of India Companies (Railways, Municipality, Atomic Energy, Electricity, Power, Water Supply, etc.) The number of resumes in each segment, are around 30,000. 3. *Infotech & Software Personnel*: MCAs, BCAs, Computer/Electronics &Telecom Engineers, etc. Around 1,00,000 resumes. 4. Company wise data are also available, around 50 reputed companies in India. *The basic purpose of this database is that the consumer can market his product/s sitting from his office, making cold calls or by mailing directly to the concerned personnel. * The Resume Database comprises of people who have maximum experience in their relevant field. The Resume Database does not contain personnel from the Accounts/Finance, Sales & Business Development department and the Banking Sector. The reason is that these people are comparatively easily accessible. Although Engineers (Technical personnel) working in sales, marketing and business development fields are available. *This Database is most suited and useful for Recruitment Consultants in ** India** and Abroad. *If the consultant is not subscribed to the expensive portals then this data would be most useful to them. *This Data is also useful to small and medium sized companies who have limited resources, who cannot employ a battery of sales personnel.* Sales people who keep on moving around in the market are susceptible to change their job. They can spare some time to apply for other jobs and attend the interview, while on outdoor field. Hence sales and marketing people keep on changing their jobs after having short stints with companies. Once a Salesman leaves a company, he also takes away with him the market information and the intellectual database, sometimes. *Frequently Asked Questions* *Why Resumes as a database?* Resumes don't lie ! A Resume gives the full detail of a person for example, the residential phone no. and address, the mobile/cell phone no., the email address, the job responsibilities, the references, etc. These details help to contact that key person with ease and perfection. Most importantly it saves time and money. *Money! How does it save money?* You save money on phone calls. For example you want to contact the Maintenance Manager of M/s.XYZ Corporation, you visit the companies website wherein you may not find the required details (like the plant address, the contact details, the contact person, the email address of that Manager, etc.). You hence call the telephone operator, who will give you the half information only (the telephone operator is normally not allowed to disclose the cell phone numbers, email IDs, etc. of key contact personnel). Ultimately to reach that department or that key man you spend some good amount of money on phone calls. *Is the database perfect and totally updated? * Honestly we believe that our database is nearly 94% correct. Because people change companies; they give the temporary home address in their resumes; sometimes email IDs are also changed; etc. But in India normally people mention their permanent address in their resumes which rarely changes. *How do we search in this huge database ? * By using the Microsoft search method, typing out the key words and selecting the particular folder in the options. The simplest method is to open each resume. *How is the supply of the database done?* We shall courier you the CD/DVD containing the database to your said address. *Still Confused? A typical example is given below * A Company manufacturing Medicines (tablets, syrups, injections, etc). Normally, the method of sales is that the company employs MRs (Medical Representatives) to promote
Re: [White-paper] Full Text Search w/ MySQL
On 8/2/06, John Sutherland <[EMAIL PROTECTED]> wrote: > I just wanted to let you know about a white-paper written by one of > the guys in my office. > > "Extending Django's database API to include full-text search" > <http://www.mercurytide.com/knowledge/white-papers/django-full-text-search> Hey John, I noticed this the other day and thought it was a great writeup. Nicely done! Adrian -- Adrian Holovaty holovaty.com | djangoproject.com --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users -~--~~~~--~~--~--~---
Tutorial: Full text search with Lupy in Django
I've written a small tutorial about using Lupy - a full text python search engine and Django: Link: http://www.rkblog.rk.edu.pl/w/p/django-lupy/ I'll more code when I'll add Lupy search support to Diamanda :) Django full-text branch isn't finished and won't be in the near and not near future from what I see... --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
How to implement full-text search in Django?
Hi all, I would like some input on implementing a full-text search engine for my website and blog. I really don't want to use Google search API or a external webservice to index my documents. Also, my blog is powered by ZODB while my main website uses Durus. Ideally, i would like to use Redis server to index my documents and Ajax/jQuery to make a search using JSON. I have considered Elasticsearch, but it requires Java, and I don't really like this. Is there any other alternatives for indexing non-relational database objects such as ZODB ? What do you think? Etienne -- Etienne Robillard tkad...@yandex.com https://www.isotopesoftware.ca/ -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at https://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/02c550ee-57c9-a7b6-128b-343693c98cbe%40yandex.com. For more options, visit https://groups.google.com/d/optout.
Public Domain ( was Re: Full Text Search w/ MySQL)
On 03/08/2006, at 10:42 PM, Andrew Durdin wrote: the code in the paper is released intothe public domain, so there's nothing stopping any interested personfrom taking it and developing it into something more suited for generalsearching.AndrewThis isn't directed at you, or your paper Andrew.I know this sounds counter-intuitive, but can you (and others) not put things in the public domain?Go have a look at http://www.rosenlaw.com/html/GL15a.pdf It's 2 pages, and VERY light reading, writtenby a lawyer and a expert on Open Source Licenses.To quote the paper: This “Give-It-Away” license provides no protection for anyone if the donated software causes harm.ie.. even if you didn't mean it, if I used a piece of public domain code in a app I write and it fails,i can still go after the person who wrote it and sue them.thanks.. that makes my chest feel much lighter. --Ian Holsman[EMAIL PROTECTED]http://economy-chat.com It's what the economists talk about --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users -~--~~~~--~~--~--~---
django dont support select_related when use full-text search?
i seems django dont support select_related when using full-text search filter. a.objects.filter(field1__search=q), it can return results successfully, but when i wrote: a.objects.select_related.filter(field1__search=q),it returns nothing. why that happen? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users -~--~~~~--~~--~--~---
Re: How to implement full-text search in Django?
Check out this library https://github.com/django-haystack/django-haystack It supports a wide range of backends. On 16/12/2017, Etienne Robillard wrote: > Hi all, > > I would like some input on implementing a full-text search engine for my > website and blog. > > I really don't want to use Google search API or a external webservice to > index my documents. > > Also, my blog is powered by ZODB while my main website uses Durus. > > Ideally, i would like to use Redis server to index my documents and > Ajax/jQuery to make a search using JSON. > > I have considered Elasticsearch, but it requires Java, and I don't > really like this. > > Is there any other alternatives for indexing non-relational database > objects such as ZODB ? > > What do you think? > > > Etienne > > > -- > Etienne Robillard > tkad...@yandex.com > https://www.isotopesoftware.ca/ > > -- > You received this message because you are subscribed to the Google Groups > "Django users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to django-users+unsubscr...@googlegroups.com. > To post to this group, send email to django-users@googlegroups.com. > Visit this group at https://groups.google.com/group/django-users. > To view this discussion on the web visit > https://groups.google.com/d/msgid/django-users/02c550ee-57c9-a7b6-128b-343693c98cbe%40yandex.com. > For more options, visit https://groups.google.com/d/optout. > -- Kind Regards, Egor Smolyakov. -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at https://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CACL14xrDjq1qGwJrdSx70qGJz%3DdFr%2BRRVj-jeDFxnVCmQWYw9Q%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: How to implement full-text search in Django?
Thanks Egor. Django-haystack looks quite flexible and well-designed. My current plan is to use redisearch plugin for Redis: http://redisearch.io/ I'm thinking that I could write a script to import into redis a list of URLs by fetching a RSS file as input and then use redisearch to perform a full-text search query and return a JSON stream. My goal is to not duplicate data between redis and zodb... What do you think? Etienne Le 2017-12-16 à 06:05, Egor Smolyakov a écrit : Check out this library https://github.com/django-haystack/django-haystack It supports a wide range of backends. On 16/12/2017, Etienne Robillard wrote: Hi all, I would like some input on implementing a full-text search engine for my website and blog. I really don't want to use Google search API or a external webservice to index my documents. Also, my blog is powered by ZODB while my main website uses Durus. Ideally, i would like to use Redis server to index my documents and Ajax/jQuery to make a search using JSON. I have considered Elasticsearch, but it requires Java, and I don't really like this. Is there any other alternatives for indexing non-relational database objects such as ZODB ? What do you think? Etienne -- Etienne Robillard tkad...@yandex.com https://www.isotopesoftware.ca/ -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at https://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/02c550ee-57c9-a7b6-128b-343693c98cbe%40yandex.com. For more options, visit https://groups.google.com/d/optout. -- Etienne Robillard tkad...@yandex.com https://www.isotopesoftware.ca/ -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at https://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/21edc431-64dd-e7f5-5d5f-4a9d522ca985%40yandex.com. For more options, visit https://groups.google.com/d/optout.
Re: Public Domain ( was Re: Full Text Search w/ MySQL)
Op vr, 04-08-2006 te 16:55 +1000, schreef Ian Holsman: > To quote the paper: This “Give-It-Away” license provides no protection > for anyone if the donated software causes harm. > > ie.. even if you didn't mean it, if I used a piece of public domain > code in a app I write and it fails, i can still go after the person > who wrote it and sue them. I guess that's related to the fact that you can't put something in the public domain according to the EU law on author's rights. You can give everyone permission to copy & (re)distribute & (re)publish it for free forever, but you can't legally say "I'm not the author anymore" or "I refuse all my rights (and plights) as an author". -- Jan Claeys --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users -~--~~~~--~~--~--~---
how to select related field in full-text search filter?
it seems like django dont support select_related when using full-text search? a.objects.filter(field1__search=str) can show the results successful but: a.objects.select_related().filter(field1__search=str) show nothing why that happen? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users -~--~~~~--~~--~--~---
Storing IP address as integer within database to remove need for full text search
Hi, I have created a ModelField called RealIPAddressField. It stores the IP address in integer form, meaning the lookups on large tables are much faster: http://djangosnippets.org/snippets/2493/ @django-developers - Do you think there is any possibility of this getting included into the core? @django-users - Hope this is of some use to someone else, would appreciate any criticisms / improvements. Cal -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Re: Storing IP address as integer within database to remove need for full text search
On Mon, Jul 18, 2011 at 9:56 AM, Cal Leeming [Simplicity Media Ltd] wrote: > It stores the IP address in integer form, meaning the lookups on large > tables are much faster: are they?hashtables shouldn't be too sensitive to key size, as long as the string size stays bounded... like on IP addresses (max 15 chars) of course, i don't know about the specific dict implementation on Python. Sounds like a job for a microbenchmark! -- Javier -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Re: Storing IP address as integer within database to remove need for full text search
Sorry, I should have been a little more specific. I meant faster lookups in terms of database index, such as MySQL with InnoDB. Cal On Mon, Jul 18, 2011 at 4:13 PM, Javier Guerra Giraldez wrote: > On Mon, Jul 18, 2011 at 9:56 AM, Cal Leeming [Simplicity Media Ltd] > wrote: > > It stores the IP address in integer form, meaning the lookups on large > > tables are much faster: > > are they?hashtables shouldn't be too sensitive to key size, as > long as the string size stays bounded... like on IP addresses (max 15 > chars) > > of course, i don't know about the specific dict implementation on > Python. Sounds like a job for a microbenchmark! > > -- > Javier > > -- > You received this message because you are subscribed to the Google Groups > "Django users" group. > To post to this group, send email to django-users@googlegroups.com. > To unsubscribe from this group, send email to > django-users+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/django-users?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Re: Storing IP address as integer within database to remove need for full text search
Or maybe you were talking about hash tables in MySQL. Either way, you're right that this would probably need some benchmarks before being approved for the core. @django-developers, if I was to provide some benchmarks, would this possibly be considered for the core? Cal On Mon, Jul 18, 2011 at 4:15 PM, Cal Leeming [Simplicity Media Ltd] < cal.leem...@simplicitymedialtd.co.uk> wrote: > Sorry, I should have been a little more specific. > > I meant faster lookups in terms of database index, such as MySQL with > InnoDB. > > Cal > > > On Mon, Jul 18, 2011 at 4:13 PM, Javier Guerra Giraldez < > jav...@guerrag.com> wrote: > >> On Mon, Jul 18, 2011 at 9:56 AM, Cal Leeming [Simplicity Media Ltd] >> wrote: >> > It stores the IP address in integer form, meaning the lookups on large >> > tables are much faster: >> >> are they?hashtables shouldn't be too sensitive to key size, as >> long as the string size stays bounded... like on IP addresses (max 15 >> chars) >> >> of course, i don't know about the specific dict implementation on >> Python. Sounds like a job for a microbenchmark! >> >> -- >> Javier >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Django users" group. >> To post to this group, send email to django-users@googlegroups.com. >> To unsubscribe from this group, send email to >> django-users+unsubscr...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/django-users?hl=en. >> >> > -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Re: Storing IP address as integer within database to remove need for full text search
On Mon, Jul 18, 2011 at 10:13 AM, Javier Guerra Giraldez wrote: > On Mon, Jul 18, 2011 at 9:56 AM, Cal Leeming [Simplicity Media Ltd] > wrote: >> It stores the IP address in integer form, meaning the lookups on large >> tables are much faster: > > are they? hashtables shouldn't be too sensitive to key size, as > long as the string size stays bounded... like on IP addresses (max 15 > chars) OTOH, for really huge database tables, making the index 4 times smaller can be a significant difference on RAM-starved servers but to fill 1G can hold something like of 50million IP addresses in text form... -- Javier -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Re: Storing IP address as integer within database to remove need for full text search
I like the idea, but I don't like the approach, it should be a subclass of PositiveIntegerField, as it is an unsigned int on DB level. Also, I agree with Łukasz that it should support both IPv6 and IPv4. -- Best wishes, Dmitry Gladkov, mailto:dmitry.glad...@gmail.com +380 91 303-37-46 On Mon, Jul 18, 2011 at 6:21 PM, Javier Guerra Giraldez wrote: > On Mon, Jul 18, 2011 at 10:13 AM, Javier Guerra Giraldez > wrote: >> On Mon, Jul 18, 2011 at 9:56 AM, Cal Leeming [Simplicity Media Ltd] >> wrote: >>> It stores the IP address in integer form, meaning the lookups on large >>> tables are much faster: >> >> are they? hashtables shouldn't be too sensitive to key size, as >> long as the string size stays bounded... like on IP addresses (max 15 >> chars) > > OTOH, for really huge database tables, making the index 4 times > smaller can be a significant difference on RAM-starved servers > but to fill 1G can hold something like of 50million IP addresses in > text form... > > -- > Javier > > -- > You received this message because you are subscribed to the Google Groups > "Django developers" group. > To post to this group, send email to django-develop...@googlegroups.com. > To unsubscribe from this group, send email to > django-developers+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/django-developers?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Re: Storing IP address as integer within database to remove need for full text search
On 18 Jul 18:28, Dmitry Gladkov wrote: > I like the idea, but I don't like the approach, it should be a > subclass of PositiveIntegerField, as it is an unsigned int on DB > level. Also, I agree with Łukasz that it should support both IPv6 and > IPv4. IPAddressField appears to be v4 only for backwards compatibilty reasons, GenericIPAddressField covers both, so in theory I've got no problems with that, other than it making IPAddressField backwards incompatible which seems silly at this point, and I'd be recommending people to be using GenericIPAddressField anyways. -- Brett Parker -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Supporting a wider range of awesome PostgreSQL features in Django - including hstore, JSON and full text search
I found this on django's blog: https://www.kickstarter.com/projects/mjtamlyn/improved-postgresql-support-in-django Very interesting! Anybody knows if they intend to redevelop everything from scratch or reuse and improve existing implementations? Federico -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/0ce88480-2143-4e40-9b09-ca47c3234b52%40googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Supporting a wider range of awesome PostgreSQL features in Django - including hstore, JSON and full text search
On Sun, Feb 23, 2014 at 10:45 PM, Federico Capoano wrote: > I found this on django's blog: > > > https://www.kickstarter.com/projects/mjtamlyn/improved-postgresql-support-in-django > > Very interesting! > > Anybody knows if they intend to redevelop everything from scratch or reuse > and improve existing implementations? > > Marc will be the ultimate authority on this, but from my conversations with him, I believe that his intention is to: * use existing implementations where possible, * merge the best parts of existing implementations where multiple options exist, * rebuild implementations that need significant improvement, and * build from scratch where no existing solutions exist. Yours, Russ Magee %-) -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAJxq8483yYq_M_yiuYA6q3its0TV2FVPb133e-yyde57b_tNew%40mail.gmail.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Supporting a wider range of awesome PostgreSQL features in Django - including hstore, JSON and full text search
Is it somehow planned that Django Users can dialog with him to express some desires ? For example, I would be very happy to see a reimplementation of the annotate and aggregate functions, as they make the queries so slow by grouping by all the fields that I always have to use raw SQL to group only by id. In one case in my admin, it is trying to group by 40 fields (I have 5 foreign keys and more than 20 fields) instead of 4 IDs and it requires 6 seconds instead of 160 ms... All the best to Marc On Monday, February 24, 2014 2:59:28 AM UTC+1, Russell Keith-Magee wrote: > > > On Sun, Feb 23, 2014 at 10:45 PM, Federico Capoano > > > wrote: > >> I found this on django's blog: >> >> >> https://www.kickstarter.com/projects/mjtamlyn/improved-postgresql-support-in-django >> >> Very interesting! >> >> Anybody knows if they intend to redevelop everything from scratch or >> reuse and improve existing implementations? >> >> > Marc will be the ultimate authority on this, but from my conversations > with him, I believe that his intention is to: > > * use existing implementations where possible, > * merge the best parts of existing implementations where multiple options > exist, > * rebuild implementations that need significant improvement, and > * build from scratch where no existing solutions exist. > > Yours, > Russ Magee %-) > -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/58ed5785-e424-42f2-b560-0692ce66fc43%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Supporting a wider range of awesome PostgreSQL features in Django - including hstore, JSON and full text search
On Mon, Mar 10, 2014 at 7:22 PM, Matthieu Rigal wrote: > Is it somehow planned that Django Users can dialog with him to express > some desires ? > > For example, I would be very happy to see a reimplementation of the > annotate and aggregate functions, as they make the queries so slow by > grouping by all the fields that I always have to use raw SQL to group only > by id. > > In one case in my admin, it is trying to group by 40 fields (I have 5 > foreign keys and more than 20 fields) instead of 4 IDs and it requires 6 > seconds instead of 160 ms... > Again, I can't speak for Marc; once the Kickstarter has finalised, I'm sure he'll be in contact with backers to describe the process he'll take. I'd be guessing it's either going to be through his blog, or through django-developers (and possibly both, depending on the level of interaction you're looking for). That said, the issue you describe sounds like it's outside the scope of what he's looking at. The annotate query you describe may be sub-optimal, but it does *work*. Marc's project is looking at adding the features that PostgreSQL has, but aren't currently accessible using Django's core feature set. This means new field types, new index types, and so on. This is all possible by making a concerted effort to use the features Django already provides, like custom fields and custom lookups. What you've described is a specific SQL query optimization that presumably would have applicability across multiple database backends. If you've got a proposal for what this would look like at an API level, feel free to propose it. However, keep in mind that Django's ORM is very deliberately *not* a SQL generation engine. There are very few "SQL" concepts at the API level, and discussing "grouping" clauses is very much in that space. You may find that the answer to your question is "use raw SQL". Yours, Russ Magee %-) -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at http://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAJxq84_hkv9AAXg-b%2BC1usw-6F%3DW22SkfasMyv4jH9Se%2BwAu7Q%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
