How to keep user search history and how to turn it into information?

2007-08-10 Thread Lukas Vlcek
Hi, I would like to keep user search history data and I am looking for some ideas/advices/recommendations. In general I would like to talk about methods of storing such data, its structure and how to turn it into valuable information. As for the structure: == For now I don't have exac

Re: How to keep user search history and how to turn it into information?

2007-08-10 Thread Lukas Vlcek
Enis, Thanks for your time. I gave a quick glance at Pig and it seems good (seems it is directly based on Hadoop which I am starting to play with :-). It obvious that a huge amount of data (like user queries or access logs) should be stored in flat files which makes it convenient for further analy

Re: How to keep user search history and how to turn it into information?

2007-08-10 Thread Lukas Vlcek
Hi Enis, On 8/10/07, Enis Soztutar <[EMAIL PROTECTED]> wrote: > > Hi, > > Lukas Vlcek wrote: > > Hi, > > > > I would like to keep user search history data and I am looking for some > > ideas/advices/recommendations. In general I would like to talk about > methods > > of storing such data, its stru

Re: How to keep user search history and how to turn it into information?

2007-08-10 Thread Enis Soztutar
Hi, Lukas Vlcek wrote: Hi, I would like to keep user search history data and I am looking for some ideas/advices/recommendations. In general I would like to talk about methods of storing such data, its structure and how to turn it into valuable information. As for the structure: ==

Re: How to keep user search history and how to turn it into information?

2007-08-10 Thread Dmitry
earch / Hostory/ Retrieval History/Cache Management... Thanks, dt www.ejinz.com Search News - Original Message - From: "Lukas Vlcek" <[EMAIL PROTECTED]> To: Sent: Friday, August 10, 2007 2:28 AM Subject: How to keep user search history and how to turn it into information?

Re: How to keep user search history and how to turn it into information?

2007-08-10 Thread Lukas Vlcek
iddle tier to catch all event concerning Users Search / Hostory/ > Retrieval History/Cache Management... > Thanks, > dt > www.ejinz.com > Search News > > - Original Message - > From: "Lukas Vlcek" <[EMAIL PROTECTED]> > To: > Sent: Friday, August

Re: How to keep user search history and how to turn it into information?

2007-08-10 Thread Enis Soztutar
Lukas Vlcek wrote: Hi Enis, Hi again, On 8/10/07, Enis Soztutar <[EMAIL PROTECTED]> wrote: Hi, Lukas Vlcek wrote: Hi, I would like to keep user search history data and I am looking for some ideas/advices/recommendations. In general I would like to talk about methods

Re: How to keep user search history and how to turn it into information?

2007-08-13 Thread Enis Soztutar
Hi, Lukas Vlcek wrote: Enis, Thanks for your time. I gave a quick glance at Pig and it seems good (seems it is directly based on Hadoop which I am starting to play with :-). It obvious that a huge amount of data (like user queries or access logs) should be stored in flat files which makes it co

Re: How to keep user search history and how to turn it into information?

2007-08-13 Thread Lukas Vlcek
Enis, thanks for excellent answer! Lukas On 8/13/07, Enis Soztutar <[EMAIL PROTECTED]> wrote: > > Hi, > > Lukas Vlcek wrote: > > Enis, > > > > Thanks for your time. > > I gave a quick glance at Pig and it seems good (seems it is directly > based > > on Hadoop which I am starting to play with :-).

Re: How to keep user search history and how to turn it into information?

2007-08-13 Thread karl wettin
13 aug 2007 kl. 12.49 skrev Lukas Vlcek: But I am looking for more IR oriented application of this information. I remember that once I read on Lucene mail list that somebody suggested utilization of previously issued user queries for suggestions of similar/other/related queries or for typo

Re: How to keep user search history and how to turn it into information?

2007-08-14 Thread Peter W.
Hey Lukas, You can get a basic demo of this working in Lucene first then make a more advanced and efficient version. First, give each document in your index a score field using NumberTools so it's sortable. When users perform a search, log the unique document_id, IP address and result position f

Re: How to keep user search history and how to turn it into information?

2007-08-14 Thread Peter W.
Lukas, One last thing, be sure to log only when a user clicks on a result and in Hadoop document_id will be a key in the map phase. Lucene related steps are the same. Best, Peter W. On Aug 14, 2007, at 1:28 PM, Peter W. wrote: When users perform a search, log the unique document_id, IP add