Re: Input raw log file

2011-01-12 Thread Peter Karich
Dinesh, it will stay 'real time' even if you convert it. Converting should be done in the millisecond range if at all measureable (e.g. if you apply streaming). Beware: To use the real features you'll need the latest trunk of solr IMHO. I've done similar log-feeding stuff here (with code!): http

Re: Input raw log file

2011-01-12 Thread Dinesh
i got some idea like creating a DIH and then doing with that.. thanks every one for the help.. hope i'll create an regex DIH i guess that's right.. -- View this message in context: http://lucene.472066.n3.nabble.com/Input-raw-log-file-tp2210043p2239947.html Sent from the Solr - User mailing list

Re: Input raw log file

2011-01-12 Thread Gora Mohanty
On Wed, Jan 12, 2011 at 12:10 PM, Dinesh wrote: > > if i convert it to CSV or XML then it will be time consuming cause the > indexing and getting data out of it should be real time.. is there any way i > can do other than this.. if not what are the ways i can convert them to CSV > and XML.. and la

Re: Input raw log file

2011-01-12 Thread Gora Mohanty
On Wed, Jan 12, 2011 at 11:50 AM, Dinesh wrote: > > I have installed and tested the sample xml file and tried indexing.. > everything went successful and when i tried with log files i got an error.. Please provide details of what you are doing, and of the error messages. How exactly are you sendi

Re: Input raw log file

2011-01-11 Thread Dennis Gearon
11 10:46:20 PM Subject: Re: Input raw log file First thing is that your raw log files solr can not understand. Solr needs data according to schema defined And also solr does not know your log file format . So you have to write a parser program that will parse your log files into a existing solr wr

Re: Input raw log file

2011-01-11 Thread Grijesh.singh
First thing is that your raw log files solr can not understand. Solr needs data according to schema defined And also solr does not know your log file format . So you have to write a parser program that will parse your log files into a existing solr writable formats .Then you can be able to index

Re: Input raw log file

2011-01-11 Thread Dinesh
if i convert it to CSV or XML then it will be time consuming cause the indexing and getting data out of it should be real time.. is there any way i can do other than this.. if not what are the ways i can convert them to CSV and XML.. and lastly which is the doc folder of solr -- View this message

Re: Input raw log file

2011-01-11 Thread Grijesh.singh
It will not work. I think your log files are not in solr Doc xml files. First thing is that your log files is raw data. you have to convert it to any of solr readable data either in solr xml DOC or CSV format to index on solr As Gora suggested to you. - Grijesh -- View this message in conte

Re: Input raw log file

2011-01-11 Thread Dinesh
i copied it to the same exampledocs folder and did #java -jar post.jar log.txt and i got SimplePostTool: version 1.2 SimplePostTool: WARNING: Make sure your XML documents are encoded in UTF-8, other encodings are not currently supported SimplePostTool: POSTing files to http://localhost:8983/solr

Re: Input raw log file

2011-01-11 Thread Grijesh.singh
How you parsed you log? Which way you gone for index of log file data? have you done any work what Gora Mohanty has suggested to you. I am local for Delhi NCR area. - Grijesh -- View this message in context: http://lucene.472066.n3.nabble.com/Input-raw-log-file-tp2210043p2239505.html Sent

Re: Input raw log file

2011-01-11 Thread Dinesh
I have installed and tested the sample xml file and tried indexing.. everything went successful and when i tried with log files i got an error.. i tried reading the schema.xml and didn't get a clear idea.. can you please help.. -- View this message in context: http://lucene.472066.n3.nabble.co

Re: Input raw log file

2011-01-11 Thread Gora Mohanty
On Tue, Jan 11, 2011 at 10:06 AM, Dinesh wrote: > > can u give an example.. like something that is currently being used.. Sorry, I do not have anything like this at hand at the moment. > > i'am an > engin

Re: Input raw log file

2011-01-10 Thread Dinesh
can u give an example.. like something that is currently being used.. i'am an engineering student and my project is to index all the real time log files from different devices and use some artificial intelligence and produce a usefull data out of it.. i'm doing this for my college.. i'm struggling

Re: Input raw log file

2011-01-08 Thread Gora Mohanty
On Sat, Jan 8, 2011 at 3:50 PM, Dinesh wrote: > > i don't have much idea about converting log into CSV and then giving it as > input.. can u please specify how to do it excatly.. [...] As the format of the raw log file is known only to you, it is difficult for someone to give you advice on this.

Re: Input raw log file

2011-01-08 Thread Dinesh
i don't have much idea about converting log into CSV and then giving it as input.. can u please specify how to do it excatly.. -- View this message in context: http://lucene.472066.n3.nabble.com/Input-raw-log-file-tp2210043p2216083.html Sent from the Solr - User mailing list archive at Nabble.co

Re: Input raw log file

2011-01-07 Thread Grijesh.singh
There is a csv update handler in solr you can use it by modifying your logfile - Grijesh -- View this message in context: http://lucene.472066.n3.nabble.com/Input-raw-log-file-tp2210043p2210673.html Sent from the Solr - User mailing list archive at Nabble.com.