Re: changing schema is dynamic or not

2008-12-04 Thread Noble Paul നോബിള്‍ नोब्ळ्
http://wiki.apache.org/solr/CoreAdmin#head-3f125034c6a64611779442539812067b8b430930 On Thu, Dec 4, 2008 at 4:06 PM, Neha Bhardwaj [EMAIL PROTECTED] wrote: Could you brief ,What exactly I need to do? -Original Message- From: Noble Paul നോബിള്‍ नोब्ळ् [mailto:[EMAIL PROTECTED] Sent

Re: Is there a clean way to determine whether a core exists?

2008-12-04 Thread Noble Paul നോബിള്‍ नोब्ळ्
SOLR-880 is an issue raised for the same On Fri, Dec 5, 2008 at 10:16 AM, Chris Hostetter [EMAIL PROTECTED] wrote: : Subject: Is there a clean way to determine whether a core exists? doesn't the CoreAdminHandler's STATUS feature make this easy? -Hoss -- --Noble Paul

Re: DataImportHandler - time stamp format in

2008-12-05 Thread Noble Paul നോബിള്‍ नोब्ळ्
I gguess you are trying to pass it in the SQL query. Tryit as it is . If oracle does not take it you can format the date according to what oracle likes http://wiki.apache.org/solr/DataImportHandler#head-5675e913396a42eb7c6c5d3c894ada5dadbb62d7 On Fri, Dec 5, 2008 at 8:09 PM, Jae Joo [EMAIL

Re: Can Solr follow links?

2008-12-05 Thread Noble Paul നോബിള്‍ नोब्ळ्
Look at http://wiki.apache.org/solr/DataImportHandler You may use an outer entity with SqlEntityProcessor and an inner entity with XPathEntityProcessor On Fri, Dec 5, 2008 at 5:35 PM, Joel Karlsson [EMAIL PROTECTED] wrote: Hello, Is there any way for Solr to follow links stored in my

Re: Delta-import hack to use last indexed id document

2008-12-06 Thread Noble Paul നോബിള്‍ नोब्ळ्
On Sat, Dec 6, 2008 at 10:59 PM, Marc Sturlese [EMAIL PROTECTED] wrote: Hey there, I am doing some hacks to some parts of the solr source. I am doing a feature for everytime I use delta import hanlder I want it to start geting info from the db starting from the last indexed document id (from

Re: Help with Solr configuration for LuSql performance comparison

2008-12-08 Thread Noble Paul നോബിള്‍ नोब्ळ्
On Tue, Dec 9, 2008 at 10:26 AM, Chris Hostetter [EMAIL PROTECTED] wrote: : First, LuSql default uses Lucene's StandardAnalyzer[4]. The Javadocs : indicates it uses StandardTokenizer[5], StandardFilter[6], : LowerCaseFilter[7], and StopFilter[8]. I have created a fieldType in : my Solr

Re: Help with Solr configuration for LuSql performance comparison

2008-12-08 Thread Noble Paul നോബിള്‍ नोब्ळ्
For the nested entities there is a feature called CachedSqlEntityProcessor to make things faster (if you have enough RAM) On Tue, Dec 9, 2008 at 10:36 AM, Noble Paul നോബിള്‍ नोब्ळ् [EMAIL PROTECTED] wrote: On Tue, Dec 9, 2008 at 10:26 AM, Chris Hostetter [EMAIL PROTECTED] wrote: : First

Re: solr performance

2008-12-09 Thread Noble Paul നോബിള്‍ नोब्ळ्
I guess this is the best idea . Let us have a new BatchHttpSolrServer which can help achieve this --Noble On Thu, Dec 4, 2008 at 7:14 PM, Yonik Seeley [EMAIL PROTECTED] wrote: On Thu, Dec 4, 2008 at 8:39 AM, Mark Miller [EMAIL PROTECTED] wrote: Kick off some indexing more than once - eg, post a

Re: Dealing with field values as key/value pairs

2008-12-10 Thread Noble Paul നോബിള്‍ नोब्ळ्
On Thu, Dec 11, 2008 at 4:41 AM, Chris Hostetter [EMAIL PROTECTED] wrote: : This is really cool. U... How does it integrate with the Data Import : Handler? my DIH knowledge is extremely limited, but i'm guessing approach #1 is trivial (there is an easy way to concat DB values to build up

Re: SolrConfig.xml Replication

2008-12-10 Thread Noble Paul നോബിള്‍ नोब्ळ्
This is a known issue and I was planning to take it up soon. https://issues.apache.org/jira/browse/SOLR-821 On Thu, Dec 11, 2008 at 5:30 AM, Jeff Newburn [EMAIL PROTECTED] wrote: I am curious as to whether there is a solution to be able to replicate solrconfig.xml with the 1.4 replication.

Re: Solr Newbie question

2008-12-10 Thread Noble Paul നോബിള്‍ नोब्ळ्
On Wed, Dec 10, 2008 at 11:00 PM, Rakesh Sinha [EMAIL PROTECTED] wrote: Hi - I am a new user of Solr tool and came across the introductory tutorial here - http://lucene.apache.org/solr/tutorial.html . I am planning to use Solr in one of my projects . I see that the tutorial mentions about

Re: Building Solr from Source

2008-12-11 Thread Noble Paul നോബിള്‍ नोब्ळ्
Solr uses ant for build install ant On Thu, Dec 11, 2008 at 7:13 PM, John Martyniak [EMAIL PROTECTED] wrote: Hi, I have downloaded Maven 2.0.9, and tried to build using mvn clean install and mvn install, nothing works. Can somebody tell me how to build solr from source? I am trying to

Re: SolrConfig.xml Replication

2008-12-11 Thread Noble Paul നോബിള്‍ नोब्ळ्
The moment the current patch is tested it will be checked in. On Thu, Dec 11, 2008 at 8:33 PM, Jeff Newburn jnewb...@zappos.com wrote: Thank you for the quick response. I will keep an eye on that to see how it progresses. On 12/10/08 8:03 PM, Noble Paul നോബിള്‍ नोब्ळ् noble.p...@gmail.com

Re: DIH - duplicate xpaths on HttpDataSource

2008-12-14 Thread Noble Paul നോബിള്‍ नोब्ळ्
one xpath can be mapped to only one name. If you want to copy one to another use something like a TemplateTransformer On Sun, Dec 14, 2008 at 7:18 AM, Erik Hatcher e...@ehatchersolutions.com wrote: I'm trying to index a blog with DIH, and have this: field column=id

Re: DIH - duplicate xpaths on HttpDataSource

2008-12-14 Thread Noble Paul നോബിള്‍ नोब्ळ्
On Sun, Dec 14, 2008 at 8:43 PM, Erik Hatcher e...@ehatchersolutions.com wrote: On Dec 14, 2008, at 6:15 AM, Noble Paul നോബിള്‍ नोब्ळ् wrote: one xpath can be mapped to only one name. That's a bizarre restriction. A shame. hmmm Understandale . XPathRecordReader is a custom minimal

Re: Parent Child Entity - DataImport

2008-12-15 Thread Noble Paul നോബിള്‍ नोब्ळ्
I do not observe anything wrong. you can also mention the 'deltaImportQuery' and try it someting like entity name=table1 pk=id query= SELECT ID,MY_GUID FROM activityLog WHERE type in (11, 15)

Re: SolrConfig.xml Replication

2008-12-15 Thread Noble Paul നോബിള്‍ नोब्ळ्
...@zappos.com wrote: Thank you for the quick response. I will keep an eye on that to see how it progresses. On 12/10/08 8:03 PM, Noble Paul നോബിള്‍ नोब्ळ् noble.p...@gmail.com wrote: This is a known issue and I was planning to take it up soon. https://issues.apache.org/jira/browse/SOLR-821

Re: Parent Child Entity - DataImport

2008-12-15 Thread Noble Paul നോബിള്‍ नोब्ळ्
ones. The only option that i'ev found so far is to do a full-import with the clean=false attribute and created_date last_indexed_date... Is there a better way? Thanks Noble Paul നോബിള്‍ नोब्ळ् wrote: I do not observe anything wrong. you can also mention the 'deltaImportQuery

Re: Using DIH, getting exception

2008-12-16 Thread Noble Paul നോബിള്‍ नोब्ळ्
you either remove the name=mySource from the dataSource tag or add dataSource=mySource in the entity 'item' On Wed, Dec 17, 2008 at 2:26 AM, Plaatje, Patrick patrick.plaa...@getronics.com wrote: Hi All, I'm trying to use the Data import handler, with the data config below (snippet):

Re: setting a log file per core with slf4

2008-12-17 Thread Noble Paul നോബിള്‍ नोब्ळ्
On Wed, Dec 17, 2008 at 10:54 PM, Ryan McKinley ryan...@gmail.com wrote: I'm not sure I understand... are you suggesting that rather then configuring our logger like this: static Logger log = LoggerFactory.getLogger(SolrCore.class); We get the Logger everytime we use it with something like:

Re: Solrj - Exception in thread main java.lang.ClassCastException: java.lang.Long cannot be cast to org.apache.solr.common.util.NamedList

2008-12-18 Thread Noble Paul നോബിള്‍ नोब्ळ्
which version of the server are you using? SolrJ documenttaion says that the binary format works only with Solr1.3 On Thu, Dec 18, 2008 at 2:49 PM, Sajith Vimukthi saji...@zone24x7.com wrote: Hi all, I used the sample code given below and tried to run with all the relevant jars. I receive

Re: Data Import Request Handler problem: Odd performance behaviour for large number of records

2008-12-18 Thread Noble Paul നോബിള്‍ नोब्ळ्
DIH does not maintain any state between two runs. So if there is a perf degradation it could be because - Solr Indexing is taking longer after you do a delete *:* - Your RAM is insufficient (your machine is swapping) On Fri, Dec 19, 2008 at 2:51 AM, Glen Newton glen.new...@gmail.com wrote:

Re: One conf folder in Multicore environment

2008-12-22 Thread Noble Paul നോബിള്‍ नोब्ळ्
solr.xml allows you to mention the other properties as well like instanceDir, config,schema in the cores/core tag So , sharing the entire conf dir may not be possible , but it is possible to share solrconfig.xml and schema.xml On Tue, Dec 23, 2008 at 8:17 AM, Otis Gospodnetic

Re: Getting request object within search component

2008-12-24 Thread Noble Paul നോബിള്‍ नोब्ळ्
The request object is not exposed. You may need to hack SolrDispatchFilter and put in the values into the request attributes On Wed, Dec 24, 2008 at 1:46 PM, Plaatje, Patrick patrick.plaa...@getronics.com wrote: Hi All, I developed my own custom search component, in which I need to get the

Re: Unable to move index file error during replication

2008-12-24 Thread Noble Paul നോബിള്‍ नोब्ळ्
Replication uses File#renameTo() method to move files from temp dir to the index dir.If this call fails replication fails. We must do a file copy if this fails. report a bug . --Noble On Wed, Dec 24, 2008 at 3:47 PM, Jaco jdevr...@gmail.com wrote: Hello, While testing out the new replication

Re: Unable to move index file error during replication

2008-12-24 Thread Noble Paul നോബിള്‍ नोब्ळ्
James thanks . If this is true the place to fix this is in ReplicationHandler#getFileList(). patch is attached. On Wed, Dec 24, 2008 at 4:04 PM, James Grant james.gr...@semantico.com wrote: I had the same problem. It turned out that the list of files from the master included duplicates. When

Re: Unable to move index file error during replication

2008-12-25 Thread Noble Paul നോബിള്‍ नोब्ळ्
next nightly build? Thanks very much for the very quick response! Jaco. 2008/12/24 Noble Paul നോബിള്‍ नोब्ळ् noble.p...@gmail.com James thanks . If this is true the place to fix this is in ReplicationHandler#getFileList(). patch is attached. On Wed, Dec 24, 2008 at 4:04 PM, James Grant

Re: Delta DataImport is not picking the modified value in DB

2008-12-29 Thread Noble Paul നോബിള്‍ नोब्ळ्
This may happen if your deltaQuery has a problem Try using the deltaImportQuery also. -- On Mon, Dec 29, 2008 at 2:57 PM, Shalin Shekhar Mangar shalinman...@gmail.com wrote: What does your data-config look like? especially the delta query part. On Mon, Dec 29, 2008 at 2:35 PM, Manupriya

Re: Delta DataImport is not picking the modified value in DB

2008-12-29 Thread Noble Paul നോബിള്‍ नोब्ळ्
did you put in the data? On Mon, Dec 29, 2008 at 3:07 PM, Manupriya manupriya.si...@gmail.com wrote: Hi Shalin, I am actually trying out the example provided at http://wiki.apache.org/solr/DataImportHandler. My db-data-config.xml is as following - dataConfig dataSource

Re: Delta DataImport is not picking the modified value in DB

2008-12-29 Thread Noble Paul നോബിള്‍ नोब्ळ्
/ /entity /entity /entity /document /dataConfig On Mon, Dec 29, 2008 at 3:30 PM, Noble Paul നോബിള്‍ नोब्ळ् noble.p...@gmail.com wrote: did you put in the data? On Mon, Dec 29, 2008 at 3:07 PM, Manupriya manupriya.si...@gmail.com wrote: Hi Shalin, I am actually trying out

Re: Delta DataImport is not picking the modified value in DB

2008-12-29 Thread Noble Paul നോബിള്‍ नोब्ळ्
:377) Another thing, I do have proper data in the DB. Thanks, Manu Noble Paul നോബിള്‍ नोब्ळ् wrote: The same example with deltaImportQuery would look as follows dataConfig dataSource driver=com.mysql.jdbc.Driver url=jdbc:mysql://localhost/solr_demo_db user=root password=root

Re: DataImportHandler - The field :xyz present in DataConfig does not have a counterpart in Solr Schema

2008-12-29 Thread Noble Paul നോബിള്‍ नोब्ळ्
These are warning messages . They do not stop anything in DIH It is very common that users make typos and that may be hard to debug. So the messages are spit out On Tue, Dec 30, 2008 at 12:06 AM, Rakesh Sinha rakesh.use...@gmail.com wrote: Hi - I am testing around with the full - import

Re: Local Solr - Error creating document with DIH

2008-12-29 Thread Noble Paul നോബിള്‍ नोब्ळ्
apparently the local solr expects everything as string. I guess we must raise an issue with them. meanwhile you can convert those double fields to String using TemplateTransformer field column=lat template=${your_entity_name.lat}/ field column=lng template=${your_entity_name.lng}/ On Tue,

Re: collectionDistribution vs SolrReplication

2009-01-04 Thread Noble Paul നോബിള്‍ नोब्ळ्
* SolrReplication does not create snapshots . So you have less cleanup to do. The script based replication results is more disk space consumption (especially if you do frequent commits) * Performance is roughly same unless you are replicating across different LAN where SolrReplication can zip and

Re: collectionDistribution vs SolrReplication

2009-01-05 Thread Noble Paul നോബിള്‍ नोब्ळ्
The default IndexDeletionPolicy just keeps the last commit only (KeepOnlyLastCommitDeletionPolicy) .Files belonging to older commits are removed. If the files are needed longer for replication, they are leased . The lease is extended 10 secs at a time. Once all the slaves have copied the lease is

Re: Deduplication patch not working in nightly build

2009-01-05 Thread Noble Paul നോബിള്‍ नोब्ळ्
looks like a bug w/ DIH with the recent fixes. --Noble On Mon, Jan 5, 2009 at 2:36 PM, Marc Sturlese marc.sturl...@gmail.com wrote: Hey there, I was using the Deduplication patch with Solr 1.3 release and everything was working perfectly. Now I upgraded to a nigthly build (20th december) to

Re: Deduplication patch not working in nightly build

2009-01-05 Thread Noble Paul നോബിള്‍ नोब्ळ्
! Shalin Shekhar Mangar wrote: Marc, I've just committed a fix which may have caused the bug. Can you use svn trunk (or the next nightly build) and confirm? On Mon, Jan 5, 2009 at 3:10 PM, Noble Paul നോബിള്‍ नोब्ळ् noble.p...@gmail.com wrote: looks like a bug w/ DIH with the recent fixes

Re: DataImport

2009-01-05 Thread Noble Paul നോബിള്‍ नोब्ळ्
The driver can be put directly into the WEB-INF/lib of the solr web app or it can be put into ${solr.home}/lib dir. or if something is really screwed up you can try the old fashioned way of putting your driver jar into JAVA_HOME/lib/ext --Noble On Tue, Jan 6, 2009 at 7:05 AM, Performance

Re: Setting up DataImportHandler for Oracle datasource on JBoss

2009-01-05 Thread Noble Paul നോബിള്‍ नोब्ळ्
the document tag and the rest of the stuff is missing in your data-config file On Tue, Jan 6, 2009 at 12:50 PM, The Flight Captain jason_sheph...@flightcentre.com wrote: I am having trouble setting up an Oracle datasource. Can anyone help me connect to the datasource? My solrconfig.xml:

Re: DataImportHandler (reading XML w/ paging)

2009-01-06 Thread Noble Paul നോബിള്‍ नोब्ळ्
Paging is possible w/ XPathEntityProcessor look at the $hasMore and $nextUrl in the documentation if you can explain better and I may be able to give a better solution eg: where is the metadata coming from and what is the datasource On Tue, Jan 6, 2009 at 11:17 PM, Jon Baer jonb...@gmail.com

Re: Setting up DataImportHandler for Oracle datasource on JBoss

2009-01-06 Thread Noble Paul നോബിള്‍ नोब्ळ्
(DataConfig.java:236) at org.apache.solr.handler.dataimport.DataImporter.loadDataConfig(DataImporter.java:170) ... 140 more Am I missing anything else? Noble Paul നോബിള്‍ नोब्ळ् wrote: the document tag and the rest of the stuff is missing in your data-config file On Tue, Jan 6

Re: DataImport

2009-01-06 Thread Noble Paul നോബിള്‍ नोब्ळ्
a config file I should look at? Noble Paul നോബിള്‍ नोब्ळ् wrote: The driver can be put directly into the WEB-INF/lib of the solr web app or it can be put into ${solr.home}/lib dir. or if something is really screwed up you can try the old fashioned way of putting your driver jar

Re: Is there any better way to configure db-data-config.xml

2009-01-07 Thread Noble Paul നോബിള്‍ नोब्ळ्
why do you have multiple root entities ? On Wed, Jan 7, 2009 at 7:48 PM, Manupriya manupriya.si...@gmail.com wrote: Hi, I am using the following schema - http://www.nabble.com/file/p21332196/table_stuct.gif 1. INSTITUTION table is the main table that has information about all the

Re: Beginner: importing own data

2009-01-08 Thread Noble Paul നോബിള്‍ नोब्ळ्
did you explore using SolrJ to index data? http://wiki.apache.org/solr/Solrj or DataImportHandler. http://wiki.apache.org/solr/DataImportHandler On Fri, Jan 9, 2009 at 3:49 AM, Otis Gospodnetic otis_gospodne...@yahoo.com wrote: Phil, The easiest thing to do at this stage in Solr learning

Re: Flipping data dirs for an (/multiple) SolrCore without affecting search / IndexReaders

2009-01-08 Thread Noble Paul നോബിള്‍ नोब्ळ्
Hi Kay, do you wish to change data dir or just indexdir. It is possible to switch indexDir w/ minimal overhead. dataDir can contain a index.properties file which can contain a property called 'index' it can point to your new index . but you will have to find a way to populate this index

Re: collectionDistribution vs SolrReplication

2009-01-16 Thread Noble Paul നോബിള്‍ नोब्ळ्
On Fri, Jan 16, 2009 at 3:37 AM, Chris Hostetter hossman_luc...@fucit.org wrote: : I would like to know the advantages of moving from: : a master-slave system using CollectionDistribution with all their .sh : scripts : http://wiki.apache.org/solr/CollectionDistribution : to: : use

Re: DIH XPathEntityProcessor fails with docs containing !DOCTYPE

2009-01-16 Thread Noble Paul നോബിള്‍ नोब्ळ्
stax parser automatically tries to fetch the DTD. How can we disable that at the parser level? On Fri, Jan 16, 2009 at 3:34 PM, Fergus McMenemie fer...@twig.me.uk wrote: Hello all, as the subject says: DIH XPathEntityProcessor fails with docs containing !DOCTYPE This is using a solr nightly

Re: DIH XPathEntityProcessor fails with docs containing !DOCTYPE

2009-01-16 Thread Noble Paul നോബിള്‍ नोब्ळ्
I have raised an issue and a patch is provided. Please confirm if it helps https://issues.apache.org/jira/browse/SOLR-964 On Fri, Jan 16, 2009 at 3:52 PM, Noble Paul നോബിള്‍ नोब्ळ् noble.p...@gmail.com wrote: stax parser automatically tries to fetch the DTD. How can we disable

Re: Having no luck with build-in replication and multicore

2009-01-16 Thread Noble Paul നോബിള്‍ नोब्ळ्
On Fri, Jan 16, 2009 at 7:14 PM, Jacob Singh jacobsi...@gmail.com wrote: Hi Shalin, Sorry, my post was unlcear. I am calling snappull from the slave, I get that part, just obfuscating the domains incorrectly :). The problem it seems is the index version. The index was a 1.3 index and I've

Re: collectionDistribution vs SolrReplication

2009-01-18 Thread Noble Paul നോബിള്‍ नोब्ळ्
http://wiki.apache.org/solr/SolrReplication#head-b77f4610e9b2f38433fdffc7f07cc9789ecabe72 On Sun, Jan 18, 2009 at 10:53 PM, Shalin Shekhar Mangar shalinman...@gmail.com wrote: On Sun, Jan 18, 2009 at 9:51 PM, Yonik Seeley ysee...@gmail.com wrote: I've not looked into the file replication

Re: Reading database parameters from outside data-config.xml

2009-01-18 Thread Noble Paul നോബിള്‍ नोब्ळ्
Unfortunately , there is no inbuilt feature which supports something like this. The problem is that there is no simple common way to achieve this. some would want JNDI, some would want a properties file and some would want it to be read from ldap/jdbc etc It is possible to use a different

Re: problem with DIH and MySQL

2009-01-20 Thread Noble Paul നോബിള്‍ नोब्ळ्
it got rolled back any exceptions on solr console? On Tue, Jan 20, 2009 at 9:07 PM, Nick Friedrich nick.friedr...@student.uni-magdeburg.de wrote: Hi, I'm new to Solr and I have a problem. I want to use DIH to index data stored in a MySQL database. I added to solrconfig.xml requestHandler

Re: Newbie Design Questions

2009-01-20 Thread Noble Paul നോബിള്‍ नोब्ळ्
On Wed, Jan 21, 2009 at 5:15 AM, Gunaranjan Chandraraju chandrar...@apple.com wrote: Hi All We are considering SOLR for a large database of XMLs. I have some newbie questions - if there is a place I can go read about them do let me know and I will go read up :) 1. Currently we are able to

Re: Solr Replication: disk space consumed on slave much higher than on master

2009-01-21 Thread Noble Paul നോബിള്‍ नोब्ळ्
the index.xxx directories are supposed to be deleted (automatically). you can safely delete them. But, I am wondering why the index files in the slave did not get deleted. By default the deletionPolicy is KeepOnlyLastCommit. On Wed, Jan 21, 2009 at 2:15 PM, Jaco jdevr...@gmail.com wrote: Hi,

Re: Solr Replication: disk space consumed on slave much higher than on master

2009-01-21 Thread Noble Paul നോബിള്‍ नोब्ळ्
On Wed, Jan 21, 2009 at 3:42 PM, Jaco jdevr...@gmail.com wrote: Thanks for the fast replies! It appears that I made a (probably classical) error... I didnt' make the change to solrconfig.xml to include the deletionPolicy when applying the upgrade. I include this now, but the slave is not

Re: How to schedule delta-import and auto commit

2009-01-21 Thread Noble Paul നോബിള്‍ नोब्ळ्
On Wed, Jan 21, 2009 at 4:31 PM, Manupriya manupriya.si...@gmail.com wrote: Hi, Our Solr server is a standalone server and some web applications send HTTP query to search and get back the results. Now I have following two requirements - 1. we want to schedule 'delta-import' at a specified

Re: Newbie Design Questions

2009-01-21 Thread Noble Paul നോബിള്‍ नोब्ळ्
(of course :) ) THanks Guna On Jan 20, 2009, at 9:37 PM, Noble Paul നോബിള്‍ नोब्ळ् wrote: On Wed, Jan 21, 2009 at 5:15 AM, Gunaranjan Chandraraju chandrar...@apple.com wrote: Hi All We are considering SOLR for a large database of XMLs. I have some newbie questions - if there is a place I can

Re: Solr Replication: disk space consumed on slave much higher than on master

2009-01-22 Thread Noble Paul നോബിള്‍ नोब्ळ्
Jeff , Do you see both the empty index. dirs as well as the extra files in the index? --Noble On Thu, Jan 22, 2009 at 10:37 PM, Jeff Newburn jnewb...@zappos.com wrote: We are seeing something very similar. Ours is intermittent and usually happens a great deal on random days. Often it seems

Re: Solr Replication: disk space consumed on slave much higher than on master

2009-01-22 Thread Noble Paul നോബിള്‍ नोब्ळ्
org.apache.solr.handler.SnapPuller copyAFile SEVERE: Unable to move index file from: /data/index.20090122101604/_8n.tvx to: /data/index/_8n.tvx On 1/22/09 10:23 AM, Noble Paul നോബിള്‍ नोब्ळ् noble.p...@gmail.com wrote: Jeff , Do you see both the empty index. dirs as well as the extra files

Re: Solr Replication: disk space consumed on slave much higher than on master

2009-01-22 Thread Noble Paul നോബിള്‍ नोब्ळ्
10:33 AM, Noble Paul നോബിള്‍ नोब्ळ् noble.p...@gmail.com wrote: This was reported by another user and was fixed recently.Are you using a recent version? --Noble On Fri, Jan 23, 2009 at 12:00 AM, Jeff Newburn jnewb...@zappos.com wrote: We have both. A majority of them are just empty

Re: Newbie Design Questions

2009-01-22 Thread Noble Paul നോബിള്‍ नोब्ळ्
On Jan 21, 2009, at 8:22 PM, Noble Paul നോബിള്‍ नोब्ळ् wrote: On Thu, Jan 22, 2009 at 7:02 AM, Gunaranjan Chandraraju chandrar...@apple.com wrote: Thanks Yes the source of data is a DB. However the xml is also posted on updates via publish framework. So I can just plug in an adapter hear

Re: Newbie Design Questions

2009-01-22 Thread Noble Paul നോബിള്‍ नोब्ळ्
) then I can plan for our development around it. Thanks Guna On Jan 22, 2009, at 11:04 AM, Noble Paul നോബിള്‍ नोब्ळ् wrote: You are out of luck if you are not using a recent version of DIH The sub entity will work only if you use the FieldReaderDataSource. Then you do not need

Re: URL-import field type?

2009-01-22 Thread Noble Paul നോബിള്‍ नोब्ळ्
where is this url coming from? what is the content type of the stream? is it plain text or html? if yes, this is a possible enhancement to DIH On Fri, Jan 23, 2009 at 4:39 AM, Paul Libbrecht p...@activemath.org wrote: Hello list, after searching around for quite a while, including in the

Re: URL-import field type?

2009-01-23 Thread Noble Paul നോബിള്‍ नोब्ळ्
based on a single stream. Hints for altnernative eagerly welcome. paul Le 23-janv.-09 à 05:45, Noble Paul നോബിള്‍ नोब्ळ् a écrit : where is this url coming from? what is the content type of the stream? is it plain text or html? if yes, this is a possible enhancement to DIH On Fri

Re: URL-import field type?

2009-01-23 Thread Noble Paul നോബിള്‍ नोब्ळ्
On Fri, Jan 23, 2009 at 2:55 PM, Paul Libbrecht p...@activemath.org wrote: Le 23-janv.-09 à 10:10, Noble Paul നോബിള്‍ नोब्ळ् a écrit : if the response is not XML ,then there is no EntityProcessor that can consume this. We may need to add one. well, even binary data such as word documents

Re: Solr Replication: disk space consumed on slave much higher than on master

2009-01-23 Thread Noble Paul നോബിള്‍ नोब्ळ्
I tested with the patch it has solved both the issues On Fri, Jan 23, 2009 at 5:00 PM, Shalin Shekhar Mangar shalinman...@gmail.com wrote: On Fri, Jan 23, 2009 at 2:12 PM, Jaco jdevr...@gmail.com wrote: Hi, I applied the patch and did some more tests - also adding some LOG.info() calls in

Re: Solr Replication: disk space consumed on slave much higher than on master

2009-01-23 Thread Noble Paul നോബിള്‍ नोब्ळ्
I have opened an issue to track this https://issues.apache.org/jira/browse/SOLR-978 On Fri, Jan 23, 2009 at 5:22 PM, Noble Paul നോബിള്‍ नोब्ळ् noble.p...@gmail.com wrote: I tested with the patch it has solved both the issues On Fri, Jan 23, 2009 at 5:00 PM, Shalin Shekhar Mangar shalinman

Re: DataImport TXT file entity processor

2009-01-24 Thread Noble Paul നോബിള്‍ नोब्ळ्
an EntityProcessor looks right to me. It may help us add more attributes if needed. PlainTextEntityProcessor looks like a good name. It can also be used to read html etc. --Noble On Sat, Jan 24, 2009 at 12:37 PM, Shalin Shekhar Mangar shalinman...@gmail.com wrote: On Sat, Jan 24, 2009 at 5:56

Re: Should I extend DIH to handle POST too?

2009-01-24 Thread Noble Paul നോബിള്‍ नोब्ळ्
That does not look like a great option. DIH looks like an overkill for this usecase. You can write a simple UpdateHandler to do that . All that you need to do is to extent ContentStreamHandlerBase and register it as an UpdateHandler On Sat, Jan 24, 2009 at 12:34 PM, Shalin Shekhar Mangar

Re: How to make Relationships work for Multi-valued Index Fields?

2009-01-24 Thread Noble Paul നോബിള്‍ नोब्ळ्
nesting of an XPathEntityProcessor into another XPathEntityProcessor is possible only if a field in an xml is a filename/url . what is the purpose of nesting like this? is it because you have multiple addresses? the possible solutions are discussed elsewhere in this thread On Sat, Jan 24, 2009 at

Re: Master failover - seeking comments

2009-01-24 Thread Noble Paul നോബിള്‍ नोब्ळ्
Did you look at the new in-built replication? http://wiki.apache.org/solr/SolrReplication#head-0e25211b6ef50373fcc2f9a6ad40380c169a5397 It can help you decide where to replicate from during runtime . Look at the snappull command you can pass the masterUrl at the time of replication. On Fri,

Re: How to make Relationships work for Multi-valued Index Fields?

2009-01-24 Thread Noble Paul നോബിള്‍ नोब्ळ्
xpath=/record/address/@state / I saw the syntax '/record/address//@state'. '//' is not supported . You will have to explicitly give the full path. --Noble On Sat, Jan 24, 2009 at 2:57 PM, Noble Paul നോബിള്‍ नोब्ळ् noble.p...@gmail.com wrote: nesting of an XPathEntityProcessor into another

Re: How to make Relationships work for Multi-valued Index Fields?

2009-01-24 Thread Noble Paul നോബിള്‍ नोब्ळ्
for searching you need to put them in a single field . use copyField in schema.xml to achieve that On Sun, Jan 25, 2009 at 7:39 AM, Gunaranjan Chandraraju chandrar...@apple.com wrote: I make this approach work with XPATH and XSL. However, this approach creates multiple fields of like this

Re: best way to extend solr webapp

2009-01-25 Thread Noble Paul നോബിള്‍ नोब्ळ्
On Mon, Jan 26, 2009 at 3:41 AM, Paul Libbrecht p...@activemath.org wrote: Is it common practice to use the maven war-overlay function so as to build, mostly, a solr webapp but with some added servlets and a few more classes (e.g. my own analyzers) ? your own analyzersa can be added w/o

Re: Setting dataDir in multicore environment

2009-01-26 Thread Noble Paul നോബിള്‍ नोब्ळ्
The behavior is expected properties set in solr.xml are not implicitly used anywhere. you will have to use those variables explicitly in solrconfig.xml/schema.xml instead of hardcoding dataDir in solrconfig.xml you can use it as a variable dataDir$$dataDir/dataDir BTW there is an issue

Re: URL-import field type?

2009-01-26 Thread Noble Paul നോബിള്‍ नोब्ळ्
On Tue, Jan 27, 2009 at 4:47 AM, Chris Hostetter hossman_luc...@fucit.org wrote: : I know I was able to imitate that in plain-lucene by crafting a particular : analyzer-filter who was only given the URL as content and who gave further the : tokens of the stream. FWIW: while taking

Re: Should I extend DIH to handle POST too?

2009-01-26 Thread Noble Paul നോബിള്‍ नोब्ळ्
enhancements will work as-is for us? Regards Guna On Jan 24, 2009, at 12:42 AM, Noble Paul നോബിള്‍ नोब्ळ् wrote: That does not look like a great option. DIH looks like an overkill for this usecase. You can write a simple UpdateHandler to do that . All that you need to do is to extent

Re: Connection mismanagement in Solrj?

2009-01-26 Thread Noble Paul നോബിള്‍ नोब्ळ्
correction wrong: make sure that you are not reusing instance of CommonsHttpSolrServer correct : make sure that you are reusing instance of CommonsHttpSolrServer On Tue, Jan 27, 2009 at 11:58 AM, Noble Paul നോബിള്‍ नोब्ळ् noble.p...@gmail.com wrote: you can set any connection parameters

Re: Connection mismanagement in Solrj?

2009-01-26 Thread Noble Paul നോബിള്‍ नोब्ळ्
are you making requests in parallel ? which ConnectionManager are you using for HttpClient? On Tue, Jan 27, 2009 at 11:58 AM, Noble Paul നോബിള്‍ नोब्ळ् noble.p...@gmail.com wrote: you can set any connection parameters for the HttpClient and pass on the instance to CommonsHttpSolrServer

Re: Setting dataDir in multicore environment

2009-01-27 Thread Noble Paul നോബിള്‍ नोब्ळ्
. I know there is a patch in the works for this issue but I can't wait for it. Is there any way to share the solrconfig.xml file and create indexes dynamically? Mark On Mon, Jan 26, 2009 at 9:02 PM, Noble Paul നോബിള്‍ नोब्ळ् noble.p...@gmail.com wrote: The behavior is expected

Re: Connection mismanagement in Solrj?

2009-01-27 Thread Noble Paul നോബിള്‍ नोब्ळ्
Underwood wunderw...@netflix.com wrote: Making requests in parallel, using the default connection manager, which is multi-threaded, and we are reusing a single CommonsHttpSolrServer for all requests. wunder On 1/26/09 10:59 PM, Noble Paul നോബിള്‍ नोब्ळ् noble.p...@gmail.com wrote: are you

Re: [dummy question] applying patch

2009-01-27 Thread Noble Paul നോബിള്‍ नोब्ळ्
since you are asking about 'batch file' , are you using windows? I recommend using TortoiseSVN to apply patch On Wed, Jan 28, 2009 at 10:05 AM, surfer10 mainp...@mail.ru wrote: i'm a little bit noob in java compiler so could you please tell me what tools are used to apply patch SOLR-236 (Field

Re: Setting dataDir in multicore environment

2009-01-27 Thread Noble Paul നോബിള്‍ नोब्ळ्
There is a patch given for SOLR-883 . On Wed, Jan 28, 2009 at 9:43 AM, Noble Paul നോബിള്‍ नोब्ळ् noble.p...@gmail.com wrote: I shall give a patch today On Tue, Jan 27, 2009 at 11:58 PM, Mark Ferguson mark.a.fergu...@gmail.com wrote: Oh I see, thanks for the clarification. Unfortunately

Re: DIH handling of missing files

2009-01-28 Thread Noble Paul നോബിള്‍ नोब्ळ्
onError=continue must help . which version of DIH are you using? onError is a Solr 1.4 feature --Noble On Thu, Jan 29, 2009 at 5:04 AM, Nathan Adams na...@umich.edu wrote: I am constructing documents from a JDBC datasource and a HTTP datasource (see data-config file below.) My problem is that

Re: newbie question --- multiple schemas

2009-01-29 Thread Noble Paul നോബിള്‍ नोब्ळ्
have two different cores and you can have separate schema for each. On Thu, Jan 29, 2009 at 1:20 PM, Cheng Zhang zhangyongji...@yahoo.com wrote: Hello, Is it possible to define more than one schema? I'm reading the example schema.xml. It seems that we can only define one schema? What about

Re: Data Directory Sync.

2009-01-29 Thread Noble Paul നോബിള്‍ नोब्ळ्
On Thu, Jan 29, 2009 at 7:27 PM, Kalidoss MM mm.kalid...@gmail.com wrote: Hi, I have a requirement like, There is a running solr and having around 10K records indexed in it. Now i have to index another set of 30K records? The 10K data already in live, And i dont have an option to

Re: How to handle database replication delay when using DataImportHandler?

2009-01-29 Thread Noble Paul നോബിള്‍ नोब्ळ्
, onImportEnd). Would those be the right spot to calculate these dates so that I could expose them to my custom function at query time? Thanks. --Gregg On Wed, Jan 28, 2009 at 11:20 PM, Noble Paul നോബിള്‍ नोब्ळ् noble.p...@gmail.com wrote: The problem you are trying to solve is that you

Re: DIH using values from solrconfig.xml inside data-config.xml

2009-02-02 Thread Noble Paul നോബിള്‍ नोब्ळ्
RegexTransformer does not replace the placeholders before processing the regex. it has to be enhanced On Mon, Feb 2, 2009 at 10:34 PM, Fergus McMenemie fer...@twig.me.uk wrote: Hello As per several postings I noted that I can define variables inside an invariants list section of the DIH

Re: DIH - Example of using $nextUrl and $hasMore

2009-02-02 Thread Noble Paul നോബിള്‍ नोब्ळ्
On Mon, Feb 2, 2009 at 11:01 PM, Jon Baer jonb...@gmail.com wrote: Yes I think what Jared mentions in the JIRA is what I was thinking about when it is recommended to always return true for $hasMore ... The transformer must know somehow when $hasMore should be true. If the transformer always

Re: DIH using values from solrconfig.xml inside data-config.xml

2009-02-02 Thread Noble Paul നോബിള്‍ नोब्ळ्
this patch must help On Mon, Feb 2, 2009 at 10:49 PM, Shalin Shekhar Mangar shalinman...@gmail.com wrote: On Mon, Feb 2, 2009 at 10:34 PM, Fergus McMenemie fer...@twig.me.uk wrote: Is there some simple escape or other syntax to be used or is this an enhancement? I guess the problem is

Re: DIH - Example of using $nextUrl and $hasMore

2009-02-02 Thread Noble Paul നോബിള്‍ नोब्ळ्
On Mon, Feb 2, 2009 at 9:20 PM, Jon Baer jonb...@gmail.com wrote: Hi, Sorry I know this exists ... If an API supports chunking (when the dataset is too large) multiple calls need to be made to complete the process. XPathEntityprocessor supports this with a transformer. If transformer

Re: DIH using values from solrconfig.xml inside data-config.xml

2009-02-02 Thread Noble Paul നോബിള്‍ नोब्ळ्
. '[...@qualifier='Date']' may not be supported. field column=date xpath=/record/metadata/da...@qualifier='Date'] dateTimeFormat=MMdd / On Mon, Feb 2, 2009 at 9:24 AM, Noble Paul നോബിള്‍ नोब्ळ् noble.p...@gmail.com wrote: this patch must help On Mon, Feb 2, 2009 at 10:49 PM, Shalin

Re: DIH using values from solrconfig.xml inside data-config.xml

2009-02-03 Thread Noble Paul നോബിള്‍ नोब्ळ्
On Wed, Feb 4, 2009 at 6:13 AM, Chris Hostetter hossman_luc...@fucit.org wrote: : The solr data field is populated properly. So I guess that bit works. : I really wish I could use xpath=//para : The limitation comes from streaming the XML instead of creating a DOM. : XPathRecordReader is a

Re: DIH, assigning multiple xpaths to the same solr field

2009-02-03 Thread Noble Paul നോബിള്‍ नोब्ळ्
it is safe to use different column names as Shalin suggested. After all a row is a map with the column value as the key. If you map multiple values to the same column it may overwrite each other. use explicit 'name' attributes On Wed, Feb 4, 2009 at 2:17 AM, Shalin Shekhar Mangar

Re: DIH - Example of using $nextUrl and $hasMore

2009-02-04 Thread Noble Paul നോബിള്‍ नोब्ळ्
, com.nhl.solr.EnumeratedEntityTransformer I guess what Im looking for is that snippet which shows how it is setup (the initial counter) ... - Jon On Mon, Feb 2, 2009 at 12:39 PM, Noble Paul നോബിള്‍ नोब्ळ् noble.p...@gmail.com wrote: On Mon, Feb 2, 2009 at 11:01 PM, Jon Baer jonb...@gmail.com wrote: Yes I think

Re: DIH using values from solrconfig.xml inside data-config.xml

2009-02-04 Thread Noble Paul നോബിള്‍ नोब्ळ्
The implementation assumed that most of the users have xml with a fixed schema. . In that case giving absolute path is not hard. This helps us deal with a large subset of usecases rather easily. We have not added all the features which are possible with a streaming parser. It is wiser to

Re: Latest on DataImportHandler and Tika?

2009-02-04 Thread Noble Paul നോബിള്‍ नोब्ळ्
We have not taken up anything yet. The idea is to create another contrib which will contain extensions to DIH which has external dependencies as SOLR-934. TikaEntityProcessor is something we wish to do but our limited bandwidth has been the problem On Thu, Feb 5, 2009 at 5:15 AM, Chris Harris

Re: DataImportHandler (DIH) and Solr 1.2

2009-02-05 Thread Noble Paul നോബിള്‍ नोब्ळ्
On Fri, Feb 6, 2009 at 11:12 AM, Amit Nithian anith...@gmail.com wrote: Few questions regarding the DataImportHandler: 1) My company is currently using Solr 1.2 and upgrading to 1.3 isn't in the immediate near term horizon, so I was wondering how difficult would it be to use the DIH and SolrJ

Re: Example Solr instance

2009-02-11 Thread Noble Paul നോബിള്‍ नोब्ळ्
this link is not working example.solrstuff.org. who is maintaining it? On Thu, Feb 12, 2009 at 7:57 AM, Mauricio Scheffer mauricioschef...@gmail.com wrote: Hi everyone. I'm developing a Solr client for .NET ( http://code.google.com/p/solrnet/) and I was wondering if I could use the Solr

Re: Is this DIH entity forEach expression OK?

2009-02-12 Thread Noble Paul നോബിള്‍ नोब्ळ्
forEach=/record | /record/mediaBlock is likely to have problem . This causes inconsistencies anyway because this is supposed to give callbacks with same data. you can definitely write a transformer which gives out multiple rows if a record contains /record/mediaBlock. the return value of a

<    1   2   3   4   5   6   7   8   9   10   >