Re: Question about sending solrconfig and schema files with java
Hi Jack,
What other informations do you need on SO ?
I added some info in comments, but i'm limited on size.
Could you comment on SO if you need something else ?
*Frédéric Esnault*
CTO / CO-FOUNDER
*SERENZIA*
57 Rue Maurice Bokanowski
92600 Asnières-sur-Seine
Tel : +33 6 49 45 53 38
Mail : fesna...@serenzia.com
2014-06-21 1:15 GMT+02:00 Jack Krupansky :
> Oops! Sorry I missed it. Please post of the rest of the info on SO as well.
>
> We'll get to it!
>
>
> -- Jack Krupansky
>
> -Original Message- From: Frederic Esnault
> Sent: Friday, June 20, 2014 7:03 PM
>
> To: solr-user@lucene.apache.org
> Subject: Re: Question about sending solrconfig and schema files with java
>
> Hi Jack, actually i posted on OS first, but got no anwser.
> Check here :
> https://stackoverflow.com/questions/24296014/datastax-
> dse-search-how-to-post-solrconfig-xml-and-schema-xml-using-java
>
> I can't see any exception in cassandra/system.log at the moment of the
> error. :(
>
>
> *Frédéric Esnault*
> CTO / CO-FOUNDER
>
> *SERENZIA*
> 57 Rue Maurice Bokanowski
> 92600 Asnières-sur-Seine
>
> Tel : +33 6 49 45 53 38
> Mail : fesna...@serenzia.com
>
>
>
>
> 2014-06-21 0:35 GMT+02:00 Jack Krupansky :
>
> Please post this issue on StackOverflow and one of us DataStax guys will
>> deal with it there, since nobody here would know much about the
>> specialized
>> way that DataStax uses for dynamic schema and config loading.
>>
>> Check your DSE server log for the 500 exception - but post it on SO since
>> it is probably not Solr-related.
>>
>> Sorry for the inconvenience!
>>
>> -- Jack Krupansky
>>
>> -Original Message- From: Frederic Esnault
>> Sent: Friday, June 20, 2014 11:50 AM
>> To: solr-user@lucene.apache.org
>> Subject: Re: Question about sending solrconfig and schema files with java
>>
>>
>> Hi Shawn,
>>
>> Actually i should say that i'm using DSE Search (ie. Datastax Enterprise
>> with SolR enabled).
>> With cURL, i'm doing like this :
>>
>> $ curl http://localhost:8983/solr/resource/nhanes_ks.nhanes/
>> solrconfig.xml
>> --data-binary @solrconfig.xml -H 'Content-type:text/xml;
>> charset=utf-8'
>>
>> $ curl http://localhost:8983/solr/resource/nhanes_ks.nhanes/schema.xml
>> --data-binary @schema.xml -H 'Content-type:text/xml; charset=utf-8'
>>
>> $ curl "http://localhost:8983/solr/admin/cores?action=CREATE&;
>> name=nhanes_ks.nhanes"
>>
>>
>> Except i'm doing this not on localhost but a remote server, and with
>> files generated in my java program (which are correct once generated,
>> i checked).
>>
>> Using HttpComponents to send them does not work, it adds weird things
>> before the file (read from the cassandra blob after insert).
>>
>> Using SolrJ to create the core does not work (cannot upload files, so
>> it's complaining about missing files).
>>
>> Using a ContentStream request fails with an internal server error (no
>> details)
>>
>>HttpSolrServer server = new HttpSolrServer(solrUrl);
>>
>>ContentStreamUpdateRequest req = new
>> ContentStreamUpdateRequest("/resources/"+solrKeyspace + "." +
>> datasetName + "/");
>>
>> req.addContentStream(new ContentStreamBase.FileStream(new
>> File("./target/classes/solrconfig.xml")));
>>
>>server.request(req);
>>
>>server.commit();
>>
>> *returned non ok status:500, message:Internal Server Error*
>>
>>
>>
>> *Frédéric Esnault*
>> CTO / CO-FOUNDER
>>
>> *SERENZIA*
>>
>> 57 Rue Maurice Bokanowski
>> 92600 Asnières-sur-Seine
>>
>> Tel : +33 6 49 45 53 38
>> Mail : fesna...@serenzia.com
>>
>>
>>
>>
>> 2014-06-20 17:34 GMT+02:00 Shawn Heisey :
>>
>> On 6/20/2014 8:46 AM, Frederic Esnault wrote:
>>
>>> > First thank you for taking the time to answer me.
>>> >
>>> > Actually i tried looking for a way to use SolrJ to upload my files, but
>>> > i
>>> > cannot find anywhere informations about how to create nodes with their
>>> > config files using SolrJ.
>>> > All websites, blogs and docs i found seem to be based on the principle
>>> that
>>> > the core already exist or that the config files are already there.
>>>
>>> You said that you know how to send the files with curl. How are you
>>> doing this? If you can do it with curl, chances are good that you can
>>> duplicate the request with HttpSolrServer in some java code.
>>>
>>> Thanks,
>>> Shawn
>>>
>>>
>>>
>>>
>>
>
Re: Question about sending solrconfig and schema files with java
Hi Jack, actually i posted on OS first, but got no anwser.
Check here :
https://stackoverflow.com/questions/24296014/datastax-dse-search-how-to-post-solrconfig-xml-and-schema-xml-using-java
I can't see any exception in cassandra/system.log at the moment of the
error. :(
*Frédéric Esnault*
CTO / CO-FOUNDER
*SERENZIA*
57 Rue Maurice Bokanowski
92600 Asnières-sur-Seine
Tel : +33 6 49 45 53 38
Mail : fesna...@serenzia.com
2014-06-21 0:35 GMT+02:00 Jack Krupansky :
> Please post this issue on StackOverflow and one of us DataStax guys will
> deal with it there, since nobody here would know much about the specialized
> way that DataStax uses for dynamic schema and config loading.
>
> Check your DSE server log for the 500 exception - but post it on SO since
> it is probably not Solr-related.
>
> Sorry for the inconvenience!
>
> -- Jack Krupansky
>
> -Original Message- From: Frederic Esnault
> Sent: Friday, June 20, 2014 11:50 AM
> To: solr-user@lucene.apache.org
> Subject: Re: Question about sending solrconfig and schema files with java
>
>
> Hi Shawn,
>
> Actually i should say that i'm using DSE Search (ie. Datastax Enterprise
> with SolR enabled).
> With cURL, i'm doing like this :
>
> $ curl http://localhost:8983/solr/resource/nhanes_ks.nhanes/solrconfig.xml
> --data-binary @solrconfig.xml -H 'Content-type:text/xml;
> charset=utf-8'
>
> $ curl http://localhost:8983/solr/resource/nhanes_ks.nhanes/schema.xml
> --data-binary @schema.xml -H 'Content-type:text/xml; charset=utf-8'
>
> $ curl "http://localhost:8983/solr/admin/cores?action=CREATE&;
> name=nhanes_ks.nhanes"
>
>
> Except i'm doing this not on localhost but a remote server, and with
> files generated in my java program (which are correct once generated,
> i checked).
>
> Using HttpComponents to send them does not work, it adds weird things
> before the file (read from the cassandra blob after insert).
>
> Using SolrJ to create the core does not work (cannot upload files, so
> it's complaining about missing files).
>
> Using a ContentStream request fails with an internal server error (no
> details)
>
>HttpSolrServer server = new HttpSolrServer(solrUrl);
>
>ContentStreamUpdateRequest req = new
> ContentStreamUpdateRequest("/resources/"+solrKeyspace + "." +
> datasetName + "/");
>
>req.addContentStream(new ContentStreamBase.FileStream(new
> File("./target/classes/solrconfig.xml")));
>
>server.request(req);
>
>server.commit();
>
> *returned non ok status:500, message:Internal Server Error*
>
>
>
> *Frédéric Esnault*
> CTO / CO-FOUNDER
>
> *SERENZIA*
>
> 57 Rue Maurice Bokanowski
> 92600 Asnières-sur-Seine
>
> Tel : +33 6 49 45 53 38
> Mail : fesna...@serenzia.com
>
>
>
>
> 2014-06-20 17:34 GMT+02:00 Shawn Heisey :
>
> On 6/20/2014 8:46 AM, Frederic Esnault wrote:
>> > First thank you for taking the time to answer me.
>> >
>> > Actually i tried looking for a way to use SolrJ to upload my files, but
>> > i
>> > cannot find anywhere informations about how to create nodes with their
>> > config files using SolrJ.
>> > All websites, blogs and docs i found seem to be based on the principle
>> that
>> > the core already exist or that the config files are already there.
>>
>> You said that you know how to send the files with curl. How are you
>> doing this? If you can do it with curl, chances are good that you can
>> duplicate the request with HttpSolrServer in some java code.
>>
>> Thanks,
>> Shawn
>>
>>
>>
>
Re: Question about sending solrconfig and schema files with java
Hi Shawn,
Actually i should say that i'm using DSE Search (ie. Datastax Enterprise
with SolR enabled).
With cURL, i'm doing like this :
$ curl http://localhost:8983/solr/resource/nhanes_ks.nhanes/solrconfig.xml
--data-binary @solrconfig.xml -H 'Content-type:text/xml;
charset=utf-8'
$ curl http://localhost:8983/solr/resource/nhanes_ks.nhanes/schema.xml
--data-binary @schema.xml -H 'Content-type:text/xml; charset=utf-8'
$ curl
"http://localhost:8983/solr/admin/cores?action=CREATE&name=nhanes_ks.nhanes";
Except i'm doing this not on localhost but a remote server, and with
files generated in my java program (which are correct once generated,
i checked).
Using HttpComponents to send them does not work, it adds weird things
before the file (read from the cassandra blob after insert).
Using SolrJ to create the core does not work (cannot upload files, so
it's complaining about missing files).
Using a ContentStream request fails with an internal server error (no details)
HttpSolrServer server = new HttpSolrServer(solrUrl);
ContentStreamUpdateRequest req = new
ContentStreamUpdateRequest("/resources/"+solrKeyspace + "." +
datasetName + "/");
req.addContentStream(new ContentStreamBase.FileStream(new
File("./target/classes/solrconfig.xml")));
server.request(req);
server.commit();
*returned non ok status:500, message:Internal Server Error*
*Frédéric Esnault*
CTO / CO-FOUNDER
*SERENZIA*
57 Rue Maurice Bokanowski
92600 Asnières-sur-Seine
Tel : +33 6 49 45 53 38
Mail : fesna...@serenzia.com
2014-06-20 17:34 GMT+02:00 Shawn Heisey :
> On 6/20/2014 8:46 AM, Frederic Esnault wrote:
> > First thank you for taking the time to answer me.
> >
> > Actually i tried looking for a way to use SolrJ to upload my files, but i
> > cannot find anywhere informations about how to create nodes with their
> > config files using SolrJ.
> > All websites, blogs and docs i found seem to be based on the principle
> that
> > the core already exist or that the config files are already there.
>
> You said that you know how to send the files with curl. How are you
> doing this? If you can do it with curl, chances are good that you can
> duplicate the request with HttpSolrServer in some java code.
>
> Thanks,
> Shawn
>
>
Re: Question about sending solrconfig and schema files with java
Hi Alexandre, Nope, I cannot access the server (well i can actually, but my users won't be able to do so), and i can't rely on an http curl call. As for the final http call indicated in the link you gave, this is my last step, but before that i need my solrconfig.xml and schema.xml uploaded via java in solr. And this is where i'm stuck. *Frédéric Esnault* CTO / CO-FOUNDER *SERENZIA* 57 Rue Maurice Bokanowski 92600 Asnières-sur-Seine Tel : +33 6 49 45 53 38 Mail : fesna...@serenzia.com 2014-06-20 17:01 GMT+02:00 Alexandre Rafalovitch : > On Fri, Jun 20, 2014 at 9:46 PM, Frederic Esnault > wrote: > > Actually i tried looking for a way to use SolrJ to upload my files, but i > > cannot find anywhere informations about how to create nodes with their > > config files using SolrJ. > > Is this something solvable with configsets? > https://cwiki.apache.org/confluence/display/solr/Config+Sets > > Regards, >Alex. > > Personal website: http://www.outerthoughts.com/ > Current project: http://www.solr-start.com/ - Accelerating your Solr > proficiency >
Re: Question about sending solrconfig and schema files with java
Hi Shawn, First thank you for taking the time to answer me. Actually i tried looking for a way to use SolrJ to upload my files, but i cannot find anywhere informations about how to create nodes with their config files using SolrJ. All websites, blogs and docs i found seem to be based on the principle that the core already exist or that the config files are already there. I tried using SolrJ anyway, using CoreAdminRequest.create(), but i can only pass a config file name and a schema file name, not the files themselves, so i don't see how to do this. Result of this try is : INFO: Sending SolR config ... 4226 [AWT-EventQueue-0] INFO org.apache.solr.client.solrj.impl.HttpClientUtil - Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: No resource solrconfig.xml for core solrks.villes_france, did you miss to upload it? at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:402) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180) at org.apache.solr.client.solrj.request.CoreAdminRequest.process(CoreAdminRequest.java:462) at org.apache.solr.client.solrj.request.CoreAdminRequest.createCore(CoreAdminRequest.java:534) at org.apache.solr.client.solrj.request.CoreAdminRequest.createCore(CoreAdminRequest.java:514) *Frédéric Esnault* CTO / CO-FOUNDER *SERENZIA* 57 Rue Maurice Bokanowski 92600 Asnières-sur-Seine Tel : +33 6 49 45 53 38 Mail : fesna...@serenzia.com 2014-06-20 15:35 GMT+02:00 Shawn Heisey : > On 6/20/2014 5:16 AM, Frederic Esnault wrote: > > I know how to send solrconfig.xml and schema.xml files to SolR using curl > > commands. > > But my problem is that i want to send them with java, and i can't find a > > way to do so. > > I used HttpComponentsand got http headers before the file begins, which > SAX > > parser does not like at all. > > > > What is the best way to send this files from a java program ? > > Chances are good that you can duplicate your curl requests with > HttpSolrServer and SolrQuery, part of solrj, which is in the Solr > download under the dist directory. > > If you are running SolrCloud, then the configs in Zookeeper are directly > accessible with Java code. You should take a look at the source code, > in ZkController#uploadConfigDir, to see how the uploadToZK methods work. > You should be able to use the SolrZkClient#makePath method, just like > uploadToZK does. > > To use SolrZKClient (or the requests similar to what you do now with > curl), you will need the solrj jar and it's dependencies. The > recommended versions of those dependencies can be found in the download, > in the dist/solrj-lib directory. To get the SolrZkClient, you would > need to establish a CloudSolrServer object, then retrieve the > ZkStateReader from the CloudSolrServer, and the SolrZkClient from the > ZkStateReader. > > Thanks, > Shawn > >
Question about sending solrconfig and schema files with java
Hi, I know how to send solrconfig.xml and schema.xml files to SolR using curl commands. But my problem is that i want to send them with java, and i can't find a way to do so. I used HttpComponentsand got http headers before the file begins, which SAX parser does not like at all. What is the best way to send this files from a java program ? What i have once i sent the file is something like that : *��: solr_admin solr_resources resource_value��--9NDJNu2AW4jtIyX6ggQAgEqI3FXp3JpDZ6 Content-Disposition: form-data; name="solrconfig.xml"; filename="solrconfig.xml" Content-Type: application/xml; charset=ISO-8859-1 Content-Transfer-Encoding:** binary*
