Re: Import data from Mysql concat issues
Tomcat setup is fine. I insist that it's Solr's issue. The whole index consists of Greek (funny characters) and solr returns them normally. The problem here is that I cannot concatenate Greek characters in "data-config.xml" (hard-coded). -- View this message in context: http://lucene.472066.n3.nabble.com/Import-data-from-Mysql-concat-issues-tp4137814p4137939.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Import data from Mysql concat issues
bq: I think that it happens at index time How do you know that? If you're looking at the results in a browser you do _not_ know that. If you're looking at the raw values in, say, SolrJ then you _might_ know that, there's still the issue of whether you're sending the docs to Solr and your servlet container is munging them. In this latter case, you're right the info is indexed that way. Either way, it's not Solr's problem. Solr works just fine with utf-8. So if the data is getting into the index with weird characters, it's your setup. If it's just a browser problem, it's _still_ a problem with your setup. FWIW, Erick On Fri, May 23, 2014 at 10:36 AM, anarchos78 wrote: > I think that it happens at index time. The reason is that when i query for > the specific field solr returns the "?" string! > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Import-data-from-Mysql-concat-issues-tp4137814p4137908.html > Sent from the Solr - User mailing list archive at Nabble.com.
Re: Import data from Mysql concat issues
I think that it happens at index time. The reason is that when i query for the specific field solr returns the "?" string! -- View this message in context: http://lucene.472066.n3.nabble.com/Import-data-from-Mysql-concat-issues-tp4137814p4137908.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Import data from Mysql concat issues
Couple of possibilities: 1> The data in Solr is fine. However, your browser is getting the proper characters back but is not set up to handle the proper character set so displays . 2> Your servlet container is not set up (either inbound or outbound) to handle the character set you're sending it. Best, Erick On Fri, May 23, 2014 at 3:18 AM, anarchos78 wrote: > Hi, > > I'm trying to index data from mysql. The indexing is successful. Then I > tried to use the mysql concat function (data-config.xml) in order to > concatenate a custom string with a field like this: *CONCAT('(', > CAST('ΤΜΗΜΑ' AS CHAR CHARACTER SET utf8), ' ', apofasi_tmima, ')') *. The > custom string ('ΤΜΗΜΑ') in Greek. Then when I try to query the field solr > returns "?" instead of "ΤΜΗΜΑ". I have also use this: *CONCAT('(', > 'ΤΜΗΜΑ', ' ', apofasi_tmima, ')')* with no success. > The "data-config.xml" file is utf-8 encoded and at the beginning there is > the "" xml directive. I have also > tried to set in the dataSource url “characterEncoding=utf8” but indexing > fails. > What am I missing here? Is there any workaround for this? > Below is a snippet from data-config.xml: > > > > convertType="false" driver="com.mysql.jdbc.Driver" > url="jdbc:mysql://127.0.0.1:3306/apofaseis?zeroDateTimeBehavior=convertToNull" > user="root" password="" name="db" /> > > > >dataSource="db" > transformer="HTMLStripTransformer" > query="select id, CONCAT_WS('', CONCAT(apofasi_number, '/', > apofasi_date, > ' ', (CASE apofasi_tmima WHEN NULL THEN '' WHEN '' THEN '' ELSE CONCAT('(', > CAST('ΤΜΗΜΑ' AS CHAR CHARACTER SET utf8), ' ', apofasi_tmima, ')') END))) AS > grid_title, CAST(CONCAT_WS('_',id,model) AS CHAR) AS solr_id, > apofasi_number, apofasi_date, apofasi_tmima, CONCAT(IFNULL(apofasi_thema, > ''), ' ', IFNULL(apofasi_description, ''), ' ', apofasi_body) AS content, > type, model, url, search_tag, last_modified, CONCAT_WS('', > CONCAT(apofasi_number, '/', apofasi_date, ' ', (CASE apofasi_tmima WHEN > NULL > THEN'' WHEN '' THEN '' ELSE CONCAT('(', CAST('ΤΜΗΜΑ' AS CHAR CHARACTER SET > utf8), ' ', apofasi_tmima, ')') END))) AS title from apofaseis _2000 where > type = 'text'" > ... > ... > > > Regards, > anarchos78 > > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Import-data-from-Mysql-concat-issues-tp4137814.html > Sent from the Solr - User mailing list archive at Nabble.com.
Import data from Mysql concat issues
Hi, I'm trying to index data from mysql. The indexing is successful. Then I tried to use the mysql concat function (data-config.xml) in order to concatenate a custom string with a field like this: *CONCAT('(', CAST('ΤΜΗΜΑ' AS CHAR CHARACTER SET utf8), ' ', apofasi_tmima, ')') *. The custom string ('ΤΜΗΜΑ') in Greek. Then when I try to query the field solr returns "?" instead of "ΤΜΗΜΑ". I have also use this: *CONCAT('(', 'ΤΜΗΜΑ', ' ', apofasi_tmima, ')')* with no success. The "data-config.xml" file is utf-8 encoded and at the beginning there is the "" xml directive. I have also tried to set in the dataSource url “characterEncoding=utf8” but indexing fails. What am I missing here? Is there any workaround for this? Below is a snippet from data-config.xml: http://lucene.472066.n3.nabble.com/Import-data-from-Mysql-concat-issues-tp4137814.html Sent from the Solr - User mailing list archive at Nabble.com.