[
https://issues.apache.org/jira/browse/SOLR-1916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492775#comment-13492775
]
Robert Muir commented on SOLR-1916:
-----------------------------------
James, thanks for looking at this!!!
It may not be a locale issue, instead a time zone issue (or both). But this
test definitely failed intermittently in the past.
For example, it failed during a Daylight savings time window (but only for
developers in Europe!) and Chris Male addressed
some of the issues in SOLR-1821.
Fortunately, Uwe Schindler has made it dead easy to identify most of these
issues: we not longer have to solely rely upon unit tests alone.
http://blog.thetaphi.de/2012/07/default-locales-default-charsets-and.html
DIH currently has 40 violations!
Try this:
{noformat}
Index: build.xml
===================================================================
--- build.xml (revision 1406757)
+++ build.xml (working copy)
@@ -250,8 +250,6 @@
</apiFileSet>
<fileset dir="${basedir}/build">
<include name="**/*.class" />
- <!-- exclude DIH for now as it is broken with Locales and Encodings:
SOLR-1916 -->
- <exclude name="contrib/solr-dataimporthandler*/**" />
</fileset>
</forbidden-apis>
</target>
{noformat}
Then run
{noformat}
rmuir@beast:~/workspace/lucene-trunk/solr$ ant check-forbidden-apis
...
-check-forbidden-java-apis:
[forbidden-apis] Reading API signatures:
/home/rmuir/workspace/lucene-trunk/lucene/tools/forbiddenApis/commons-io.txt
[forbidden-apis] Reading API signatures:
/home/rmuir/workspace/lucene-trunk/lucene/tools/forbiddenApis/executors.txt
[forbidden-apis] Reading API signatures:
/home/rmuir/workspace/lucene-trunk/lucene/tools/forbiddenApis/jdk-deprecated.txt
[forbidden-apis] Reading API signatures:
/home/rmuir/workspace/lucene-trunk/lucene/tools/forbiddenApis/jdk.txt
[forbidden-apis] Loading classes to check...
[forbidden-apis] Scanning for API signatures and dependencies...
[forbidden-apis] Forbidden method invocation:
java.text.DecimalFormatSymbols#<init>()
[forbidden-apis] in
org.apache.solr.handler.dataimport.TestNumberFormatTransformer
(TestNumberFormatTransformer.java:36)
[forbidden-apis] Forbidden method invocation:
java.text.DecimalFormatSymbols#<init>()
[forbidden-apis] in
org.apache.solr.handler.dataimport.TestNumberFormatTransformer
(TestNumberFormatTransformer.java:37)
[forbidden-apis] Forbidden method invocation:
java.text.MessageFormat#<init>(java.lang.String)
[forbidden-apis] in org.apache.solr.handler.dataimport.DebugLogger
(DebugLogger.java:52)
[forbidden-apis] Forbidden method invocation:
java.text.SimpleDateFormat#<init>(java.lang.String)
[forbidden-apis] in
org.apache.solr.handler.dataimport.TestDateFormatTransformer
(TestDateFormatTransformer.java:43)
[forbidden-apis] Forbidden method invocation:
java.text.SimpleDateFormat#<init>(java.lang.String)
[forbidden-apis] in
org.apache.solr.handler.dataimport.TestDateFormatTransformer
(TestDateFormatTransformer.java:66)
[forbidden-apis] Forbidden method invocation:
java.text.SimpleDateFormat#<init>(java.lang.String)
[forbidden-apis] in org.apache.solr.handler.dataimport.MailEntityProcessor
(MailEntityProcessor.java:88)
[forbidden-apis] Forbidden method invocation: java.lang.String#getBytes()
[forbidden-apis] in org.apache.solr.handler.dataimport.TestDocBuilder2
(TestDocBuilder2.java:250)
[forbidden-apis] Forbidden method invocation: java.lang.String#getBytes()
[forbidden-apis] in org.apache.solr.handler.dataimport.TestDocBuilder2
(TestDocBuilder2.java:251)
[forbidden-apis] Forbidden method invocation: java.lang.String#getBytes()
[forbidden-apis] in org.apache.solr.handler.dataimport.TestDocBuilder2
(TestDocBuilder2.java:252)
[forbidden-apis] Forbidden method invocation: java.lang.String#getBytes()
[forbidden-apis] in org.apache.solr.handler.dataimport.TestDocBuilder2
(TestDocBuilder2.java:257)
[forbidden-apis] Forbidden method invocation:
java.text.SimpleDateFormat#<init>(java.lang.String)
[forbidden-apis] in org.apache.solr.handler.dataimport.DataImporter$3
(DataImporter.java:490)
[forbidden-apis] Forbidden method invocation:
java.lang.String#format(java.lang.String,java.lang.Object[])
[forbidden-apis] in org.apache.solr.handler.dataimport.DocBuilder
(DocBuilder.java:711)
[forbidden-apis] Forbidden method invocation:
java.lang.String#format(java.lang.String,java.lang.Object[])
[forbidden-apis] in org.apache.solr.handler.dataimport.DocBuilder
(DocBuilder.java:717)
[forbidden-apis] Forbidden method invocation:
java.lang.String#format(java.lang.String,java.lang.Object[])
[forbidden-apis] in org.apache.solr.handler.dataimport.DocBuilder
(DocBuilder.java:725)
[forbidden-apis] Forbidden method invocation:
java.lang.String#format(java.lang.String,java.lang.Object[])
[forbidden-apis] in org.apache.solr.handler.dataimport.DocBuilder
(DocBuilder.java:727)
[forbidden-apis] Forbidden method invocation:
java.io.InputStreamReader#<init>(java.io.InputStream)
[forbidden-apis] in org.apache.solr.handler.dataimport.FieldReaderDataSource
(FieldReaderDataSource.java:109)
[forbidden-apis] Forbidden method invocation: java.lang.String#getBytes()
[forbidden-apis] in
org.apache.solr.handler.dataimport.TestFileListWithLineEntityProcessor
(TestFileListWithLineEntityProcessor.java:36)
[forbidden-apis] Forbidden method invocation: java.lang.String#getBytes()
[forbidden-apis] in
org.apache.solr.handler.dataimport.TestFileListWithLineEntityProcessor
(TestFileListWithLineEntityProcessor.java:37)
[forbidden-apis] Forbidden method invocation: java.lang.String#getBytes()
[forbidden-apis] in
org.apache.solr.handler.dataimport.TestFileListWithLineEntityProcessor
(TestFileListWithLineEntityProcessor.java:38)
[forbidden-apis] Forbidden method invocation:
java.text.SimpleDateFormat#<init>(java.lang.String)
[forbidden-apis] in org.apache.solr.handler.dataimport.TestVariableResolver
(TestVariableResolver.java:92)
[forbidden-apis] Forbidden method invocation:
java.text.SimpleDateFormat#<init>(java.lang.String)
[forbidden-apis] in org.apache.solr.handler.dataimport.TestVariableResolver
(TestVariableResolver.java:101)
[forbidden-apis] Forbidden method invocation:
java.text.SimpleDateFormat#<init>(java.lang.String)
[forbidden-apis] in org.apache.solr.handler.dataimport.TestVariableResolver
(TestVariableResolver.java:107)
[forbidden-apis] Forbidden method invocation:
java.text.SimpleDateFormat#<init>(java.lang.String)
[forbidden-apis] in org.apache.solr.handler.dataimport.TestVariableResolver
(TestVariableResolver.java:138)
[forbidden-apis] Forbidden method invocation:
java.text.SimpleDateFormat#<init>(java.lang.String)
[forbidden-apis] in org.apache.solr.handler.dataimport.TestVariableResolver
(TestVariableResolver.java:147)
[forbidden-apis] Forbidden method invocation:
java.text.SimpleDateFormat#<init>(java.lang.String)
[forbidden-apis] in org.apache.solr.handler.dataimport.EvaluatorBag$4
(EvaluatorBag.java:164)
[forbidden-apis] Forbidden method invocation:
java.util.GregorianCalendar#<init>()
[forbidden-apis] in org.apache.solr.handler.dataimport.TestEvaluatorBag
(TestEvaluatorBag.java:122)
[forbidden-apis] Forbidden method invocation:
java.text.SimpleDateFormat#<init>(java.lang.String)
[forbidden-apis] in org.apache.solr.handler.dataimport.TestEvaluatorBag
(TestEvaluatorBag.java:125)
[forbidden-apis] Forbidden method invocation:
java.util.GregorianCalendar#<init>()
[forbidden-apis] in org.apache.solr.handler.dataimport.TestEvaluatorBag
(TestEvaluatorBag.java:128)
[forbidden-apis] Forbidden method invocation:
java.text.SimpleDateFormat#<init>(java.lang.String)
[forbidden-apis] in org.apache.solr.handler.dataimport.TestEvaluatorBag
(TestEvaluatorBag.java:135)
[forbidden-apis] Forbidden method invocation: java.lang.String#getBytes()
[forbidden-apis] in
org.apache.solr.handler.dataimport.TestFileListEntityProcessor
(TestFileListEntityProcessor.java:43)
[forbidden-apis] Forbidden method invocation: java.lang.String#getBytes()
[forbidden-apis] in
org.apache.solr.handler.dataimport.TestFileListEntityProcessor
(TestFileListEntityProcessor.java:44)
[forbidden-apis] Forbidden method invocation: java.lang.String#getBytes()
[forbidden-apis] in
org.apache.solr.handler.dataimport.TestFileListEntityProcessor
(TestFileListEntityProcessor.java:45)
[forbidden-apis] Forbidden method invocation: java.lang.String#getBytes()
[forbidden-apis] in
org.apache.solr.handler.dataimport.TestFileListEntityProcessor
(TestFileListEntityProcessor.java:140)
[forbidden-apis] Forbidden method invocation: java.lang.String#getBytes()
[forbidden-apis] in
org.apache.solr.handler.dataimport.TestFileListEntityProcessor
(TestFileListEntityProcessor.java:141)
[forbidden-apis] Forbidden method invocation: java.lang.String#getBytes()
[forbidden-apis] in
org.apache.solr.handler.dataimport.TestFileListEntityProcessor
(TestFileListEntityProcessor.java:142)
[forbidden-apis] Forbidden method invocation: java.lang.String#getBytes()
[forbidden-apis] in
org.apache.solr.handler.dataimport.TestFileListEntityProcessor
(TestFileListEntityProcessor.java:164)
[forbidden-apis] Forbidden method invocation: java.lang.String#getBytes()
[forbidden-apis] in
org.apache.solr.handler.dataimport.TestFileListEntityProcessor
(TestFileListEntityProcessor.java:179)
[forbidden-apis] Forbidden method invocation: java.lang.String#getBytes()
[forbidden-apis] in
org.apache.solr.handler.dataimport.TestFileListEntityProcessor
(TestFileListEntityProcessor.java:180)
[forbidden-apis] Forbidden method invocation: java.lang.String#getBytes()
[forbidden-apis] in
org.apache.solr.handler.dataimport.TestFileListEntityProcessor
(TestFileListEntityProcessor.java:181)
[forbidden-apis] Forbidden method invocation:
java.io.InputStreamReader#<init>(java.io.InputStream)
[forbidden-apis] in org.apache.solr.handler.dataimport.FileDataSource
(FileDataSource.java:128)
[forbidden-apis] Scanned 2080 (and 648 related) class file(s) for forbidden API
invocations (in 1.08s), 40 error(s).
BUILD FAILED
/home/rmuir/workspace/lucene-trunk/solr/build.xml:243: Check for forbidden API
calls failed, see log.
{noformat}
> investigate DIH use of default locale
> -------------------------------------
>
> Key: SOLR-1916
> URL: https://issues.apache.org/jira/browse/SOLR-1916
> Project: Solr
> Issue Type: Task
> Components: contrib - DataImportHandler
> Affects Versions: 3.1, 4.0-ALPHA
> Reporter: Robert Muir
> Assignee: Robert Muir
> Fix For: 4.1
>
> Attachments: SOLR-1916.patch
>
>
> This is a spinoff from LUCENE-2466.
> In this issue I changed my locale to various locales and found some problems
> in Lucene/Solr triggered by use of the default Locale.
> I noticed some use of the default-locale for Date operations in DIH
> (TimeZone.getDefault/Locale.getDefault) and, while no tests fail, I think it
> might be better to support a locale parameter for this.
> The wiki documents that numeric parsing can support localized numerics
> formats: http://wiki.apache.org/solr/DataImportHandler#NumberFormatTransformer
> In both cases, I don't think we should ever use the default Locale. If no
> Locale is provided, I find that new Locale("") <-- Unicode Root Locale, is a
> better default for a server situation in a lot of cases, as it won't change
> depending on the computer, or perhaps we just make Locale params mandatory
> for this.
> Finally, in both cases, if localized numbers/dates are explicitly supported,
> I think we should come up with a test strategy to ensure everything is
> working. One idea is to do something similar to or make use of Lucene's
> LocalizedTestCase.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
