subject:"failed tests because of wrong locale"

Re: failed tests because of wrong locale

2008-08-14 Thread Stefan Oestreicher


Shalin Shekhar Mangar schrieb:

Hi Stefan,

Do you mind opening a jira issue and attaching these patches there?
  

not at all: https://issues.apache.org/jira/browse/SOLR-700

For NumberFormatTransformer, I guess we should add another attribute for
locale which should be used for parsing. The reason is that the data you are
processing may be coming from a different locale than the system locale.
What do you think?
  

I totally agree.

On Thu, Aug 14, 2008 at 2:33 AM, Stefan Oestreicher <
[EMAIL PROTECTED]> wrote:

  

Hi,

I just ran the unit tests on my system (uses a german locale) and the
DateFieldTest and LegacyDateFieldTest failed because the expected decimal
format used "." as separator but the actual result was formatted using the
"," separator.
This happens because the ISO8601CanonicalDateFormat class which is defined
inside the DateField class doesn't specify the US locale when creating the
formatter used for milliseconds.
I attached a patch which fixes that problem
(schema.DateField-locale.patch.txt).

TestNumberFormatTransformer in contrib/dataimporthandler also failed
because handler.dataimport.NumberFormatTransformer uses a NumberFormatter
which relies on the system locale. But I think in this case it's
intentional, so I modified the test case to use the grouping separator of
the system locale
(handler.dataimport.TestNumberFormatTransformer-locale.patch.txt).

After applying those changes all tests succeeded.

kind regards,

Stefan

### Eclipse Workspace Patch 1.0
#P solr
Index:
contrib/dataimporthandler/src/test/java/org/apache/solr/handler/dataimport/TestNumberFormatTransformer.java
===
---
contrib/dataimporthandler/src/test/java/org/apache/solr/handler/dataimport/TestNumberFormatTransformer.java
(revision 685675)
+++
contrib/dataimporthandler/src/test/java/org/apache/solr/handler/dataimport/TestNumberFormatTransformer.java
(working copy)
@@ -19,6 +19,7 @@
 import org.junit.Assert;
 import org.junit.Test;

+import java.text.DecimalFormatSymbols;
 import java.util.ArrayList;
 import java.util.List;
 import java.util.Map;
@@ -32,6 +33,9 @@
 * @since solr 1.3
 */
 public class TestNumberFormatTransformer {
+  private static final char GRP_SEP = new DecimalFormatSymbols()
+.getGroupingSeparator();
+
  @Test
  @SuppressWarnings("unchecked")
  public void testTransformRow_SingleNumber() {
@@ -40,7 +44,7 @@
NumberFormatTransformer.FORMAT_STYLE,
NumberFormatTransformer.NUMBER));
Context c = AbstractDataImportHandlerTest.getContext(null, null, null,
0,
l, null);
-Map m = AbstractDataImportHandlerTest.createMap("num", "123,567");
+Map m = AbstractDataImportHandlerTest.createMap("num",
"123"+GRP_SEP+"567");
new NumberFormatTransformer().transformRow(m, c);
Assert.assertEquals(new Long(123567), m.get("num"));
  }
@@ -56,8 +60,8 @@
NumberFormatTransformer.FORMAT_STYLE,
NumberFormatTransformer.NUMBER));

List inputs = new ArrayList();
-inputs.add("123,567");
-inputs.add("245,678");
+inputs.add("123"+GRP_SEP+"567");
+inputs.add("245"+GRP_SEP+"678");
Map row = AbstractDataImportHandlerTest.createMap("inputs", inputs);

VariableResolverImpl resolver = new VariableResolverImpl();

### Eclipse Workspace Patch 1.0
#P solr
Index: src/java/org/apache/solr/schema/DateField.java
===
--- src/java/org/apache/solr/schema/DateField.java  (revision 685675)
+++ src/java/org/apache/solr/schema/DateField.java  (working copy)
@@ -31,6 +31,7 @@
 import java.util.Date;
 import java.util.TimeZone;
 import java.util.Locale;
+import java.text.DecimalFormatSymbols;
 import java.text.SimpleDateFormat;
 import java.text.DateFormat;
 import java.text.NumberFormat;
@@ -245,7 +246,8 @@
protected NumberFormat millisParser
  = NumberFormat.getIntegerInstance(Locale.US);

-protected NumberFormat millisFormat = new DecimalFormat(".###");
+protected NumberFormat millisFormat = new DecimalFormat(".###",
+  new DecimalFormatSymbols(Locale.US));

public ISO8601CanonicalDateFormat() {
  super("-MM-dd'T'HH:mm:ss", Locale.US);
@@ -295,7 +297,8 @@
  ISO8601CanonicalDateFormat c
= (ISO8601CanonicalDateFormat) super.clone();
  c.millisParser = NumberFormat.getIntegerInstance(Locale.US);
-  c.millisFormat = new DecimalFormat(".###");
+  c.millisFormat = new DecimalFormat(".###",
+new DecimalFormatSymbols(Locale.US));
  return c;
}
  }

Re: failed tests because of wrong locale

2008-08-14 Thread Shalin Shekhar Mangar

Hi Stefan,

Do you mind opening a jira issue and attaching these patches there?

For NumberFormatTransformer, I guess we should add another attribute for
locale which should be used for parsing. The reason is that the data you are
processing may be coming from a different locale than the system locale.
What do you think?

On Thu, Aug 14, 2008 at 2:33 AM, Stefan Oestreicher <
[EMAIL PROTECTED]> wrote:

> Hi,
>
> I just ran the unit tests on my system (uses a german locale) and the
> DateFieldTest and LegacyDateFieldTest failed because the expected decimal
> format used "." as separator but the actual result was formatted using the
> "," separator.
> This happens because the ISO8601CanonicalDateFormat class which is defined
> inside the DateField class doesn't specify the US locale when creating the
> formatter used for milliseconds.
> I attached a patch which fixes that problem
> (schema.DateField-locale.patch.txt).
>
> TestNumberFormatTransformer in contrib/dataimporthandler also failed
> because handler.dataimport.NumberFormatTransformer uses a NumberFormatter
> which relies on the system locale. But I think in this case it's
> intentional, so I modified the test case to use the grouping separator of
> the system locale
> (handler.dataimport.TestNumberFormatTransformer-locale.patch.txt).
>
> After applying those changes all tests succeeded.
>
> kind regards,
>
> Stefan
>
> ### Eclipse Workspace Patch 1.0
> #P solr
> Index:
> contrib/dataimporthandler/src/test/java/org/apache/solr/handler/dataimport/TestNumberFormatTransformer.java
> ===
> ---
> contrib/dataimporthandler/src/test/java/org/apache/solr/handler/dataimport/TestNumberFormatTransformer.java
> (revision 685675)
> +++
> contrib/dataimporthandler/src/test/java/org/apache/solr/handler/dataimport/TestNumberFormatTransformer.java
> (working copy)
> @@ -19,6 +19,7 @@
>  import org.junit.Assert;
>  import org.junit.Test;
>
> +import java.text.DecimalFormatSymbols;
>  import java.util.ArrayList;
>  import java.util.List;
>  import java.util.Map;
> @@ -32,6 +33,9 @@
>  * @since solr 1.3
>  */
>  public class TestNumberFormatTransformer {
> +  private static final char GRP_SEP = new DecimalFormatSymbols()
> +.getGroupingSeparator();
> +
>   @Test
>   @SuppressWarnings("unchecked")
>   public void testTransformRow_SingleNumber() {
> @@ -40,7 +44,7 @@
> NumberFormatTransformer.FORMAT_STYLE,
> NumberFormatTransformer.NUMBER));
> Context c = AbstractDataImportHandlerTest.getContext(null, null, null,
> 0,
> l, null);
> -Map m = AbstractDataImportHandlerTest.createMap("num", "123,567");
> +Map m = AbstractDataImportHandlerTest.createMap("num",
> "123"+GRP_SEP+"567");
> new NumberFormatTransformer().transformRow(m, c);
> Assert.assertEquals(new Long(123567), m.get("num"));
>   }
> @@ -56,8 +60,8 @@
> NumberFormatTransformer.FORMAT_STYLE,
> NumberFormatTransformer.NUMBER));
>
> List inputs = new ArrayList();
> -inputs.add("123,567");
> -inputs.add("245,678");
> +inputs.add("123"+GRP_SEP+"567");
> +inputs.add("245"+GRP_SEP+"678");
> Map row = AbstractDataImportHandlerTest.createMap("inputs", inputs);
>
> VariableResolverImpl resolver = new VariableResolverImpl();
>
> ### Eclipse Workspace Patch 1.0
> #P solr
> Index: src/java/org/apache/solr/schema/DateField.java
> ===
> --- src/java/org/apache/solr/schema/DateField.java  (revision 685675)
> +++ src/java/org/apache/solr/schema/DateField.java  (working copy)
> @@ -31,6 +31,7 @@
>  import java.util.Date;
>  import java.util.TimeZone;
>  import java.util.Locale;
> +import java.text.DecimalFormatSymbols;
>  import java.text.SimpleDateFormat;
>  import java.text.DateFormat;
>  import java.text.NumberFormat;
> @@ -245,7 +246,8 @@
> protected NumberFormat millisParser
>   = NumberFormat.getIntegerInstance(Locale.US);
>
> -protected NumberFormat millisFormat = new DecimalFormat(".###");
> +protected NumberFormat millisFormat = new DecimalFormat(".###",
> +  new DecimalFormatSymbols(Locale.US));
>
> public ISO8601CanonicalDateFormat() {
>   super("-MM-dd'T'HH:mm:ss", Locale.US);
> @@ -295,7 +297,8 @@
>   ISO8601CanonicalDateFormat c
> = (ISO8601CanonicalDateFormat) super.clone();
>   c.millisParser = NumberFormat.getIntegerInstance(Locale.US);
> -  c.millisFormat = new DecimalFormat(".###");
> +  c.millisFormat = new DecimalFormat(".###",
> +new DecimalFormatSymbols(Locale.US));
>   return c;
> }
>   }
>
>


-- 
Regards,
Shalin Shekhar Mangar.

failed tests because of wrong locale

2008-08-13 Thread Stefan Oestreicher


Hi,

I just ran the unit tests on my system (uses a german locale) and the 
DateFieldTest and LegacyDateFieldTest failed because the expected 
decimal format used "." as separator but the actual result was formatted 
using the "," separator.
This happens because the ISO8601CanonicalDateFormat class which is 
defined inside the DateField class doesn't specify the US locale when 
creating the formatter used for milliseconds.
I attached a patch which fixes that problem 
(schema.DateField-locale.patch.txt).


TestNumberFormatTransformer in contrib/dataimporthandler also failed 
because handler.dataimport.NumberFormatTransformer uses a 
NumberFormatter which relies on the system locale. But I think in this 
case it's intentional, so I modified the test case to use the grouping 
separator of the system locale 
(handler.dataimport.TestNumberFormatTransformer-locale.patch.txt).


After applying those changes all tests succeeded.

kind regards,

Stefan
### Eclipse Workspace Patch 1.0
#P solr
Index: 
contrib/dataimporthandler/src/test/java/org/apache/solr/handler/dataimport/TestNumberFormatTransformer.java
===
--- 
contrib/dataimporthandler/src/test/java/org/apache/solr/handler/dataimport/TestNumberFormatTransformer.java
 (revision 685675)
+++ 
contrib/dataimporthandler/src/test/java/org/apache/solr/handler/dataimport/TestNumberFormatTransformer.java
 (working copy)
@@ -19,6 +19,7 @@
 import org.junit.Assert;
 import org.junit.Test;
 
+import java.text.DecimalFormatSymbols;
 import java.util.ArrayList;
 import java.util.List;
 import java.util.Map;
@@ -32,6 +33,9 @@
  * @since solr 1.3
  */
 public class TestNumberFormatTransformer {
+  private static final char GRP_SEP = new DecimalFormatSymbols()
+.getGroupingSeparator();
+  
   @Test
   @SuppressWarnings("unchecked")
   public void testTransformRow_SingleNumber() {
@@ -40,7 +44,7 @@
 NumberFormatTransformer.FORMAT_STYLE, 
NumberFormatTransformer.NUMBER));
 Context c = AbstractDataImportHandlerTest.getContext(null, null, null, 0,
 l, null);
-Map m = AbstractDataImportHandlerTest.createMap("num", "123,567");
+Map m = AbstractDataImportHandlerTest.createMap("num", 
"123"+GRP_SEP+"567");
 new NumberFormatTransformer().transformRow(m, c);
 Assert.assertEquals(new Long(123567), m.get("num"));
   }
@@ -56,8 +60,8 @@
 NumberFormatTransformer.FORMAT_STYLE, 
NumberFormatTransformer.NUMBER));
 
 List inputs = new ArrayList();
-inputs.add("123,567");
-inputs.add("245,678");
+inputs.add("123"+GRP_SEP+"567");
+inputs.add("245"+GRP_SEP+"678");
 Map row = AbstractDataImportHandlerTest.createMap("inputs", inputs);
 
 VariableResolverImpl resolver = new VariableResolverImpl();
### Eclipse Workspace Patch 1.0
#P solr
Index: src/java/org/apache/solr/schema/DateField.java
===
--- src/java/org/apache/solr/schema/DateField.java  (revision 685675)
+++ src/java/org/apache/solr/schema/DateField.java  (working copy)
@@ -31,6 +31,7 @@
 import java.util.Date;
 import java.util.TimeZone;
 import java.util.Locale;
+import java.text.DecimalFormatSymbols;
 import java.text.SimpleDateFormat;
 import java.text.DateFormat;
 import java.text.NumberFormat;
@@ -245,7 +246,8 @@
 protected NumberFormat millisParser
   = NumberFormat.getIntegerInstance(Locale.US);
 
-protected NumberFormat millisFormat = new DecimalFormat(".###");
+protected NumberFormat millisFormat = new DecimalFormat(".###", 
+  new DecimalFormatSymbols(Locale.US));
 
 public ISO8601CanonicalDateFormat() {
   super("-MM-dd'T'HH:mm:ss", Locale.US);
@@ -295,7 +297,8 @@
   ISO8601CanonicalDateFormat c
 = (ISO8601CanonicalDateFormat) super.clone();
   c.millisParser = NumberFormat.getIntegerInstance(Locale.US);
-  c.millisFormat = new DecimalFormat(".###");
+  c.millisFormat = new DecimalFormat(".###", 
+new DecimalFormatSymbols(Locale.US));
   return c;
 }
   }

Re: failed tests because of wrong locale

Re: failed tests because of wrong locale

failed tests because of wrong locale

3 matches

Site Navigation

Mail list logo

Footer information