[
https://issues.apache.org/jira/browse/PARQUET-1647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17768334#comment-17768334
]
ASF GitHub Bot commented on PARQUET-1647:
-----------------------------------------
zhangjiashen commented on code in PR #1142:
URL: https://github.com/apache/parquet-mr/pull/1142#discussion_r1335131705
##########
parquet-common/src/test/java/org/apache/parquet/util/TestFloat16.java:
##########
@@ -0,0 +1,89 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.parquet.util;
+
+import org.junit.Test;
+
+import static org.junit.Assert.assertEquals;
+import static org.apache.parquet.util.Float16.*;
+
+public class TestFloat16
+{
+ @Test
+ public void testFloat16ToFloat() {
+ // Zeroes, NaN and infinities
+ assertEquals(0.0f, toFloat(toFloat16(0.0f)), 0.0f);
+ assertEquals(-0.0f, toFloat(toFloat16(-0.0f)), 0.0f);
+ assertEquals(Float.NaN, toFloat(toFloat16(Float.NaN)), 0.0f);
+ assertEquals(Float.POSITIVE_INFINITY,
toFloat(toFloat16(Float.POSITIVE_INFINITY)), 0.0f);
+ assertEquals(Float.NEGATIVE_INFINITY,
toFloat(toFloat16(Float.NEGATIVE_INFINITY)), 0.0f);
+ // Known values
+ assertEquals(1.0009765625f, toFloat(toFloat16(1.0009765625f)), 0.0f);
+ assertEquals(-2.0f, toFloat(toFloat16(-2.0f)), 0.0f);
+ assertEquals(6.1035156e-5f, toFloat(toFloat16(6.10352e-5f)), 0.0f); //
Inexact
+ assertEquals(65504.0f, toFloat(toFloat16(65504.0f)), 0.0f);
+ assertEquals(0.33325195f, toFloat(toFloat16(1.0f / 3.0f)), 0.0f); //
Inexact
+ // Denormals (flushed to +/-0)
+ assertEquals(6.097555e-5f, toFloat(toFloat16(6.09756e-5f)), 0.0f);
+ assertEquals(5.9604645e-8f, toFloat(toFloat16(5.96046e-8f)), 0.0f);
+ assertEquals(-6.097555e-5f, toFloat(toFloat16(-6.09756e-5f)), 0.0f);
+ assertEquals(-5.9604645e-8f, toFloat(toFloat16(-5.96046e-8f)), 0.0f);
+ }
+
+ @Test
+ public void testFloatToFloat16() {
+ // Zeroes, NaN and infinities
+ assertEquals(POSITIVE_ZERO, toFloat16(0.0f));
+ assertEquals(NEGATIVE_ZERO, toFloat16(-0.0f));
+ assertEquals(NaN, toFloat16(Float.NaN));
Review Comment:
Good suggestion, added!
> [Java] support for Arrow's float16
> ----------------------------------
>
> Key: PARQUET-1647
> URL: https://issues.apache.org/jira/browse/PARQUET-1647
> Project: Parquet
> Issue Type: Improvement
> Components: parquet-format, parquet-thrift
> Reporter: The Alchemist
> Priority: Minor
>
> h2. DESCRIPTION
>
> I'm wondering if there's any interest in supporting Arrow's {{float16}} type
> in Parquet.
> There seem to be one or two {{float16}} / {{halffloat}} tickets here (e.g.,
> PARQUET-1403) but nothing that speaks to adding half-float support to Parquet
> in-general.
>
> h2. PLANS
> I'm able to spend some time on this, if someone points me in the right
> direction.
>
> # Add the {{HALFFLOAT}} or {{FLOAT16}} enum (any preferred naming
> convention?) to
> [https://github.com/apache/parquet-format/blob/master/src/main/thrift/parquet.thrift#L32]
> # Add {{HALFFLOAT}} to {{org.apache.parquet.schema.PrimitiveType}}
> # Add {{HALFFLOAT}} support to
> {{org.apache.parquet.arrow.schema.SchemaConverter}}
> # Add encoding for new type at {{org.apache.parquet.column.Encoding}}
> # ??
> If anyone has any interest in this, pointers, or comments, they would be
> greatly appreciated!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
