[
https://issues.apache.org/jira/browse/DRILL-5846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16343770#comment-16343770
]
ASF GitHub Bot commented on DRILL-5846:
---------------------------------------
Github user vrozov commented on a diff in the pull request:
https://github.com/apache/drill/pull/1060#discussion_r164519147
--- Diff:
exec/memory/base/src/main/java/org/apache/drill/exec/util/MemoryUtils.java ---
@@ -0,0 +1,186 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.drill.exec.util;
+
+import java.lang.reflect.Field;
+import java.nio.ByteOrder;
+
+import sun.misc.Unsafe;
+
+/** Exposes advanced Memory Access APIs for Little-Endian / Unaligned
platforms */
+@SuppressWarnings("restriction")
+public final class MemoryUtils {
+
+ // Ensure this is a little-endian hardware */
+ static {
+ if (ByteOrder.nativeOrder() != ByteOrder.LITTLE_ENDIAN) {
+ throw new IllegalStateException("Drill only runs on LittleEndian
systems.");
+ }
+ }
+
+ /** Java's unsafe object */
+ private static Unsafe UNSAFE;
+
+ static {
+ try {
+ Field theUnsafe = Unsafe.class.getDeclaredField("theUnsafe");
+ theUnsafe.setAccessible(true);
+ UNSAFE = (Unsafe) theUnsafe.get(null);
+ }
+ catch (Exception e) {
+ throw new RuntimeException(e);
+ }
+ }
+
+ /** Byte arrays offset */
+ private static final long BYTE_ARRAY_OFFSET =
UNSAFE.arrayBaseOffset(byte[].class);
+
+ /** Number of bytes in a long */
+ public static final int LONG_NUM_BYTES = 8;
+ /** Number of bytes in an int */
+ public static final int INT_NUM_BYTES = 4;
+ /** Number of bytes in a short */
+ public static final int SHORT_NUM_BYTES = 2;
+
+//----------------------------------------------------------------------------
+// APIs
+//----------------------------------------------------------------------------
+
+ /**
+ * @param data source byte array
+ * @param index index within the byte array
+ * @return short value starting at data+index
+ */
+ public static short getShort(byte[] data, int index) {
--- End diff --
@sachouche It is not clear why using MemoryUtils will make difference for
hotspot in using cache or 8bit instruction set. It will be used for both
MemoryUtils and DrillBuf.
> Improve Parquet Reader Performance for Flat Data types
> -------------------------------------------------------
>
> Key: DRILL-5846
> URL: https://issues.apache.org/jira/browse/DRILL-5846
> Project: Apache Drill
> Issue Type: Improvement
> Components: Storage - Parquet
> Affects Versions: 1.11.0
> Reporter: salim achouche
> Assignee: salim achouche
> Priority: Major
> Labels: performance
> Fix For: 1.13.0
>
>
> The Parquet Reader is a key use-case for Drill. This JIRA is an attempt to
> further improve the Parquet Reader performance as several users reported that
> Parquet parsing represents the lion share of the overall query execution. It
> tracks Flat Data types only as Nested DTs might involve functional and
> processing enhancements (e.g., a nested column can be seen as a Document;
> user might want to perform operations scoped at the document level that is no
> need to span all rows). Another JIRA will be created to handle the nested
> columns use-case.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
