[GitHub] [incubator-hudi] smarthi commented on a change in pull request #1159: [HUDI-479] Eliminate or Minimize use of Guava if possible
smarthi commented on a change in pull request #1159: [HUDI-479] Eliminate or
Minimize use of Guava if possible
URL: https://github.com/apache/incubator-hudi/pull/1159#discussion_r399324299
##
File path:
hudi-common/src/main/java/org/apache/hudi/common/util/FileIOUtils.java
##
@@ -91,4 +94,29 @@ public static void writeStringToFile(String str, String
filePath) throws IOExcep
out.flush();
out.close();
}
+
+ /**
+ * Closes a {@link Closeable}, with control over whether an {@code
IOException} may be thrown.
+ * @param closeable the {@code Closeable} object to be closed, or null,
+ * in which case this method does nothing.
+ * @param swallowIOException if true, don't propagate IO exceptions thrown
by the {@code close} methods.
+ *
+ * @throws IOException if {@code swallowIOException} is false and {@code
close} throws an {@code IOException}.
+ */
+ public static void close(@Nullable Closeable closeable, boolean
swallowIOException)
+ throws IOException {
+if (closeable == null) {
+ return;
+}
+try {
+ closeable.close();
+} catch (IOException e) {
+ if (!swallowIOException) {
+throw e;
+ }
+}
+ }
+
+ /** Maximum loop count when creating temp directories. */
+ private static final int TEMP_DIR_ATTEMPTS = 1;
Review comment:
I think we do - this is getting called in Metrics.java - but I guess it
could be dispensed with if we modify Metrics.java - haven't looked at it
closely enough - will take that up.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-hudi] smarthi commented on a change in pull request #1159: [HUDI-479] Eliminate or Minimize use of Guava if possible
smarthi commented on a change in pull request #1159: [HUDI-479] Eliminate or
Minimize use of Guava if possible
URL: https://github.com/apache/incubator-hudi/pull/1159#discussion_r398910413
##
File path:
hudi-common/src/main/java/org/apache/hudi/common/util/CollectionUtils.java
##
@@ -0,0 +1,132 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.common.util;
+
+import java.util.Collections;
+import java.util.Iterator;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+
+public class CollectionUtils {
+ /**
+ * Determines whether two iterators contain equal elements in the same
order. More specifically,
+ * this method returns {@code true} if {@code iterator1} and {@code
iterator2} contain the same
+ * number of elements and every element of {@code iterator1} is equal to the
corresponding element
+ * of {@code iterator2}.
+ *
+ * Note that this will modify the supplied iterators, since they will
have been advanced some
+ * number of elements forward.
+ */
+ public static boolean elementsEqual(Iterator iterator1, Iterator
iterator2) {
+while (iterator1.hasNext()) {
+ if (!iterator2.hasNext()) {
+return false;
+ }
+ Object o1 = iterator1.next();
+ Object o2 = iterator2.next();
+ if (!Objects.equals(o1, o2)) {
+return false;
+ }
+}
+return !iterator2.hasNext();
+ }
+
+ @SafeVarargs
+ public static Set createSetFromElements(final T... elements) {
+return Stream.of(elements).collect(Collectors.toSet());
+ }
+
+ public static Map createImmutableMap(final K key, final V value)
{
+return Collections.unmodifiableMap(Collections.singletonMap(key, value));
+ }
+
+ @SafeVarargs
+ public static List createImmutableList(final T... elements) {
+return
Collections.unmodifiableList(Stream.of(elements).collect(Collectors.toList()));
+ }
+
+ public static Map createImmutableMap(final Map map) {
+return Collections.unmodifiableMap(map);
+ }
+
+ @SafeVarargs
+ public static Set createImmutableSet(final T... elements) {
+return
Collections.unmodifiableSet(Stream.of(elements).collect(Collectors.toSet()));
+ }
+
+ public static Set createImmutableSet(final Set set) {
+return Collections.unmodifiableSet(set);
+ }
+
+ public static List createImmutableList(final List list) {
+return Collections.unmodifiableList(list);
+ }
+
+ private static Object[] checkElementsNotNull(Object... array) {
+return checkElementsNotNull(array, array.length);
+ }
+
+ private static Object[] checkElementsNotNull(Object[] array, int length) {
+for (int i = 0; i < length; i++) {
+ checkElementNotNull(array[i], i);
+}
+return array;
+ }
+
+ private static Object checkElementNotNull(Object element, int index) {
+if (element == null) {
+ throw new NullPointerException("at index " + index);
+}
+return element;
+ }
+
+ public static class Maps {
Review comment:
Fixed it as part of this PR including removing CollectionUtils.Maps -
HUDI-737 can be closed.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-hudi] smarthi commented on a change in pull request #1159: [HUDI-479] Eliminate or Minimize use of Guava if possible
smarthi commented on a change in pull request #1159: [HUDI-479] Eliminate or
Minimize use of Guava if possible
URL: https://github.com/apache/incubator-hudi/pull/1159#discussion_r398333676
##
File path:
hudi-common/src/main/java/org/apache/hudi/common/util/ReflectionUtils.java
##
@@ -90,16 +99,58 @@ public static Object loadClass(String clazz, Object...
constructorArgs) {
}
/**
- * Return stream of top level class names in the same class path as
passed-in class.
- *
- * @param clazz
+ * Scans all classes accessible from the context class loader
+ * which belong to the given package and subpackages.
+ *
+ * @param clazz class
+ * @return Stream of Class names in package
*/
- public static Stream getTopLevelClassesInClasspath(Class clazz) {
+ public static Stream getTopLevelClassesInClasspath(Class clazz) {
Review comment:
yes - let's raise a jira
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-hudi] smarthi commented on a change in pull request #1159: [HUDI-479] Eliminate or Minimize use of Guava if possible
smarthi commented on a change in pull request #1159: [HUDI-479] Eliminate or
Minimize use of Guava if possible
URL: https://github.com/apache/incubator-hudi/pull/1159#discussion_r398332715
##
File path:
hudi-common/src/main/java/org/apache/hudi/common/util/CollectionUtils.java
##
@@ -0,0 +1,132 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.common.util;
+
+import java.util.Collections;
+import java.util.Iterator;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+
+public class CollectionUtils {
+ /**
+ * Determines whether two iterators contain equal elements in the same
order. More specifically,
+ * this method returns {@code true} if {@code iterator1} and {@code
iterator2} contain the same
+ * number of elements and every element of {@code iterator1} is equal to the
corresponding element
+ * of {@code iterator2}.
+ *
+ * Note that this will modify the supplied iterators, since they will
have been advanced some
+ * number of elements forward.
+ */
+ public static boolean elementsEqual(Iterator iterator1, Iterator
iterator2) {
+while (iterator1.hasNext()) {
+ if (!iterator2.hasNext()) {
+return false;
+ }
+ Object o1 = iterator1.next();
+ Object o2 = iterator2.next();
+ if (!Objects.equals(o1, o2)) {
+return false;
+ }
+}
+return !iterator2.hasNext();
+ }
+
+ @SafeVarargs
+ public static Set createSetFromElements(final T... elements) {
+return Stream.of(elements).collect(Collectors.toSet());
+ }
+
+ public static Map createImmutableMap(final K key, final V value)
{
+return Collections.unmodifiableMap(Collections.singletonMap(key, value));
+ }
+
+ @SafeVarargs
+ public static List createImmutableList(final T... elements) {
+return
Collections.unmodifiableList(Stream.of(elements).collect(Collectors.toList()));
+ }
+
+ public static Map createImmutableMap(final Map map) {
+return Collections.unmodifiableMap(map);
+ }
+
+ @SafeVarargs
+ public static Set createImmutableSet(final T... elements) {
+return
Collections.unmodifiableSet(Stream.of(elements).collect(Collectors.toSet()));
+ }
+
+ public static Set createImmutableSet(final Set set) {
+return Collections.unmodifiableSet(set);
+ }
+
+ public static List createImmutableList(final List list) {
+return Collections.unmodifiableList(list);
+ }
+
+ private static Object[] checkElementsNotNull(Object... array) {
+return checkElementsNotNull(array, array.length);
+ }
+
+ private static Object[] checkElementsNotNull(Object[] array, int length) {
+for (int i = 0; i < length; i++) {
+ checkElementNotNull(array[i], i);
+}
+return array;
+ }
+
+ private static Object checkElementNotNull(Object element, int index) {
+if (element == null) {
+ throw new NullPointerException("at index " + index);
+}
+return element;
+ }
+
+ public static class Maps {
Review comment:
+1 - this was the closest to replacing Guava's ImmutableMap Builder pattern
that was being widely used across the project.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-hudi] smarthi commented on a change in pull request #1159: [HUDI-479] Eliminate or Minimize use of Guava if possible
smarthi commented on a change in pull request #1159: [HUDI-479] Eliminate or
Minimize use of Guava if possible
URL: https://github.com/apache/incubator-hudi/pull/1159#discussion_r362275520
##
File path:
hudi-client/src/main/java/org/apache/hudi/io/compact/HoodieRealtimeTableCompactor.java
##
@@ -125,7 +125,7 @@
config.getCompactionReverseLogReadEnabled(),
config.getMaxDFSStreamBufferSize(),
config.getSpillableMapBasePath());
if (!scanner.iterator().hasNext()) {
- return Lists.newArrayList();
+ return new ArrayList<>();
Review comment:
Agreed.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-hudi] smarthi commented on a change in pull request #1159: [HUDI-479] Eliminate or Minimize use of Guava if possible
smarthi commented on a change in pull request #1159: [HUDI-479] Eliminate or
Minimize use of Guava if possible
URL: https://github.com/apache/incubator-hudi/pull/1159#discussion_r362275520
##
File path:
hudi-client/src/main/java/org/apache/hudi/io/compact/HoodieRealtimeTableCompactor.java
##
@@ -125,7 +125,7 @@
config.getCompactionReverseLogReadEnabled(),
config.getMaxDFSStreamBufferSize(),
config.getSpillableMapBasePath());
if (!scanner.iterator().hasNext()) {
- return Lists.newArrayList();
+ return new ArrayList<>();
Review comment:
Agreed.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
