[jira] [Commented] (HUDI-124) Ensure third party libs are compatible with ASF policy
[ https://issues.apache.org/jira/browse/HUDI-124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16920261#comment-16920261 ] leesf commented on HUDI-124: Fixed via master: 5f9fa82f47e1cc14a22b869250fe23c8f9c033cd > Ensure third party libs are compatible with ASF policy > -- > > Key: HUDI-124 > URL: https://issues.apache.org/jira/browse/HUDI-124 > Project: Apache Hudi (incubating) > Issue Type: Sub-task > Components: asf-migration >Reporter: Vinoth Chandar >Assignee: BALAJI VARADARAJAN >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > [http://www.apache.org/legal/resolved.html] -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HUDI-128) Setup infra for nightly/snapshot releases
[ https://issues.apache.org/jira/browse/HUDI-128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16920260#comment-16920260 ] leesf commented on HUDI-128: Fixed via master: c265b4948fd78b2d9050b67d18c20b4caa8aceab > Setup infra for nightly/snapshot releases > - > > Key: HUDI-128 > URL: https://issues.apache.org/jira/browse/HUDI-128 > Project: Apache Hudi (incubating) > Issue Type: Sub-task > Components: asf-migration >Reporter: Vinoth Chandar >Assignee: BALAJI VARADARAJAN >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HUDI-222) Rename main class path to org.apache.hudi.timeline.service.TimelineService in run_server.sh
[
https://issues.apache.org/jira/browse/HUDI-222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16920259#comment-16920259
]
leesf commented on HUDI-222:
Fixed via master: 00cfe72c5d56dbaa2e170a087cf04b1e56a87bc3
> Rename main class path to org.apache.hudi.timeline.service.TimelineService in
> run_server.sh
> ---
>
> Key: HUDI-222
> URL: https://issues.apache.org/jira/browse/HUDI-222
> Project: Apache Hudi (incubating)
> Issue Type: Bug
>Reporter: leesf
>Assignee: leesf
>Priority: Major
> Labels: pull-request-available
> Fix For: 0.6.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> The current main class path in run_server.sh is
> {code:java}
> com.uber.hoodie.timeline.service.TimelineService
> {code}
> , however, it should changed be
> {code:java}
> org.apache.hudi.timeline.service.TimelineService{code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HUDI-229) Fix mvn notice:generate issue in windows
[
https://issues.apache.org/jira/browse/HUDI-229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16920257#comment-16920257
]
leesf commented on HUDI-229:
Fixed via master: 40dd4dd637a44f532af9143bf588dc36a3ced729
> Fix mvn notice:generate issue in windows
>
>
> Key: HUDI-229
> URL: https://issues.apache.org/jira/browse/HUDI-229
> Project: Apache Hudi (incubating)
> Issue Type: Bug
> Components: newbie
>Reporter: leesf
>Assignee: leesf
>Priority: Major
> Labels: pull-request-available
> Fix For: 0.5.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> When running mvn notice:generate from top-level module in windows, the
> following exception occurs.
> {code:java}
> Failed to execute goal org.jasig.maven:maven-notice-plugin:1.1.0:generate
> (default-cli) on project hudi: Resource not found in file system, classpath
> or URL: file://F:/community
> /incubator-hudi/release/config/license-mappings.xml{code}
> Note:F:/community/incubator-hudi is the base path of the project.
> To fix the issue, we need to remove file:// in configuration.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HUDI-225) Create Hudi Timeline Server Fat Jar
[ https://issues.apache.org/jira/browse/HUDI-225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16920258#comment-16920258 ] leesf commented on HUDI-225: Fixed via master: 5c2da6051effb38795011fc1d296ef9bd38f1208 > Create Hudi Timeline Server Fat Jar > > > Key: HUDI-225 > URL: https://issues.apache.org/jira/browse/HUDI-225 > Project: Apache Hudi (incubating) > Issue Type: Task > Components: Deployment, newbie >Reporter: BALAJI VARADARAJAN >Assignee: leesf >Priority: Minor > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > We need to add maven module under packaging named hudi-timeline-server-bundle > to bundle timeline service > In the pom, add the following shading configurations > > org.apache.maven.plugins > maven-shade-plugin > 2.4 > > true > > > *:* > > META-INF/*.SF > META-INF/*.DSA > META-INF/*.RSA > > > > > > > package > > shade > > > > implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer" > /> > implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer"> > org.apache.hudi.timeline.service.TimelineService > > > > > > > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HUDI-227) DeltaStreamer Improvements : Commit empty input batch with progressing checkpoints and allow users to override configs through properties
[ https://issues.apache.org/jira/browse/HUDI-227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16920256#comment-16920256 ] leesf commented on HUDI-227: Fixed via master: 376b59ae5ff46a77941c6e6608a82b8ceb6523ef > DeltaStreamer Improvements : Commit empty input batch with progressing > checkpoints and allow users to override configs through properties > -- > > Key: HUDI-227 > URL: https://issues.apache.org/jira/browse/HUDI-227 > Project: Apache Hudi (incubating) > Issue Type: New Feature > Components: deltastreamer >Reporter: BALAJI VARADARAJAN >Priority: Minor > Labels: pull-request-available > Fix For: 0.5.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Original PR: By [~eisig] > [https://github.com/apache/incubator-hudi/pull/805/] > [https://github.com/apache/incubator-hudi/pull/806] > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HUDI-230) Add missing Apache License in some files
[ https://issues.apache.org/jira/browse/HUDI-230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16920255#comment-16920255 ] leesf commented on HUDI-230: Fixed via master: 8b150a3c6bf76e8029fda82a8a4e181b7a026980 > Add missing Apache License in some files > > > Key: HUDI-230 > URL: https://issues.apache.org/jira/browse/HUDI-230 > Project: Apache Hudi (incubating) > Issue Type: Sub-task >Reporter: leesf >Assignee: leesf >Priority: Minor > Labels: pull-request-available > Fix For: 0.5.0 > > Time Spent: 20m > Remaining Estimate: 0h > > In some files, such as hudi-cli.sh, run_sync_tool.sh, run_hoodie_app.sh, > license-mappings.xml, checkstyle.xml, checkstyle-suppressions.xml, > eclipse-java-google-style.xml, intellij-java-google-style.xml, > scalastyle-config.xml, run_travis_tests.sh, .travis.yml, _config.yaml. > We need to add missing apache license. > cc [~vbalaji] [~vinoth] -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HUDI-231) Support Nested ordering field types in deltastreamer
[ https://issues.apache.org/jira/browse/HUDI-231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16920254#comment-16920254 ] leesf commented on HUDI-231: Fixed via master: 555dd55c16b2df504214f2bdfe84a6b106f61904 > Support Nested ordering field types in deltastreamer > > > Key: HUDI-231 > URL: https://issues.apache.org/jira/browse/HUDI-231 > Project: Apache Hudi (incubating) > Issue Type: Improvement > Components: deltastreamer >Reporter: BALAJI VARADARAJAN >Priority: Major > Labels: pull-request-available > Fix For: 0.5.0 > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Closed] (HUDI-231) Support Nested ordering field types in deltastreamer
[ https://issues.apache.org/jira/browse/HUDI-231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leesf closed HUDI-231. -- Fix Version/s: 0.5.0 Resolution: Fixed > Support Nested ordering field types in deltastreamer > > > Key: HUDI-231 > URL: https://issues.apache.org/jira/browse/HUDI-231 > Project: Apache Hudi (incubating) > Issue Type: Improvement > Components: deltastreamer >Reporter: BALAJI VARADARAJAN >Priority: Major > Labels: pull-request-available > Fix For: 0.5.0 > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [incubator-hudi] leesf commented on a change in pull request #870: [HUDI-164] Fixes incorrect averageBytesPerRecord
leesf commented on a change in pull request #870: [HUDI-164] Fixes incorrect averageBytesPerRecord URL: https://github.com/apache/incubator-hudi/pull/870#discussion_r319738963 ## File path: hudi-client/src/main/java/org/apache/hudi/table/HoodieCopyOnWriteTable.java ## @@ -18,17 +18,13 @@ package org.apache.hudi.table; +import static org.apache.hudi.common.util.HoodieWriteClientUtils.averageBytesPerRecord; + import com.google.common.hash.Hashing; import java.io.IOException; import java.io.Serializable; import java.nio.charset.StandardCharsets; -import java.util.ArrayList; -import java.util.Collections; -import java.util.HashMap; -import java.util.Iterator; -import java.util.List; -import java.util.Map; -import java.util.Set; +import java.util.*; Review comment: Could you split this import into seperate imports? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-hudi] leesf commented on issue #868: HUDI-180 : Adding support for hive registration using metastore along with JDBC
leesf commented on issue #868: HUDI-180 : Adding support for hive registration using metastore along with JDBC URL: https://github.com/apache/incubator-hudi/pull/868#issuecomment-526872481 @n3nash Thanks for opening this PR, I have left some minor comments. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-hudi] leesf commented on a change in pull request #868: HUDI-180 : Adding support for hive registration using metastore along with JDBC
leesf commented on a change in pull request #868: HUDI-180 : Adding support for
hive registration using metastore along with JDBC
URL: https://github.com/apache/incubator-hudi/pull/868#discussion_r319738890
##
File path: hudi-hive/src/main/java/org/apache/hudi/hive/HoodieHiveClient.java
##
@@ -458,19 +507,69 @@ public boolean doesTableExist() {
* @param s SQL to execute
*/
public void updateHiveSQL(String s) {
-Statement stmt = null;
+if (syncConfig.useJdbc) {
+ Statement stmt = null;
+ try {
+stmt = connection.createStatement();
+LOG.info("Executing SQL " + s);
+stmt.execute(s);
+ } catch (SQLException e) {
+throw new HoodieHiveSyncException("Failed in executing SQL " + s, e);
+ } finally {
+closeQuietly(null, stmt);
+ }
+} else {
+ updateHiveSQLUsingHiveDriver(s);
+}
+ }
+
+ /**
+ * Execute a update in hive using Hive Driver
+ *
+ * @param sql SQL statement to execute
+ */
+ public CommandProcessorResponse updateHiveSQLUsingHiveDriver(String sql)
throws HoodieHiveSyncException {
+List responses =
updateHiveSQLs(Arrays.asList(sql));
+return responses.get(responses.size() - 1);
+ }
+
+ private List updateHiveSQLs(List sqls)
throws HoodieHiveSyncException {
+SessionState ss = null;
+org.apache.hadoop.hive.ql.Driver hiveDriver = null;
+List responses = new ArrayList<>();
try {
- stmt = connection.createStatement();
- LOG.info("Executing SQL " + s);
- stmt.execute(s);
-} catch (SQLException e) {
- throw new HoodieHiveSyncException("Failed in executing SQL " + s, e);
+ final long startTime = System.currentTimeMillis();
+ ss = SessionState.start(configuration);
+ hiveDriver = new org.apache.hadoop.hive.ql.Driver(configuration);
+ final long endTime = System.currentTimeMillis();
+ LOG.info("Time taken to start SessionState and create Driver: {} ms",
(endTime - startTime));
+ for (String sql : sqls) {
+final long start = System.currentTimeMillis();
+responses.add(hiveDriver.run(sql));
+final long end = System.currentTimeMillis();
+LOG.info("Time taken to execute [{}]: {} ms", sql, (end - start));
+ }
+} catch (Exception e) {
+ throw new HoodieHiveSyncException("Failed in executing SQL", e);
} finally {
- closeQuietly(null, stmt);
+ if (ss != null) {
+try {
+ ss.close();
+} catch (IOException ie) {
+ LOG.error("Error while closing SessionState: {}", ie);
Review comment:
May be LOG.error("Error while closing SessionState.", ie); ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-hudi] leesf commented on a change in pull request #868: HUDI-180 : Adding support for hive registration using metastore along with JDBC
leesf commented on a change in pull request #868: HUDI-180 : Adding support for
hive registration using metastore along with JDBC
URL: https://github.com/apache/incubator-hudi/pull/868#discussion_r319738883
##
File path: hudi-hive/src/main/java/org/apache/hudi/hive/HoodieHiveClient.java
##
@@ -458,19 +507,69 @@ public boolean doesTableExist() {
* @param s SQL to execute
*/
public void updateHiveSQL(String s) {
-Statement stmt = null;
+if (syncConfig.useJdbc) {
+ Statement stmt = null;
+ try {
+stmt = connection.createStatement();
+LOG.info("Executing SQL " + s);
+stmt.execute(s);
+ } catch (SQLException e) {
+throw new HoodieHiveSyncException("Failed in executing SQL " + s, e);
+ } finally {
+closeQuietly(null, stmt);
+ }
+} else {
+ updateHiveSQLUsingHiveDriver(s);
+}
+ }
+
+ /**
+ * Execute a update in hive using Hive Driver
+ *
+ * @param sql SQL statement to execute
+ */
+ public CommandProcessorResponse updateHiveSQLUsingHiveDriver(String sql)
throws HoodieHiveSyncException {
+List responses =
updateHiveSQLs(Arrays.asList(sql));
+return responses.get(responses.size() - 1);
+ }
+
+ private List updateHiveSQLs(List sqls)
throws HoodieHiveSyncException {
+SessionState ss = null;
+org.apache.hadoop.hive.ql.Driver hiveDriver = null;
+List responses = new ArrayList<>();
try {
- stmt = connection.createStatement();
- LOG.info("Executing SQL " + s);
- stmt.execute(s);
-} catch (SQLException e) {
- throw new HoodieHiveSyncException("Failed in executing SQL " + s, e);
+ final long startTime = System.currentTimeMillis();
+ ss = SessionState.start(configuration);
+ hiveDriver = new org.apache.hadoop.hive.ql.Driver(configuration);
+ final long endTime = System.currentTimeMillis();
+ LOG.info("Time taken to start SessionState and create Driver: {} ms",
(endTime - startTime));
+ for (String sql : sqls) {
+final long start = System.currentTimeMillis();
+responses.add(hiveDriver.run(sql));
+final long end = System.currentTimeMillis();
+LOG.info("Time taken to execute [{}]: {} ms", sql, (end - start));
+ }
+} catch (Exception e) {
+ throw new HoodieHiveSyncException("Failed in executing SQL", e);
} finally {
- closeQuietly(null, stmt);
+ if (ss != null) {
+try {
+ ss.close();
+} catch (IOException ie) {
+ LOG.error("Error while closing SessionState: {}", ie);
+}
+ }
+ if (hiveDriver != null) {
+try {
+ hiveDriver.close();
+} catch (Exception e) {
+ LOG.error("Error while closing hiveDriver: {}", e);
Review comment:
May be LOG.error("Error while closing hiveDriver.", e); ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-hudi] leesf commented on a change in pull request #868: HUDI-180 : Adding support for hive registration using metastore along with JDBC
leesf commented on a change in pull request #868: HUDI-180 : Adding support for
hive registration using metastore along with JDBC
URL: https://github.com/apache/incubator-hudi/pull/868#discussion_r319738852
##
File path: hudi-hive/src/main/java/org/apache/hudi/hive/HoodieHiveClient.java
##
@@ -116,6 +126,19 @@ public HoodieHiveClient(HiveSyncConfig cfg, HiveConf
configuration, FileSystem f
activeTimeline =
metaClient.getActiveTimeline().getCommitsTimeline().filterCompletedInstants();
}
+ private static void closeQuietly(ResultSet resultSet, Statement stmt) {
+try {
+ if (stmt != null) {
+stmt.close();
+ }
+ if (resultSet != null) {
+resultSet.close();
+ }
+} catch (SQLException e) {
Review comment:
If stmt.close() throws SQLException. the resultSet would not be closed.
Could it be the following?
```
try {
if (stmt != null) {
stmt.close();
}
} catch (SQLException e) {
// log
}
try {
if (resultSet != null) {
resultSet.close();
}
} catch (SQLException e) {
// log
}
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Closed] (HUDI-230) Add missing Apache License in some files
[ https://issues.apache.org/jira/browse/HUDI-230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leesf closed HUDI-230. -- Resolution: Fixed > Add missing Apache License in some files > > > Key: HUDI-230 > URL: https://issues.apache.org/jira/browse/HUDI-230 > Project: Apache Hudi (incubating) > Issue Type: Sub-task >Reporter: leesf >Assignee: leesf >Priority: Minor > Labels: pull-request-available > Fix For: 0.5.0 > > Time Spent: 20m > Remaining Estimate: 0h > > In some files, such as hudi-cli.sh, run_sync_tool.sh, run_hoodie_app.sh, > license-mappings.xml, checkstyle.xml, checkstyle-suppressions.xml, > eclipse-java-google-style.xml, intellij-java-google-style.xml, > scalastyle-config.xml, run_travis_tests.sh, .travis.yml, _config.yaml. > We need to add missing apache license. > cc [~vbalaji] [~vinoth] -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HUDI-233) Redo log statements using {} variable substitution
[
https://issues.apache.org/jira/browse/HUDI-233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16920228#comment-16920228
]
leesf commented on HUDI-233:
[~vinoth] Fine, after you have completed the dependency removal, we would
continue discuss it further.
> Redo log statements using {} variable substitution
> --
>
> Key: HUDI-233
> URL: https://issues.apache.org/jira/browse/HUDI-233
> Project: Apache Hudi (incubating)
> Issue Type: Bug
> Components: newbie, Performance
>Affects Versions: 0.5.0
>Reporter: Vinoth Chandar
>Assignee: leesf
>Priority: Major
>
> Currently we are not employing variable substitution aggresively in the
> project. ala
> {code:java}
> LogManager.getLogger(SomeName.class.getName()).info("Message: {}, Detail:
> {}", message, detail);
> {code}
> This can improve performance since the string concatenation is deferrable to
> when the logging is actually in effect.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HUDI-233) Redo log statements using {} variable substitution
[
https://issues.apache.org/jira/browse/HUDI-233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16920180#comment-16920180
]
Vinoth Chandar commented on HUDI-233:
-
[~xleesf] I am working on thinning the amount of dependencies we have in Hudi
to reduce probability of clashing with external systems/runtime environments it
integrates with. One of them I just removed is slf4j.. While I am not against
the specific framework (I obviously pretty good), I feel we need to be lean
towards having our helper wrappers for this sort of stuff vs bringing in new
dependencies just for this..
Can we revisit this in a week or two when I have better idea about how this
will affect shading/bundling? thoughts?
> Redo log statements using {} variable substitution
> --
>
> Key: HUDI-233
> URL: https://issues.apache.org/jira/browse/HUDI-233
> Project: Apache Hudi (incubating)
> Issue Type: Bug
> Components: newbie, Performance
>Affects Versions: 0.5.0
>Reporter: Vinoth Chandar
>Assignee: leesf
>Priority: Major
>
> Currently we are not employing variable substitution aggresively in the
> project. ala
> {code:java}
> LogManager.getLogger(SomeName.class.getName()).info("Message: {}, Detail:
> {}", message, detail);
> {code}
> This can improve performance since the string concatenation is deferrable to
> when the logging is actually in effect.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HUDI-143) GCS: Jackson Databind Issue seen in query side - Presto/Hive
[ https://issues.apache.org/jira/browse/HUDI-143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16920173#comment-16920173 ] BALAJI VARADARAJAN commented on HUDI-143: - [~bhasudha] If you list presto-bundle and other Hudi jars, you might see files like the ones below: META-INF/services/javax.ws.rs.ext.MessageBodyReader META-INF/services/javax.ws.rs.ext.MessageBodyWriter These needs to be omitted from jars. > GCS: Jackson Databind Issue seen in query side - Presto/Hive > > > Key: HUDI-143 > URL: https://issues.apache.org/jira/browse/HUDI-143 > Project: Apache Hudi (incubating) > Issue Type: Bug > Components: hackathon, Hive Integration, Presto Integration >Reporter: BALAJI VARADARAJAN >Assignee: Bhavani Sudha Saktheeswaran >Priority: Major > Labels: gcs-parity > > “”” > com.fasterxml.jackson.databind.ObjectMapper.setDefaultPropertyInclusion(Lcom/fasterxml/jackson/annotation/JsonInclude$Value;)Lcom/fasterxml/jackson/databind/ObjectMapper; > java.lang.NoSuchMethodError: > com.fasterxml.jackson.databind.ObjectMapper.setDefaultPropertyInclusion(Lcom/fasterxml/jackson/annotation/JsonInclude$Value;)Lcom/fasterxml/jackson/databind/ObjectMapper; > “”” > _Status_ : Fixed by adding exclusion filter to exclude > META-INF/services/javax.* -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HUDI-143) GCS: Jackson Databind Issue seen in query side - Presto/Hive
[ https://issues.apache.org/jira/browse/HUDI-143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16920113#comment-16920113 ] Bhavani Sudha Saktheeswaran commented on HUDI-143: -- [~vbalaji] seems like this issue is related to - [https://github.com/apache/incubator-hudi/pull/818] > GCS: Jackson Databind Issue seen in query side - Presto/Hive > > > Key: HUDI-143 > URL: https://issues.apache.org/jira/browse/HUDI-143 > Project: Apache Hudi (incubating) > Issue Type: Bug > Components: hackathon, Hive Integration, Presto Integration >Reporter: BALAJI VARADARAJAN >Assignee: Bhavani Sudha Saktheeswaran >Priority: Major > Labels: gcs-parity > > “”” > com.fasterxml.jackson.databind.ObjectMapper.setDefaultPropertyInclusion(Lcom/fasterxml/jackson/annotation/JsonInclude$Value;)Lcom/fasterxml/jackson/databind/ObjectMapper; > java.lang.NoSuchMethodError: > com.fasterxml.jackson.databind.ObjectMapper.setDefaultPropertyInclusion(Lcom/fasterxml/jackson/annotation/JsonInclude$Value;)Lcom/fasterxml/jackson/databind/ObjectMapper; > “”” > _Status_ : Fixed by adding exclusion filter to exclude > META-INF/services/javax.* -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [incubator-hudi] vinothchandar edited a comment on issue #869: Hudi Spark error when spark bundle jar is added to spark's classpath
vinothchandar edited a comment on issue #869: Hudi Spark error when spark bundle jar is added to spark's classpath URL: https://github.com/apache/incubator-hudi/issues/869#issuecomment-526823124 @umehrot2 noted. I am still working on the bundles.. Once I get to spark-bundle, I will look into this as well. Little suprised how it repros on all versions specially in EMR. Does it happen on Apache Spark standalone? Currently recommendation is to use --jars anyway. Anyways, for now, can you try excluding `org.scala-lang:scala-library` and see if that helps. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-hudi] vinothchandar commented on issue #869: Hudi Spark error when spark bundle jar is added to spark's classpath
vinothchandar commented on issue #869: Hudi Spark error when spark bundle jar is added to spark's classpath URL: https://github.com/apache/incubator-hudi/issues/869#issuecomment-526823124 @umehrot2 noted. I am still working on the bundles.. Once I get to spark-bundle, I will look into this as well. Little suprised how it repros on all versions specially in EMR. Does it happen on Apache Spark standalone? Anyways, for now, can you try excluding `org.scala-lang:scala-library` and see if that helps. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-hudi] smdahmed commented on issue #859: Hudi upsert after a delete in partition will cause valid records inserted to disappear.
smdahmed commented on issue #859: Hudi upsert after a delete in partition will cause valid records inserted to disappear. URL: https://github.com/apache/incubator-hudi/issues/859#issuecomment-526821478 Vinoth, Thanks for all the work and help. I have a common routine through which I handle inserts and upserts. Upserts after insert work fine. I will recheck and revert back to you This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HUDI-228) Add Jira Conventions to contributing/community pages of HUDI
[ https://issues.apache.org/jira/browse/HUDI-228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16920060#comment-16920060 ] vinoyang commented on HUDI-228: --- +1 for [~xleesf] 's suggestion > Add Jira Conventions to contributing/community pages of HUDI > > > Key: HUDI-228 > URL: https://issues.apache.org/jira/browse/HUDI-228 > Project: Apache Hudi (incubating) > Issue Type: Improvement > Components: asf-migration, newbie >Reporter: BALAJI VARADARAJAN >Priority: Minor > > When filing or updating a JIRA for Apache Hudi, kindly make sure > > (a) the issue type and versions (when resolving the ticket) are set correctly. > (b) Also, the summary needs to be descriptive enough to catch the essence of > the problem/features. > (c) Capture the version of Hoodie/Spark/Hive/Hadoop/Cloud environments in the > ticket > > When opening a github PR corresponding to the JIRA, reference Jira-id in the > commit message > -- This message was sent by Atlassian Jira (v8.3.2#803003)
