[jira] [Commented] (ARROW-11135) Using Maven Central artifacts as dependencies produce runtime errors
[ https://issues.apache.org/jira/browse/ARROW-11135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17327023#comment-17327023 ] Julian Hyde commented on ARROW-11135: - [~projjal], Thanks for stepping up to help. I really appreciate it. In case I didn't make it clear: We have used the ursa-labs nightly build, and in fact we checked one of the jars into a git branch during development. But in order to merge into Calcite's master branch (from which releases will be made), we can only depend on released artifacts in Maven central. Our criteria are: * master branch MUST NOT contain binary files (e.g. .jar); * a source release MUST NOT contain binary files; * master branch and release builds MUST NOT depend on snapshot or nightly artifacts, only released artifacts; * code SHOULD run on at least Linux and macOS (windows would be nice, but it's not feasible); * all platforms SHOULD use the same jar file; * artifacts SHOULD be in maven central (another repo might be possible). These are fairly typical for an ASF project, I think. They are designed to reduce friction for people who want to use or develop this feature. > Using Maven Central artifacts as dependencies produce runtime errors > > > Key: ARROW-11135 > URL: https://issues.apache.org/jira/browse/ARROW-11135 > Project: Apache Arrow > Issue Type: Bug > Components: Java >Affects Versions: 2.0.0, 3.0.0 >Reporter: Michael Mior >Assignee: Projjal Chanda >Priority: Major > > I'm working on connecting Arrow/Gandiva with Apache Calcite. Overall the > integration is working well, but I'm having issues . As [suggested on the > mailing > list|https://lists.apache.org/thread.html/r93a4fedb499c746917ab8d62cf5a8db8c93a7f24bc9fac81f90bedaa%40%3Cuser.arrow.apache.org%3E], > using Dremio's public artifacts solves the problem. Between two Apache > projects however, there would be strong preference to use Apache artifacts as > a dependency. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-11135) Using Maven Central artifacts as dependencies produce runtime errors
[ https://issues.apache.org/jira/browse/ARROW-11135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17326307#comment-17326307 ] Projjal Chanda commented on ARROW-11135: [~julianhyde] The gandiva lib in ursa-labs nightly mac and linux jars are built with static protobuf dependency (see [here|https://github.com/apache/arrow/blob/7c4a07eeddf885189f7a6ad01ba9f98945bfa022/dev/tasks/gandiva-jars/build-cpp-osx.sh#L39] and [here)|https://github.com/apache/arrow/blob/7c4a07eeddf885189f7a6ad01ba9f98945bfa022/dev/tasks/gandiva-jars/build-cpp-linux.sh#L45]. It seems the [release gandiva jar|https://github.com/apache/arrow/blob/7c4a07eeddf885189f7a6ad01ba9f98945bfa022/dev/release/post-11-java.sh#L45] is not build similarly, and the supported platform for the released gandiva jar depends on the release manager's machine. Agree with [~kou] about the changes needed on gandiva release process. We need to - 1. Build arrow-gandiva and native libraries in all supported platforms and collect all the libraries in a single jar (Currently we create separate jars for macos and linux) 2. Release process should push the built arrow-gandiva jar instead of building it on RM's machine. I will try to get this done. > Using Maven Central artifacts as dependencies produce runtime errors > > > Key: ARROW-11135 > URL: https://issues.apache.org/jira/browse/ARROW-11135 > Project: Apache Arrow > Issue Type: Bug > Components: Java >Affects Versions: 2.0.0, 3.0.0 >Reporter: Michael Mior >Priority: Major > > I'm working on connecting Arrow/Gandiva with Apache Calcite. Overall the > integration is working well, but I'm having issues . As [suggested on the > mailing > list|https://lists.apache.org/thread.html/r93a4fedb499c746917ab8d62cf5a8db8c93a7f24bc9fac81f90bedaa%40%3Cuser.arrow.apache.org%3E], > using Dremio's public artifacts solves the problem. Between two Apache > projects however, there would be strong preference to use Apache artifacts as > a dependency. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-11135) Using Maven Central artifacts as dependencies produce runtime errors
[ https://issues.apache.org/jira/browse/ARROW-11135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320752#comment-17320752 ] Kouhei Sutou commented on ARROW-11135: -- Gandiva developers such as [~praveenbingo] may be interested in this. > Using Maven Central artifacts as dependencies produce runtime errors > > > Key: ARROW-11135 > URL: https://issues.apache.org/jira/browse/ARROW-11135 > Project: Apache Arrow > Issue Type: Bug > Components: Java >Affects Versions: 2.0.0, 3.0.0 >Reporter: Michael Mior >Priority: Major > > I'm working on connecting Arrow/Gandiva with Apache Calcite. Overall the > integration is working well, but I'm having issues . As [suggested on the > mailing > list|https://lists.apache.org/thread.html/r93a4fedb499c746917ab8d62cf5a8db8c93a7f24bc9fac81f90bedaa%40%3Cuser.arrow.apache.org%3E], > using Dremio's public artifacts solves the problem. Between two Apache > projects however, there would be strong preference to use Apache artifacts as > a dependency. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-11135) Using Maven Central artifacts as dependencies produce runtime errors
[ https://issues.apache.org/jira/browse/ARROW-11135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320525#comment-17320525 ] Julian Hyde commented on ARROW-11135: - This issue is going to be a blocker for getting CALCITE-2040 merged into master as a "beta" feature. For our purposes, the artifact(s) have to be in Maven central, and have to work on one OS (either macOS or Linux) preferably both. Currently the Maven central artifacts don't work for either. [~kou] and [~wesm], Any suggestions who we could nag to get this fixed for 4.0.0? I know contributors are volunteers, but it's a shame to have Calcite contributions languishing. If we can't find anyone, that's fine, but I want to be able to say we tried. > Using Maven Central artifacts as dependencies produce runtime errors > > > Key: ARROW-11135 > URL: https://issues.apache.org/jira/browse/ARROW-11135 > Project: Apache Arrow > Issue Type: Bug > Components: Java >Affects Versions: 2.0.0, 3.0.0 >Reporter: Michael Mior >Priority: Major > > I'm working on connecting Arrow/Gandiva with Apache Calcite. Overall the > integration is working well, but I'm having issues . As [suggested on the > mailing > list|https://lists.apache.org/thread.html/r93a4fedb499c746917ab8d62cf5a8db8c93a7f24bc9fac81f90bedaa%40%3Cuser.arrow.apache.org%3E], > using Dremio's public artifacts solves the problem. Between two Apache > projects however, there would be strong preference to use Apache artifacts as > a dependency. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-11135) Using Maven Central artifacts as dependencies produce runtime errors
[
https://issues.apache.org/jira/browse/ARROW-11135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17319551#comment-17319551
]
Julian Hyde commented on ARROW-11135:
-
[~kou], Agreed. Static linking would be better. In fact static linking seems to
work on the Linux jar (I did not need to install protobuf on my Linux machine).
The basic problem statement should be 'make the
{{/usr/local/opt/protobuf/lib/libprotobuf.24.dylib}} error go away'.
> Using Maven Central artifacts as dependencies produce runtime errors
>
>
> Key: ARROW-11135
> URL: https://issues.apache.org/jira/browse/ARROW-11135
> Project: Apache Arrow
> Issue Type: Bug
> Components: Java
>Affects Versions: 2.0.0, 3.0.0
>Reporter: Michael Mior
>Priority: Major
>
> I'm working on connecting Arrow/Gandiva with Apache Calcite. Overall the
> integration is working well, but I'm having issues . As [suggested on the
> mailing
> list|https://lists.apache.org/thread.html/r93a4fedb499c746917ab8d62cf5a8db8c93a7f24bc9fac81f90bedaa%40%3Cuser.arrow.apache.org%3E],
> using Dremio's public artifacts solves the problem. Between two Apache
> projects however, there would be strong preference to use Apache artifacts as
> a dependency.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (ARROW-11135) Using Maven Central artifacts as dependencies produce runtime errors
[
https://issues.apache.org/jira/browse/ARROW-11135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17318952#comment-17318952
]
Kouhei Sutou commented on ARROW-11135:
--
I don't agree with the former.
I agree with the latter.
{{libgandiva_jni}} should not depend on other libraries (should be linked with
other libraries statically). Could you try .jar at
https://github.com/ursacomputing/crossbow/releases/tag/nightly-2021-04-09-0-github-gandiva-jar-osx
?
We need to improve our release process to resolve them. The current our release
process generates Java packages on release manager's environment:
https://github.com/apache/arrow/blob/master/dev/release/01-perform.sh
The release manager for 3.0.0 used macOS. So arrow-gandiva 3.0.0 works only on
macOS.
We should build arrow-gandiva and native libraries (for macOS, Linux and
Windows) for it on CI (we can use macOS, Linux and Windows on CI) and collect
native libraries for all supported platforms into one arrow-gandiva.jar. Our
release process should just pushes the built arrow-gandiva.jar instead of
building arrow-gandiva.ja on release manager's machine.
We'll release 4.0.0 soon. This improvement will not be included in 4.0.0 if no
volunteers that work on this soon.
> Using Maven Central artifacts as dependencies produce runtime errors
>
>
> Key: ARROW-11135
> URL: https://issues.apache.org/jira/browse/ARROW-11135
> Project: Apache Arrow
> Issue Type: Bug
> Components: Java
>Affects Versions: 2.0.0, 3.0.0
>Reporter: Michael Mior
>Priority: Major
>
> I'm working on connecting Arrow/Gandiva with Apache Calcite. Overall the
> integration is working well, but I'm having issues . As [suggested on the
> mailing
> list|https://lists.apache.org/thread.html/r93a4fedb499c746917ab8d62cf5a8db8c93a7f24bc9fac81f90bedaa%40%3Cuser.arrow.apache.org%3E],
> using Dremio's public artifacts solves the problem. Between two Apache
> projects however, there would be strong preference to use Apache artifacts as
> a dependency.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (ARROW-11135) Using Maven Central artifacts as dependencies produce runtime errors
[
https://issues.apache.org/jira/browse/ARROW-11135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17318933#comment-17318933
]
Julian Hyde commented on ARROW-11135:
-
I think this issue boils down to two problems:
* The install documentation should state that you need to install protobuf on
macOS. That is the cause of the
{{/usr/local/opt/protobuf/lib/libprotobuf.24.dylib}} error.
* The artifacts in Maven Central only support macOS. They should support Linux
and macOS.
Do you agree?
> Using Maven Central artifacts as dependencies produce runtime errors
>
>
> Key: ARROW-11135
> URL: https://issues.apache.org/jira/browse/ARROW-11135
> Project: Apache Arrow
> Issue Type: Bug
> Components: Java
>Affects Versions: 2.0.0
>Reporter: Michael Mior
>Priority: Major
>
> I'm working on connecting Arrow/Gandiva with Apache Calcite. Overall the
> integration is working well, but I'm having issues . As [suggested on the
> mailing
> list|https://lists.apache.org/thread.html/r93a4fedb499c746917ab8d62cf5a8db8c93a7f24bc9fac81f90bedaa%40%3Cuser.arrow.apache.org%3E],
> using Dremio's public artifacts solves the problem. Between two Apache
> projects however, there would be strong preference to use Apache artifacts as
> a dependency.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (ARROW-11135) Using Maven Central artifacts as dependencies produce runtime errors
[
https://issues.apache.org/jira/browse/ARROW-11135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17318931#comment-17318931
]
Julian Hyde commented on ARROW-11135:
-
There are no missing packages. But the install instructions should probably say:
* The Gandiva library only works on macOS, and requires that you manually
install protobuf 2.5.
By the way, I compared which files are in the 3.0.0 release jar (which works on
macOS) and the 3.0.0-SNAPSHOT jar (which works on Linux).
{noformat}
$ diff -u <(tar tvf ./arrow-gandiva-3.0.0-SNAPSHOT.jar |awk '{print $NF}'|sort)
<(tar tvf ./arrow-gandiva-3.0.0.jar |awk '{print $NF}'|sort)
--- /dev/fd/63 2021-04-11 12:25:09.0 -0700
+++ /dev/fd/62 2021-04-11 12:25:09.0 -0700
@@ -11,7 +11,10 @@
META-INF/maven/org.apache.arrow.gandiva/arrow-gandiva/pom.xml
Types.proto
git.properties
-libgandiva_jni.so
+libgandiva_jni.300.0.0.dylib
+libgandiva_jni.300.dylib
+libgandiva_jni.a
+libgandiva_jni.dylib
org/
org/apache/
org/apache/arrow/
@@ -188,3 +191,8 @@
org/apache/arrow/gandiva/ipc/GandivaTypes$TreeNode.class
org/apache/arrow/gandiva/ipc/GandivaTypes$TreeNodeOrBuilder.class
org/apache/arrow/gandiva/ipc/GandivaTypes.class
+release/
+release/libgandiva_jni.300.0.0.dylib
+release/libgandiva_jni.300.dylib
+release/libgandiva_jni.a
+release/libgandiva_jni.dylib
{noformat}
It would be awesome if, in the next release, the jar contained ALL of those
files, and then I suppose it would work on both Linux and macOS.
> Using Maven Central artifacts as dependencies produce runtime errors
>
>
> Key: ARROW-11135
> URL: https://issues.apache.org/jira/browse/ARROW-11135
> Project: Apache Arrow
> Issue Type: Bug
> Components: Java
>Affects Versions: 2.0.0
>Reporter: Michael Mior
>Priority: Major
>
> I'm working on connecting Arrow/Gandiva with Apache Calcite. Overall the
> integration is working well, but I'm having issues . As [suggested on the
> mailing
> list|https://lists.apache.org/thread.html/r93a4fedb499c746917ab8d62cf5a8db8c93a7f24bc9fac81f90bedaa%40%3Cuser.arrow.apache.org%3E],
> using Dremio's public artifacts solves the problem. Between two Apache
> projects however, there would be strong preference to use Apache artifacts as
> a dependency.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (ARROW-11135) Using Maven Central artifacts as dependencies produce runtime errors
[ https://issues.apache.org/jira/browse/ARROW-11135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17318611#comment-17318611 ] Kouhei Sutou commented on ARROW-11135: -- I'm sorry but I'm not familiar with Java packages. Could you list missing packages in https://arrow.apache.org/install/#java-packages ? Only Gandiva jar for macOS? > Using Maven Central artifacts as dependencies produce runtime errors > > > Key: ARROW-11135 > URL: https://issues.apache.org/jira/browse/ARROW-11135 > Project: Apache Arrow > Issue Type: Bug > Components: Java >Affects Versions: 2.0.0 >Reporter: Michael Mior >Priority: Major > > I'm working on connecting Arrow/Gandiva with Apache Calcite. Overall the > integration is working well, but I'm having issues . As [suggested on the > mailing > list|https://lists.apache.org/thread.html/r93a4fedb499c746917ab8d62cf5a8db8c93a7f24bc9fac81f90bedaa%40%3Cuser.arrow.apache.org%3E], > using Dremio's public artifacts solves the problem. Between two Apache > projects however, there would be strong preference to use Apache artifacts as > a dependency. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-11135) Using Maven Central artifacts as dependencies produce runtime errors
[ https://issues.apache.org/jira/browse/ARROW-11135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17318605#comment-17318605 ] Julian Hyde commented on ARROW-11135: - If I need to install protobuf 2.5. and the Gandiva jar only works on macOS, it seems that the [one-line installation instruction|https://arrow.apache.org/install/#java-packages] is incomplete. > Using Maven Central artifacts as dependencies produce runtime errors > > > Key: ARROW-11135 > URL: https://issues.apache.org/jira/browse/ARROW-11135 > Project: Apache Arrow > Issue Type: Bug > Components: Java >Affects Versions: 2.0.0 >Reporter: Michael Mior >Priority: Major > > I'm working on connecting Arrow/Gandiva with Apache Calcite. Overall the > integration is working well, but I'm having issues . As [suggested on the > mailing > list|https://lists.apache.org/thread.html/r93a4fedb499c746917ab8d62cf5a8db8c93a7f24bc9fac81f90bedaa%40%3Cuser.arrow.apache.org%3E], > using Dremio's public artifacts solves the problem. Between two Apache > projects however, there would be strong preference to use Apache artifacts as > a dependency. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-11135) Using Maven Central artifacts as dependencies produce runtime errors
[ https://issues.apache.org/jira/browse/ARROW-11135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17318599#comment-17318599 ] Julian Hyde commented on ARROW-11135: - An update. I am working on the same branch as [~mmior] (see CALCITE-2040). I have been able to successfully run on Linux using Michael's [commit e9954|https://github.com/julianhyde/calcite/commit/e99547f371f100014e981e5a500dc04887806708]. But I noticed that Apache Arrow Gandiva 3.0.0 was released on 2021/01/25 and the artifacts are on [Maven central|https://search.maven.org/artifact/org.apache.arrow.gandiva/arrow-gandiva/3.0.0/jar], so I have been trying to switch to that version and run on macOS. Some background: In order to merge CALCITE-2040 to Calcite's master branch as a beta feature, we must use artifacts from Maven central, rather than from Dremio's maven repo, or Usra labs' snapshots, or a .jar committed to Calcite's git repo; I don't mind as much whether the artifacts are for Linux or macOS. I am running into errors which I think mean I need to install protobuf2.5 on my Mac. I'll send another update when I make some progress. > Using Maven Central artifacts as dependencies produce runtime errors > > > Key: ARROW-11135 > URL: https://issues.apache.org/jira/browse/ARROW-11135 > Project: Apache Arrow > Issue Type: Bug > Components: Java >Affects Versions: 2.0.0 >Reporter: Michael Mior >Priority: Major > > I'm working on connecting Arrow/Gandiva with Apache Calcite. Overall the > integration is working well, but I'm having issues . As [suggested on the > mailing > list|https://lists.apache.org/thread.html/r93a4fedb499c746917ab8d62cf5a8db8c93a7f24bc9fac81f90bedaa%40%3Cuser.arrow.apache.org%3E], > using Dremio's public artifacts solves the problem. Between two Apache > projects however, there would be strong preference to use Apache artifacts as > a dependency. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-11135) Using Maven Central artifacts as dependencies produce runtime errors
[
https://issues.apache.org/jira/browse/ARROW-11135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17267509#comment-17267509
]
Michael Mior commented on ARROW-11135:
--
My mistake. I missed that the JAR was macOS only. The tests pass and all is
fine on openJDK 11. However, on openJDK 8 and 15, I get the following error
after the tests complete which still causes the build to fail. However, I
assume this may be because I did not correctly close some Gandiva resources. I
have edited the code to try to properly free all resources, and I'll watch what
happens with this build.
{{pure virtual method called}}
{{ terminate called without an active exception}}
> Using Maven Central artifacts as dependencies produce runtime errors
>
>
> Key: ARROW-11135
> URL: https://issues.apache.org/jira/browse/ARROW-11135
> Project: Apache Arrow
> Issue Type: Bug
> Components: Java
>Affects Versions: 2.0.0
>Reporter: Michael Mior
>Priority: Major
>
> I'm working on connecting Arrow/Gandiva with Apache Calcite. Overall the
> integration is working well, but I'm having issues . As [suggested on the
> mailing
> list|https://lists.apache.org/thread.html/r93a4fedb499c746917ab8d62cf5a8db8c93a7f24bc9fac81f90bedaa%40%3Cuser.arrow.apache.org%3E],
> using Dremio's public artifacts solves the problem. Between two Apache
> projects however, there would be strong preference to use Apache artifacts as
> a dependency.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (ARROW-11135) Using Maven Central artifacts as dependencies produce runtime errors
[
https://issues.apache.org/jira/browse/ARROW-11135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1722#comment-1722
]
Kouhei Sutou commented on ARROW-11135:
--
The {{.jar}} is for macOS.
It seems that the Travis CI build was ran on Linux. Could you try the {{.jar}}
at
https://github.com/ursacomputing/crossbow/releases/tag/nightly-2021-01-16-0-github-gandiva-jar-ubuntu
for Linux?
> Using Maven Central artifacts as dependencies produce runtime errors
>
>
> Key: ARROW-11135
> URL: https://issues.apache.org/jira/browse/ARROW-11135
> Project: Apache Arrow
> Issue Type: Bug
> Components: Java
>Affects Versions: 2.0.0
>Reporter: Michael Mior
>Priority: Major
>
> I'm working on connecting Arrow/Gandiva with Apache Calcite. Overall the
> integration is working well, but I'm having issues . As [suggested on the
> mailing
> list|https://lists.apache.org/thread.html/r93a4fedb499c746917ab8d62cf5a8db8c93a7f24bc9fac81f90bedaa%40%3Cuser.arrow.apache.org%3E],
> using Dremio's public artifacts solves the problem. Between two Apache
> projects however, there would be strong preference to use Apache artifacts as
> a dependency.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (ARROW-11135) Using Maven Central artifacts as dependencies produce runtime errors
[ https://issues.apache.org/jira/browse/ARROW-11135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17266408#comment-17266408 ] Michael Mior commented on ARROW-11135: -- [~kou] It's the JAR from the link you posted above. I just included a copy in a subfolder for now for testing. > Using Maven Central artifacts as dependencies produce runtime errors > > > Key: ARROW-11135 > URL: https://issues.apache.org/jira/browse/ARROW-11135 > Project: Apache Arrow > Issue Type: Bug > Components: Java >Affects Versions: 2.0.0 >Reporter: Michael Mior >Priority: Major > > I'm working on connecting Arrow/Gandiva with Apache Calcite. Overall the > integration is working well, but I'm having issues . As [suggested on the > mailing > list|https://lists.apache.org/thread.html/r93a4fedb499c746917ab8d62cf5a8db8c93a7f24bc9fac81f90bedaa%40%3Cuser.arrow.apache.org%3E], > using Dremio's public artifacts solves the problem. Between two Apache > projects however, there would be strong preference to use Apache artifacts as > a dependency. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-11135) Using Maven Central artifacts as dependencies produce runtime errors
[
https://issues.apache.org/jira/browse/ARROW-11135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17266406#comment-17266406
]
Kouhei Sutou commented on ARROW-11135:
--
Do you know the {{.jar}} URL that is used in the build?
> Using Maven Central artifacts as dependencies produce runtime errors
>
>
> Key: ARROW-11135
> URL: https://issues.apache.org/jira/browse/ARROW-11135
> Project: Apache Arrow
> Issue Type: Bug
> Components: Java
>Affects Versions: 2.0.0
>Reporter: Michael Mior
>Priority: Major
>
> I'm working on connecting Arrow/Gandiva with Apache Calcite. Overall the
> integration is working well, but I'm having issues . As [suggested on the
> mailing
> list|https://lists.apache.org/thread.html/r93a4fedb499c746917ab8d62cf5a8db8c93a7f24bc9fac81f90bedaa%40%3Cuser.arrow.apache.org%3E],
> using Dremio's public artifacts solves the problem. Between two Apache
> projects however, there would be strong preference to use Apache artifacts as
> a dependency.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (ARROW-11135) Using Maven Central artifacts as dependencies produce runtime errors
[ https://issues.apache.org/jira/browse/ARROW-11135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17266136#comment-17266136 ] Michael Mior commented on ARROW-11135: -- [~kou] Thanks for clarifying. Unfortunately, the [tests failed|https://travis-ci.com/github/apache/calcite/jobs/469237125] with the nightly JAR. > Using Maven Central artifacts as dependencies produce runtime errors > > > Key: ARROW-11135 > URL: https://issues.apache.org/jira/browse/ARROW-11135 > Project: Apache Arrow > Issue Type: Bug > Components: Java >Affects Versions: 2.0.0 >Reporter: Michael Mior >Priority: Major > > I'm working on connecting Arrow/Gandiva with Apache Calcite. Overall the > integration is working well, but I'm having issues . As [suggested on the > mailing > list|https://lists.apache.org/thread.html/r93a4fedb499c746917ab8d62cf5a8db8c93a7f24bc9fac81f90bedaa%40%3Cuser.arrow.apache.org%3E], > using Dremio's public artifacts solves the problem. Between two Apache > projects however, there would be strong preference to use Apache artifacts as > a dependency. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-11135) Using Maven Central artifacts as dependencies produce runtime errors
[ https://issues.apache.org/jira/browse/ARROW-11135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17260147#comment-17260147 ] Kouhei Sutou commented on ARROW-11135: -- Yes. > Using Maven Central artifacts as dependencies produce runtime errors > > > Key: ARROW-11135 > URL: https://issues.apache.org/jira/browse/ARROW-11135 > Project: Apache Arrow > Issue Type: Bug > Components: Java >Affects Versions: 2.0.0 >Reporter: Michael Mior >Priority: Major > > I'm working on connecting Arrow/Gandiva with Apache Calcite. Overall the > integration is working well, but I'm having issues . As [suggested on the > mailing > list|https://lists.apache.org/thread.html/r93a4fedb499c746917ab8d62cf5a8db8c93a7f24bc9fac81f90bedaa%40%3Cuser.arrow.apache.org%3E], > using Dremio's public artifacts solves the problem. Between two Apache > projects however, there would be strong preference to use Apache artifacts as > a dependency. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-11135) Using Maven Central artifacts as dependencies produce runtime errors
[ https://issues.apache.org/jira/browse/ARROW-11135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17260099#comment-17260099 ] Michael Mior commented on ARROW-11135: -- Without knowing the root of the issue, it's hard to say. To clarify, is the nightly JAR you posted based off what is already on master? That is, should the artifacts generated at the next Arrow release fix the issue? I'll also note that the failure persists on Windows but I don't know if this was ever a goal. > Using Maven Central artifacts as dependencies produce runtime errors > > > Key: ARROW-11135 > URL: https://issues.apache.org/jira/browse/ARROW-11135 > Project: Apache Arrow > Issue Type: Bug > Components: Java >Affects Versions: 2.0.0 >Reporter: Michael Mior >Priority: Major > > I'm working on connecting Arrow/Gandiva with Apache Calcite. Overall the > integration is working well, but I'm having issues . As [suggested on the > mailing > list|https://lists.apache.org/thread.html/r93a4fedb499c746917ab8d62cf5a8db8c93a7f24bc9fac81f90bedaa%40%3Cuser.arrow.apache.org%3E], > using Dremio's public artifacts solves the problem. Between two Apache > projects however, there would be strong preference to use Apache artifacts as > a dependency. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-11135) Using Maven Central artifacts as dependencies produce runtime errors
[ https://issues.apache.org/jira/browse/ARROW-11135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17260080#comment-17260080 ] Kouhei Sutou commented on ARROW-11135: -- Should we close this issue and open another issue for the exception on shutdown problem? Or should we keep using this issue for the problem? > Using Maven Central artifacts as dependencies produce runtime errors > > > Key: ARROW-11135 > URL: https://issues.apache.org/jira/browse/ARROW-11135 > Project: Apache Arrow > Issue Type: Bug > Components: Java >Affects Versions: 2.0.0 >Reporter: Michael Mior >Priority: Major > > I'm working on connecting Arrow/Gandiva with Apache Calcite. Overall the > integration is working well, but I'm having issues . As [suggested on the > mailing > list|https://lists.apache.org/thread.html/r93a4fedb499c746917ab8d62cf5a8db8c93a7f24bc9fac81f90bedaa%40%3Cuser.arrow.apache.org%3E], > using Dremio's public artifacts solves the problem. Between two Apache > projects however, there would be strong preference to use Apache artifacts as > a dependency. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-11135) Using Maven Central artifacts as dependencies produce runtime errors
[ https://issues.apache.org/jira/browse/ARROW-11135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17259890#comment-17259890 ] Michael Mior commented on ARROW-11135: -- [~kou] That JAR seems to work as well on my development machine (using arrow-memory-netty and arrow-vector 2.0.0). I haven't tried this on Travis CI yet, but I'll note that while the tests passed with the Dremio release, I did get a Gandiva exception on shutdown that still caused the build to fail (although this may be an unrelated issue). > Using Maven Central artifacts as dependencies produce runtime errors > > > Key: ARROW-11135 > URL: https://issues.apache.org/jira/browse/ARROW-11135 > Project: Apache Arrow > Issue Type: Bug > Components: Java >Affects Versions: 2.0.0 >Reporter: Michael Mior >Priority: Major > > I'm working on connecting Arrow/Gandiva with Apache Calcite. Overall the > integration is working well, but I'm having issues . As [suggested on the > mailing > list|https://lists.apache.org/thread.html/r93a4fedb499c746917ab8d62cf5a8db8c93a7f24bc9fac81f90bedaa%40%3Cuser.arrow.apache.org%3E], > using Dremio's public artifacts solves the problem. Between two Apache > projects however, there would be strong preference to use Apache artifacts as > a dependency. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-11135) Using Maven Central artifacts as dependencies produce runtime errors
[ https://issues.apache.org/jira/browse/ARROW-11135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17259321#comment-17259321 ] Kouhei Sutou commented on ARROW-11135: -- Could you try .jar at https://github.com/ursa-labs/crossbow/releases/nightly-2021-01-05-0-github-gandiva-jar-osx ? > Using Maven Central artifacts as dependencies produce runtime errors > > > Key: ARROW-11135 > URL: https://issues.apache.org/jira/browse/ARROW-11135 > Project: Apache Arrow > Issue Type: Bug > Components: Java >Affects Versions: 2.0.0 >Reporter: Michael Mior >Priority: Major > > I'm working on connecting Arrow/Gandiva with Apache Calcite. Overall the > integration is working well, but I'm having issues . As [suggested on the > mailing > list|https://lists.apache.org/thread.html/r93a4fedb499c746917ab8d62cf5a8db8c93a7f24bc9fac81f90bedaa%40%3Cuser.arrow.apache.org%3E], > using Dremio's public artifacts solves the problem. Between two Apache > projects however, there would be strong preference to use Apache artifacts as > a dependency. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-11135) Using Maven Central artifacts as dependencies produce runtime errors
[
https://issues.apache.org/jira/browse/ARROW-11135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17259317#comment-17259317
]
Kouhei Sutou commented on ARROW-11135:
--
{quote}
Starting with a fresh clone of the repository, on my macOS machine I
use for development, I get the following which seems to be related to
a protobuf version mismatch:
java.lang.UnsatisfiedLinkError:
/private/var/folders/0h/n0jrz4qn5x37gj25sxx4z2wmgp/T/libgandiva_jni.dylib43feded5-f265-4a6d-b1f7-2aa56e94f392:
dlopen(/private/var/folders/0h/n0jrz4qn5x37gj25sxx4z2wmgp/T/libgandiva_jni.dylib43feded5-f265-4a6d-b1f7-2aa56e94f392,
1): Library not loaded:
/usr/local/opt/protobuf/lib/libprotobuf.24.dylib
Referenced from:
/private/var/folders/0h/n0jrz4qn5x37gj25sxx4z2wmgp/T/libgandiva_jni.dylib43feded5-f265-4a6d-b1f7-2aa56e94f392
{quote}
> Using Maven Central artifacts as dependencies produce runtime errors
>
>
> Key: ARROW-11135
> URL: https://issues.apache.org/jira/browse/ARROW-11135
> Project: Apache Arrow
> Issue Type: Bug
> Components: Java
>Affects Versions: 2.0.0
>Reporter: Michael Mior
>Priority: Major
>
> I'm working on connecting Arrow/Gandiva with Apache Calcite. Overall the
> integration is working well, but I'm having issues . As [suggested on the
> mailing
> list|https://lists.apache.org/thread.html/r93a4fedb499c746917ab8d62cf5a8db8c93a7f24bc9fac81f90bedaa%40%3Cuser.arrow.apache.org%3E],
> using Dremio's public artifacts solves the problem. Between two Apache
> projects however, there would be strong preference to use Apache artifacts as
> a dependency.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
