No, we don't even use (nor have we ever used) Hive in our flows. It's
just there and we didn't want to modify the NiFi download. Should this
not even happen if we're not using it?
On 10/13/20 4:24 PM, Matt Burgess wrote:
Ugh now I remember, that version of Hive uses a version of Snappy that
doesn’t create a unique path under /tmp, do you have multiple
PutHiveStreaming processors in the flow? I don’t think that works
because we can’t load a single native library into multiple classloaders.
On Oct 13, 2020, at 6:15 PM, Russell Bateman <r...@windofkeltia.com>
wrote:
I see
-rwxr-xr-x. 1 nifi nifi 48432 Oct 13 13:48
snappy-1.0.5-libsnappyjava.so
in //tmp/. Therefore, not a permissions issue? Launching this way works:
$ ( export
LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/tmp/snappy-1.0.5-libsnappyjava.so"
&& /opt/nifi/bin/nifi.sh start )
but it's not something we want to do (in case that shared object
disappears from //tmp/).
On 10/13/20 3:42 PM, Matt Burgess wrote:
IIRC this is likely a permissions issue, Xerial Snappy tries to
unzip the native library to the location pointed to by
“java.io.tempdir” which on *nix defaults to /tmp. Does the NiFi user
have write access to that directory? If not you can change the Java
temp dir or set it specifically for Snappy (I don’t have the
property on hand but a quick Google should find it)
Regards,
Matt
Sent from my iPhone
On Oct 13, 2020, at 5:36 PM, Russell Bateman
<r...@windofkeltia.com> wrote:
Should I be seeing this in the log of a vanilla NiFi installation
on CentOS?
ERROR [main] org.apache.nifi.NiFi Failure to launch NiFi due to
java.util.ServiceConfigurationError:
org.apache.nifi.processor.Processor: Provider
org.apache.nifi.processors.hive.PutHiveStreaming could not be
initialized
.
.
.
Caused by: org.xerial.snappy.SnappyError:
[FAILED_TO_LOAD_NATIVE_LIBRARY] null
"Snappy" makes me think Ubuntu. This is CentOS. Did I pull down the
wrong /nifi-1.11.4-bin.zip/?
