[
https://issues.apache.org/jira/browse/NIFI-4944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Burgess updated NIFI-4944:
-------------------------------
Description:
When data coming into PutHiveStreaming is compressed with Snappy, then multiple
instances of PutHiveStreaming in a flow can cause a failure, the log often
shows the following:
{{org.apache.nifi.processors.hive.PutHiveStreaming$$Lambda$510/1467586448@68a5884d
failed to process due to org.xerial.snappy.SnappyError:
[FAILED_TO_LOAD_NATIVE_LIBRARY] null; rolling back session: {}}}
This is due to a race condition in Snappy 1.0.5 (the version used by the Hive
NAR) where two classloaders can try to define the native loader class, thus the
second one would fail, giving the error above.
The proposed solution is to guarantee that Snappy is loaded before this
situation is encountered (i.e. before the InstanceClassLoaders are created).
was:
{{When data coming into PutHiveStreaming is compressed with Snappy, then
multiple instances of PutHiveStreaming in a flow can cause a failure, the log
often shows the following: }}
{{
org.apache.nifi.processors.hive.PutHiveStreaming$$Lambda$510/1467586448@68a5884d
failed to process due to org.xerial.snappy.SnappyError:
[FAILED_TO_LOAD_NATIVE_LIBRARY] null; rolling back session: {} }}
This is due to a race condition in Snappy 1.0.5 (the version used by the Hive
NAR) where two classloaders can try to define the native loader class, thus the
second one would fail, giving the error above.
The proposed solution is to guarantee that Snappy is loaded before this
situation is encountered (i.e. before the InstanceClassLoaders are created).
> PutHiveStreaming multiple instances with Snappy fail intermittently
> -------------------------------------------------------------------
>
> Key: NIFI-4944
> URL: https://issues.apache.org/jira/browse/NIFI-4944
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Reporter: Matt Burgess
> Assignee: Matt Burgess
> Priority: Major
>
> When data coming into PutHiveStreaming is compressed with Snappy, then
> multiple instances of PutHiveStreaming in a flow can cause a failure, the log
> often shows the following:
> {{org.apache.nifi.processors.hive.PutHiveStreaming$$Lambda$510/1467586448@68a5884d
> failed to process due to org.xerial.snappy.SnappyError:
> [FAILED_TO_LOAD_NATIVE_LIBRARY] null; rolling back session: {}}}
> This is due to a race condition in Snappy 1.0.5 (the version used by the Hive
> NAR) where two classloaders can try to define the native loader class, thus
> the second one would fail, giving the error above.
> The proposed solution is to guarantee that Snappy is loaded before this
> situation is encountered (i.e. before the InstanceClassLoaders are created).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
