Re: Nifi 1.0.0 compatibility with Hive 1.1.0
name of an >> >>> enumeration constant >> >>> [ERROR] >> >>> /nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors >> /src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcFlowF >> ileWriter.java:[689,43] >> >>> error: cannot find symbol >> >>> [ERROR] symbol: variable BloomFilter >> >>> [ERROR] location: class OrcProto >> >>> [ERROR] >> >>> /nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors >> /src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcFlowF >> ileWriter.java:[690,43] >> >>> error: cannot find symbol >> >>> [ERROR] symbol: variable BloomFilterIndex >> >>> [ERROR] location: class OrcProto >> >>> [ERROR] >> >>> /nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors >> /src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcFlowF >> ileWriter.java:[691,87] >> >>> error: cannot find symbol >> >>> [ERROR] symbol: variable BLOOM_FILTER >> >>> [ERROR] location: class Kind >> >>> [ERROR] >> >>> /nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors >> /src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcFlowF >> ileWriter.java:[692,34] >> >>> error: cannot find symbol >> >>> [ERROR] symbol: class BloomFilterIO >> >>> [ERROR] location: class TreeWriter >> >>> [ERROR] >> >>> /nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors >> /src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcFlowF >> ileWriter.java:[726,52] >> >>> error: incompatible types: PositionedOutputStream cannot be converted >> to >> >>> OutStream >> >>> [ERROR] >> >>> /nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors >> /src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcFlowF >> ileWriter.java:[728,50] >> >>> error: incompatible types: PositionedOutputStream cannot be converted >> to >> >>> OutStream >> >>> [ERROR] >> >>> /nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors >> /src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcFlowF >> ileWriter.java:[804,19] >> >>> error: cannot find symbol >> >>> [ERROR] symbol: method setWriterTimezone(String) >> >>> [ERROR] location: variable builder of type Builder >> >>> [ERROR] >> >>> /nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors >> /src/main/java/org/apache/hadoop/hive/ql/io/orc/NiFiOrcU >> tils.java:[450,32] >> >>> error: cannot find symbol >> >>> >> >>> do you have any idea? >> >>> >> >>> Thanks, >> >>> Yari >> >>> >> >>> On 8 September 2016 at 20:42, Peter Wicks (pwicks) > > >> >>> wrote: >> >>>> >> >>>> Also, ORC File support was pulled out into its own library on the >> HIVE >> >>>> side. >> >>>> >> >>>> If you are willing to compile and run your own version you might >> need to >> >>>> include orc-core as a MVN dependency: >> >>>> https://mvnrepository.com/artifact/org.apache.orc/orc-core/1.2.0. >> >>>> >> >>>> >> >>>> >> >>>> >> >>>> >> >>>> From: Andre [mailto:andre-li...@fucs.org] >> >>>> Sent: Thursday, September 08, 2016 4:51 AM >> >>>> To: users@nifi.apache.org >> >>>> Subject: Re: Nifi 1.0.0 compatibility with Hive 1.1.0 >> >>>> >> >>>> >> >>>> >> >>>> Yari, >> >>>> >> >>>> >> >>>> >> >>>> Is there any chance you can cherry pick commit >> >>>> 80224e3e5ed7ee7b09c4985a920a7fa393bff26c and try again? >> >>>> >> >>>> >> >>>> >> >>>> Post 1.0.0 there have been some changes to streamline compilation >> using >> >>>> vendor provided libraries. >> >>>> >> >>>> >> >>>> >> >>>> Cheers >> >>>> >> >>>> >> >>>> >> >>>> On Thu, Sep 8, 2016 at 8:44 PM, Yari Marchetti >> >>>> wrote: >> >>>> >> >>>> Hello, >> >
Re: Nifi 1.0.0 compatibility with Hive 1.1.0
:[692,34]
> >>> error: cannot find symbol
> >>> [ERROR] symbol: class BloomFilterIO
> >>> [ERROR] location: class TreeWriter
> >>> [ERROR]
> >>> /nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-
> processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/
> OrcFlowFileWriter.java:[726,52]
> >>> error: incompatible types: PositionedOutputStream cannot be converted
> to
> >>> OutStream
> >>> [ERROR]
> >>> /nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-
> processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/
> OrcFlowFileWriter.java:[728,50]
> >>> error: incompatible types: PositionedOutputStream cannot be converted
> to
> >>> OutStream
> >>> [ERROR]
> >>> /nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-
> processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/
> OrcFlowFileWriter.java:[804,19]
> >>> error: cannot find symbol
> >>> [ERROR] symbol: method setWriterTimezone(String)
> >>> [ERROR] location: variable builder of type Builder
> >>> [ERROR]
> >>> /nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-
> processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/
> NiFiOrcUtils.java:[450,32]
> >>> error: cannot find symbol
> >>>
> >>> do you have any idea?
> >>>
> >>> Thanks,
> >>> Yari
> >>>
> >>> On 8 September 2016 at 20:42, Peter Wicks (pwicks)
> >>> wrote:
> >>>>
> >>>> Also, ORC File support was pulled out into its own library on the HIVE
> >>>> side.
> >>>>
> >>>> If you are willing to compile and run your own version you might need
> to
> >>>> include orc-core as a MVN dependency:
> >>>> https://mvnrepository.com/artifact/org.apache.orc/orc-core/1.2.0.
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> From: Andre [mailto:andre-li...@fucs.org]
> >>>> Sent: Thursday, September 08, 2016 4:51 AM
> >>>> To: users@nifi.apache.org
> >>>> Subject: Re: Nifi 1.0.0 compatibility with Hive 1.1.0
> >>>>
> >>>>
> >>>>
> >>>> Yari,
> >>>>
> >>>>
> >>>>
> >>>> Is there any chance you can cherry pick commit
> >>>> 80224e3e5ed7ee7b09c4985a920a7fa393bff26c and try again?
> >>>>
> >>>>
> >>>>
> >>>> Post 1.0.0 there have been some changes to streamline compilation
> using
> >>>> vendor provided libraries.
> >>>>
> >>>>
> >>>>
> >>>> Cheers
> >>>>
> >>>>
> >>>>
> >>>> On Thu, Sep 8, 2016 at 8:44 PM, Yari Marchetti
> >>>> wrote:
> >>>>
> >>>> Hello,
> >>>>
> >>>> I'd like to use Nifi 1.0.0 with Hive 1.1.0 (on CDH 5.5.2) but after
> some
> >>>> investigation I realised that the hive-jdbc driver included in Nifi is
> >>>> incompatible with the Hive version we're using (1.1.0 on CDH 5.5.2)
> as I'm
> >>>> getting the error:
> >>>>
> >>>>
> >>>>
> >>>> org.apache.hive.jdbc.HiveConnection Error opening session
> >>>>
> >>>> org.apache.thrift.TApplicationException: Required field
> >>>> 'client_protocol' is unset! Struct:TOpenSessionReq(client_
> protocol:null,
> >>>> configuration:{use:database=unifieddata})
> >>>>
> >>>>
> >>>>
> >>>> So I just tried to recompile Nifi using the Cloudera profile 5.5.2 but
> >>>> compilation is failing:
> >>>>
> >>>>
> >>>>
> >>>> [ERROR] Failed to execute goal
> >>>> org.apache.maven.plugins:maven-compiler-plugin:3.2:compile
> (default-compile)
> >>>> on project nifi-hive-processors: Compilation failure: Compilation
> failure:
> >>>>
> >>>> [ERROR]
> >>>> /home/matteo/git/nifi/nifi-nar-bundles/nifi-hive-bundle/
> nifi-hive-processors/src/main/java/org/apache/hadoop/hive/
> ql/io/orc/NiFiOrcUtils.java:[26,43]
> >>>> error: package org.apache.hadoop.hive.ql.io.filters does not exist
> >>>>
> >>>> [ERROR]
> >>>> /home/matteo/git/nifi/nifi-nar-bundles/nifi-hive-bundle/
> nifi-hive-processors/src/main/java/org/apache/hadoop/hive/
> ql/io/orc/OrcFlowFileWriter.java:[45,43]
> >>>> error: package org.apache.hadoop.hive.ql.io.filters does not exist
> >>>>
> >>>> [ERROR]
> >>>> /home/matteo/git/nifi/nifi-nar-bundles/nifi-hive-bundle/
> nifi-hive-processors/src/main/java/org/apache/hadoop/hive/
> ql/io/orc/OrcFlowFileWriter.java:[643,24]
> >>>> error: cannot find symbol
> >>>>
> >>>> [ERROR] symbol: class BloomFilterIO
> >>>>
> >>>> [ERROR] location: class TreeWriter
> >>>>
> >>>> [ERROR]
> >>>> /home/matteo/git/nifi/nifi-nar-bundles/nifi-hive-bundle/
> nifi-hive-processors/src/main/java/org/apache/hadoop/hive/
> ql/io/orc/OrcFlowFileWriter.java:[645,30]
> >>>> error: cannot find symbol
> >>>>
> >>>> [ERROR] symbol: class BloomFilterIndex
> >>>>
> >>>> [ERROR] location: class OrcProto
> >>>>
> >>>> [ERROR]
> >>>> /home/matteo/git/nifi/nifi-nar-bundles/nifi-hive-bundle/
> nifi-hive-processors/src/main/java/org/apache/hadoop/hive/
> ql/io/orc/OrcFlowFileWriter.java:[646,30]
> >>>> error: cannot find symbol
> >>>>
> >>>> [ERROR] symbol: class BloomFilter
> >>>>
> >>>> [ERROR] location: class OrcProto
> >>>>
> >>>> [ERROR]
> >>>> /home/matteo/git/nifi/nifi-nar-bundles/nifi-hive-bundle/
> nifi-hive-processors/src/main/java/org/apache/hadoop/hive/
> ql/io/orc/NiFiOrcUtils.java:[450,32]
> >>>> error: cannot find symbol
> >>>>
> >>>> [ERROR] symbol: variable BloomFilterIO
> >>>>
> >>>> [ERROR] location: class NiFiOrcUtils
> >>>>
> >>>> [ERROR]
> >>>> /home/matteo/git/nifi/nifi-nar-bundles/nifi-hive-bundle/
> nifi-hive-processors/src/main/java/org/apache/hadoop/hive/
> ql/io/orc/OrcFlowFileWriter.java:[200,20]
> >>>> error: cannot find symbol
> >>>>
> >>>> [ERROR] symbol: variable OrcUtils
> >>>>
> >>>> [ERROR] location: class OrcFlowFileWriter
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> Is there any way to get Nifi to work with Hive 1.1.0 and CDH 5.5.2?
> >>>>
> >>>>
> >>>>
> >>>> Thanks,
> >>>>
> >>>> Yari
> >>>>
> >>>>
> >>>
> >>>
> >>
> >
>
Re: Nifi 1.0.0 compatibility with Hive 1.1.0
FileWriter.java:[804,19]
>>> error: cannot find symbol
>>> [ERROR] symbol: method setWriterTimezone(String)
>>> [ERROR] location: variable builder of type Builder
>>> [ERROR]
>>> /nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/NiFiOrcUtils.java:[450,32]
>>> error: cannot find symbol
>>>
>>> do you have any idea?
>>>
>>> Thanks,
>>> Yari
>>>
>>> On 8 September 2016 at 20:42, Peter Wicks (pwicks)
>>> wrote:
>>>>
>>>> Also, ORC File support was pulled out into its own library on the HIVE
>>>> side.
>>>>
>>>> If you are willing to compile and run your own version you might need to
>>>> include orc-core as a MVN dependency:
>>>> https://mvnrepository.com/artifact/org.apache.orc/orc-core/1.2.0.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> From: Andre [mailto:andre-li...@fucs.org]
>>>> Sent: Thursday, September 08, 2016 4:51 AM
>>>> To: users@nifi.apache.org
>>>> Subject: Re: Nifi 1.0.0 compatibility with Hive 1.1.0
>>>>
>>>>
>>>>
>>>> Yari,
>>>>
>>>>
>>>>
>>>> Is there any chance you can cherry pick commit
>>>> 80224e3e5ed7ee7b09c4985a920a7fa393bff26c and try again?
>>>>
>>>>
>>>>
>>>> Post 1.0.0 there have been some changes to streamline compilation using
>>>> vendor provided libraries.
>>>>
>>>>
>>>>
>>>> Cheers
>>>>
>>>>
>>>>
>>>> On Thu, Sep 8, 2016 at 8:44 PM, Yari Marchetti
>>>> wrote:
>>>>
>>>> Hello,
>>>>
>>>> I'd like to use Nifi 1.0.0 with Hive 1.1.0 (on CDH 5.5.2) but after some
>>>> investigation I realised that the hive-jdbc driver included in Nifi is
>>>> incompatible with the Hive version we're using (1.1.0 on CDH 5.5.2) as I'm
>>>> getting the error:
>>>>
>>>>
>>>>
>>>> org.apache.hive.jdbc.HiveConnection Error opening session
>>>>
>>>> org.apache.thrift.TApplicationException: Required field
>>>> 'client_protocol' is unset! Struct:TOpenSessionReq(client_protocol:null,
>>>> configuration:{use:database=unifieddata})
>>>>
>>>>
>>>>
>>>> So I just tried to recompile Nifi using the Cloudera profile 5.5.2 but
>>>> compilation is failing:
>>>>
>>>>
>>>>
>>>> [ERROR] Failed to execute goal
>>>> org.apache.maven.plugins:maven-compiler-plugin:3.2:compile
>>>> (default-compile)
>>>> on project nifi-hive-processors: Compilation failure: Compilation failure:
>>>>
>>>> [ERROR]
>>>> /home/matteo/git/nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/NiFiOrcUtils.java:[26,43]
>>>> error: package org.apache.hadoop.hive.ql.io.filters does not exist
>>>>
>>>> [ERROR]
>>>> /home/matteo/git/nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcFlowFileWriter.java:[45,43]
>>>> error: package org.apache.hadoop.hive.ql.io.filters does not exist
>>>>
>>>> [ERROR]
>>>> /home/matteo/git/nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcFlowFileWriter.java:[643,24]
>>>> error: cannot find symbol
>>>>
>>>> [ERROR] symbol: class BloomFilterIO
>>>>
>>>> [ERROR] location: class TreeWriter
>>>>
>>>> [ERROR]
>>>> /home/matteo/git/nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcFlowFileWriter.java:[645,30]
>>>> error: cannot find symbol
>>>>
>>>> [ERROR] symbol: class BloomFilterIndex
>>>>
>>>> [ERROR] location: class OrcProto
>>>>
>>>> [ERROR]
>>>> /home/matteo/git/nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcFlowFileWriter.java:[646,30]
>>>> error: cannot find symbol
>>>>
>>>> [ERROR] symbol: class BloomFilter
>>>>
>>>> [ERROR] location: class OrcProto
>>>>
>>>> [ERROR]
>>>> /home/matteo/git/nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/NiFiOrcUtils.java:[450,32]
>>>> error: cannot find symbol
>>>>
>>>> [ERROR] symbol: variable BloomFilterIO
>>>>
>>>> [ERROR] location: class NiFiOrcUtils
>>>>
>>>> [ERROR]
>>>> /home/matteo/git/nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcFlowFileWriter.java:[200,20]
>>>> error: cannot find symbol
>>>>
>>>> [ERROR] symbol: variable OrcUtils
>>>>
>>>> [ERROR] location: class OrcFlowFileWriter
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Is there any way to get Nifi to work with Hive 1.1.0 and CDH 5.5.2?
>>>>
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> Yari
>>>>
>>>>
>>>
>>>
>>
>
Re: Nifi 1.0.0 compatibility with Hive 1.1.0
Thanks, Andre. This is the command I'm using: mvn install -DskipTests -Pcloudera -Dhadoop.version=2.6.0-cdh5.5.2 -Dhive.version=1.1.0-cdh5.5.2 Adding as dependency: org.apache.orc orc-core 1.1.0 Yari On 9 September 2016 at 13:23, Andre wrote: > Yari, > > Can you please confirm what command line are you using to compile? > > Happy yo try to reproduce. > > Cheers > > On Fri, Sep 9, 2016 at 8:49 PM, Yari Marchetti < > yari.marche...@buongiorno.com> wrote: > >> Hi Peter/Andre, >> I cherrypicked the commit and added the dependency on: >> >> >> org.apache.orc >> orc-core >> 1.1.0 >> >> >> (tried also with version 1.2.0) but stil no luck, as I keep on getting >> compilation errors: >> >> [ERROR] /nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors >> /src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcFlowFileWriter.java:[200,20] >> error: cannot find symbol >> [ERROR] symbol: variable OrcUtils >> [ERROR] location: class OrcFlowFileWriter >> [ERROR] /nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors >> /src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcFlowFileWriter.java:[203,20] >> error: cannot find symbol >> [ERROR] symbol: variable OrcUtils >> [ERROR] location: class OrcFlowFileWriter >> [ERROR] /nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors >> /src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcFlowFileWriter.java:[313,45] >> error: cannot find symbol >> [ERROR] symbol: method loadClass(String) >> [ERROR] location: class JavaUtils >> [ERROR] /nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors >> /src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcFlowFileWriter.java:[494,21] >> error: an enum switch case label must be the unqualified name of an >> enumeration constant >> [ERROR] /nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors >> /src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcFlowFileWriter.java:[689,43] >> error: cannot find symbol >> [ERROR] symbol: variable BloomFilter >> [ERROR] location: class OrcProto >> [ERROR] /nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors >> /src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcFlowFileWriter.java:[690,43] >> error: cannot find symbol >> [ERROR] symbol: variable BloomFilterIndex >> [ERROR] location: class OrcProto >> [ERROR] /nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors >> /src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcFlowFileWriter.java:[691,87] >> error: cannot find symbol >> [ERROR] symbol: variable BLOOM_FILTER >> [ERROR] location: class Kind >> [ERROR] /nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors >> /src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcFlowFileWriter.java:[692,34] >> error: cannot find symbol >> [ERROR] symbol: class BloomFilterIO >> [ERROR] location: class TreeWriter >> [ERROR] /nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors >> /src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcFlowFileWriter.java:[726,52] >> error: incompatible types: PositionedOutputStream cannot be converted to >> OutStream >> [ERROR] /nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors >> /src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcFlowFileWriter.java:[728,50] >> error: incompatible types: PositionedOutputStream cannot be converted to >> OutStream >> [ERROR] /nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors >> /src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcFlowFileWriter.java:[804,19] >> error: cannot find symbol >> [ERROR] symbol: method setWriterTimezone(String) >> [ERROR] location: variable builder of type Builder >> [ERROR] /nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors >> /src/main/java/org/apache/hadoop/hive/ql/io/orc/NiFiOrcUtils.java:[450,32] >> error: cannot find symbol >> >> do you have any idea? >> >> Thanks, >> Yari >> >> On 8 September 2016 at 20:42, Peter Wicks (pwicks) >> wrote: >> >>> Also, ORC File support was pulled out into its own library on the HIVE >>> side. >>> >>> If you are willing to compile and run your own version you might need to >>> include orc-core as a MVN dependency: https://mvnrepository.com/arti >>> fact/org.apache.orc/orc-core/1.2.0. >>> >>> >>> >>> >>> >>> *From:* Andre [mailto:andre-li...@fucs.org] >>> *Sent:* Thursday, September 08, 2016 4:51 AM >>> *To:* users@nifi.apache.org >>> *Subject:* Re: Nifi 1
Re: Nifi 1.0.0 compatibility with Hive 1.1.0
Yari, Can you please confirm what command line are you using to compile? Happy yo try to reproduce. Cheers On Fri, Sep 9, 2016 at 8:49 PM, Yari Marchetti < yari.marche...@buongiorno.com> wrote: > Hi Peter/Andre, > I cherrypicked the commit and added the dependency on: > > > org.apache.orc > orc-core > 1.1.0 > > > (tried also with version 1.2.0) but stil no luck, as I keep on getting > compilation errors: > > [ERROR] /nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive- > processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/ > OrcFlowFileWriter.java:[200,20] error: cannot find symbol > [ERROR] symbol: variable OrcUtils > [ERROR] location: class OrcFlowFileWriter > [ERROR] /nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive- > processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/ > OrcFlowFileWriter.java:[203,20] error: cannot find symbol > [ERROR] symbol: variable OrcUtils > [ERROR] location: class OrcFlowFileWriter > [ERROR] /nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive- > processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/ > OrcFlowFileWriter.java:[313,45] error: cannot find symbol > [ERROR] symbol: method loadClass(String) > [ERROR] location: class JavaUtils > [ERROR] /nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive- > processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/ > OrcFlowFileWriter.java:[494,21] error: an enum switch case label must be > the unqualified name of an enumeration constant > [ERROR] /nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive- > processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/ > OrcFlowFileWriter.java:[689,43] error: cannot find symbol > [ERROR] symbol: variable BloomFilter > [ERROR] location: class OrcProto > [ERROR] /nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive- > processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/ > OrcFlowFileWriter.java:[690,43] error: cannot find symbol > [ERROR] symbol: variable BloomFilterIndex > [ERROR] location: class OrcProto > [ERROR] /nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive- > processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/ > OrcFlowFileWriter.java:[691,87] error: cannot find symbol > [ERROR] symbol: variable BLOOM_FILTER > [ERROR] location: class Kind > [ERROR] /nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive- > processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/ > OrcFlowFileWriter.java:[692,34] error: cannot find symbol > [ERROR] symbol: class BloomFilterIO > [ERROR] location: class TreeWriter > [ERROR] /nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive- > processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/ > OrcFlowFileWriter.java:[726,52] error: incompatible types: > PositionedOutputStream cannot be converted to OutStream > [ERROR] /nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive- > processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/ > OrcFlowFileWriter.java:[728,50] error: incompatible types: > PositionedOutputStream cannot be converted to OutStream > [ERROR] /nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive- > processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/ > OrcFlowFileWriter.java:[804,19] error: cannot find symbol > [ERROR] symbol: method setWriterTimezone(String) > [ERROR] location: variable builder of type Builder > [ERROR] /nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive- > processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/NiFiOrcUtils.java:[450,32] > error: cannot find symbol > > do you have any idea? > > Thanks, > Yari > > On 8 September 2016 at 20:42, Peter Wicks (pwicks) > wrote: > >> Also, ORC File support was pulled out into its own library on the HIVE >> side. >> >> If you are willing to compile and run your own version you might need to >> include orc-core as a MVN dependency: https://mvnrepository.com/arti >> fact/org.apache.orc/orc-core/1.2.0. >> >> >> >> >> >> *From:* Andre [mailto:andre-li...@fucs.org] >> *Sent:* Thursday, September 08, 2016 4:51 AM >> *To:* users@nifi.apache.org >> *Subject:* Re: Nifi 1.0.0 compatibility with Hive 1.1.0 >> >> >> >> Yari, >> >> >> >> Is there any chance you can cherry pick commit >> 80224e3e5ed7ee7b09c4985a920a7fa393bff26c >> and try again? >> >> >> >> Post 1.0.0 there have been some changes to streamline compilation using >> vendor provided libraries. >> >> >> >> Cheers >> >> >> >> On Thu, Sep 8, 2016 at 8:44 PM, Yari Marchetti < >> yari.marche...@buongiorno.com> wrote: >> >> Hello, >> >> I'd like to use Nifi 1.0.0 with Hive 1.1.0 (on CDH 5.5.2) but
Re: Nifi 1.0.0 compatibility with Hive 1.1.0
Hi Peter/Andre,
I cherrypicked the commit and added the dependency on:
org.apache.orc
orc-core
1.1.0
(tried also with version 1.2.0) but stil no luck, as I keep on getting
compilation errors:
[ERROR]
/nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcFlowFileWriter.java:[200,20]
error: cannot find symbol
[ERROR] symbol: variable OrcUtils
[ERROR] location: class OrcFlowFileWriter
[ERROR]
/nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcFlowFileWriter.java:[203,20]
error: cannot find symbol
[ERROR] symbol: variable OrcUtils
[ERROR] location: class OrcFlowFileWriter
[ERROR]
/nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcFlowFileWriter.java:[313,45]
error: cannot find symbol
[ERROR] symbol: method loadClass(String)
[ERROR] location: class JavaUtils
[ERROR]
/nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcFlowFileWriter.java:[494,21]
error: an enum switch case label must be the unqualified name of an
enumeration constant
[ERROR]
/nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcFlowFileWriter.java:[689,43]
error: cannot find symbol
[ERROR] symbol: variable BloomFilter
[ERROR] location: class OrcProto
[ERROR]
/nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcFlowFileWriter.java:[690,43]
error: cannot find symbol
[ERROR] symbol: variable BloomFilterIndex
[ERROR] location: class OrcProto
[ERROR]
/nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcFlowFileWriter.java:[691,87]
error: cannot find symbol
[ERROR] symbol: variable BLOOM_FILTER
[ERROR] location: class Kind
[ERROR]
/nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcFlowFileWriter.java:[692,34]
error: cannot find symbol
[ERROR] symbol: class BloomFilterIO
[ERROR] location: class TreeWriter
[ERROR]
/nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcFlowFileWriter.java:[726,52]
error: incompatible types: PositionedOutputStream cannot be converted to
OutStream
[ERROR]
/nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcFlowFileWriter.java:[728,50]
error: incompatible types: PositionedOutputStream cannot be converted to
OutStream
[ERROR]
/nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcFlowFileWriter.java:[804,19]
error: cannot find symbol
[ERROR] symbol: method setWriterTimezone(String)
[ERROR] location: variable builder of type Builder
[ERROR]
/nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/NiFiOrcUtils.java:[450,32]
error: cannot find symbol
do you have any idea?
Thanks,
Yari
On 8 September 2016 at 20:42, Peter Wicks (pwicks)
wrote:
> Also, ORC File support was pulled out into its own library on the HIVE
> side.
>
> If you are willing to compile and run your own version you might need to
> include orc-core as a MVN dependency: https://mvnrepository.com/
> artifact/org.apache.orc/orc-core/1.2.0.
>
>
>
>
>
> *From:* Andre [mailto:andre-li...@fucs.org]
> *Sent:* Thursday, September 08, 2016 4:51 AM
> *To:* users@nifi.apache.org
> *Subject:* Re: Nifi 1.0.0 compatibility with Hive 1.1.0
>
>
>
> Yari,
>
>
>
> Is there any chance you can cherry pick commit
> 80224e3e5ed7ee7b09c4985a920a7fa393bff26c and try again?
>
>
>
> Post 1.0.0 there have been some changes to streamline compilation using
> vendor provided libraries.
>
>
>
> Cheers
>
>
>
> On Thu, Sep 8, 2016 at 8:44 PM, Yari Marchetti <
> yari.marche...@buongiorno.com> wrote:
>
> Hello,
>
> I'd like to use Nifi 1.0.0 with Hive 1.1.0 (on CDH 5.5.2) but after some
> investigation I realised that the hive-jdbc driver included in Nifi is
> incompatible with the Hive version we're using (1.1.0 on CDH 5.5.2) as I'm
> getting the error:
>
>
>
> org.apache.hive.jdbc.HiveConnection Error opening session
>
> org.apache.thrift.TApplicationException: Required field 'client_protocol'
> is unset! Struct:TOpenSessionReq(client_protocol:null,
> configuration:{use:database=unifieddata})
>
>
>
> So I just tried to recompile Nifi using the Cloudera profile 5.5.2 but
> compilation is failing:
>
>
>
> [ERROR] Failed to execute goal org.apache.maven.plugins:
> maven-compiler-plugin:3.2:compile (default-compile) on project
> nifi-hive-processors: C
RE: Nifi 1.0.0 compatibility with Hive 1.1.0
Also, ORC File support was pulled out into its own library on the HIVE side.
If you are willing to compile and run your own version you might need to
include orc-core as a MVN dependency:
https://mvnrepository.com/artifact/org.apache.orc/orc-core/1.2.0.
From: Andre [mailto:andre-li...@fucs.org]
Sent: Thursday, September 08, 2016 4:51 AM
To: users@nifi.apache.org
Subject: Re: Nifi 1.0.0 compatibility with Hive 1.1.0
Yari,
Is there any chance you can cherry pick commit
80224e3e5ed7ee7b09c4985a920a7fa393bff26c and try again?
Post 1.0.0 there have been some changes to streamline compilation using vendor
provided libraries.
Cheers
On Thu, Sep 8, 2016 at 8:44 PM, Yari Marchetti
mailto:yari.marche...@buongiorno.com>> wrote:
Hello,
I'd like to use Nifi 1.0.0 with Hive 1.1.0 (on CDH 5.5.2) but after some
investigation I realised that the hive-jdbc driver included in Nifi is
incompatible with the Hive version we're using (1.1.0 on CDH 5.5.2) as I'm
getting the error:
org.apache.hive.jdbc.HiveConnection Error opening session
org.apache.thrift.TApplicationException: Required field 'client_protocol' is
unset! Struct:TOpenSessionReq(client_protocol:null,
configuration:{use:database=unifieddata})
So I just tried to recompile Nifi using the Cloudera profile 5.5.2 but
compilation is failing:
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.2:compile (default-compile) on
project nifi-hive-processors: Compilation failure: Compilation failure:
[ERROR]
/home/matteo/git/nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/NiFiOrcUtils.java:[26,43]
error: package
org.apache.hadoop.hive.ql.io<http://org.apache.hadoop.hive.ql.io>.filters does
not exist
[ERROR]
/home/matteo/git/nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcFlowFileWriter.java:[45,43]
error: package
org.apache.hadoop.hive.ql.io<http://org.apache.hadoop.hive.ql.io>.filters does
not exist
[ERROR]
/home/matteo/git/nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcFlowFileWriter.java:[643,24]
error: cannot find symbol
[ERROR] symbol: class BloomFilterIO
[ERROR] location: class TreeWriter
[ERROR]
/home/matteo/git/nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcFlowFileWriter.java:[645,30]
error: cannot find symbol
[ERROR] symbol: class BloomFilterIndex
[ERROR] location: class OrcProto
[ERROR]
/home/matteo/git/nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcFlowFileWriter.java:[646,30]
error: cannot find symbol
[ERROR] symbol: class BloomFilter
[ERROR] location: class OrcProto
[ERROR]
/home/matteo/git/nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/NiFiOrcUtils.java:[450,32]
error: cannot find symbol
[ERROR] symbol: variable BloomFilterIO
[ERROR] location: class NiFiOrcUtils
[ERROR]
/home/matteo/git/nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcFlowFileWriter.java:[200,20]
error: cannot find symbol
[ERROR] symbol: variable OrcUtils
[ERROR] location: class OrcFlowFileWriter
Is there any way to get Nifi to work with Hive 1.1.0 and CDH 5.5.2?
Thanks,
Yari
Re: Nifi 1.0.0 compatibility with Hive 1.1.0
Yari,
Is there any chance you can cherry pick
commit 80224e3e5ed7ee7b09c4985a920a7fa393bff26c and try again?
Post 1.0.0 there have been some changes to streamline compilation using
vendor provided libraries.
Cheers
On Thu, Sep 8, 2016 at 8:44 PM, Yari Marchetti <
yari.marche...@buongiorno.com> wrote:
> Hello,
> I'd like to use Nifi 1.0.0 with Hive 1.1.0 (on CDH 5.5.2) but after some
> investigation I realised that the hive-jdbc driver included in Nifi is
> incompatible with the Hive version we're using (1.1.0 on CDH 5.5.2) as I'm
> getting the error:
>
> org.apache.hive.jdbc.HiveConnection Error opening session
> org.apache.thrift.TApplicationException: Required field 'client_protocol'
> is unset! Struct:TOpenSessionReq(client_protocol:null,
> configuration:{use:database=unifieddata})
>
> So I just tried to recompile Nifi using the Cloudera profile 5.5.2 but
> compilation is failing:
>
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.2:compile
> (default-compile) on project nifi-hive-processors: Compilation failure:
> Compilation failure:
> [ERROR] /home/matteo/git/nifi/nifi-nar-bundles/nifi-hive-bundle/nifi
> -hive-processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/NiFiOrcUtils.java:[26,43]
> error: package org.apache.hadoop.hive.ql.io.filters does not exist
> [ERROR] /home/matteo/git/nifi/nifi-nar-bundles/nifi-hive-bundle/nifi
> -hive-processors/src/main/java/org/apache/hadoop/hive/ql/io/
> orc/OrcFlowFileWriter.java:[45,43] error: package
> org.apache.hadoop.hive.ql.io.filters does not exist
> [ERROR] /home/matteo/git/nifi/nifi-nar-bundles/nifi-hive-bundle/nifi
> -hive-processors/src/main/java/org/apache/hadoop/hive/ql/io/
> orc/OrcFlowFileWriter.java:[643,24] error: cannot find symbol
> [ERROR] symbol: class BloomFilterIO
> [ERROR] location: class TreeWriter
> [ERROR] /home/matteo/git/nifi/nifi-nar-bundles/nifi-hive-bundle/nifi
> -hive-processors/src/main/java/org/apache/hadoop/hive/ql/io/
> orc/OrcFlowFileWriter.java:[645,30] error: cannot find symbol
> [ERROR] symbol: class BloomFilterIndex
> [ERROR] location: class OrcProto
> [ERROR] /home/matteo/git/nifi/nifi-nar-bundles/nifi-hive-bundle/nifi
> -hive-processors/src/main/java/org/apache/hadoop/hive/ql/io/
> orc/OrcFlowFileWriter.java:[646,30] error: cannot find symbol
> [ERROR] symbol: class BloomFilter
> [ERROR] location: class OrcProto
> [ERROR] /home/matteo/git/nifi/nifi-nar-bundles/nifi-hive-bundle/nifi
> -hive-processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/NiFiOrcUtils.java:[450,32]
> error: cannot find symbol
> [ERROR] symbol: variable BloomFilterIO
> [ERROR] location: class NiFiOrcUtils
> [ERROR] /home/matteo/git/nifi/nifi-nar-bundles/nifi-hive-bundle/nifi
> -hive-processors/src/main/java/org/apache/hadoop/hive/ql/io/
> orc/OrcFlowFileWriter.java:[200,20] error: cannot find symbol
> [ERROR] symbol: variable OrcUtils
> [ERROR] location: class OrcFlowFileWriter
>
>
> Is there any way to get Nifi to work with Hive 1.1.0 and CDH 5.5.2?
>
> Thanks,
> Yari
>
