Hi Matt, #1 Thank you very much!!! PutHive3Streaming works after compilation :)
BTW: compilation with options mvn -T C2.0 clean install -Phortonworks -Dhive.version=3.1.0.3.0.0.0-1634 -Dhive.hadoop.version=3.1.0.3.0.0.0-1634 -Dhadoop.version=3.1.0.3.0.0.0-1634 -Pinclude-hive3 -DskipTests -e throws errors like [ERROR] symbol: class MemoryManager [ERROR] location: class NiFiOrcUtils [ERROR] /home/compiler/nifi/nifi-master/nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/NiFiOrcUtils.java:[478,32] error: cannot find symbol so I compiled using: mvn -T C2.0 clean install -Phortonworks -Pinclude-hive3 -DskipTests -e I hope its proper command. #2 Please advice how to make PutHive3Streaming use specified username and password for authorization when contacting with hive? Currently all requests are made as anonymous. I have ranger for authorization, but cluster is not kerberised. In HiveConnectionPool (e.g. for HiveQL) there is a place for username and password for query that works as required. In PutHive3Streaming there is no place to put credentials. Regards, Mike > -----Original Message----- > From: Matt Burgess <mattyb...@apache.org> > Sent: Monday, July 30, 2018 2:45 PM > To: users@nifi.apache.org > Subject: Re: PutHiveStreaming + Hive 3.0 > > Mike, > > That error usually indicates a Thrift version mismatch, which in this case is > pretty much expected since PutHiveStreaming uses version 1.2.1 and HDP 3.0 > uses 3.0.0+. As of NiFi 1.7.0 you can add the "-Pinclude-hive3" profile in > your > Maven build and it will add a full set of Hive 3-compatible processors to the > assembly, or you can just go into nifi-nar-bundles/nifi-hive-bundle and build > from there, there will be a nifi-hive3-nar created in nifi-hive3-nar/target. > The > processors include SelectHive3QL, PutHive3QL, PutHive3Streaming, and > PutORC, as well as a Hive3ConnectionPool for the JDBC processors > (Select/Put). > > Regards, > Matt > > On Mon, Jul 30, 2018 at 8:18 AM Michal Tomaszewski > <michal.tomaszew...@cca.pl> wrote: > > > > Hi Pierre, > > nifi-app.log enclosed. > > > > Regards, > > Mike > > > > > > From: Pierre Villard <pierre.villard...@gmail.com> > > Sent: Monday, July 30, 2018 11:40 AM > > To: users@nifi.apache.org > > Subject: Re: PutHiveStreaming + Hive 3.0 > > > > Hi Mike, > > > > By any chance, could you share the full stack trace from nifi-app.log? > > > > Thanks, > > Pierre > > > > 2018-07-30 11:19 GMT+02:00 Michal Tomaszewski > <mailto:michal.tomaszew...@cca.pl>: > > Hello, > > Is PutHiveStreaming processor working with Hive 3.0? > > We installed Hortonworks HDP 3.0, compiled newest NiFi 1.8 snapshot and > got errors: > > Failed connecting to Hive endpoint table: hivetest3 at thrift://servername. > > NiFi has access to all site.xml's. Configuration is exactly the same as with > previous Hive server, that worked properly. > > > > Regards, > > Mike ________________________________________ Uwaga: Treść niniejszej wiadomości może być poufna i objęta zakazem jej ujawniania. Jeśli czytelnik tej wiadomości nie jest jej zamierzonym adresatem, pracownikiem lub pośrednikiem upoważnionym do jej przekazania adresatowi, informujemy że wszelkie rozprowadzanie, rozpowszechnianie lub powielanie niniejszej wiadomości jest zabronione. Jeśli otrzymałeś tę wiadomość omyłkowo, proszę bezzwłocznie odesłać ją nadawcy, a samą wiadomość usunąć z komputera. Dziękujemy. ________________________________ Note: The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited.If you have received this communication in error, please notify the sender immediately by replying to the message and deleting it from your computer. Thank you. ________________________________
