Hi,
I am not sure if the official built docker contains TEZ, but if you try to
build it by yourself, you can have a look at here:
https://github.com/apache/hive/blob/master/packaging/src/docker/Dockerfile#L41
To use TEZ you need to place it inside the docker container and configure
it:
Download from here:
https://archive.apache.org/dist/tez/0.10.2/apache-tez-0.10.2-bin.tar.gz
and configure it with:
export TEZ_HOME="/tez"
export TEZ_CONF_DIR="/hive/conf/"
export HADOOP_CLASSPATH="$TEZ_HOME/*:$TEZ_HOME/lib/*:$HADOOP_CLASSPATH"
The above Dockerfile seems to do all that job, so try to build docker image
with with the official Dockerfile, it should work.
Regards,
Zoltan Ratkai
On Wed, Jan 10, 2024 at 8:45 AM Ayush Saxena wrote:
> Hive on MR3 isn’t an official Apache Hive thing, not even an Apache OS
> thing, so, it is a vendor product just being tried to advertised in the
> ‘Apache’ Hive space
>
> So, it can be all mess, filled with security issues or bugs & we Apache
> Hive for the record aren’t responsible for that neither do we endorse usage
> of that or anything outside the scope of Apache
>
> -Ayush
>
> On 10-Jan-2024, at 1:09 PM, Sungwoo Park wrote:
>
>
> As far as I know, Hive-Tez supports local mode, but does not standalone
> mode (like Spark). Hive-MR3 supports standalone mode, so you can run it in
> any type of cluster.
>
> --- Sungwoo
>
> On Wed, Jan 10, 2024 at 4:22 PM Sanjay Gupta wrote:
>
>> I can run hive with mr engine in local mode. Does Hive + Tez also
>> works in standalone mode ?
>>
>> On Tue, Jan 9, 2024 at 11:08 PM Sungwoo Park wrote:
>> >
>> > Hello,
>> >
>> > I don't have an answer to your problem, but if your goal is to quickly
>> test Hive 3 using Docker, there is an alternative way which uses Hive on
>> MR3.
>> >
>> > https://mr3docs.datamonad.com/docs/quick/docker/
>> >
>> > You can also run Hive on MR3 on Kubernetes.
>> >
>> > Thanks,
>> >
>> > --- Sungwoo
>> >
>> >
>> >
>> > On Wed, Jan 10, 2024 at 3:25 PM Sanjay Gupta
>> wrote:
>> >>
>> >> Hi,
>> >> Using following docker container to run meta , hiveserver2
>> >>
>> >> https://hub.docker.com/r/apache/hive
>> >> https://github.com/apache/hive/blob/master/packaging/src/docker/
>> >>
>> >> I have configured hive-site.xml to se S3
>> >> When I set in hive.execution.engine to mr hive-site.xml, hive is
>> >> running fine and I can perform queries but setting to tez fails with
>> >> error.
>> >> There is no hdfs but it is running in local mode.
>> >>
>> >>
>> >> hive.execution.engine
>> >> tez
>> >>
>> >>
>> >> Any idea how to fix this issue ?
>> >>
>> >> hive
>> >> SLF4J: Actual binding is of type
>> [org.apache.logging.slf4j.Log4jLoggerFactory]
>> >> Hive Session ID = 03368207-1904-4c4c-b63e-b29dd28e0a71
>> >>
>> >> Logging initialized using configuration in
>> >> jar:file:/opt/hive/lib/hive-common-3.1.3.jar!/hive-log4j2.properties
>> >> Async: true
>> >> Exception in thread "main" java.lang.NoClassDefFoundError:
>> >> org/apache/tez/dag/api/TezConfiguration
>> >> at
>> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:661)
>> >> at
>> org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:591)
>> >> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:747)
>> >> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:683)
>> >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> >> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>> >> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> >> at java.lang.reflect.Method.invoke(Method.java:498)
>> >> at org.apache.hadoop.util.RunJar.run(RunJar.java:308)
>> >> at org.apache.hadoop.util.RunJar.main(RunJar.java:222)
>> >> Caused by: java.lang.ClassNotFoundException:
>> >> org.apache.tez.dag.api.TezConfiguration
>> >> at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
>> >> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
>> >> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
>> >> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
>> >>
>> >>
>> >> --
>> >>
>> >> Thanks
>> >> Sanjay Gupta
>>
>>
>>
>> --
>>
>> Thanks
>> Sanjay Gupta
>>
>
