Re: Hive using Spark engine vs native spark with hive integration.

2020-10-06 Thread Ricardo Martinelli de Oliveira
re control to add logic and also control some of the > performance features, for example things like caching/evicting etc. > > > > > > Any advice on this is much appreciated. > > > > > > Thanks, > > -Manu > > > -- Ricardo Martinelli De Oliveira

Should SHOW TABLES statement return a hive-compatible output?

2020-09-29 Thread Ricardo Martinelli de Oliveira
d make Thrift Server resolve a SHOW TABLES EXTENDED statement to return what SparkSQL returns? [1] https://github.com/dropbox/PyHive/issues/146 [2] https://spark.apache.org/docs/latest/sql-distributed-sql-engine.html -- Ricardo Martinelli De Oliveira Data Engineer, AI CoE Red Hat Brazil <https:

Re: Is there any good Docker container / compose with spark 2.4+ and YARN 2.8.2+

2020-09-16 Thread Ricardo Martinelli de Oliveira
+ distribution then you will find this script. On Wed, Sep 16, 2020 at 7:55 AM Ivan Petrov wrote: > Hi, > looking for a ready to use docker-container that has inside: > - spark 2.4 or higher > - yarn 2.8.2 or higher > I'm looking for a way to submit spark jobs on yarn. > -- Ric

Fwd: [Spark SQL Thrift Server] Persistence errors with PostgreSQL and MySQL in 2.4.3

2019-06-06 Thread Ricardo Martinelli de Oliveira
BC Driver version(42.2.5) and postgres, but the versions are the same as in spark 2.2.3 (where I have a working environment). Anyone have any ideas about what can cause these issues? -- Ricardo Martinelli De Oliveira -- Ricardo Martinelli De Oliveira Data Engineer, AI CoE Red Hat Brazil <ht