Hi,
I have just set up the Griffin Docker image and it seems to work ok, I am able
to view the sample data that comes by default.
Now, I would like to test a bit the metrics things against a subset of a table
that I have in our Hive instance;
In particular the configuration is as follows:
- Hiv
Hi Enrico,
Griffin service only need to get metadata from hive metastore service, it
doesn't fetch hive table data actually.
Griffin measure, which runs on spark cluster, needs to fetch hive table
data, you need to pass the AWS credentials to it when submit. I recommend
you try the shell-submit wa
Liu
Date: Wednesday, April 11, 2018 at 3:34 AM
To: "dev@griffin.incubator.apache.org" ,
Enrico D'Urso
Subject: Re: Griffin on Docker - modify Hive metastore Uris
Hi Enrico,
Griffin service only need to get metadata from hive metastore service, it
doesn't fetch hive table da
Hadoop-aws?
Many thanks,
Enrico
From: Lionel Liu
Date: Wednesday, April 11, 2018 at 3:34 AM
To: "dev@griffin.incubator.apache.org" ,
Enrico D'Urso
Subject: Re: Griffin on Docker - modify Hive metastore Uris
Hi Enrico,
Griffin service only need to get metadata from hive me
nel Liu , "dev@griffin.incubator.apache.org"
>
> Subject: Re: Griffin on Docker - modify Hive metastore Uris
>
> Hi Lionel,
>
> Thank you for your email.
>
> Right now, I am testing Spark cluster using the Spark-shell available on
> your Docker image. I just
t;dev@griffin.incubator.apache.org" ,
Enrico D'Urso
Subject: Re: Griffin on Docker - modify Hive metastore Uris
Hi Enrico,
I think you need to copy hive-site.xml into spark config directory, or
explicitly set hive-site.xml in spark-shell command line.
Because spark shell creates its sqlContext when start
u
> Date: Friday, April 13, 2018 at 10:20 AM
> To: "dev@griffin.incubator.apache.org" ,
> Enrico D'Urso
> Subject: Re: Griffin on Docker - modify Hive metastore Uris
>
> Hi Enrico,
>
>
> I think you need to copy hive-site.xml into spark config directory, o
180423t221106.941z-58moytj7/bk_date=2016-12-13/xyz
>
> Anyone has tested the Docker image to work with S3 instead of HDFS?
>
>
> Thanks,
>
> Enrico
> From: Lionel Liu
> Date: Friday, April 13, 2018 at 10:20 AM
> To: "dev@griffin.i
>,
> > Enrico D'Urso
> > Subject: Re: Griffin on Docker - modify Hive metastore Uris
> >
> > Hi Enrico,
> >
> >
> > I think you need to copy hive-site.xml into spark config directory,
> or
> > explicitly set hive-
ExecutionException:
> > java.io.FileNotFoundException: File s3://XX-common/XXXX_dm/
> > _trip_details/ctp-20180423t221106.941z-58moytj7/
> bk_date=2016-12-13
> > does not exist.
> >
> > I checked on s3 and it does exists, although there is an
und exception:
> >
> > Caused by: java.util.concurrent.ExecutionException:
> > java.io.FileNotFoundException: File s3://XX-common/_dm/
> > _trip_details/ctp-20180423t221106.941z-58moytj7/
> bk_date=2016-12
ctp-
> > > 20180423t221106.941z-58moytj7/bk_date=2016-12-13/xyz
> > >
> > > Anyone has tested the Docker image to work with S3 instead
> of HDFS?
> > >
> > >
> > >
> Hi,
> > >
> > > Yes, we did all of those things.
> > > Spark has the correct Hive metastore URI set and also has
> the right
> > > credentials for S3 (where the data is actually stored
> Thanks,
> > >
> > > Enrico
> > >
> > >
> > > On 4/25/18, 2:32 AM, "William Guo"
> wrote:
> > >
> > > hi Enrico,
> >
14 matches
Mail list logo
