Maybe using ranger or sentry would be the better choice to intercept those calls?
> On 7. Apr 2017, at 16:32, Alvaro Brandon <[email protected]> wrote: > > I was going through the SparkContext.textFile() and I was wondering at that > point does Spark communicates with HDFS. Since when you download Spark > binaries you also specify the Hadoop version you will use, I'm guessing it > has its own client that calls HDFS wherever you specify it in the > configuration files. > > The goal is to instrument and log all the calls that Spark does to HDFS. > Which class or classes perform these operations? > > --------------------------------------------------------------------- To unsubscribe e-mail: [email protected]
