Re: Spark Metrics: custom source/sink configurations not getting recognized

2016-11-28 Thread Matthew Dailey
can't read 'propName' from >>>> 'mySink') since they are initialized >>>> <https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/deploy/master/Master.scala#L84> >>>> by >>>> Spark's metric system. >>>> >>>> Is there a (programmatic) way to have 'metrics.properties' shipped >>>> before executors initialize >>>> <https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/SparkEnv.scala#L335> >>>> ? >>>> >>>> Here's my SO question >>>> <https://stackoverflow.com/questions/39340080/spark-metrics-custom-source-sink-configurations-not-getting-recognized> >>>> . >>>> >>>> Thanks, >>>> >>>> KP >>>> >>> >>> >>> >> >

Re: Spark Metrics: custom source/sink configurations not getting recognized

2016-09-08 Thread map reduced
ster/core/src/main/scala/org/apache/spark/deploy/master/Master.scala#L84> >>> by >>> Spark's metric system. >>> >>> Is there a (programmatic) way to have 'metrics.properties' shipped >>> before executors initialize >>> <https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/SparkEnv.scala#L335> >>> ? >>> >>> Here's my SO question >>> <https://stackoverflow.com/questions/39340080/spark-metrics-custom-source-sink-configurations-not-getting-recognized> >>> . >>> >>> Thanks, >>> >>> KP >>> >> >> >> >

Re: Spark Metrics: custom source/sink configurations not getting recognized

2016-09-07 Thread map reduced
spark/deploy/master/Master.scala#L84> >> by >> Spark's metric system. >> >> Is there a (programmatic) way to have 'metrics.properties' shipped before >> executors initialize >> <https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/SparkEnv.scala#L335> >> ? >> >> Here's my SO question >> <https://stackoverflow.com/questions/39340080/spark-metrics-custom-source-sink-configurations-not-getting-recognized> >> . >> >> Thanks, >> >> KP >> > > >

Re: Spark Metrics: custom source/sink configurations not getting recognized

2016-09-07 Thread Benjamin Kim
em. > > Is there a (programmatic) way to have 'metrics.properties' shipped before > executors initialize > <https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/SparkEnv.scala#L335> > ? > > Here's my SO question > <https://stackoverflow.com/questions/39340080/spark-metrics-custom-source-sink-configurations-not-getting-recognized>. > > Thanks, > > KP > >

Re: Spark Metrics: custom source/sink configurations not getting recognized

2016-09-06 Thread map reduced
k/deploy/master/Master.scala#L84> > by > Spark's metric system. > > Is there a (programmatic) way to have 'metrics.properties' shipped before > executors initialize > <https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/SparkEnv.scala#

Spark Metrics: custom source/sink configurations not getting recognized

2016-09-05 Thread map reduced
cala/org/apache/spark/SparkEnv.scala#L335> ? Here's my SO question <https://stackoverflow.com/questions/39340080/spark-metrics-custom-source-sink-configurations-not-getting-recognized> . Thanks, KP