Re: Spark Streaming Log4j Inside Eclipse

[Ashish Soni]

I am using Java Streaming context and it doesnt have method setLogLevel and
also i have tried by passing VM argument in eclipse and it doesnt work

JavaStreamingContext jssc = new JavaStreamingContext(sparkConf,
Durations.seconds(2));

Ashish

On Tue, Sep 29, 2015 at 7:23 AM, Adrian Tanase <atan...@adobe.com> wrote:

> You should set exta java options for your app via Eclipse project and
> specify something like
>
>  -Dlog4j.configuration=file:/tmp/log4j.properties
>
> Sent from my iPhone
>
> On 28 Sep 2015, at 18:52, Shixiong Zhu <zsxw...@gmail.com> wrote:
>
> You can use JavaSparkContext.setLogLevel to set the log level in your
> codes.
>
> Best Regards,
> Shixiong Zhu
>
> 2015-09-28 22:55 GMT+08:00 Ashish Soni <asoni.le...@gmail.com>:
>
>> I am not running it using spark submit , i am running locally inside
>> Eclipse IDE , how i set this using JAVA Code
>>
>> Ashish
>>
>> On Mon, Sep 28, 2015 at 10:42 AM, Adrian Tanase <atan...@adobe.com>
>> wrote:
>>
>>> You also need to provide it as parameter to spark submit
>>>
>>> http://stackoverflow.com/questions/28840438/how-to-override-sparks-log4j-properties-per-driver
>>>
>>> From: Ashish Soni
>>> Date: Monday, September 28, 2015 at 5:18 PM
>>> To: user
>>> Subject: Spark Streaming Log4j Inside Eclipse
>>>
>>> I need to turn off the verbose logging of Spark Streaming Code when i am
>>> running inside eclipse i tried creating a log4j.properties file and placed
>>> inside /src/main/resources but i do not see it getting any effect , Please
>>> help as not sure what else needs to be done to change the log at DEBUG or
>>> WARN
>>>
>>
>>
>

Re: Spark Streaming Log4j Inside Eclipse

[Adrian Tanase]

You should set exta java options for your app via Eclipse project and specify 
something like

 -Dlog4j.configuration=file:/tmp/log4j.properties

Sent from my iPhone

On 28 Sep 2015, at 18:52, Shixiong Zhu 
<zsxw...@gmail.com<mailto:zsxw...@gmail.com>> wrote:

You can use JavaSparkContext.setLogLevel to set the log level in your codes.


Best Regards,

Shixiong Zhu

2015-09-28 22:55 GMT+08:00 Ashish Soni 
<asoni.le...@gmail.com<mailto:asoni.le...@gmail.com>>:
I am not running it using spark submit , i am running locally inside Eclipse 
IDE , how i set this using JAVA Code

Ashish

On Mon, Sep 28, 2015 at 10:42 AM, Adrian Tanase 
<atan...@adobe.com<mailto:atan...@adobe.com>> wrote:
You also need to provide it as parameter to spark submit
http://stackoverflow.com/questions/28840438/how-to-override-sparks-log4j-properties-per-driver

From: Ashish Soni
Date: Monday, September 28, 2015 at 5:18 PM
To: user
Subject: Spark Streaming Log4j Inside Eclipse

I need to turn off the verbose logging of Spark Streaming Code when i am 
running inside eclipse i tried creating a log4j.properties file and placed 
inside /src/main/resources but i do not see it getting any effect , Please help 
as not sure what else needs to be done to change the log at DEBUG or WARN

Re: Spark Streaming Log4j Inside Eclipse

[Shixiong Zhu]

I mean JavaSparkContext.setLogLevel. You can use it like this:

JavaStreamingContext jssc = new JavaStreamingContext(sparkConf,
Durations.seconds(2));
jssc.sparkContext().setLogLevel(...);



Best Regards,
Shixiong Zhu

2015-09-29 22:07 GMT+08:00 Ashish Soni <asoni.le...@gmail.com>:

> I am using Java Streaming context and it doesnt have method setLogLevel
> and also i have tried by passing VM argument in eclipse and it doesnt work
>
> JavaStreamingContext jssc = new JavaStreamingContext(sparkConf,
> Durations.seconds(2));
>
> Ashish
>
> On Tue, Sep 29, 2015 at 7:23 AM, Adrian Tanase <atan...@adobe.com> wrote:
>
>> You should set exta java options for your app via Eclipse project and
>> specify something like
>>
>>  -Dlog4j.configuration=file:/tmp/log4j.properties
>>
>> Sent from my iPhone
>>
>> On 28 Sep 2015, at 18:52, Shixiong Zhu <zsxw...@gmail.com> wrote:
>>
>> You can use JavaSparkContext.setLogLevel to set the log level in your
>> codes.
>>
>> Best Regards,
>> Shixiong Zhu
>>
>> 2015-09-28 22:55 GMT+08:00 Ashish Soni <asoni.le...@gmail.com>:
>>
>>> I am not running it using spark submit , i am running locally inside
>>> Eclipse IDE , how i set this using JAVA Code
>>>
>>> Ashish
>>>
>>> On Mon, Sep 28, 2015 at 10:42 AM, Adrian Tanase <atan...@adobe.com>
>>> wrote:
>>>
>>>> You also need to provide it as parameter to spark submit
>>>>
>>>> http://stackoverflow.com/questions/28840438/how-to-override-sparks-log4j-properties-per-driver
>>>>
>>>> From: Ashish Soni
>>>> Date: Monday, September 28, 2015 at 5:18 PM
>>>> To: user
>>>> Subject: Spark Streaming Log4j Inside Eclipse
>>>>
>>>> I need to turn off the verbose logging of Spark Streaming Code when i
>>>> am running inside eclipse i tried creating a log4j.properties file and
>>>> placed inside /src/main/resources but i do not see it getting any effect ,
>>>> Please help as not sure what else needs to be done to change the log at
>>>> DEBUG or WARN
>>>>
>>>
>>>
>>
>

Re: Spark Streaming Log4j Inside Eclipse

[Ashish Soni]

I am not running it using spark submit , i am running locally inside
Eclipse IDE , how i set this using JAVA Code

Ashish

On Mon, Sep 28, 2015 at 10:42 AM, Adrian Tanase <atan...@adobe.com> wrote:

> You also need to provide it as parameter to spark submit
>
> http://stackoverflow.com/questions/28840438/how-to-override-sparks-log4j-properties-per-driver
>
> From: Ashish Soni
> Date: Monday, September 28, 2015 at 5:18 PM
> To: user
> Subject: Spark Streaming Log4j Inside Eclipse
>
> I need to turn off the verbose logging of Spark Streaming Code when i am
> running inside eclipse i tried creating a log4j.properties file and placed
> inside /src/main/resources but i do not see it getting any effect , Please
> help as not sure what else needs to be done to change the log at DEBUG or
> WARN
>

Re: Spark Streaming Log4j Inside Eclipse

[Adrian Tanase]

You also need to provide it as parameter to spark submit
http://stackoverflow.com/questions/28840438/how-to-override-sparks-log4j-properties-per-driver

From: Ashish Soni
Date: Monday, September 28, 2015 at 5:18 PM
To: user
Subject: Spark Streaming Log4j Inside Eclipse

I need to turn off the verbose logging of Spark Streaming Code when i am 
running inside eclipse i tried creating a log4j.properties file and placed 
inside /src/main/resources but i do not see it getting any effect , Please help 
as not sure what else needs to be done to change the log at DEBUG or WARN

Re: Spark Streaming Log4j Inside Eclipse

[Shixiong Zhu]

You can use JavaSparkContext.setLogLevel to set the log level in your codes.

Best Regards,
Shixiong Zhu

2015-09-28 22:55 GMT+08:00 Ashish Soni <asoni.le...@gmail.com>:

> I am not running it using spark submit , i am running locally inside
> Eclipse IDE , how i set this using JAVA Code
>
> Ashish
>
> On Mon, Sep 28, 2015 at 10:42 AM, Adrian Tanase <atan...@adobe.com> wrote:
>
>> You also need to provide it as parameter to spark submit
>>
>> http://stackoverflow.com/questions/28840438/how-to-override-sparks-log4j-properties-per-driver
>>
>> From: Ashish Soni
>> Date: Monday, September 28, 2015 at 5:18 PM
>> To: user
>> Subject: Spark Streaming Log4j Inside Eclipse
>>
>> I need to turn off the verbose logging of Spark Streaming Code when i am
>> running inside eclipse i tried creating a log4j.properties file and placed
>> inside /src/main/resources but i do not see it getting any effect , Please
>> help as not sure what else needs to be done to change the log at DEBUG or
>> WARN
>>
>
>

Spark Streaming Log4j Inside Eclipse

[Ashish Soni]

Hi All ,

I need to turn off the verbose logging of Spark Streaming Code when i am
running inside eclipse i tried creating a log4j.properties file and placed
inside /src/main/resources but i do not see it getting any effect , Please
help as not sure what else needs to be done to change the log at DEBUG or
WARN

Ashish