Re: Silencing messages from Ivy when calling spark-submit

[Nicholas Chammas]

Looks like the only way to change the Ivy log setting, given how we're
using it in Spark, is via code
.
This makes me think: People often have trouble

(77K
views) silencing undesired output from Spark.

Would it make sense to have a convenience option in spark-submit that lets
you set a global log level for Ivy, log4j, and anything else? i.e. Without
needing to edit any separate files or look at other projects' documentation.

It sure would be handy for everyday use, but I wonder if there is a design
reason we wouldn't want to add something like that.

On Tue, Mar 6, 2018 at 1:14 PM Bryan Cutler  wrote:

> Cool, hopefully it will work.  I don't know what setting that would be
> though, but it seems like it might be somewhere under here
> http://ant.apache.org/ivy/history/latest-milestone/settings/outputters.html.
> It's pretty difficult to sort through the docs, and I often found myself
> looking at the source to understand some settings.  If you happen to figure
> out the answer, please report back here.  I'm sure others would find it
> useful too.
>
> Bryan
>
> On Mon, Mar 5, 2018 at 3:50 PM, Nicholas Chammas <
> nicholas.cham...@gmail.com> wrote:
>
>> Oh, I didn't know about that. I think that will do the trick.
>>
>> Would you happen to know what setting I need? I'm looking here
>> , but
>> it's a bit overwhelming. I'm basically looking for a way to set the overall
>> Ivy log level to WARN or higher.
>>
>> Nick
>>
>> On Mon, Mar 5, 2018 at 2:11 PM Bryan Cutler  wrote:
>>
>>> Hi Nick,
>>>
>>> Not sure about changing the default to warnings only because I think
>>> some might find the resolution output useful, but you can specify your own
>>> ivy settings file with "spark.jars.ivySettings" to point to your
>>> ivysettings.xml file.  Would that work for you to configure it there?
>>>
>>> Bryan
>>>
>>> On Mon, Mar 5, 2018 at 8:20 AM, Nicholas Chammas <
>>> nicholas.cham...@gmail.com> wrote:
>>>
 I couldn’t get an answer anywhere else, so I thought I’d ask here.

 Is there a way to silence the messages that come from Ivy when you call
 spark-submit with --packages? (For the record, I asked this question on
 Stack Overflow .)

 Would it be a good idea to configure Ivy by default to only output
 warnings or errors?

 Nick
 ​

>>>
>>>
>

Re: Silencing messages from Ivy when calling spark-submit

[Bryan Cutler]

Cool, hopefully it will work.  I don't know what setting that would be
though, but it seems like it might be somewhere under here
http://ant.apache.org/ivy/history/latest-milestone/settings/outputters.html.
It's pretty difficult to sort through the docs, and I often found myself
looking at the source to understand some settings.  If you happen to figure
out the answer, please report back here.  I'm sure others would find it
useful too.

Bryan

On Mon, Mar 5, 2018 at 3:50 PM, Nicholas Chammas  wrote:

> Oh, I didn't know about that. I think that will do the trick.
>
> Would you happen to know what setting I need? I'm looking here
> , but
> it's a bit overwhelming. I'm basically looking for a way to set the overall
> Ivy log level to WARN or higher.
>
> Nick
>
> On Mon, Mar 5, 2018 at 2:11 PM Bryan Cutler  wrote:
>
>> Hi Nick,
>>
>> Not sure about changing the default to warnings only because I think some
>> might find the resolution output useful, but you can specify your own ivy
>> settings file with "spark.jars.ivySettings" to point to your
>> ivysettings.xml file.  Would that work for you to configure it there?
>>
>> Bryan
>>
>> On Mon, Mar 5, 2018 at 8:20 AM, Nicholas Chammas <
>> nicholas.cham...@gmail.com> wrote:
>>
>>> I couldn’t get an answer anywhere else, so I thought I’d ask here.
>>>
>>> Is there a way to silence the messages that come from Ivy when you call
>>> spark-submit with --packages? (For the record, I asked this question on
>>> Stack Overflow .)
>>>
>>> Would it be a good idea to configure Ivy by default to only output
>>> warnings or errors?
>>>
>>> Nick
>>> ​
>>>
>>
>>

Re: Silencing messages from Ivy when calling spark-submit

[Nicholas Chammas]

Oh, I didn't know about that. I think that will do the trick.

Would you happen to know what setting I need? I'm looking here
, but
it's a bit overwhelming. I'm basically looking for a way to set the overall
Ivy log level to WARN or higher.

Nick

On Mon, Mar 5, 2018 at 2:11 PM Bryan Cutler  wrote:

> Hi Nick,
>
> Not sure about changing the default to warnings only because I think some
> might find the resolution output useful, but you can specify your own ivy
> settings file with "spark.jars.ivySettings" to point to your
> ivysettings.xml file.  Would that work for you to configure it there?
>
> Bryan
>
> On Mon, Mar 5, 2018 at 8:20 AM, Nicholas Chammas <
> nicholas.cham...@gmail.com> wrote:
>
>> I couldn’t get an answer anywhere else, so I thought I’d ask here.
>>
>> Is there a way to silence the messages that come from Ivy when you call
>> spark-submit with --packages? (For the record, I asked this question on
>> Stack Overflow .)
>>
>> Would it be a good idea to configure Ivy by default to only output
>> warnings or errors?
>>
>> Nick
>> ​
>>
>
>

Re: Silencing messages from Ivy when calling spark-submit

[Bryan Cutler]

Hi Nick,

Not sure about changing the default to warnings only because I think some
might find the resolution output useful, but you can specify your own ivy
settings file with "spark.jars.ivySettings" to point to your
ivysettings.xml file.  Would that work for you to configure it there?

Bryan

On Mon, Mar 5, 2018 at 8:20 AM, Nicholas Chammas  wrote:

> I couldn’t get an answer anywhere else, so I thought I’d ask here.
>
> Is there a way to silence the messages that come from Ivy when you call
> spark-submit with --packages? (For the record, I asked this question on
> Stack Overflow .)
>
> Would it be a good idea to configure Ivy by default to only output
> warnings or errors?
>
> Nick
> ​
>

Silencing messages from Ivy when calling spark-submit

[Nicholas Chammas]

I couldn’t get an answer anywhere else, so I thought I’d ask here.

Is there a way to silence the messages that come from Ivy when you call
spark-submit with --packages? (For the record, I asked this question on
Stack Overflow .)

Would it be a good idea to configure Ivy by default to only output warnings
or errors?

Nick
​