bq. t is super easy to change the log level of one particular class or package
If anyone has prod logs laying around collected at INFO level and could provide any counts of what packages the most frequent log messages come from, please attach the data to SOLR-11934. I've seen some cases where just changing the log levels for a single class from INFO to WARN reduced the log volume tremendously (90% or so). That was an app that had a high update volume of a single doc at a time so doesn't reflect other scenarios. I'll be glad to analyze any shareable log files, but understand that sharing log files often violates corporate policy. My suspicion is that there are just a few classes that could be configured at WARN level that would reduce the volume of log messages greatly. That's really a separate question from what level we log messages at, much of the logging in the code is specified at levels suitable for troubleshooting rather than an ops perspective. But that's a discussion for whenever I/we get to SOLR-11934. I think configuring some classes to WARN by default or just providing them commented-out in the log4j2.xml files would be an easy win. ****************** bq. We can find a way to distinguish between dev/prod usage and only warn about file descriptors, suspiciously low heap, lack of SSL etc if Solr is bound to a public IP address. Don't want to bikeshed this too much, so take this for what it's worth. Seeing it for the first time when you deploy to prod is too late IMO. I suppose we're looking through two different lenses. We bounce around from client to client, so my lens is how often people new to Solr and/or operations folks who get the finished app from development miss things like this. And then spend time in panic mode because their prod system broke. And how long we've spent on the phone with them before remembering to check this (it's part of a standard checklist now of course, but still)... And how it tarnishes their opinion of Solr. I'd be less worried about this if we could throw errors that pointed to this issue, but the errors that come out when ulimits are exceeded are obscure and varied. Sure, the same problem can occur if the devs set the variable in solr.in.sh then forget about it and don't tell ops to bump these limits, but at least some of the time the problem will be avoided with the message front and center. Given the cost to devs is to set a single system var I think it's a price worth paying. All IMO and reflecting my experience of course. Best, Erick On Wed, Nov 21, 2018 at 12:40 AM Jan Høydahl <[email protected]> wrote: > > bq. I don't want to see file limit warnings on a dev laptop > > "If you no longer wish to see this warning, set SOLR_ULIMIT_CHECKS to > false in your profile or solr.in.sh" > > Not having these limits set properly has caused weird errors for a lot > of people, but I agree they are annoying at times so there's a way to > shut them off. > > > Note my wording - "on a dev laptop". We can find a way to distinguish between > dev/prod > usage and only warn about file descriptors, suspiciously low heap, lack of > SSL etc if > Solr is bound to a public IP address. > > The only logging config files you should have to worry about are in > ./solr/server/resources..... > > But yeah, there are several Solr JIRAs lying around that I'll break > some time loose _sometime_ to work on to try to make logging more > rational. Unfortunately as Shawn says, there's always tension. When > running "normally", I pretty much only want to see WARN messages. Then > every time I want to diagnose something, I want to see _everything_. > And telling the user to change their logging and call me again next > time the problem happens is not very satisfying for anyone. > > > It is super easy to change the log level of one particular class or package > selectively, that's why we have log4j2.xml and also the Admin's Logging > levels menu. > The occational need to see all of the 150 jar files loaded on startup should > not be > solved by spewing out that for first-time users of Solr who really just care > for whether > the server started OK or not. > > Agree that adding commented-out snippets in the logging config could be a > good way to quickly let people debug "ClassNotFound issues", "Indexing > issues", > "Faceting issues", "Security issues" etc. We could have pre-canned configs > that > would upper the logging of classes of particular interest for each case. > > Jan --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
