Re: Upgrading findbugs
Findbugs warnings are cleaned up at YARN side. Thanks for the contribution from Li Lu and Varun Saxena! On Thu, Dec 18, 2014 at 2:44 PM, Haohui Mai h...@hortonworks.com wrote: So far we made great progress on fixing findbugs warnings. We're free of findbugs warnings in hdfs, nfs, and a couple other sub projects. There are two findbugs warnings left in hadoop-common. I saw there are some progresses on the YARN side as well. Thanks very much for the contributors (Brandon Li, Li Lu, and many others) that have worked on this. We have a cleaner code base now, and the newer findbugs can help us to catch more issues during pre-commits. :-) I plan to finish the remaining work on in 2.7 timeframe. Thanks again for contribution. Thanks, Haohui On Tue, Dec 9, 2014 at 2:16 AM, Steve Loughran ste...@hortonworks.com wrote: +1 to upgrade. regarding the newly surfacing issues, I'd recommend we look at them and see which are critical problems and fix them. One of the conclusions I got from the building is that there are a lot of javac and javadoc warnings that everyone ignores. Sitting down to fix them is time consuming and doesn't directly fix anything or add new features —but it keeps the code cleaner, which is something we want to encourage -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- Zhijie Shen Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: Upgrading findbugs
+1 to upgrade. regarding the newly surfacing issues, I'd recommend we look at them and see which are critical problems and fix them. One of the conclusions I got from the building is that there are a lot of javac and javadoc warnings that everyone ignores. Sitting down to fix them is time consuming and doesn't directly fix anything or add new features —but it keeps the code cleaner, which is something we want to encourage -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: Upgrading findbugs
Thanks for initiating this, Haohui. +1 to upgrading findbugs version. Inline. On Mon, Dec 8, 2014 at 9:57 PM, Haohui Mai h...@hortonworks.com wrote: Hi, The recent changes on moving to Java 7 triggers a bug in findbug ( http://sourceforge.net/p/findbugs/bugs/918), which causes all pre-commit runs (e.g., HADOOP-11287) to fail. The current version of findbugs (1.3.9) used by Hadoop is released in 2009. Given that: (1) The current bug that we hit are fixed by a later version of findbug. (2) A newer findbug (3.0.0) is required to analyze Hadoop that is compiled against Java 8. (3) Newer findbugs are capable of catching more bugs. :-) Is it a good time to consider upgrading findbugs, which gives us better tools on ensuring the quality of the code case? I ran findbugs 3.0.0 against trunk today. It reported 111 warnings for hadoop-common, 44 for HDFS and 40+ for YARN. Many of them are possible NPEs, resource leaks, and ignored exception which are indeed bugs and are worthwhile to address. However, one issue that needs to be considered is that how to deal with the additional warnings reported by the newer findbugs without breaking the Jenkins pre-commit runs. Personally I can see three possible routes if we decide to upgrade findbugs: (1) Fix all warnings before upgrading to newer findbugs. This might take a while. We might want to use the newer findbugs sooner? (2) Add all new warnings to the exclude list and fix them slowly. I have my doubts on how soon we fix these warnings unless we make the associated JIRAs (assuming we have one per exclude) blockers for the next release. A findbugs Fix It day would be ideal to get this done. (3) Update test-patch.sh to make sure that new code won't introduce any new findbugs warnings. Seems the best, especially if test-patch.sh shows the warnings, but doesn't -1 unless there are new findbugs warnings. This way, the contributor can choose to fix related warnings at the least. I proposed upgrading to findbugs 2.0.2 and fixing new warnings in HADOOP-10476, which could be dated backed to April, 2014. I volunteer to accelerate the effort if it is required. Thoughts? Regards, Haohui -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- Karthik Kambatla Software Engineer, Cloudera Inc. http://five.sentenc.es
