+1 agree How do we tackle this ? On Sun, Sep 1, 2024 at 7:49 AM Avik Ganguly <a...@fynarfin.io> wrote:
> Still, when eg. 2 tests fail out of the 1100+ on a Github build, it's a >> horror to find out which ones failed, and our best bet is to text-search >> for "FAILED" in the 17000+ lines of output. > > +1 > > Items like this should be high priority in CI improvements epic. The > overall failed test history, flaky test history, log tracing problem > (actual result is often not sufficient to quickly debug and linking logs to > a failed test case is important) were historically connected to getting PRs > merged in time. CircleCI has helped us with the first two parts of this > problem. > > just by splitting up the code as is and moving some classes to different >> project modules is not really taking into account the interdependencies... >> if we don't address those then the project modules are not really helpful >> (actually it's adds more work to the Gradle builds) > > I had overestimated what Fineract modularization design efforts have > achieved. Would be interesting to know more about the design here as > "splitting up the code as is" can be helped with more clarity. > > > On Thu, Aug 8, 2024 at 8:02 PM Aleksandar Vidakovic < > chee...@monkeysintown.com> wrote: > >> Kristof, >> >> ... great overview, so first: thanks for that. Some notes... >> >> Repeatable builds: we should consider this as a major bug and not just >> "gloss over" it by executing the build multiple times... stuff like this >> should halt all current development on other features and everyone (or at >> least one person) should focus on fixing this as soon as possible. I would >> consider this also a show stopper for a release. >> >> Dirty checks: we were actually at one point (2+ years?) not too bad here, >> but it seems that this is now actually worse than before... my bet is that >> we introduced some performance hogs by splitting up the code base into >> separate project modules; this didn't really fix the main problem of the >> source code (too many interdependencies between packages aka "modules"); >> just by splitting up the code as is and moving some classes to different >> project modules is not really taking into account the interdependencies... >> if we don't address those then the project modules are not really helpful >> (actually it's adds more work to the Gradle builds), neither for publishing >> them on Maven central nor chopping up the code into smaller pieces and >> taking full advantage of caching. >> >> Caching: disabling caching is just not such a good idea... at all... >> again: I would consider this a major bug and halt all machines until it's >> fixed; also: consider this a show stopper for releases. >> >> Console readability: we've talked about this already on occasions (there >> might be even a mailing list message)... on every upgrade of the >> dependencies (including the quality assurance tools like Errorprone etc.) >> the screws get tightened (read: rules get stricter); I remember that after >> one upgrade (maybe last year, maybe longer ago) I had to disable some rule >> enforcements, because the reported style and quality checks literally >> exploded; holding back on Errorprone (and similar) upgrades is obviously >> not a solution; I tried to be conservative (just disabled those that >> brought the build to a halt aka that were reported as errors), but as we >> can see there are enough warnings left to drive everyone crazy; btw: >> disabling the rules with certain annotations is neither a solution, that's >> also avoiding to fix the quality issues. >> >> Suggestions: >> >> - let's make an effort to reduce the interdependencies between >> packages aka "modules" before we rip the code apart and move it to >> separate >> project modules; if we can't do that (backwards compatibility, not allowed >> to "break eggs" etc.) then let's leave the code in one place as is... I >> see >> no added value of having classes in different jar files when it hurts >> build >> performance >> - let's spend time and fix those style and quality issues, not just >> pile feature on feature on feature... I am pretty sure that everyone can >> grab a handful of those warnings and fix them as part of their PRs (even >> if >> it's not one that is explicitly dedicated to quality assurance), in other >> words: everyone look a bit left and right to see what ELSE you can fix/add >> in addition to your tasks (chores vs cherry picking)... fixing everything >> in one PR is obviously not practical; we'll have a thousand files >> changed... this will interrupt everyone else quite significantly and is a >> nightmare to review >> - there might be less than a handful of Gradle build constraints that >> are again too paranoid and mess up the build; I think it's worth looking >> at >> the OpenAPI yaml build... that one had issues in the past with proper >> dirty >> checks, but there might be other/new constraints that are equally >> unhelpful >> >> My 2 cents >> >> Aleks >> >> >> >> On Thu, Aug 8, 2024 at 3:47 PM Kristof Jozsa <kristof.jo...@gmail.com> >> wrote: >> >>> Hi all, >>> >>> After working with Fineract for some months now, I collected several >>> issues related to the Gradle build system we have in place. I categorized >>> my pain points, and would like to initiate a discussion on the topic with >>> the community. My usage scenarios include building locally with >>> IntelliJ+Gradle wrapper, building locally from CLI and using the Github >>> build from the web UI. >>> >>> The problems I gathered fall into the following categories: >>> >>> - *REPEATABLE BUILDS:* starting Fineract from IntelliJ always fails for >>> the first time with: >>> Caused by: java.lang.IllegalArgumentException: Couldn't bind >>> FineractProperties to the 'fineract' property >>> and simply trying it again always makes it wor0k. >>> >>> - *DIRTY CHECKS*: modifying a single file (eg. put a newline character >>> into the Loan class) and restarting Fineract makes it "build" for 17-18 >>> seconds (checking all modules to be up-to-date) before starting it again. >>> This happens on my 16 core Intel i7 HQ CPU with 64Gb of RAM, eating up to >>> 800% CPU and spinning up all the vents in my rig while doing so. >>> >>> - *INTEGRATION TESTS*: the same dirty check problem happens when >>> working with tests as well. Adding a blank line to an existing test class >>> and re-running the test again happens after a ~20 seconds "rebuild" >>> process. This leads to painfully slow code-test cycles at local >>> development. >>> >>> - *RESOURCE REQUIREMENTS*: local team reports that in cases the Gradle >>> daemon repeatedly grows up to 8Gb memory usage. That causes issues with dev >>> boxes having 16Gb or even less RAM.. Even using my high-spec dev box, I >>> needed to add GRADLE_OPTS="-Dorg.gradle.workers.max=12 >>> -Dorg.gradle.priority=low" to keep my desktop responsible during local >>> builds. >>> >>> - *CACHING*: I see many team members and mailing list members >>> recommending `--no-build-cache --no-daemon` settings when invoking >>> gradle, to avoid issues with our build. This kind of denies even any >>> potential adventages of using Gradle over more stable build tools. Note, >>> that I do not use these settings (so my issues are definetely not caused by >>> this), but I keep getting this recommendation for a reason. >>> >>> - *CONSOLE READABILITY*: I also see many fellow devs suggesting running >>> gradle with `--console=plain` to get some chance to read the logs in a >>> sensible way. Still, when eg. 2 tests fail out of the 1100+ on a Github >>> build, it's a horror to find out which ones failed, and our best bet is to >>> text-search for "FAILED" in the 17000+ lines of output. >>> >>> - *REPORTING COMPILE ERRORS*: related to the previous point, check >>> https://github.com/kjozsa/fineract/actions/runs/10302318345/job/28515733682 >>> for a perfect example. Here at line 1489 Gradle shows >>> > Task :fineract-loan:compileJava FAILED >>> and shows a couple of warnings below, but the actual error is shown at >>> line 1393 which appears to be part of >>> > Task :fineract-document:modernizer >>> which is clearly misleading, and your best bet to find the actual error >>> on Github is to search for "error: ", but NOT using Ctrl+F, but the >>> search box at the top right corner. That's ridiculous - a sensible build >>> tool should communicate build errors in a straightforward way. >>> >>> I'd like to ask you to share your experiences and pain points with the >>> current build that we have and use every day. Once we have a clearer >>> understanding of the areas that need improvement, we can discuss potential >>> strategies to revise our build system and ensure it meets basic >>> requirements.To be fair, please also make sure you share any recent >>> positive experiences related to Fineract builds, if you have any. >>> >>> Thanks, >>> Kristof >>> >>>
