> Can you explain more about how the conversion process works? I added conversion guidelines to the PR that only adds Kotlin support without converting anything < https://github.com/apache/trafficcontrol/pull/7071>. Don't use the GitHub preview since it does not render the code blocks or images. You can see a compiled preview of the Kotlin guidelines at < https://atc.zrhoffman.net/kotlin>.
If anyone is willing to read through the guidelines to see if it makes sense (or, better yet, go through those guidelines, convert a small class, and report back with how it went), that would help immensely. > What are the risks associated with it (likelihood of introducing a bug, risks of having mixed Java+Kotlin, etc...). When following the appropriate steps to convert the codebase from Kotlin to Java, there is little risk of introducing a new bug, because the code will not even compile unless you have examined the conversion thoroughly. For example, if one were to convert the `TrafficRouter.java` class only to Kotlin, Kotlin would produce 3 errors and 24 warnings needing to be resolved. However, if one follows the documented steps by converting the whole TR codebase to Kotlin, keeping `TrafficRouter.kt`, and reverting the rest of the newly-converted classes, Kotlin finds 240 errors and 49 warnings needing to be resolved, nearly all of which have to do with null safety ambiguities (see a diff between these 2 methods for `TrafficRouter.kt` at < https://github.com/zrhoffman/trafficcontrol/commit/tr-kotlin-reliable-conversion>). So - The resultant difference between converting a single class and following the guidelines is night and day - If not very much care was taken, the code will not even compile, tests will not pass, the result will not be PRed as-is, and new bugs will not be introduced. That's not to discourage people from taking on an ambitious task like converting the entire TrafficRouter class to Kotlin, but it takes more care than a button click, and trying smaller classes first is probably more realistic. Besides the stability that Kotlin itself brings, because we already have good unit and integration test coverage in Traffic Router, any code that does manage to compile but is not a successful conversion will almost certainly fail tests. > risks of having mixed Java+Kotlin, etc... No risk at runtime. The Java bytecode compiled from Java and Kotlin are indistinguishable at runtime, and simply having added Kotlin support does not change any existing Java code. Also, when using a Kotlin class from Java, there is no perceptible difference from a developer's perspective. > Also, how long could the community expect the transition to take and what are the mechanics (separate branch, etc...)? Because Java and Kotlin are totally interoperable, conversion PRs targeting a branch other than the master branch does not make sense. Classes can be converted incrementally, one per PR (or a few, in the case of very small classes), and at no point does that leave Traffic Router in a half-baked state before all classes are converted. Under a focused effort to convert Traffic Router, converting the open source TR codebase could be completed in a matter of months. A less-focused conversion effort could take perhaps a year, but regardless, if any internal forks of Traffic Router exist, they should continue to work under either a partial or complete Kotlin rewrite. -Zach On Wed, Jun 23, 2021 at 7:44 PM Eric Friedrich <fri...@apache.org> wrote: > Rawlin, ocket8888- > > Thanks for pointing out some of the additional benefits Kotlin has over > Java- I didn't know about the null check differences. > > > Can you explain more about how the conversion process works? > > What are the risks associated with it (likelihood of introducing a bug, > risks of having mixed Java+Kotlin, etc...). > > > Also, how long could the community expect the transition to take and what > are the mechanics (separate branch, etc...)? > > > --Eric > > > > On Wed, Jun 23, 2021 at 2:37 PM ocket 8888 <ocket8...@gmail.com> wrote: > > > As someone who avoids looking at TR code as much as possible, I'd say > it's > > certainly true that the codebase is a rat's nest, and switching to Kotlin > > won't automagically fix that. I do think, though, that it being Java > > exacerbates the problem because of how verbose Java is. I, personally, > > would not want to refactor TR's Java codebase; that just doesn't sound > like > > fun to me. If it were Kotlin, though, I'd be much more willing to try to > > improve it. > > > > On Wed, Jun 23, 2021 at 10:15 AM Rawlin Peters <raw...@apache.org> > wrote: > > > > > > Rather, I think the TR code itself is badly in need of a clean up, > > > to improve modularity and to give some thought to extensibility. > > > > > > I agree, but it's very difficult to make a business case for "improve > > > modularity and extensibility" without a concrete use case behind it > > > that can justify doing that refactoring beforehand. Whereas converting > > > from Java to Kotlin can be done fairly easily on a piecemeal basis > > > over time by devs in their free time. > > > > > > > The solution to that is not easy, nor is it fast. It's a process of > > > making > > > small, incremental improvements to the cleanliness of TR with minimal > > risk > > > to stability. > > > > > > That actually sounds like what would happen during a conversion to > > > Kotlin. In fact, that's basically what I did when I converted > > > TrafficRouter.java to Kotlin yesterday: > > > https://github.com/rawlinp/trafficcontrol/commits/kotlin-tr. The > > > auto-conversion was pretty straightforward -- there were only 3 errors > > > I had to fix manually for it to compile. The rest was just fixing most > > > of the warnings/suggestions pointed out by IntelliJ, most of which > > > helped clean things up a bit, making it more readable and less > > > verbose. Combing over the results of the conversion like that is also > > > a great way to find even more things that can be cleaned up, which in > > > most cases would go ignored because no one is just combing over TR > > > looking for things to clean up. > > > > > > > Somewhat less of a concern is shrinking the pool of developers by > > > choosing > > > a less mature, less widely adopted language. I understand Kotlin is on > > the > > > rise and is in demand but is still dwarfed by the 20+ years of Java. > > > > > > I agree this is also less of a concern -- I think Java devs can learn > > > Kotlin pretty quickly because Kotlin is basically just an enhanced > > > version of Java. Also, the community did choose to replace the > > > original Traffic Ops with a language 20+ years it's junior, and that > > > was a tremendous amount of work compared to what converting Java to > > > Kotlin would be. > > > > > > One of the main tenets of a CDN is reliability, and all of the > > > features around null safety in Kotlin > > > (https://kotlinlang.org/docs/null-safety.html) can seriously help out > > > there, especially if the entire codebase was Kotlin. While the > > > codebase is a mix of the two, there is more chance of a > > > NullPointerException coming from Java code because all Java references > > > are nullable, and Kotlin needs to maintain that interoperability. Some > > > of the nastier bugs in TR have stemmed from NullPointerExceptions due > > > to not knowing that an argument was nullable, and with Kotlin's > > > built-in nullable vs non-nullable types, that problem is way less > > > likely to occur. > > > > > > If we were to develop TR 2.0 for instance (instead of just converting > > > TR 1.0), I'd hope we could use a language like Kotlin which has solved > > > the Billion Dollar Mistake. > > > > > > - Rawlin > > > > > > On Tue, Jun 22, 2021 at 9:14 PM Eric Friedrich <fri...@apache.org> > > wrote: > > > > > > > > Zach- > > > > > > > > First of all, thanks for trying to build a bigger tent here, I > really > > > > appreciate that effort. > > > > > > > > (Personal comments, not as chair) > > > > > > > > I agree that Traffic Router is difficult to work on, and that > presents > > a > > > > large barrier to entry. However, I don't think that is a result of > > > language > > > > choice. Rather, I think the TR code itself is badly in need of a > clean > > > up, > > > > to improve modularity and to give some thought to extensibility. > > > > > > > > Recently one of my coworkers added a new feature to TR. He is > > incredibly > > > > experienced with Java, but had many problems trying to create a clean > > > > integration. > > > > > > > > Converting TR to Kotlin wouldn't improve or solve any of these > issues. > > I > > > > fear that even as we try to attract new contributors, they would be > > > > frustrated with the state of TR and not continue as ongoing > > contributors. > > > > > > > > The solution to that is not easy, nor is it fast. It's a process of > > > making > > > > small, incremental improvements to the cleanliness of TR with minimal > > > risk > > > > to stability. > > > > > > > > > > > > Somewhat less of a concern is shrinking the pool of developers by > > > choosing > > > > a less mature, less widely adopted language. I understand Kotlin is > on > > > the > > > > rise and is in demand but is still dwarfed by the 20+ years of Java. > > > > > > > > --Eric > > > > > > > > > > > > > > > > On Tue, Jun 22, 2021, 7:14 PM Chris Lemmons <alfic...@gmail.com> > > wrote: > > > > > > > > > I don't have a strong opinion on Kotlin; I haven't used it to any > > real > > > > > degree. I do know it interoperates with the Java API and compiles > > down > > > > > to the JVM. So in terms of the openness of the platform, it's not > any > > > > > more "open" than Oracle's OpenJDK. It does compile down to native > > > > > code, but then we'd need to replace (or convert and maintain) all > our > > > > > dependent libraries. > > > > > > > > > > So if Kotlin helps with maintainability, I say we go for it. I > don't > > > > > know that we're winning any additional openness we don't already > > have, > > > > > though. > > > > > > > > > > On Tue, Jun 22, 2021 at 4:44 PM Taylor Frey <taylor.f...@gmail.com > > > > > wrote: > > > > > > > > > > > > I too +1 > > > > > > > > > > > > My experience porting a Java project to Kotlin was specific to > > > Android in > > > > > > the past (re: years ago). I recall my general impression of > Kotlin > > to > > > > > have > > > > > > some extraneous and unnecessary syntactic sugar (What's up with > the > > > > > > for/loop keywords?), but it was an overall solid language. I > > > appreciate > > > > > the > > > > > > terseness and brevity of things like variable declarations w/ > > > implicit > > > > > data > > > > > > types and `data classes`, which addresses a long standing > complaint > > > about > > > > > > Java's overly verbose syntax. Language aside, the automated > porting > > > back > > > > > > then wasn't exactly perfect, but a few things have changed since > > > then; 1) > > > > > > the language specification has solidified over the years, 2) > > tooling > > > has > > > > > > matured, and 3) is manageable to manually intervene if necessary. > > > > > > > > > > > > > As a side note, because of Oracle's aggressive legal practices, > > and > > > > > > because > > > > > > they really do want you to think that they own Java (see: > > Google > > > LLC > > > > > v. > > > > > > Oracle America, Inc.), using Java in Apache Traffic Control > is > > > not > > > > > > really > > > > > > in the spirit of free software. Kotlin is an Apache > > 2.0-licensed > > > > > > project, > > > > > > so switching to Kotlin would be a true FOSS win for the ATC > > > > > community. > > > > > > > > > > > > I think the true benefit is moving away from the Java API, which > is > > > under > > > > > > such contention, without changing underlying implementation (such > > as > > > a > > > > > > transition to another, non-JVM, language might incur). > > > > > > > > > > > > On Tue, Jun 22, 2021 at 11:42 AM Chatterjee, Srijeet > > > > > > <srijeet_chatter...@comcast.com.invalid> wrote: > > > > > > > > > > > > > +1 > > > > > > > Sounds like a fun project! > > > > > > > > > > > > > > --Srijeet > > > > > > > > > > > > > > On 6/21/21, 8:43 PM, "Zach Hoffman" <zrhoff...@apache.org> > > wrote: > > > > > > > > > > > > > > Hi Traffic Control! > > > > > > > > > > > > > > As you all may know, the current implementation of Traffic > > > Router > > > > > runs > > > > > > > on > > > > > > > Java 8. While coding directly in Java is still done, there > > are > > > some > > > > > > > reasons > > > > > > > to be critical of Java. > > > > > > > > > > > > > > Java often has very verbose syntax when trying to > accomplish > > > simple > > > > > > > things. > > > > > > > This has made the Traffic Router codebase grow larger over > > > time, > > > > > > > leading to > > > > > > > maintainability issues. Also, Java suffers from lack of > > > > > compile-time > > > > > > > safety > > > > > > > in some ways that more modern languages do not. For these > > > reasons, > > > > > and > > > > > > > because Java is a very old language, some developers who > may > > > > > otherwise > > > > > > > be > > > > > > > interested in contributing to Apache Traffic Control may be > > > > > > > disinterested > > > > > > > contributing to Traffic Router in particular, simply > because > > > they > > > > > want > > > > > > > to > > > > > > > work with more modern languages that do not have the issues > > > that > > > > > Java > > > > > > > does. > > > > > > > > > > > > > > Kotlin, a JVM language devised by Jetbrains with the goal > of > > > > > tackling > > > > > > > these > > > > > > > problems, has exploded in popularity in recent years, and > > > Kotlin is > > > > > > > becoming a standard way to use the JVM, with Google > choosing > > > it as > > > > > the > > > > > > > preferred language to for developing Android apps. More > > > > > importantly, > > > > > > > as the > > > > > > > Intellij IDEA Kotlin plugin lets you convert Java sources > to > > > Kotlin > > > > > > > (either > > > > > > > at the file level or an entire project, or anywhere in > > > between), > > > > > > > Kotlin has > > > > > > > become a low-effort way to decrease tech debt across large > > Java > > > > > > > projects. > > > > > > > > > > > > > > Back to Traffic Router: In order to gauge how we would > > benefit > > > from > > > > > > > converting Traffic Router to Kotlin (it really is a matter > of > > > > > > > *converting*, > > > > > > > not rewriting), I converted all of the tests in the Traffic > > > Router > > > > > > > "Core" > > > > > > > module to Kotlin. You can see the result here (as you can > see > > > from > > > > > > > GitHub > > > > > > > Actions, the TR tests all pass): < > > > > > > > > > > > > > > > > > > > > > > > > > https://urldefense.com/v3/__https://github.com/zrhoffman/trafficcontrol/commits/tr-kotlin__;!!CQl3mcHX2A!UCz457i9stX5A-AP8jvvWo21Rqa43JRsC1jQt_fo5nb07raZxByUR9gAGVPZTI4tGhalAel2$ > > > > > > > > > > > > > > > > > > > > > > Some observations: > > > > > > > - Without any putting any effort into optimizing the > existing > > > code, > > > > > > > lines > > > > > > > of code shrunk from 10572 in Java to 10217 in Kotlin. See: > < > > > > > > > > > > > > > > > > > > > > > > > > > https://urldefense.com/v3/__https://github.com/zrhoffman/trafficcontrol/commit/5add06bccc__;!!CQl3mcHX2A!UCz457i9stX5A-AP8jvvWo21Rqa43JRsC1jQt_fo5nb07raZxByUR9gAGVPZTI4tGqGQTevX$ > > > > > > > > > > > > > > > - Because it now includes the Kotlin standard library, the > > > Traffic > > > > > > > Router > > > > > > > RPM size increases from 48MB to 58MB. > > > > > > > - The converted sources benefit from Kotlin's null safety > and > > > type > > > > > > > safety > > > > > > > (avoiding wildcard types). See < > > > > > > > > > > > > > > > > > > > > > > > > > https://urldefense.com/v3/__https://github.com/zrhoffman/trafficcontrol/commit/0c2eec39d9__;!!CQl3mcHX2A!UCz457i9stX5A-AP8jvvWo21Rqa43JRsC1jQt_fo5nb07raZxByUR9gAGVPZTI4tGjjMNj-F$ > > > > > > > > (Kotlin > > > > > > > error resolution) and < > > > > > > > > > > > > > > > > > > > > > > > > > https://urldefense.com/v3/__https://github.com/zrhoffman/trafficcontrol/commit/d8a0fd3bca__;!!CQl3mcHX2A!UCz457i9stX5A-AP8jvvWo21Rqa43JRsC1jQt_fo5nb07raZxByUR9gAGVPZTI4tGoysImMb$ > > > > > > > > (Kotlin > > > > > > > warning resolution) for the changes involved. > > > > > > > > > > > > > > As a side note, because of Oracle's aggressive legal > > > practices, and > > > > > > > because > > > > > > > they really do want you to think that they own Java (see: > > > Google > > > > > LLC v. > > > > > > > Oracle America, Inc.), using Java in Apache Traffic Control > > is > > > not > > > > > > > really > > > > > > > in the spirit of free software. Kotlin is an Apache > > > 2.0-licensed > > > > > > > project, > > > > > > > so switching to Kotlin would be a true FOSS win for the ATC > > > > > community. > > > > > > > > > > > > > > So, converting the rest of Traffic Router would be > relatively > > > fast > > > > > and > > > > > > > easy, would result in *no* functionality gained or lost, > > would > > > > > leave TR > > > > > > > usage totally unchanged, would make development more > > enjoyable > > > for > > > > > > > current > > > > > > > TR devs, and would maybe catch the interest of additional > > devs > > > who > > > > > > > otherwise may not be interested in Traffic Router. > > > > > > > > > > > > > > How does this sound? > > > > > > > > > > > > > > -Zach > > > > > > > > > > > > > > > > > > > > > > > > >
