I just looked at the examples.
https://github.com/JetBrains/kotlin-spark-api/tree/master/examples/src/main/kotlin/org/jetbrains/spark/api/examples
These look v nice! V concise yet flexible. I like the ability to do
inline *side-effects. *E.g. caching or printing or showDs()
package org.jetbrains.spark.api.examples
import org.apache.spark.sql.Row
import org.jetbrains.spark.api.*
fun main() {
withSpark {
val sd = dsOf(1, 2, 3)
sd.createOrReplaceTempView("ds")
spark.sql("select * from ds")
.withCached {
println("asList: ${toList<Int>()}")
println("asArray: ${toArray<Int>().contentToString()}")
this
}
.to<Int>()
.withCached {
println("typed collect: " + (collect() as
Array<Int>).contentToString())
println("type collectAsList: " + collectAsList())
}
dsOf(1, 2, 3)
.map { c(it, it + 1, it + 2) }
.to<Row>()
.select("_1")
.collectAsList()
.forEach { println(it) }
}
}
So that shows some of the niceness of kotlin: intuitive type conversion
`to<Int>`/`to<Row>` and `dsOf( list)`- and also the inlining of the side
effects. Overall concise and pleasant to read.
On Tue, 14 Jul 2020 at 12:18, Stephen Boesch <java...@gmail.com> wrote:
> I started with scala/spark in 2012 and scala has been my go-to language
> for six years. But I heartily applaud this direction. Kotlin is more like a
> simplified Scala - with the benefits that brings - than a simplified java.
> I particularly like the simplified / streamlined collections classes.
>
> Really looking forward to this development.
>
> On Tue, 14 Jul 2020 at 10:42, Maria Khalusova <kafoos...@gmail.com> wrote:
>
>> Hi folks,
>>
>> We would love your feedback on the new Kotlin Spark API that we are
>> working on: https://github.com/JetBrains/kotlin-spark-api.
>>
>> Why Kotlin Spark API? Kotlin developers can already use Kotlin with the
>> existing Apache Spark Java API, however they cannot take full advantage of
>> Kotlin language features. With Kotlin Spark API, you can use Kotlin data
>> classes and lambda expressions.
>>
>> The API also adds some helpful extension functions. For example, you can
>> use `withCached` to perform arbitrary transformations on a Dataset and not
>> worry about the Dataset unpersisting at the end.
>>
>> If you like Kotlin and would like to try the API, we've prepared a Quick
>> Start Guide to help you set up all the needed dependencies in no time using
>> either Maven or Gradle:
>> https://github.com/JetBrains/kotlin-spark-api/blob/master/docs/quick-start-guide.md
>>
>> In the repo, you’ll also find a few code examples to get an idea of what
>> the API looks like:
>> https://github.com/JetBrains/kotlin-spark-api/tree/master/examples/src/main/kotlin/org/jetbrains/spark/api/examples
>>
>> We’d love to see your feedback in the project’s GitHub issues:
>> https://github.com/JetBrains/kotlin-spark-api/issues.
>>
>>
>> Thanks!
>>
>>
>>
