Hi Stephen, Thank you so much for finding time for looking at our examples! Yes, we've tried to implement as clean design of API as possible and are constantly looking for ways to make it even more readable, clear and friendly.
And as Maria already stated we welcome any feedback! On 20/07/14 01:55PM, Stephen Boesch wrote: > I just looked at the examples. > https://github.com/JetBrains/kotlin-spark-api/tree/master/examples/src/main/kotlin/org/jetbrains/spark/api/examples > These look v nice! V concise yet flexible. I like the ability to do > inline *side-effects. *E.g. caching or printing or showDs() > > package org.jetbrains.spark.api.examples > import org.apache.spark.sql.Row > import org.jetbrains.spark.api.* > > fun main() { > withSpark { > val sd = dsOf(1, 2, 3) > sd.createOrReplaceTempView("ds") > spark.sql("select * from ds") > .withCached { > println("asList: ${toList<Int>()}") > println("asArray: ${toArray<Int>().contentToString()}") > this > } > .to<Int>() > .withCached { > println("typed collect: " + (collect() as > Array<Int>).contentToString()) > println("type collectAsList: " + collectAsList()) > } > > dsOf(1, 2, 3) > .map { c(it, it + 1, it + 2) } > .to<Row>() > .select("_1") > .collectAsList() > .forEach { println(it) } > } > } > > > So that shows some of the niceness of kotlin: intuitive type conversion > `to<Int>`/`to<Row>` and `dsOf( list)`- and also the inlining of the side > effects. Overall concise and pleasant to read. > > > On Tue, 14 Jul 2020 at 12:18, Stephen Boesch <java...@gmail.com> wrote: > > > I started with scala/spark in 2012 and scala has been my go-to language > > for six years. But I heartily applaud this direction. Kotlin is more like a > > simplified Scala - with the benefits that brings - than a simplified java. > > I particularly like the simplified / streamlined collections classes. > > > > Really looking forward to this development. > > > > On Tue, 14 Jul 2020 at 10:42, Maria Khalusova <kafoos...@gmail.com> wrote: > > > >> Hi folks, > >> > >> We would love your feedback on the new Kotlin Spark API that we are > >> working on: https://github.com/JetBrains/kotlin-spark-api. > >> > >> Why Kotlin Spark API? Kotlin developers can already use Kotlin with the > >> existing Apache Spark Java API, however they cannot take full advantage of > >> Kotlin language features. With Kotlin Spark API, you can use Kotlin data > >> classes and lambda expressions. > >> > >> The API also adds some helpful extension functions. For example, you can > >> use `withCached` to perform arbitrary transformations on a Dataset and not > >> worry about the Dataset unpersisting at the end. > >> > >> If you like Kotlin and would like to try the API, we've prepared a Quick > >> Start Guide to help you set up all the needed dependencies in no time using > >> either Maven or Gradle: > >> https://github.com/JetBrains/kotlin-spark-api/blob/master/docs/quick-start-guide.md > >> > >> In the repo, you’ll also find a few code examples to get an idea of what > >> the API looks like: > >> https://github.com/JetBrains/kotlin-spark-api/tree/master/examples/src/main/kotlin/org/jetbrains/spark/api/examples > >> > >> We’d love to see your feedback in the project’s GitHub issues: > >> https://github.com/JetBrains/kotlin-spark-api/issues. > >> > >> > >> Thanks! > >> > >> > >> -- Regards, Pasha Big Data Tools @ JetBrains
signature.asc
Description: PGP signature
