Kafka streams has a lower learning curve and if your source data is in
kafka topics it is pretty simple to integrate it with.
It can run like a library inside your main programs.
So as compared to spark streams
1. Is much simpler to implement.
2. Is not much heavy on hardware unlike spark.
On the downside
1. It is not elastic. You need to anticipate before hand on volume of data
you will have. Very difficult to add and reduce topic partitions later on.
2. The partition key is very important if you need to run multiple
instances of streams application and certain instance processing certain
partitions only.
In case you need aggregation on a different key you may need to
re-partition the data to a new topic and run new streams app against that.
So yes if you have good idea about your data and if it comes from kafka and
you want to build something quick without much hardware kafka streams is a
way to go.
We had first tried spark streaming but given hardware limitation and
complexity of fetching data from mongodb we decided kafka streams as way to
go forward.
Thanks
Sachin
On Wed, Oct 11, 2017 at 1:01 PM, Mich Talebzadeh <mich.talebza...@gmail.com>
wrote:
> Hi,
>
> Has anyone had an experience of using Kafka streams versus Spark?
>
> I am not familiar with Kafka streams concept except that it is a set of
> libraries.
>
> Any feedback will be appreciated.
>
> Regards,
>
> Mich
>
>
>
> LinkedIn *
> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>
>
>
> http://talebzadehmich.wordpress.com
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
