I'm considering whether or not it is worth introducing Spark at my new company. The data is no-where near Hadoop size at this point (it sits in an RDS Postgres cluster).
I'm wondering at which point it is worth the overhead of adding the Spark infrastructure (deployment scripts, monitoring, etc).
