Re: Testing Apache Spark applications

2018-11-15 Thread Lars Albertsson
My previous answers to this question can be found in the archives, along with some other responses: http://apache-spark-user-list.1001560.n3.nabble.com/testing-frameworks-td32251.html https://www.mail-archive.com/user%40spark.apache.org/msg48032.html I have made a couple of presentations on the

Re: Testing Apache Spark applications

2018-11-15 Thread Vitaliy Pisarev
Hard to answer in a succinct manner but I'll give it a shot. Cucumber is a tool for writing *Behaviour* Driven Tests (closely related to behaviour driven development, BDD). It is not a mere *technical* approach to testing but a mindset, a way of work and a different (different, whether it is

Re: Testing Apache Spark applications

2018-11-15 Thread ☼ R Nair
Sparklens from qubole is a good source. Other tests are to be handled by developer. Best, Ravi On Thu, Nov 15, 2018, 12:45 PM Hi all, > > > > How are you testing your Spark applications? > > We are writing features by using Cucumber. This is testing the behaviours. > Is this called functional

Testing Apache Spark applications

2018-11-15 Thread Omer.Ozsakarya
Hi all, How are you testing your Spark applications? We are writing features by using Cucumber. This is testing the behaviours. Is this called functional test or integration test? We are also planning to write unit tests. For instance we have a class like below. It has one method. This methos