Hello, We are looking for a developer to help us with a small ETL project using Spark and Kubernetes. Here are some of the requirements:
1. We need a REST API to run and schedule jobs. We would prefer this done in Node.js but can be done using Java. The REST API will not be available to the public. 2. We need an easy way to create new jobs in Java without deploying the whole server again. 3. We want jobs deployed/ran using Kubernetes. 4. Must be able to scale to 1000s of ETL jobs. 5. Source for data will be one REST API. 6. Destination for data will be one Couchbase Database cluster. (Couchbase also uses a REST API) 7. I am not sure how many records will be processed per job. 8. The data is mostly sales related data. I know there are commercial ETL solutions that do everything I want. We are looking for something simple and do not need a fancy UI to describe our ETL. We want to use Spark and Java to programmatically describe out ETL jobs. Please let me know if you are interested. Thanks, Warren Bell -- ********************************************************************** This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. They may not be disseminated or distributed to persons or entities other than the ones intended without the authority of the sender. If you have received this email in error or are not the intended recipient, you may not use, copy, disseminate or distribute it. Delete it immediately from your system and notify the sender promptly by email that you have done so. This footnote also confirms that this email message has been scanned for the presence of computer viruses. ********************************************************************** Please consider the environment before printing --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org
