Felix Kizhakkel Jose created SPARK-32583:
--------------------------------------------
Summary: PySpark Structured Streaming Testing Support
Key: SPARK-32583
URL: https://issues.apache.org/jira/browse/SPARK-32583
Project: Spark
Issue Type: Question
Components: PySpark
Affects Versions: 2.4.4
Reporter: Felix Kizhakkel Jose
Hello,
I have investigated a lot but couldn't get any help or resource on how to
+{color:#172b4d}*test my pyspark Structured Streaming pipeline job*{color}+
(ingesting from Kafka topics to S3) and how to build Continuous Integration
(CI)/Continuous Deployment (CD).
1. Is it possible to test (unit test, integration test) pyspark structured
stream?
2. How to build Continuous Integration (CI)/Continuous Deployment (CD)?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
