Python

---

Testing and debugging

Testing Pyspark:

• https://www.confessionsofadataguy.com/introduction-to-unit-testing-with-pyspark/

• test PySpark code with Pytest

---

How to add tests to your data pipeline:

• https://www.startdataengineering.com/post/how-to-add-tests-to-your-data-pipeline/

How to add integration tests to your data pipeline:

• https://www.startdataengineering.com/post/python-datapipeline-integration-test/

---

---