Learning Patterns Your Source for Quality Technology Courseware

Introduction to Spark 3 with Python: Lab Setup Instructions (Linux: Your Environment)

Below are the standard requirements for this course. If you have any questions or issues, please contact us.

Important Note: Student lab files are required on each computer used for the course. The links for these are not in this lab setup, and you should receive them separately.

Other notes:

Hardware and classroom setup.

Each student and the instructor shall have a work environment that fulfills the listed requirements.

Lab Files: Each student and instructor must have lab files installed (links to these files are generally sent separately via e-mail).

Other instructor requirements for the classroom

Install Java Development Kit – JDK 11 (11.0.x)

Install Python 3.9

PySpark Environment Setup

export PYSPARK_PYTHON=/usr/bin/python3.9
export SPARK_LABS=$HOME/spark-labs-python