Spark and Machine Learning at Scale Training

Chapter 1: Introduction to Spark

• Big Data and the Analytics Process
• What is Big Data?
• Volume
• Velocity
• Variety
• Veracity
• Too large to fit into memory
• Big data and analytic process
• Scaling and Distributed Computing
• How to Actually Scale?
• Bring the Data to the Compute
• Bring the Compute to the Data
• Introduction to the Spark Platform
• History of Spark and Hadoop
• Spark vs. Hadoop MapReduce
• Supported Languages
• Pandas API on Spark
• Spark Architecture: Cluster Manager
• Standalone cluster manager
• Apache Hadoop YARN
• Apache Mesos
• Spark Architecture: Driver Process
• Spark Architecture: Executor Process and Workers
• Spark Building Blocks
• Spark SQL and the Catalyst

Chapter 2: Introduction to Spark – Setting up a Spark Environment

•  Set Up On-Premise Spark Environment (Ubuntu 20.04, Docker)
• Set Up DataBricks Community Cloud and Compute Cluster
• Set Up EMR Cluster and Attach Notebook

Chapter 3: Basic Spark Operations and Transformations

• Spark Session and Context
• Loading Data
• Actions and Transformations
• More on Actions in Spark
• More on Transformations in Spark
• Persistence and Caching

Chapter 4 – Introduction to Spark SQL

• What is Spark SQL?
• Uniform Data Access with Spark SQL
• Integration with cloud storage
• Using JDBC Sources
• Hive Integration
• What is a DataFrame?
• Creating a DataFrame in PySpark
• Commonly Used DataFrame Methods and Properties in PySpark
• Grouping and Aggregation in PySpark
• The \”DataFrame to RDD\” Bridge in PySpark
• The SQLContext Object
• Examples of Spark SQL / DataFrame (PySpark Example)
• Converting an RDD to a DataFrame Example
• Example of Reading / Writing a JSON File
• Performance, Scalability, and Fault-tolerance of Spark SQL

Chapter 5: Spark’s ML libraries – Lecture: Introduction to Spark’s ML libraries

• Spark MLlib
• Algorithms
• Classification
• Binary Classification
• Multi-Class Classification
• Multi-Label Classification
• Imbalanced Classification
• Regression
• Linear Regression
• Simple Linear Regression
• Multiple Linear Regression
• Polynomial Regression
• Support Vector Regression
• Decision Tree Regression
• Random Forest Regression
• Feature Engineering
• TF-IDF – PySpark example
• Word2Vec – PySpark example
• Count Vectorizer – PySpark example
• Feature Transformers of Spark MLlib
• Tokenizer – PySpark example
• Stopwords Remover
• Stopwords Remover – PySpark example
• N-gram – PySpark example
• Binarizer – PySpark example
• Principal Component Analysis
• What is PCA used for?
• Advantages and disdvantagesof PCA
• PCA – PySpark example
• String Indexing – PySpark example
• Why One-Hot Encoding is used for nominal data?
• One-Hot Encoding – PySpark Example
• Bucketizer – PySpark example
• Standardization and Normalization
• Difference between Standardization and Normalization
• Standard Scaler
• Robust Scaler
• Min Max Scaler
• Max Abs Scaler
• Imputer
• Feature Selectors in Spark MLlib
• Vector Slicer – PySpark example
• Chi-Squared selection – PySpark example
• Univariate Feature Selector
• Variance Threshold Selector
• Locality Sensitive Hashing
• Locality Sensitive Hashing in Spark MLlib
• LSH Operations
• Locality Sensitive Hashing in Spark MLlib
• Bucketed Random Projection for Euclidean Distance
• MinHash for Jaccard Distance
• Pipeline
• Transformer
• Estimator
• Persistence
• Introduction to Hyperparameter Tuning
• Hyperparameter tuning methods
• Random Search
• Grid Search
• Bayesian Optimisation
• Hyperparameter Tuning with Spark

Chapter 6: Streaming and Graphs

• Stream Analytics
• Tools for Stream Analytics: Kafka, Storm, Flink, Spark
• Timestamps in stream analytics
• Windowing Operations

Chapter 7: Deploying Spark ML Artifacts – Introduction to deploying Spark ML Artifacts

• How the Spark system works
• What is Deployment?
• Spark Deployment Artifacts
• Packaging Spark (ML) for Production
• Deploy Spark ML to EMR
• Deploy Spark (ML) with Sagamaker
• Serving and Updating Spark ML Models
• Model Versioning with AWS Model Registry

Chapter 8: Machine learning at Scale – Introduction to Machine Learning at Scale

• Introduction to Scalability
• Common Reasons for Scaling Up ML Systems
• How to Avoid Scaling Infrastructure?
• Benefits of ML at Scale
• Challenges in ML Scalability
• Data Complexities – Challenges
• ML System Engineering – Challenges
• Integration Risks – Challenges
• Collaboration Issues – Challenges

Chapter 9: Machine learning at Scale – Distributed Training of Machine Learning models

• Introduction to Distributed Training
• Data Parallelism
• Steps of Data Parallelism
• Data Parallelism vs. Random Forest
• Model Parallelism
• Frameworks for Implementing Distributed ML
• Introduction to Distributed Training vs. Distributed Inference
• Introduction to Training
• Introduction to Inference
• Key components of Inference
• Inference Challenges
• Training vs. Inference
• Introduction to GPUs
• Inference – Hardware
• AWS Inferentia Chip vs GPU

Chapter 10: Machine learning at Scale – Hyperparameter tuning and model selection at scale

• Hyperparameter Tuning at Scale
• Hyperparameter Tuning Challenges
• Distributed Hyperparameter Tuning
• Bayesian Optimization
• Distributed Hyperparameter Tuning
• Spark Based Tools
• TensorFlowOnSpark
• Advantages of TensorFlowOnSpark
• BigDL
• Advantages of BigDL
• Horovod
• Advantages of Horovod
  H2O Sparkling Water
  Advantages of Sparkling Water over H2O