Artificial Neural Networks, Machine Learning, Deep Thinking Training Course

DAY 1 - ARTIFICIAL NEURAL NETWORKS

Introduction and ANN Structure.

• Comparison of biological and artificial neurons.
• Core models of ANNs.
• Activation functions utilized in ANNs.
• Common categories of network architectures.

Mathematical Foundations and Learning mechanisms.

• Review of vector and matrix algebra.
• Concepts of state-space analysis.
• Principles of optimization.
• Error-driven learning techniques.
• Memory-based learning methods.
• Hebbian learning principles.
• Competitive learning strategies.

Single layer perceptrons.

• Perceptron architecture and training.
• Pattern classification and Bayes' classifiers.
• Using perceptrons as pattern classifiers.
• Convergence properties of perceptrons.
• Constraints and limitations of perceptrons.

Feedforward ANN.

• Architecture of multi-layer feedforward networks.
• The backpropagation algorithm.
• Training dynamics and convergence in backpropagation.
• Functional approximation via backpropagation.
• Practical considerations and design challenges in backpropagation.

Radial Basis Function Networks.

• Pattern separability and interpolation techniques.
• Theory of regularization.
• Regularization methods applied to RBF networks.
• Design and training of RBF networks.
• Approximation capabilities of RBFs.

Competitive Learning and Self organizing ANN.

• Standard clustering methodologies.
• Learning Vector Quantization (LVQ).
• Algorithms and architectures for competitive learning.
• Self-organizing feature maps.
• Characteristics of feature maps.

Fuzzy Neural Networks.

• Neuro-fuzzy system integration.
• Foundations of fuzzy sets and logic.
• Designing fuzzy systems.
• Construction of fuzzy ANNs.

Applications

• Discussion of real-world Neural Network examples, including their benefits and associated challenges.

DAY -2 MACHINE LEARNING

• The PAC Learning Framework
  • Guarantees for finite hypothesis sets – consistent case
  • Guarantees for finite hypothesis sets – inconsistent case
  • Generalizations
    • Deterministic vs. Stochastic scenarios
    • Bayes error noise
    • Estimation and approximation errors
    • Model selection strategies
• Rademacher Complexity and VC-Dimension
• Bias-Variance tradeoff
• Regularisation techniques
• Managing Over-fitting
• Model Validation
• Support Vector Machines
• Kriging (Gaussian Process regression)
• PCA and Kernel PCA
• Self Organisation Maps (SOM)
• Kernel induced vector space
  • Mercer Kernels and Kernel-induced similarity metrics
• Reinforcement Learning

DAY 3 - DEEP LEARNING

This section builds upon the concepts covered on Day 1 and Day 2

• Logistic and Softmax Regression
• Sparse Autoencoders
• Vectorization, PCA, and Whitening
• Self-Taught Learning
• Deep Network Architectures
• Linear Decoders
• Convolution and Pooling Operations
• Sparse Coding
• Independent Component Analysis
• Canonical Correlation Analysis
• Demonstrations and Practical Applications