【454】ML-DL相关链接

GD(梯度下降)和SGD(随机梯度下降)

机器学习中的Bias和Variance

机器学习之判别式模型和生成式模型

笔记 | 什么是Cross Entropy

熵与信息增益

简单谈谈Cross Entropy Loss

Softmax 和 Softmax-loss的推演

---

Victor Zhou Blogs

• Machine Learning for Beginners: An Introduction to Neural Networks
  • 1. Building Blocks: Neurons
    • A Simple Example
    • Coding a Neuron
  • 2. Combining Neurons into a Neural Network
    • An Example: Feedforward
    • Coding a Neural Network: Feedforward
  • 3. Training a Neural Network, Part 1
    
    • Loss
    • An Example Loss Calculation
    • Code: MSE Loss
  • 4. Training a Neural Network, Part 2
    
    • Example: Calculating the Partial Derivative
    • Training: Stochastic Gradient Descent
    • Code: A Complete Neural Network
    • Now What? (A bunch of learning meterials)
• CNNs, Part 1: An Introduction to Convolutional Neural Networks
  • 1. Motivation
    • Reason 1: Images are Big
    • Reason 2: Positions can change
  • 2. Dataset
  • 3. Convolutions
    
    • 3.1 How is this useful?
    • 3.2 Padding
    • 3.3 Conv Layers
    • 3.4 Implementing Convolution
  • 4. Pooling
    • 4.1 Implementing Pooling
  • 5. Softmax
• CNNs, Part 2: Training a Convolutional Neural Network
  • 1. Setting the Stage
  • 2. Training Overview
  • 3. Backprop: Softmax
    • 最终的计算公式：$L=-ln(p_c)$，其中$p_c$指正确的类的预测概率
    • 正常的 cross-entropy loss 公式：$L=-sum_{x} p(x) ln(q(x))$，但是里面真实值除了正确的类都是 0，所以都可以消去，真实值的 概率 为 1，所以最终结果如上
  • 4. Backprop: Max Pooling
  • 5. Backprop: Conv
  • 6. Training a CNN
  • What Now?

---