Neuro-Dynamic Programming

Table of Contents:


  1. Introduction

    1. Cost-to-go Approximations in Dynamic Programming
    2. Approximation Architectures
    3. Simulation and Training
    4. Neuro-Dynamic Programming
    5. Notes and Sources

  2. Dynamic Programming

    1. Introduction
    2. Stochastic Shortest Path Problems
    3. Discounted Problems
    4. Problem Formulation and Examples
    5. Notes and Sources

  3. Neural Network Architectures and Training

    1. Architectures for Approximation
    2. Neural Network Training
    3. Notes and Sources

  4. Stochastic Iterative Algorithms

    1. The Basic Model
    2. Convergence Based on a Smooth Potential Function
    3. Convergence under Contraction or Monotonicity Assumptions
    4. The ODE Approach
    5. Notes and Sources

  5. Simulation Methods for a Lookup Table Representation

    1. Some Aspects of Monte Carlo Simulation
    2. Policy Evaluation by Monte Carlo Simulation
    3. Temporal Difference Methods
    4. Optimistic Policy Iteration
    5. Simulation-Based Value Iteration
    6. Q-Learning
    7. Notes and Sources

  6. Approximate DP with Cost-to-Go Function Approximation

    1. Generic Issues - From Parameters to Policies
    2. Approximate Policy Iteration
    3. Approximate Policy Evaluation Using TD(lambda)
    4. Optimistic Policy Iteration
    5. Approximate Value Iteration
    6. Q-Learning and Advantage Updating
    7. Value Iteration with State Aggregation
    8. Euclidean Contractions and Optimal Stopping
    9. Value Iteration with Representative States
    10. Bellman Error Methods
    11. Continuous States and the Slope of the Cost-to-Go
    12. Approximate Linear Programming
    13. Overview
    14. Notes and Sources

  7. Extensions

    1. Average Cost per Stage Problems
    2. Dynamic Games
    3. Parallel Computation Issues
    4. Notes and Sources

  8. Case Studies

    1. Parking
    2. Football
    3. Tetris
    4. Combinatorial Optimization - Maintenance and Repair
    5. Dynamic Channel Allocation
    6. Backgammon
    7. Notes and Sources

  9. Appendix A: Mathematical Review

  10. Appendix B: On Probability Theory and Markov Chains

  11. References

  12. Index


[Return to Athena Scientific Homepage]
info@athenasc.com