开源巨献：27个深度强化学习算法的实例项目-FlyAI

开源巨献：27个深度强化学习算法的实例项目

深度学习机器学习

最后更新 2020-11-20 15:37 阅读 8067

最后更新 2020-11-20 15:37

阅读 8067

深度学习机器学习

##FlyAI文献翻译
**英文原文：**[：Deep Reinforcement Learning Nanodegree Algorithms](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms)

> **标签** ：强化学习

![](https://dataset.flyai.com/壁纸.jpg)

Here you can find several projects dedicated to the Deep Reinforcement Learning methods. The projects are deployed in the matrix form: [env x model], where env is the environment to be solved, and model is the model/algorithm which solves this environment. In some cases,the same environment is resolved by several algorithms. All projects are presented as a jupyter notebook containing training log.

The following environments are supported:

AntBulletEnv, Bipedalwalker, CarRacing, CartPole, Crawler, HalfCheetahBulletEnv, HopperBulletEnv, LunarLander, LunarLanderContinuous, Markov Decision 6x6, Minitaur, Minitaur with Duck, Pong, Navigation, Reacher, Snake, Tennis, Waker2DBulletEnv.

>在这里，您可以找到几个致力于“深度强化学习”方法的项目。 项目以矩阵形式部署：[env x model]，其中env是要解决的环境，而model是解决该环境的模型/算法。 在某些情况下，可以通过几种算法来解决同一环境。 所有项目均以包含培训日志的Jupyter笔记本的形式呈现。

>支持以下环境：

>AntBulletEnv, Bipedalwalker, CarRacing, CartPole, Crawler, HalfCheetahBulletEnv, HopperBulletEnv, LunarLander, LunarLanderContinuous, Markov Decision 6x6, Minitaur, Minitaur with Duck, Pong, Navigation, Reacher, Snake, Tennis, Waker2DBulletEnv.

Four environments (Navigation, Crawler, Reacher, Tennis) are solved in the framework of the [Udacity Deep Reinforcement Learning Nanodegree Program](https://www.udacity.com/course/deep-reinforcement-learning-nanodegree--nd893 "Udacity Deep Reinforcement Learning Nanodegree Program").

>在Udacity深度强化学习学位计划的框架内解决了四个环境（导航，爬虫，到达者，网球）。

[Monte-Carlo Methods](https://medium.com/zsalloum/monte-carlo-in-reinforcement-learning-the-easy-way-564c53010511 "Monte-Carlo Methods")

In Monte Carlo (MC), we play episodes of the game until we reach the end, we grab the rewards collected on the way and move backward to the start of the episode. We repeat this method a sufficient number of times and we average the value of each state.

>蒙特卡洛方法

>在蒙特卡洛（MC）中，我们玩游戏的情节直到到达终点，我们从途中获得了奖励然后返回情节的开始。 我们重复此方法至足够的次数，然后平均每个状态的值

[Temporal Difference Methods and Q-learning](https://en.wikipedia.org/wiki/Temporal_difference_learning "Temporal Difference Methods and Q-learning")

[Reinforcement Learning in Continuous Space (Deep Q-Network)](https://en.wikipedia.org/wiki/Reinforcement_learning)

[Function Approximation and Neural Network](https://medium.com/biffures/rl-course-by-david-silver-lectures-5-to-7-576188d3b033)

>- 时差方法与Q学习

>- 连续空间中的强化学习（深度Q网络）

>- 函数逼近和神经网络

[The Universal Approximation Theorem (UAT) states](https://towardsdatascience.com/the-approximation-power-of-neural-networks-with-python-codes-ddfc250bdb58 "The Universal Approximation Theorem (UAT) states") that feed-forward neural networks containing a single hidden layer with a finite number of nodes can be used to approximate any continuous function provided rather mild assumptions about the form of the activation function are satisfied.

>通用逼近定理（UAT）规定，只要满足有关激活函数形式的轻微假设，就可以使用包含具有有限数量节点的单个隐藏层的前馈神经网络来近似任何连续函数。

[Policy-Based Methods](https://towardsdatascience.com/policy-based-reinforcement-learning-the-easy-way-8de9a3356083 "Policy-Based Methods"), [Hill-Climbing](https://en.wikipedia.org/wiki/Hill_climbing "Hill-Climbing"),[ Simulating Annealing](https://medium.com/macromoltek/machine-learning-and-simulated-annealing-588b2e70d0cc " Simulating Annealing")
Random-restart hill-climbing is a surprisingly effective algorithm in many cases. Simulated annealing is a good probabilistic technique because it does not accidentally think a local extrema is a global extrema.

>基于策略的方法，爬山，模拟退火

>在许多情况下，随机重启爬山是一种出奇的有效算法。 模拟退火是一种很好的概率技术，因为它不会偶然错误地将局部极值作为全局极值。

[Policy-Gradient Methods](https://lilianweng.github.io/lil-log/2018/04/08/policy-gradient-algorithms.html "Policy-Gradient Methods"), [REINFORCE](https://medium.com/samkirkiles/reinforce-policy-gradients-from-scratch-in-numpy-6a09ae0dfe12 "REINFORCE"), [PPO](https://arxiv.org/abs/1707.06347 "PPO")
Define a performance measure J(\theta) to maximaze. Learn policy paramter \theta throgh approximate gradient ascent.

>策略渐变方法，REINFORCE，PPO

>定义一个性能指标J（\ theta）以使其最大化。 通过近似梯度上升来学习策略参数\ theta。

[Actor-Critic Methods](https://towardsdatascience.com/soft-actor-critic-demystified-b8427df61665 "Actor-Critic Methods"), [A3C](https://medium.com/emergent-future/simple-reinforcement-learning-with-tensorflow-part-8-asynchronous-actor-critic-agents-a3c-c88f72a5e9f2 "A3C"), [A2C](https://hackernoon.com/intuitive-rl-intro-to-advantage-actor-critic-a2c-4ff545978752 "A2C"), [DDPG](https://medium.com/amitpatel.gt/policy-gradients-1edbbbc8de6b "DDPG"), [TD3](https://arxiv.org/abs/1802.09477 "TD3"),[ SAC](https://towardsdatascience.com/soft-actor-critic-demystified-b8427df61665 " SAC")

The key difference from A2C is the Asynchronous part. A3C consists of multiple independent agents(networks) with their own weights, who interact with a different copy of the environment in parallel. Thus, they can explore a bigger part of the state-action space in much less time.

>关键行为法，A3C，A2C，DDPG，TD3，SAC

>A3C与A2C的主要区别在于异步部分。 A3C由具有权重的多个独立代理（网络）组成，它们与环境的不同副本并行进行交互。 因此，他们可以在更少的时间内探索状态-行动空间的更大部分。

Projects, models and methods

>项目，模型和方法

[AntBulletEnv, Soft Actor-Critic (SAC)](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms/tree/master/Ant-PyBulletEnv-Soft-Actor-Critic "AntBulletEnv, Soft Actor-Critic (SAC)")

[BipedalWalker, Twin Delayed DDPG (TD3)](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms/tree/master/BipedalWalker-TwinDelayed-DDPG%20(TD3)

[BipedalWalker, PPO, Vectorized Environment](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms/blob/master/BipedalWalker-PPO-VectorizedEnv "BipedalWalker, PPO, Vectorized Environment")

[BipedalWalker, Soft Actor-Critic (SAC)](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms/tree/master/BipedalWalker-Soft-Actor-Critic "BipedalWalker, Soft Actor-Critic (SAC)")

[BipedalWalker, A2C, Vectorized Environment](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms/tree/master/BipedalWalker-A2C-VectorizedEnv "BipedalWalker, A2C, Vectorized Environment")

[CarRacing with PPO, Learning from Raw Pixels](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms/blob/master/CarRacing-From-Pixels-PPO "CarRacing with PPO, Learning from Raw Pixels")

[CartPole, Policy Based Methods, Hill Climbing](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms/tree/master/CartPole-Policy-Based-Hill-Climbing "CartPole, Policy Based Methods, Hill Climbing")

[CartPole, Policy Gradient Methods, REINFORCE](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms/tree/master/CartPole-Policy-Gradient-Reinforce "CartPole, Policy Gradient Methods, REINFORCE")

[Cartpole, DQN](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms/tree/master/Cartpole-Deep-Q-Learning "Cartpole, DQN")

[Cartpole, Double DQN](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms/tree/master/Cartpole-Double-Deep-Q-Learning "Cartpole, Double DQN")

[HalfCheetahBulletEnv, Twin Delayed DDPG (TD3)](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms/tree/master/HalfCheetahBulletEnv-TD3 "HalfCheetahBulletEnv, Twin Delayed DDPG (TD3)")

[HopperBulletEnv, Twin Delayed DDPG (TD3)](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms/tree/master/HopperBulletEnv_v0-TD3 "HopperBulletEnv, Twin Delayed DDPG (TD3)")

[HopperBulletEnv, Soft Actor-Critic (SAC)](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms/tree/master/HopperBulletEnv-v0-SAC "HopperBulletEnv, Soft Actor-Critic (SAC)")

[LunarLander-v2, DQN](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms/tree/master/LunarLander-v2-DQN "LunarLander-v2, DQN")

[LunarLanderContinuous-v2, DDPG](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms/tree/master/LunarLanderContinuous-v2-DDPG "LunarLanderContinuous-v2, DDPG")

[Markov Decision Process, Monte-Carlo, Gridworld 6x6](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms/tree/master/Markov-Decision-Process_6x6 "Markov Decision Process, Monte-Carlo, Gridworld 6x6")

[MinitaurBulletEnv, Soft Actor-Critic (SAC)](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms/tree/master/Minitaur-Soft-Actor-Critic "MinitaurBulletEnv, Soft Actor-Critic (SAC)")

[MinitaurBulletDuckEnv, Soft Actor-Critic (SAC)](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms/tree/master/MinitaurDuck-Soft-Actor-Critic "MinitaurBulletDuckEnv, Soft Actor-Critic (SAC)")

[Pong, Policy Gradient Methods, PPO](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms/tree/master/Pong-Policy-Gradient-PPO "Pong, Policy Gradient Methods, PPO")

[Pong, Policy Gradient Methods, REINFORCE](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms/tree/master/Pong-Policy-Gradient-REINFORCE "Pong, Policy Gradient Methods, REINFORCE")

[Snake, DQN, Pygame](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms/tree/master/Snake-Pygame-DQN "Snake, DQN, Pygame")

[Udacity Project 1: Navigation, DQN, ReplayBuffer](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms/tree/master/Project-1_Navigation-DQN "Udacity Project 1: Navigation, DQN, ReplayBuffer")

[Udacity Project 2: Continuous Control-Reacher, DDPG](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms/tree/master/Project-2_Continuous-Control-Reacher-DDPG "Udacity Project 2: Continuous Control-Reacher, DDPG"), environment[ Reacher (Double-Jointed-Arm)](https://github.com/Unity-Technologies/ml-agents/blob/master/docs/Learning-Environment-Examples.md#reacher " Reacher (Double-Jointed-Arm)")

[Udacity Project 2: Continuous Control-Crawler, PPO](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms/tree/master/Project-2_Continuous-Control-Crawler-PPO "Udacity Project 2: Continuous Control-Crawler, PPO"), environment [Crawler](https://github.com/Unity-Technologies/ml-agents/blob/master/docs/Learning-Environment-Examples.md#crawler "Crawler")

[Udacity Project 3: Collaboration_Competition-Tennis, Multi-agent DDPG](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms/tree/master/Project-3_Collaboration_Competition-Tennis-Maddpg "Udacity Project 3: Collaboration_Competition-Tennis, Multi-agent DDPG"), environment [Tennis](https://github.com/Unity-Technologies/ml-agents/blob/master/docs/Learning-Environment-Examples.md#tennis "Tennis")

[Walker2DBulletEnv, Twin Delayed DDPG (TD3)](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms/tree/master/Walker2DBulletEnv-v0_TD3 "Walker2DBulletEnv, Twin Delayed DDPG (TD3)")

[Walker2DBulletEnv, Soft Actor-Critic (SAC)](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms/tree/master/Walker2DBulletEnv-v0_SAC "Walker2DBulletEnv, Soft Actor-Critic (SAC)")

###Projects with DQN and Double DQN

[Cartpole, DQN](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms/tree/master/Cartpole-Deep-Q-Learning "Cartpole, DQN")

[Cartpole, Double DQN](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms/tree/master/Cartpole-Double-Deep-Q-Learning "Cartpole, Double DQN")

[LunarLander-v2, DQN](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms/tree/master/LunarLander-v2-DQN "LunarLander-v2, DQN")

[Navigation, DQN](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms/tree/master/Project-1_Navigation-DQN "Navigation, DQN")

[Snake, DQN, Pygame](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms/tree/master/Snake-Pygame-DQN "Snake, DQN, Pygame")

###Projects with PPO

[Pong](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms/tree/master/Pong-Policy-Gradient-PPO "Pong"), 8个并行代理

[CarRacing](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms/tree/master/CarRacing-From-Pixels-PPO "CarRacing"), 单一代理，从像素中学习

[C r a w l e r ](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms/tree/master/Project-2_Continuous-Control-Crawler-PPO "C r a w l e r "), 12 个并行代理

[BipedalWalker](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms/tree/master//BipedalWalker-PPO-VectorizedEnv "BipedalWalker"), 16 个并行代理

###Projects with TD3

[BipedalWalker](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms/tree/master/BipedalWalker-TwinDelayed-DDPG%20(TD3)

[HalfChhetahBulletEnv](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms/tree/master/HalfCheetahBulletEnv-TD3 "HalfChhetahBulletEnv")

[HopperBulletEnv](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms/tree/master/HopperBulletEnv_v0-TD3 "HopperBulletEnv")

[Walker2DBulletEnv](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms/tree/master/Walker2DBulletEnv-v0_TD3 "Walker2DBulletEnv")

###Projects with Soft Actor-Critic (SAC)

[AntBulletEnv](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms/tree/master/Ant-PyBulletEnv-Soft-Actor-Critic "AntBulletEnv")

[BipedalWalker](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms/tree/master/BipedalWalker-Soft-Actor-Critic "BipedalWalker")

[HopperBulletEnv](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms/tree/master/HopperBulletEnv-v0-SAC "HopperBulletEnv")

[MinitaurBulletEnv](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms/tree/master/Minitaur-Soft-Actor-Critic "MinitaurBulletEnv")

[MinitaurBulletDuckEnv](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms/tree/master/MinitaurDuck-Soft-Actor-Critic "MinitaurBulletDuckEnv")

[Walker2dBulletEnv](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms/tree/master/Walker2DBulletEnv-v0_SAC "Walker2dBulletEnv")

###BipedalWalker,与不同模型的混合

[BipedalWalker, Twin Delayed DDPG (TD3)](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms/tree/master/BipedalWalker-TwinDelayed-DDPG%20(TD3)

[BipedalWalker, Soft-Actor-Critic (SAC)](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms/tree/master/BipedalWalker-Soft-Actor-Critic "BipedalWalker, Soft-Actor-Critic (SAC)")

###CartPole与不同模型的混合

[Cartpole with Deep Q-Learning](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms/tree/master/Cartpole-Deep-Q-Learning "Cartpole with Deep Q-Learning")

[Cartpole with Doouble Deep Q-Learning](https://github.com/Rafael1s/Deep-Reinforcement-Learning-Algorithms/tree/master/Cartpole-Double-Deep-Q-Learning "Cartpole with Doouble Deep Q-Learning")

###更多链接

有关Policy-Gradient Methods策略梯度方法，参见 1, 2, 3.

有关 REINFORCE，参见 1, 2, 3.

有关 PPO，参见 1, 2, 3, 4, 5.

有关 DDPG，参见 1, 2.

有关 Actor-Critic Methods和A3C，参见 1, 2, 3, 4.

有关 TD3，参见 1, 2, 3

有关 SAC，参见 1, 2, 3, 4, 5

有关 A2C，参见 1, 2, 3, 4, 5

###Papers on TowardsDataScience

>###TowardsDataScience网站上的文章

[How does the Bellman equation work in Deep Reinforcement Learning?](https://towardsdatascience.com/how-the-bellman-equation-works-in-deep-reinforcement-learning-5301fe41b25a "How does the Bellman equation work in Deep Reinforcement Learning?")

>贝尔曼方程式在深度强化学习中如何工作？

[A pair of interrelated neural networks in Deep Q-Network](https://towardsdatascience.com/a-pair-of-interrelated-neural-networks-in-dqn-f0f58e09b3c4 "A pair of interrelated neural networks in Deep Q-Network")

>深度Q网络中一对相互关联的神经网络

[Three aspects of Deep Reinforcement Learning: noise, overestimation and exploration](https://towardsdatascience.com/three-aspects-of-deep-rl-noise-overestimation-and-exploration-122ffb4bb92b "Three aspects of Deep Reinforcement Learning: noise, overestimation and exploration")

>深度强化学习的三个方面：噪声，高估和探索

###我在上述项目中开发的相关视频

[Four BipedalWalker Gaits](https://www.youtube.com/watch?v=PFixqZEYKh4 "Four BipedalWalker Gaits")

[BipedalWalker by Training Stages](https://www.youtube.com/watch?v=g01mIFbxVns "BipedalWalker by Training Stages")

[CarRacing by Training Stages](https://www.youtube.com/watch?v=55buBR2pPdc "CarRacing by Training Stages")

[Lucky Hopper](https://www.youtube.com/watch?v=Ipctq89yLB0 "Lucky Hopper")

[Martian Ant](https://www.youtube.com/watch?v=s7aMZ1bbQgk "Martian Ant")

[Lunar Armada](https://www.youtube.com/watch?v=6O6g9LCWvIs "Lunar Armada")

[Wooden Snake](https://www.youtube.com/watch?v=-T4wQirNDRo "Wooden Snake")

[Walking through the chess fields](https://www.youtube.com/watch?v=qUT3TznKWAk "Walking through the chess fields")

[Artificial snake on the way](https://www.youtube.com/watch?v=-jNfUrVniNg "Artificial snake on the way")

[Learned Long Snake](https://www.youtube.com/watch?v=Tt1rqWTR8ZA "Learned Long Snake")

[Such a fast cheetah](https://www.youtube.com/watch?v=Q-FchLEZKRk "Such a fast cheetah")

[Four stages of Minitaur training](https://www.youtube.com/watch?v=uEAqyEwvi54 "Four stages of Minitaur training")

本作品采用知识共享署名-非商业性使用-相同方式共享 4.0 国际许可协议进行许可，转载请附上原文出处链接和本声明。
本文链接地址：https://flyai.com/article/761

讨论

500字

表情

发送

删除确认

是否删除该条评论？

取消删除