Department of Electrical Engineering
Control Robotics and Machine Learning Lab
Technion - Israel Institute of Technology
המעבדה לבקרה רובוטיקה ולמידה חישובית
Advanced optimization methods for LSTMs
Background
Deep reinforcement learning (DRL) methods such as the Deep Q-Network (DQN) have achieved state-of-the-art results in a variety of challenging, high-dimensional domains. This success is mainly attributed to the power of deep neural networks to learn rich domain representations for approximating the value function or policy.
Recent architectures incorporate memory modules such as Long-Short Term Memory (LSTM) / Gated Recurrent Units (GRUs) which are examples of recurrent elements.
While using these units helps improve performance, and in some scenarios overcome the lack of markovity of the environment, these units remain hard to train.
This project will focus on understanding and improving the training phase of RNNs, using methods similar (but not limited) to 'Shallow Updates for Deep Reinforcement Learning'
Project Goal
Understand the training procedure and limitations of RNNs. Suggest and implement methods to improve performance / convergence rate / stability.
Project steps
-
Understand the DQN framework.
-
Get a basic understanding of various optimization techniques.
-
Implement and assess the effects of various optimization techniques.
-
Suggest and test your own ideas!
Required knowledge
-
Strong programming skills.
-
Any knowledge in DL and RL is an advantage.
-
Any knowledge in optimization is an advantage.
Environment
-
Torch / TensorFlow / PyTorch
Comments and links
