Deep learning based approaches for imitation learning.
MetadataShow full item record
HUSSEIN, A. 2018. Deep learning based approaches for imitation learning. Robert Gordon University, PhD thesis.
Imitation learning refers to an agent's ability to mimic a desired behaviour by learning from observations. The field is rapidly gaining attention due to recent advances in computational and communication capabilities as well as rising demand for intelligent applications. The goal of imitation learning is to describe the desired behaviour by providing demonstrations rather than instructions. This enables agents to learn complex behaviours with general learning methods that require minimal task specific information. However, imitation learning faces many challenges. The objective of this thesis is to advance the state of the art in imitation learning by adopting deep learning methods to address two major challenges of learning from demonstrations. Firstly, representing the demonstrations in a manner that is adequate for learning. We propose novel Convolutional Neural Networks (CNN) based methods to automatically extract feature representations from raw visual demonstrations and learn to replicate the demonstrated behaviour. This alleviates the need for task specific feature extraction and provides a general learning process that is adequate for multiple problems. The second challenge is generalizing a policy over unseen situations in the training demonstrations. This is a common problem because demonstrations typically show the best way to perform a task and don't offer any information about recovering from suboptimal actions. Several methods are investigated to improve the agent's generalization ability based on its initial performance. Our contributions in this area are three fold. Firstly, we propose an active data aggregation method that queries the demonstrator in situations of low confidence. Secondly, we investigate combining learning from demonstrations and reinforcement learning. A deep reward shaping method is proposed that learns a potential reward function from demonstrations. Finally, memory architectures in deep neural networks are investigated to provide context to the agent when taking actions. Using recurrent neural networks addresses the dependency between the state-action sequences taken by the agent. The experiments are conducted in simulated environments on 2D and 3D navigation tasks that are learned from raw visual data, as well as a 2D soccer simulator. The proposed methods are compared to state of the art deep reinforcement learning methods. The results show that deep learning architectures can learn suitable representations from raw visual data and effectively map them to atomic actions. The proposed methods for addressing generalization show improvements over using supervised learning and reinforcement learning alone. The results are thoroughly analysed to identify the benefits of each approach and situations in which it is most suitable.
Permalink for this recordhttp://hdl.handle.net/10059/3117
Collections in which this item appears
Except where otherwise noted, this item's license is described as https://creativecommons.org/licenses/by-nc/4.0 ; © Ahmed Hussein
Showing items related by title, author, creator and subject.
Fulford, Heather; Bailey, Moira (Academic Conferences and Publishing International Limited http://www.proceedings.com/24280.html, 2014-09-18)FULFORD, H. and BAILEY, M. 2014. Journals and jottings on entrepreneurial learning journeys. In Galbraith, B. (ed.) Proceedings of the 9th European conference on innovation and entrepreneurship (ECIE 2014), 18-19 September 2014, Belfast, UK. Reading: Academic Conferences Ltd [online], pages 198-206. Available from: http://www.proceedings.com/24280.htmlReview of relevant literature highlighted that entrepreneurs need help to reflect on, and make sense of, the challenges and opportunities that occur during the entrepreneurial process. For students who are unfamiliar with ...
Hussein, Ahmed; Gaber, Mohamed Medhat; Elyan, Eyad; Jayne, Chrisina (ACM https://doi.org/10.1145/3054912, 2017-04-11)HUSSEIN, A., GABER, M.M., ELYAN, E. and JAYNE, C. 2017 Imitation learning: a survey of learning methods. ACM computing surveys [online], 50(2), article 21. Available from: https://doi.org/10.1145/3054912Imitation learning techniques aim to mimic human behavior in a given task. An agent (a learning machine) is trained to perform a task from demonstrations by learning a mapping between observations and actions. The idea of ...
Hussein, Ahmed; Elyan, Eyad; Gaber, Mohamed Medhat; Jayne, Chrisina (Springer https://doi.org/10.1007/s00521-017-3241-z, 2017-12-04)HUSSEIN, A., ELYAN, E., GABER, M.M. and JAYNE, C. 2018. Deep imitation learning for 3D navigation tasks. Neural computing and applications [online], 29(7), pages 389-404. Available from: https://doi.org/10.1007/s00521-017-3241-zDeep learning techniques have shown success in learning from raw high dimensional data in various applications. While deep reinforcement learning is recently gaining popularity as a method to train intelligent agents, ...