Student Team Project: Implementing AI Agents for Atari Games in ECGR 4117/5117

The ECGR 4117/5117 AI for Robotics and Automation course offers students hands-on experience in artificial intelligence and reinforcement learning through an exciting team project.

In this project, teams of up to four students will implement AI agents that play Atari games, aiming to replicate the groundbreaking work presented in the high-impact 2015 Nature article by Mnih, V. et al., titled “Human-level control through deep reinforcement learning.” This research demonstrated that AI agents, trained using deep reinforcement learning, can outperform human players in many Atari games, marking a significant milestone in artificial intelligence and receiving significant attention from researchers with over 34,000 citations. This problem continues to receive attention from the AI community with more impressive recent results from OpenAI’s ChatGPT and Google’s DeepMind AI agents.

Project Overview

Students will develop deep reinforcement learning models that autonomously learn to play classic Atari games such as Pong, Breakout, and Space Invaders. By leveraging deep Q-networks (DQNs) and training their AI agents using reinforcement learning techniques, students will gain practical insights into the implementation of intelligent systems.

The project will involve:

• Setting up an environment using OpenAI Gym and the Atari Learning Environment.
• Implementing and training different reinforcement learning networks using the PyTorch AI design framework and OpenAI’s stable baselines including deep Q-networks.
• Applying Optuna to automate fine-tuning network hyperparameters that optimize the learning process and performance of the AI agents.
• Evaluating and comparing results against human benchmarks and the original findings from Mnih et al.

Student Experience and Learning Outcomes

Through this project, students will gain valuable experience in reinforcement learning, a key technique in modern AI development. They will deepen their understanding of:
● Neural Networks: Implementing deep learning models and understanding their structure and training process.
● Reinforcement Learning: Applying concepts such as exploration vs. exploitation, reward maximization, and Q-learning.
● AI System Development: Learning how to structure AI projects, troubleshoot issues, and optimize training efficiency.
● Team Collaboration: Working effectively in a small team, dividing tasks, and integrating different components into a final working system.

Beyond technical skills, students often find this project engaging and rewarding due to its interactive nature. Seeing an AI agent improve over time and eventually master a game provides a tangible demonstration of how AI can learn autonomously. Many students appreciate the challenge of debugging and optimizing their models, as well as the sense of accomplishment when their agent finally makes decisions consistent with human-level play.

Real-World Applications and Career Impact

The knowledge and skills gained through this project extend well beyond academia. Reinforcement learning is a powerful tool used in various fields, including:

• Autonomous Systems: The principles learned can be applied to robotics, self-driving cars, and drone navigation, where AI must learn optimal strategies for movement and decision-making.
• Finance and Trading: Reinforcement learning is increasingly used in algorithmic trading, portfolio management, and risk assessment.
• Healthcare and Medicine: AI-driven diagnosis, personalized treatment plans, and robotic-assisted surgeries benefit from reinforcement learning approaches.
• Industrial Automation: Smart manufacturing, robotic process automation, and predictive maintenance leverage AI for efficiency improvements.

Many students who have participated in similar projects have used their experience to secure internships and full-time positions in AI-focused roles. The project provides a strong foundation for further research, entrepreneurship, and innovation in artificial intelligence and automation.

Conclusion

The AI for Robotics and Automation student project offers an immersive and practical learning experience, enabling students to implement and experiment with cutting-edge AI techniques. By replicating and extending the work of Mnih et al., students gain a deeper appreciation of reinforcement learning while preparing for careers in AI-driven industries.

The combination of hands-on implementation, problem-solving, and teamwork makes this project a cornerstone of the course, bridging the gap between theoretical AI concepts and real-world applications.