A research project using a modified version of the video game Call of Duty 4: Modern Warfare is helping students in the Marine Corps University’s Sergeants School train thanks to the collaborative efforts of researchers at the University of Memphis and Virginia Tech.
Led by principal investigator Louis Hickman, assistant professor of industrial-organizational psychology at Virginia Tech; Ryan P. McMahan, director for the Center for Human-Computer Interaction; and Brandon Booth, assistant professor in the Department of Computer Science at the University of Memphis, the team’s goal is to develop and study a training simulation environment and automated after-action review process for students at Sergeants’ School.
Preliminary results show that students participating in the structured scenarios — through planning, game play with comms, and after-action review — are improving their scores on established downstream tests as part of their usual evaluation.
Game scenarios are specifically designed to test planning and execution effectiveness, coordination, communication and contingency response.
“Game technologies provide 3D and immersive environments where we can carefully design the challenges students face without dictating how they solve them,” Booth said. “Students have the freedom to adapt and improvise their initial approaches, but those adaptations can create new challenges or complications that require fast responses and place additional demands on their ability to adapt and lead.”
Then after each scenario, students participate in an after-action review process that highlights what they did well and what they could improve upon.
“After-action reviews are widely used as a reflection tool in the military and linked with efficiency gains as trainees practice skills through repetition,” Booth said. “Our research looks at how technology can make those reviews more useful more often while also helping us better understand how individuals and teams plan, communicate and respond during training. The platform gives us an opportunity to conduct research that has a direct, real-world impact.”
According to Booth, the research team explored a lot of possibilities for training scenario platforms, ranging from slow-paced ultra realistic combat training simulators like Arma to fast-paced action games like League of Legends.
“We wanted to find an experience that supported cognitive training through careful scenario design and team-based coordination more than we needed a platform that provided a familiar environment for training, like a first-person shooter,” Booth said. “Call of Duty was the perfect choice because it gave us both, and, has a modding community that has generated custom and customizable maps and bots (AI-controlled opposing forces).”
or Booth, video games were a natural choice given his background developing entertainment games, fitness games and serious games with military applications. Before becoming a professor, he worked as a game developer for almost ten years across those areas. Today, in addition to teaching, Booth is director of the CUBES Lab (Computational Understanding of Behaviors, Experiences, and States) within the Department of Computer Science where he mentors students on multimodal AI, affective computing and human-centered computing.
One of them is doctorate student and research assistant, Shaeekh Al Jahan, who was the core student who implemented game training/scenario designs and ensured they were functioning as smoothly as possible in addition to making a separate GSC (Game Script Code) file for each map.
“I have been a fan of Call of Duty 4 game since its release in 2007. After so many years, the prospect of turning a favorite game into research work and obtaining its real-life outcomes grabbed my attention first,” Jahan said. “While designing the scenario, I thought of various challenges and decision-making strategies a person and team go through. We want to find out in high stake scenarios, how individuals and teams plan, lead and execute actions to tackle various situation-based challenges. The analysis will open the door to utilize Video Language Model, verify perception, comprehension and collective understanding through cognitive concepts like situational awareness and shared mental model. Finally, with multimodal fusion, we will aim to generate automated After Action Review (AAR). I hope participants at Marine Corps University are enjoying the custom game to understand and enhance their skills. We plan to advance with more features included in the upcoming days.”
The two-year project has plans to expand the research to other sites and continue into a third year.
