“Abstract—This paper delineates the intricate process of crafting an Augmented Reality…
- User
“Abstract—This paper delineates the intricate process of crafting an Augmented Reality (AR)-enriched version of the Subway Surfers game, engineered with an emphasis on action recognition and the leverage of Artificial Intelligence (AI) principles, with the primary objective of boosting children’s enthusiasm towards physical activity. The gameplay, fundamentally predicated on advanced computer vision methodologies for discerning player kinesthetics, and reinforced with machine learning tactics for modulating the intricacy of the game in accordance with player capabilities, offers an immersive and engaging interface. This innovative amalgamation serves to not only catalyze children’s interest in participating in active exercises, but also introduces a playful aspect to it. The procedural development of the game required the cohesive assimilation of a diverse spectrum of technologies, encompassing Unity for game development, TensorFlow for implementing machine learning algorithms, and Vuforia for crafting the AR elements. A preliminary study, conducted to assess the efficacy of the game in fostering a pro-sport attitude in children, reported encouraging outcomes. Given the potential of the game to incite physical activity among young users, it could be construed as a promising antidote to sedentarism and a potent catalyst for endorsing a healthier lifestyle. Keywords—Augmented reality; computer vision; game development; action detection; action classification; machine learning I. INTRODUCTION Physical inactivity has become a major health concern, particularly among children, with global estimates suggesting that more than 80% of adolescents worldwide are not meeting the recommended levels of physical activity [1]. The rise of sedentary lifestyles and the prevalence of screen-based activities have been identified as major contributors to this trend, with video games being a prominent example. However, video games have the potential to be a tool for promoting physical activity if they are designed with this goal in mind [2]. One such game is Subway Surfers, a popular endless runner game that has been downloaded over two billion times [3]. The game’s objective is to run through a subway system, collecting coins and avoiding obstacles. While the game has been praised for its addictive gameplay and engaging visuals, it does not require physical activity and could contribute to sedentary behavior. To address this issue, this paper presents the development of a Subway Surfers game with augmented reality (AR) based on action recognition and artificial intelligence (AI) to increase children’s motivation for sports. The game utilizes computer vision algorithms to recognize the player’s physical movements and uses machine learning techniques to adjust the game’s difficulty level accordingly. The AR feature provides a more immersive and engaging gameplay experience, encouraging children to participate in physical activity while having fun. The use of AR in video games has gained significant attention in recent years due to its potential to create immersive and interactive experiences. AR involves overlaying digital content onto the real world, allowing users to interact with virtual objects in real-time [4]. This technology has been used in various applications, including education, entertainment, and advertising [5]. In the context of video games, AR has been used to create physically active gameplay experiences, with some studies suggesting that AR games can increase physical activity levels [6]. In addition to AR, the game incorporates action recognition and AI to create a personalized gameplay experience. Action recognition involves using computer vision algorithms to detect and classify human actions, such as running, jumping, and squatting [7]. By recognizing the player’s movements, the game can adjust the gameplay difficulty level to provide a challenge that matches the player’s physical abilities. This personalized approach is important as it can help to maintain the player’s engagement and motivation, which are critical factors in promoting physical activity [8]. The use of AI in the game also allows for real-time adjustments to the gameplay. Machine learning techniques, such as reinforcement learning, can be used to train the game’s AI to optimize the gameplay experience for the player [9]. By learning from the player’s actions and adjusting the gameplay accordingly, the game can create a more challenging and engaging experience. This approach also allows the game to adapt to the player’s progress, ensuring that the gameplay remains challenging and motivating. (IJACSA) International Journal of Advanced Computer Science and Applications, Vol. 14, No. 8, 2023 683 | P a g e www.ijacsa.thesai.org The development of the game involved the integration of various technologies, including Unity, TensorFlow, and Vuforia. Unity is a game engine that provides a development environment for creating 2D and 3D games. TensorFlow is a popular machine learning library that was used to implement the game’s action recognition and AI algorithms. Vuforia is an AR platform that was used to create the game’s AR features [10]. To evaluate the effectiveness of the game in increasing children’s motivation for sports, a pilot study was conducted. The study involved 20 children aged between 7 and 12 years old, who played the game for 30 minutes. The study found that the game was effective in increasing the children’s motivation for physical activity, as measured by self-reported enjoyment and willingness to engage in physical activity. The study also found that the personalized gameplay experience created by the game’s action recognition and AI features was a key factor in maintaining the children’s engagement and motivation [11]. The potential of the Subway Surfers game with AR and AI to promote physical activity among children suggests that it can be a valuable tool in combating sedentary lifestyles and promoting a healthier lifestyle. By creating an immersive and engaging gameplay experience that encourages physical activity, the game can provide an alternative to traditional forms of exercise that may be less appealing to children. Furthermore, the game’s personalized approach can help to maintain the player’s engagement and motivation, which are critical factors in promoting long-term behavior change [12]. The development of this game also highlights the potential of technology in promoting physical activity. With the widespread availability of smartphones and tablets, mobile games have the potential to reach a large audience and provide a convenient and accessible way to promote physical activity. The integration of AR and AI technologies can further enhance the effectiveness of these games by creating personalized and engaging experiences. In conclusion, this paper presented the development of a Subway Surfers game with augmented reality based on action recognition and artificial intelligence to increase children’s motivation for sports. The game utilizes AR to create an immersive and engaging gameplay experience and uses action recognition and AI to provide a personalized gameplay experience that matches the player’s physical abilities. The pilot study conducted to evaluate the game’s effectiveness in increasing children’s motivation for sports yielded promising results, highlighting the potential of this game as a tool for promoting physical activity among children. The development of this game also demonstrates the potential of technology in promoting physical activity and provides a roadmap for the development of future mobile games with a focus on promoting physical activity. II. RELATED WORKS The use of technology to promote physical activity among children has been a topic of interest in recent years. Mobile games, in particular, have been identified as a promising tool for promoting physical activity, as they have the potential to reach a large audience and provide an engaging and interactive experience. Several studies have explored the effectiveness of mobile games in promoting physical activity among children [13-15]. Next study developed a mobile game that used augmented reality to create an immersive and engaging gameplay experience [16]. The game encouraged children to engage in physical activity by requiring them to perform various movements and exercises in order to progress through the game. The study found that the game was effective in increasing children’s motivation for physical activity, with participants reporting higher levels of enjoyment and engagement compared to traditional forms of exercise. Similarly, another study developed a mobile game that used a pedometer to track children’s physical activity levels [17]. The game provided feedback to the children based on their physical activity levels, with rewards and incentives provided for achieving certain goals. The study found that the game was effective in increasing children’s physical activity levels, with participants reporting higher levels of engagement and motivation compared to traditional forms of exercise. The use of artificial intelligence in mobile games has also been explored as a means of providing personalized and adaptive gameplay experiences. A study by Paliokas et al. (2020) developed a mobile game that used artificial intelligence to adapt the gameplay experience based on the player’s physical abilities [18]. The game used motion sensors to track the player’s movements and adjust the difficulty of the game accordingly. The study found that the personalized gameplay experience created by the AI technology was effective in maintaining the player’s engagement and motivation. The use of augmented reality in mobile games has also been identified as a promising tool for promoting physical activity. A study by Ding & Wang et al. (2019) developed a mobile game that used augmented reality to create an immersive and engaging gameplay experience [19]. The game required players to physically move around and interact with virtual objects in order to progress through the game. The study found that the game was effective in increasing children’s motivation for physical activity, with participants reporting higher levels of enjoyment and engagement compared to traditional forms of exercise. The Subway Surfers game, which was originally developed in 2021 by Wang and SYBO Games, has become one of the most popular mobile games worldwide, with over two billion downloads [20]. The game’s popularity can be attributed to its engaging gameplay, which requires players to run, jump, and dodge obstacles in a subway environment. The game’s simple controls and colorful graphics have also contributed to its appeal among children. Several studies have explored the effectiveness of the Subway Surfers game in promoting physical activity among children. A study by Pascoal et al. (2020) found that the game was effective in increasing children’s physical activity levels, with participants reporting higher levels of enjoyment and engagement compared to traditional forms of exercise [21]. (IJACSA) International Journal of Advanced Computer Science and Applications, Vol. 14, No. 8, 2023 684 | P a g e www.ijacsa.thesai.org Another study by Blattgerste et al. (2019) found that the game was effective in increasing children’s motivation for physical activity, with participants reporting higher levels of enjoyment and satisfaction compared to traditional forms of exercise [22]. However, despite the game’s popularity and potential for promoting physical activity, it has not yet been fully explored in terms of its potential for utilizing AR and AI technologies. This paper presents the development of a Subway Surfers game with augmented reality based on action recognition and artificial intelligence, which aims to further enhance the game’s potential for promoting physical activity among children [23]. The incorporation of AR technology into the Subway Surfers game has the potential to create a more immersive and engaging gameplay experience by allowing the player to interact with virtual objects in the real world. This technology can be used to encourage physical activity by requiring the player to physically move around and interact with virtual objects in order to progress through the game. In addition, the use of AI technology can be used to create a personalized and adaptive gameplay experience based on the player’s physical abilities. This technology can be used to track the player’s movements and adjust the difficulty of the game accordingly, ensuring that the player remains engaged and motivated throughout the gameplay experience. Several studies have explored the use of AR and AI technologies in promoting physical activity among children. A study by Lam et al. (2020) developed an AR-based game that used AI technology to adjust the difficulty of the game based on the player’s physical abilities [24]. The study found that the personalized gameplay experience created by the AI technology was effective in increasing children’s motivation for physical activity. Another study by Chen (2020) developed an AR-based game that required children to physically move around and interact with virtual objects in order to progress through the game [25]. The study found that the game was effective in increasing children’s physical activity levels, with participants reporting higher levels of enjoyment and engagement compared to traditional forms of exercise. The development of the Subway Surfers game with AR and AI technologies builds on these previous studies by creating a gameplay experience that is both engaging and physically active. By combining the popular and familiar gameplay of Subway Surfers with the immersive and interactive experience provided by AR and AI technologies, this game has the potential to significantly increase children’s motivation for physical activity. Furthermore, the use of action recognition technology in the development of the game adds another level of interactivity and engagement. This technology can be used to track the player’s movements and actions, allowing the game to provide immediate feedback and rewards based on the player’s performance. This feedback can be used to encourage the player to continue engaging in physical activity and improving their performance. In conclusion, the development of a Subway Surfers game with AR based on action recognition and AI technology has the potential to significantly increase children’s motivation for physical activity. The combination of the popular and familiar gameplay of Subway Surfers with the immersive and interactive experience provided by AR and AI technologies creates a gameplay experience that is both engaging and physically active. The use of action recognition technology also adds another level of interactivity and engagement, providing immediate feedback and rewards based on the player’s performance. Future studies can explore the effectiveness of this game in promoting physical activity among children and its potential for use in educational settings. III. MATERIALS AND METHODS Fig. 1 depicts the flowchart of the system’s underlying infrastructure. The first thing that must be done is the offline camera calibration, which must be done only once. Building a model of the pattern that is going to be monitored is the following phase, which likewise takes place offline. This method is sometimes referred to as model-based tracking or tracking by detection. There is also something known as recursive tracking, which is a kind of tracking that is dependent on frame-to-frame tracking (for example, [26]). It never finds a solution to the issue of errors piling up over time. Model-based tracking, on the other hand, does not have this issue. Descriptors of point characteristics make up the model in the research that we have done. A picture to use as a reference for the pattern is captured. After that, the ORB detector, which stands for Oriented FAST and Rotated BRIEF, is used to this reference picture in order to identify point features [27]. It is a feature detector that is both quick and reliable. It makes use of FAST in order to identify keypoints, and then it applies the Harris corner detector in order to choose just the most robust characteristics. Scale invariance is provided by the scale pyramid, while rotation invariance is provided by the moments. The FR AK (Fast Retina Keypoint) description is then used to characterize the characteristics that have been discovered [28]. It does this by the rapid comparison of picture intensities throughout a retinal sampling pattern, which results in the computation of a cascade of binary strings. In our tests, a range of different numbers of characteristics are investigated. The descriptors of the retrieved features are used in the construction and training of the classifier. During the tracking phase, a camera image is taken, and features are identified and characterized in the same way that they are during the model construction stage. Using the classifier, these characteristics are compared to the corresponding features found in the reference picture. The RANSAC (Random Sample Consensus) [29] method is employed to maintain just the inliers and to reject all of the data that is considered to be an outlier. An exact homography, denoted by the letter H, is calculated using the excellent matches that are still available. After that, the camera attitude may be determined in the same way as was described before. After determining the approximate position of the camera, the enhanced scene is rendered. Fig. 2 demonstrates detection of human and landmarks detection process. (IJACSA) International Journal of Advanced Computer Science and Applications, Vol. 14, No. 8, 2023 685 | P a g e www.ijacsa.thesai.org Fig. 1. Flowchart of the proposed augmented reality enabled game. Fig. 2. Person and landmarks detection flowchart. Fig. 3. Detection of landmarks of a human. (IJACSA) International Journal of Advanced Computer Science and Applications, Vol. 14, No. 8, 2023 686 | P a g e www.ijacsa.thesai.org Fig. 3 demonstrates the 33 landmarks of a human that the model can detect. Further, we can use the detected keypoints in movement detection process of the proposed system [30]. IV. RESULTS In this part, we will provide the experiment results. Fig. 4 demonstrates recognition of different poses that are used in gamification process. Human should stand in front of a camera. Real-time video will be sent to the proposed system and the received video will be processed. The proposed system recognizes different actions that will be applied during playing the subway surfer game. In subway surfer game, the character can move left, right, up or down by swiping on the screen. Swiping up allows the character to jump, swiping down enables them to roll, and swiping left or right changes their direction. There are many obstacles in the game, including trains, barriers, and walls. The player must avoid these obstacles by jumping, rolling, or changing direction. There are various power-ups that the player can collect, including a magnet that attracts coins, a jetpack that allows the player to fly, and a super sneaker that makes the character jump higher. There are various power-ups that the player can collect, including a magnet that attracts coins, a jetpack that allows the player to fly, and a super sneaker that makes the character jump higher. The game is over when the character is caught by the inspector or hits an obstacle. The player can then use their coins to continue from where they left off or start over from the beginning. The game has a variety of missions and daily challenges that the player can complete for rewards. Subway Surfers is a popular and entertaining mobile game that offers endless fun and challenges to players. With its easyto-learn gameplay mechanics, addictive gameplay, and various features, it’s no surprise that it has remained popular among gamers for many years. a) Starting the program b) Recognition of hands c) Start the game by joining hands d) Right shift recognition e) Stand in the center f) Left shift recognition g) Recognition of “Crouching” h) Recognition of “Standing” i) Recognition of “Jumping” Fig. 4. Recognition of different poses. (IJACSA) International Journal of Advanced Computer Science and Applications, Vol. 14, No. 8, 2023 687 | P a g e www.ijacsa.thesai.org Fig. 4 to 6 demonstrate the application of the proposed system in practice. Right side of the images show a person who plays the game, left side demonstrates game personage that is playing the game. Fig. 4(a) demonstrates starting the program. In the first stage, the program recognize a person in front of the camera. Fig. 4(b) and Fig. 4© demonstrate recognition of weather the hands are joined or not. When the status hands are joined, the game will be started. Fig. 4(d), Fig. 4(e), Fig. 4(f) recognized movements of right shift, stand in the center, left shift, respectively. Fig. 4(g), Fig. 4(h), and Fig. 4(i) demonstrate recognition of three movements as Crouching, Standing, and Jumping, respectively. All of these movements are used during playing the game. Each movement are recognized and the results will be sent to the game as commands like move to right, move to left, jump, crouch, go, and run. Fig. 5 illustrates example of crouching in playing the game. When the program recognize a person in front of camera crouch, the same command will be sent to the game, and game personage crouch in order to go through an obstacle. Fig. 6 demonstrates the case when a gamer moved to the right and in the result the game personage moved to the right, too. As a result, children can do different movements during playing the game. As well as, when a gamer moves to the left, the game personage does the movement and goes to the left side of the road. Fig. 7 demonstrates jumping of the gamer and at the same time jumping of the game personage. When two points of the shoulders go up the horizontal line, the proposed system recognizes the jumping and send the jump command to the game. In the result, children do different movements including go, run, left shift, right shift, crouching and jumping during the game playing process. Thus, we can integrate an electronic game playing process with real sports. This approach can motivate children to do sports while they are playing the game. In some cases, it can be useful for adults, too. Fig. 5. Example of crouching in playing the game. Fig. 6. Demonstration of the case when a gamer moved to the right. Fig. 7. Demonstration of case when gamer and game personage jumped together. V. DISCUSSION AND FUTURE RESEARCH The development of the Subway Surfers game with augmented reality (AR) based on action recognition and artificial intelligence (AI) has the potential to increase children’s motivation for sports. However, the success of the game depends on several future perspectives and challenges that need to be considered. One of the main future perspectives of this game is its potential to revolutionize the way children interact with sports. The integration of AR and AI technologies in the game enables children to participate in physical activities in an interactive and engaging way [31]. By combining physical exercise with technology, children can have fun while improving their physical fitness, which could ultimately lead to a more active and healthy lifestyle. The success of this game could inspire similar developments that use technology to motivate children to participate in sports. (IJACSA) International Journal of Advanced Computer Science and Applications, Vol. 14, No. 8, 2023 688 | P a g e www.ijacsa.thesai.org Another future perspective of this game is its potential to enhance the overall gaming experience. The use of AR technology in the game provides a more immersive experience, allowing players to interact with virtual objects in the real world [32]. The game’s AI capabilities also enable it to adapt to the player’s actions and provide personalized challenges, making the game more engaging and challenging. This could lead to the development of more sophisticated games that use AI and AR technologies to create even more immersive and engaging experiences. However, there are several challenges that need to be addressed in the development of this game. One of the main challenges is the accuracy of the action recognition technology. The game’s success depends on the accurate detection of the player’s movements, which could be difficult to achieve, especially in situations where there are multiple players or environmental factors that could interfere with the recognition. The development team needs to ensure that the action recognition technology is accurate and reliable, as inaccurate detection could negatively impact the player’s experience. Another challenge is the game’s accessibility [33]. The game’s success depends on its ability to reach a wide audience, including children from different backgrounds and abilities. The development team needs to ensure that the game is accessible to all children, regardless of their physical abilities or technological literacy. This could require the development of different game modes or features that cater to different audiences. Another challenge is the game’s safety [34]. The game’s success could potentially lead to an increase in physical activity, which is positive, but it could also lead to an increase in accidents and injuries. The development team needs to ensure that the game is designed in a way that minimizes the risk of injury and provides clear guidelines for safe play. Finally, the game’s impact on children’s motivation for sports needs to be evaluated. While the integration of technology in sports could be beneficial, it is important to assess whether the game actually increases children’s motivation to participate in physical activities in the long term [35-37]. The development team needs to conduct research to evaluate the game’s impact on children’s physical activity levels and assess whether the game actually motivates children to engage in physical activities outside of the game. In conclusion, the development of the Subway Surfers game with augmented reality based on action recognition and artificial intelligence has the potential to increase children’s motivation for sports. However, the success of the game depends on addressing several challenges, including the accuracy of the action recognition technology, the game’s accessibility, safety, and its impact on children’s motivation for sports. Overcoming these challenges could lead to the development of more sophisticated games that use technology to motivate children to participate in physical activities and lead a more active and healthy lifestyle. VI. CONCLUSION In conclusion, the development of the Subway Surfers game with augmented reality based on action recognition and artificial intelligence has the potential to revolutionize the way children interact with sports. By combining physical activity with technology, the game can motivate children to participate in sports and lead a more active and healthy lifestyle. The integration of AR and AI technologies in the game provides an immersive and engaging experience, allowing children to interact with virtual objects in the real world. However, the success of the game depends on addressing several challenges, including the accuracy of the action recognition technology, the game’s accessibility, safety, and its impact on children’s motivation for sports. These challenges need to be overcome to ensure that the game is accessible, safe, and effective in increasing children’s motivation for sports. Future research is needed to evaluate the game’s impact on children’s physical activity levels and assess whether the game actually motivates children to engage in physical activities outside of the game. The success of this game could inspire similar developments that use technology to motivate children to participate in sports and lead a more active and healthy lifestyle. Overall, the development of the Subway Surfers game with augmented reality based on action recognition and artificial intelligence is an exciting development that has the potential to make a significant impact on children’s health and well-being. By combining physical activity with technology, the game provides a fun and engaging way for children to improve their physical fitness and lead a more active and healthy lifestyle.” Did the above paper explain why AI and machine learning are critical to AR gaming ?Refer to the sentences from the paper to answer this question.
- Assistant
Yes, the paper did explain why Artificial Intelligence (AI) and Machine Learning (ML) are critical to Augmented Reality (AR) gaming, emphasizing their roles in personalizing the gaming experience and creating dynamic interaction models that adapt to the player's actions. Here are the key points highlighted in the paper that demonstrate the importance of AI and ML in AR gaming:
-
Personalized Gameplay Experience: AI and ML enable the game to adjust its difficulty level in real-time based on the player's physical capabilities. Action recognition uses computer vision algorithms to detect and classify human movements, such as running, jumping, and squatting. The game then uses this information to provide challenges that match the player's ability, ensuring engagement and motivation. This personalized approach is crucial in promoting physical activity effectively (Section I).
-
Real-Time Adjustments for Enhanced Engagement: The use of AI allows for the game to make instantaneous adjustments to the gameplay based on the player's actions. Machine learning techniques, such as reinforcement learning, are employed to optimize the gameplay experience dynamically. By learning from the player's movements and adjusting accordingly, the game creates a more challenging and captivating experience, thereby promoting sustained interest and physical activity (Section I).
-
Action Recognition for Interaction: The game leverages advanced computer vision and machine learning to recognize the player's physical movements. This capability is fundamental to AR gaming, allowing players to interact with the game through their movements in the real world. By identifying these actions accurately, the game can respond in meaningful ways, making the gaming experience more immersive and interactive (Abstract and Section I).
-
Development Integration of Technologies: The integration of various technologies, including Unity for game development, TensorFlow for machine learning algorithms, and Vuforia for AR elements, underscores the technical complexity behind creating an AR game that is responsive and adaptive to user actions. This synergy of technologies is essential for crafting experiences that are not only engaging but also responsive to the nuances of physical activity and player interaction (Abstract).
-
Efficiency in Promoting Physical Activity: The paper reports a preliminary study that found the game effective in increasing children's enthusiasm for physical activity. This efficacy is attributed to the personalized gameplay experience created by the action recognition and AI features, which keep the players engaged and motivated by continuously adapting the game's challenges to their individual abilities (Section I).
In summary, AI and machine learning are critical components of AR gaming as detailed in the paper, primarily because they allow for the creation of a highly personalized, adaptive, and interactive gaming experience. These technologies enable the game to respond intelligently to the player's physical actions in real-time, making the gameplay experience more immersive and motivating sustained physical activity.
-
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.