Kawasaki Robotics engineer Kippei Matsuda achieved a remarkable feat by winning the “NFL Health & Safety – Helmet Assignment” competition in 2022, a fiercely competitive event hosted by the National Football League (NFL) and Amazon.com, Inc. on Kaggle, a global AI data analysis competition platform. We spoke with Matsuda about how he won the competition against ambitious AI developers globally and how Kawasaki Robotics is currently developing the technology for various solutions.
Describe the NFL challenge, the solution, and what made it exciting?
The competition aimed to identify player collisions, with more than 1,000 data scientists worldwide competing to build an accurate solution using NFL game footage and sensor information. American Football is known to be one of the toughest, physically demanding sports in the world involving high-impact contact, and although players wear protective gear, injuries are prevalent. Head collisions, in particular, often result in serious injuries or disabilities, and it’s been challenging to find ways to reduce the impact of collisions along with effective treatment after an injury occurs.
If we could accurately identify which players suffered head impacts during a match, we could effectively administer treatment and advance the research into the effects of helmets and how to mitigate the impact. Doing this type of research manually would be extremely time-consuming, so the NFL held this competition to use AI technology to solve this problem.
The key to winning this competition was the successful integration of two distinct data types, video and sensor information. We utilized video images to identify player collisions and estimated player positions using sensors attached to the players. The development of an AI system that meticulously analyzed and predicted even the smallest changes in player position and posture, such as crouching or falling, set us apart. This innovation significantly improved our accuracy compared to other participants. As a result, our processing speed was 83 times faster than manual operation, and tasks that took 3-4 days could now be completed in just 2 hours, which was greatly appreciated.
Kippei Matsuda
In general, AI image analysis involves detecting objects in images, but in this competition, we had to consider the three-dimensional (3D) positions of the players on the field on the screen, which was an exciting challenge that we had never done before. It is difficult to analyze data because you have to look at it over and over again, but the NFL video was so powerful that I enjoyed watching it over and over again. Thanks to this, by the end of the competition, I was able to imagine the players’ movements just by looking at the titles of the videos.
What made you participate in the NFL challenge & what were the results?
It all started with me thinking it could be part of my studies. Theoretical aspects can be learned from books and other sources, but knowing how to use actual data and run a simulation is difficult. As a developer, touching and analyzing data and repeating trial and error is essential. I was attracted by the competition as it provided materials that led to practical learning.
“Participating in the competition allowed me to put AI development into practice”
Kippei Matsuda
Honestly, I never felt the competition itself was hard; what was difficult was finding time for my studies and family, as worked on this during my personal time. When I was playing with my children at the park, I would suddenly think, “Maybe I could do that part this way,” and it would bother me. It was hard to relax.
I was on the train, and I was shaking. Since the start of this project, I did not think I could win, but when I did, I was thrilled. I have no complaints about winning and was happy to share the news with everyone around me. I could hardly get any work done that day!
How do you work with AI vision today at Kawasaki Robotics?
I am currently involved in developing products that utilize AI vision specifically for robotics. AI analyzes images captured by cameras and processes them in various ways; for example, in our depalletization solution, it processes images of the product that need to be unloaded. Depalletizing solutions are used to improve unloading efficiency at distribution centers and factories. The Depalletizing Solution is equipped with 3D AI vision and is capable of highly sophisticated analysis of the cargo it handles.
Kawasaki Robotics depalletizing solution is equipped with 3D AI vision and is capable of highly sophisticated analysis of the cargo it handles. By specializing in unloading, we have achieved high performance at a low cost.
Kippei Matsuda
Manual unloading is very costly and time-consuming, and conventional robotic solutions are not flexible enough to handle the work. In this respect, our depalletizing solution has succeeded in increasing the accuracy and speed of automated unloading operations compared to conventional solutions.
For example, with conventional robotic systems solutions, all package sizes and shapes must be registered before being picked. If packages of unregistered shapes make it into the workflow, they cannot be processed. Our depalletizing solution however requires only the smallest and largest sizes to be registered, and all packages can be processed.
In conventional robotic systems the robot needs to know the correct size and shape of the product being handled. A camera is required in order to recognize and confirm the product size and shape. If it’s confirmed as correct the robot arm will pick the product, to teach the system this is a time-consuming process.
Our depalletizing solution requires no prior product registration other than the minimum and maximum dimensions, significantly reducing teaching time.
Kippei Matsuda
The main reason why Kawasaki Robotics’ depalletization solution has been successful is that we pair low-cost camera hardware with our robust AI software to process complex product images without a bunch of ad-ons. We achieved a high-performing and easy-to-use solution by focusing on developing a depalletizing system, and by narrowing down the functions, we achieved better cost performance than other companies’ products, making it easier for companies to introduce the product.
How will depalletizing and AI solutions continue to develop?
I believe that the efficiency of depalletizing solutions will increase as more and more data is collected and AI learning progresses. For example, one of the difficulties in developing a depalletizing solution was the strings and tapes on the surface of packages. When AI sees them, it may mistake the strings for the boundaries of the cardboard. It may then decide that the box is smaller than it actually is and take it by mistake. However, if the system is used at various sites in the future and data is collected, it will be able to learn multiple variations of packages. Then, even if a box has strings, tape, or stickers, the robot can unload it appropriately based on its past experience without being misled. Collecting a lot of good-quality data for the robot will be necessary.
I also think that AI, called the infrastructure model, will be key. One example is ChatGPT, which has been attracting much attention in recent years. Since the underlying model is trained based on a large amount of data, it has a very high recognition capability, a kind of common sense. Fundamental models can handle a variety of information, including text, images, and sound, and have the potential to dramatically expand the use of robots, not only in logistics. In the future, I would like to expand the use of AI by making good use of data and the basic model.
Click here for Depalletizing Solution product page(Japanese)