New Delhi, December 18
Meta (formerly Facebook) researchers have developed an artificial intelligence (AI) system that can automatically identify and animate human figures in children’s drawings with a high success rate and without any human assistance.
“Humans” in children’s drawings come in many different shapes, colors, sizes, and scales, with few similarities in body symmetry, morphology, and point of view.
“We are excited to announce a first-of-its-kind method to automatically animate children’s hand-drawn figures of people and human characters (a character with two arms, two legs, a head, etc.) that bring these drawings life in minutes using AI,” Meta researchers said in a statement.
By uploading the artwork to its prototype system, parents and children can experience the excitement of seeing their designs become moving characters that dance, hop and jump.
“They can even upload their cartoons to share with friends and family. If parents want, they can also submit those cartoons to help improve the AI model,” Meta said.
The first step in animating children’s drawings of people is to distinguish the human figures from the background and other types of characters in the image.
Object detection using existing techniques works quite well on children’s drawings, but segmentation masks are not accurate enough to use for animation.
“To solve this problem, we instead use the bounding boxes obtained from the object detector and apply a series of morphological operations and image processing steps to obtain masks,” the researchers explained.
They use Meta AI’s convolutional neural network-based object detection model, “Mask R-CNN”, to extract human characters in a child’s drawing for processing.
“Mask R-CNN” is pre-trained on one of the largest publicly available segmentation datasets, but it’s made up of photos of real-world objects, not drawings.
“We invited our colleagues at Meta to share and animate their children’s works using our system, and we got around 1,000 drawings that helped us train the AI,” the researchers said.
To animate the 2D figures using 3D motion capture, the researchers took advantage of the fact that many children draw using what is known as twisted perspective.
“We take advantage of this perspective in our move retargeting step. Independently for lower and upper body, we automatically determine whether the move is more recognizable from a front or side view,” the Meta team said. HAVE.
One day, an AI system could take a complex drawing and then instantly create a detailed cartoon using multiple interacting fantasy characters and background elements.
“With the AR glasses, these stories could even seem to come to life in the real world, dancing or talking with the child who drew it moments earlier. The possibilities are as limitless as the human imagination,” said said the researchers. IANS