“AI Creativity: AI.R Taletorium, New Storytelling Realm” presented by Nikolic, Liu and Bertin
Symposium:
- ISEA2022: 27th International Symposium on Electronic Art
-
More presentations from ISEA2022:
Session Title:
- Nonhuman Creativity and Agency
Presentation Title:
- AI Creativity: AI.R Taletorium, New Storytelling Realm
Presenter(s):
Venue(s):
Abstract:
While stories provide an effective and concise way to share experiences, visual content acts as a more comprehensive and universal communication tool that transcends barriers between user groups with different cultural backgrounds and physical preferences. However, the meaningful learning process for coherent story visualization is a challenging task in AI studies. This paper proposes AI.R Taletorium, an intelligent artistic fairytale visualizer that interacts with users in real-time. Furthermore, language and mental development are key aspects of early childhood education.
Our system is collaborative and enables common creative experiences between groups of children at remote locations. We aimed at offering an entertaining and imaginative platform for children to communicate with each other, improving their psychological health, especially under extreme circumstances such as the COVID- 19 pandemic. The system comprises two novel modules, a character-centric storyline generator and a visualizer in the form of doodlers. We connect the modules with characters and build a bidirectional link to allow kids to participate in the fairy tale creation process by simple adding their drawings.
Our key contribution is as follows:
– A novel multimodal fairytale co-creation interface based on interactive text-to-image transfer and vice-visa.
– The system presents a novel fusion method between a learning-based language model and rule-based graph update, allowing for more flexibility in story generation and visualization with limited data.
– Proposing a CLIP transformed story graph as an intermediate representation to transcend the barrier between digital storytelling content and insufficient illustration data (automatic AI fairy tales storytelling case).
We fuse the rule-based language model with a learning-based generation model into a unified visualization system to allow for more flexibility during generation. AI.R Taletorium can visualize complex fairy tale scenes with stylized characters and rare relationships with our fused framework.
