🕺🏻 Body Oracle

Ke Huang, Danlin Huang, Botao Amber Hu
2023-2025

Keywords:
Speculative Design, Chinese Oracle-bone Inscriptions, Intercorporeality, Bodily Awareness, Computational Linguistics, Cultural Heritage, Alternative Present, Mixed Reality, AI Generation

/ Concept /
In our modern information age, we face a deficit in bodily awareness during digital communication, despite Mehrabian's 7-38-55 rule suggesting that body language conveys 55% of meaning. Our current vocabulary struggles to describe complex bodily phenomena such as proprioception and social proprioception—the awareness of one's own and others' spatial body positioning. Body Oracle is a computational linguistic speculative design project, introducing an alterative language system of AI-generated hieroglyphic characters representing corporeal and intercorporeal poses inspired by ancient Chinese Oracle-Bone Inscriptions. These intuitive, cross-linguistic, cross-cultural signs—akin to emojis—aim to expand our capacity to conceptualize, understand, and communicate bodily and inter-bodily experiences. Drawing on linguistic relativity—the idea that language shapes worldview—we propose that these new hieroglyphic signs will function as a sociolinguistic mediator, influencing collective semantic cognition of bodily awareness. By envisioning an alternative present, Body Oracle challenges us to reconsider how embodied communication can transform our shared understanding.

/ "Body Oracle" Caster /
Body Oracle Caster is an interactive AI-generation tool. It features a screen and a camera that immediately transforms user’ movements into AI-generated Body Oracle characters. These characters mimic the pictorial style of ancient Oracle Bone Inscriptions (OBI) while reflecting each user’s unique motion.
To construct a comprehensive dataset, volunteers were instructed to replicate a series of poses derived from authentic OBI characters, capturing both individual and collaborative postures. The system employs a multi-step process underpinned by three core components.

Pose Estimation: The camera feed is processed through MoveNet—a TensorFlow Lite model that facilitates real-time pose estimation. Once the system determines that a user is maintaining a stable pose (i.e., minimal keypoint movement), the extracted skeleton data proceed to the next stage.
Initial Rendering (pix2pix cGAN): A conditional generative adversarial network (cGAN) based on the pix2pix framework is then applied to the 2D skeletal arrangement. This model was trained on paired pose–character data, enabling it to transform skeletal configurations into coarse, hand-drawn visuals suggestive of OBI glyphs.
Style Refinement (Neural Style Transfer): The pix2pix output is further refined using a neural style transfer network. This module incorporates oracle-inspired stylistic elements, adjusting line weight, curvature, and brush-like texture to enhance the resemblance to ancient script. The resulting pictograms retain anatomically relevant features while aligning with canonical OBI aesthetics.

/ "Body Oracle" Archive /
Body Oracle Archive is a continually expanding collection of somatic hieroglyphic characters. When the Caster detects a pose that does not match any existing character in the Archive, it employs cGANs and Neural Style Transfer to create a new symbol that adheres to OBI aesthetics. A large language model (GPT4o) then assigns a corresponding pronunciation based on ancient Chinese phonetics. Over several months, we collected extensive pose data from both public participants and professional dancers, yielding a comprehensive movement dictionary that links each character to its pose and pronunciation.

/ "Body Oracle" Translator /
Body Oracle Translator is a handheld mixed reality (MR) device that helps audiences experience the Body Oracle language system. It recognizes and analyzes body poses in real time, then references the Archive to display the corresponding Body Oracle characters while providing audible pronunciation of the recognized characters.
We created a binocular-style, oracle bone–inspired MR mask to encase the HoloKit, a smartphone-based, open-source, optical see-through stereoscopic headset. The translator’s shell is made from a ready-made oracle bone. An iPhone is mounted inside, and the mask design leaves the phone’s camera unobstructed, enabling environment sensing while preserving the headset’s optical see-through view for users.

When a user holds this MR translator up to a human body, a fine-tuned YOLO11 Pose Estimation model (running on the iPhone) identifies the individual’s pose in real time and compares it against previously generated Body Oracle characters in the Archive. By pressing the button at the top of the phone’s screen, the device audibly pronounces the recognized character. The resulting character and its pronunciation are displayed at the center of the translator.