[AI]legory of the Cave

Collaborators: Skye Gao, Leonie Dong, Cathy Fang, Alejandro Lopez-Rodriguez

Overview

In Plato's Allegory of the Cave, prisoners chained to a wall perceive only shadows—and mistake them for reality. We extend this concept in our piece Plato's [Al]legory of the Cave, an interactive installation featuring an LLM-controlled character who exists only as a shadow. The piece begins dark. A physical light switch invites interaction; flipping it on illuminates the stage and awakens the shadow man. His singular goal is to turn off the light that defines his existence. Humans place objects in his path—blocks, frames, and stairs—to obstruct him. He kicks, pushes, sometimes gives up, then begins again. The cycle repeats as he discovers changes to his world that he cannot comprehend.

Concept

In Plato's Allegory of the Cave, prisoners are chained facing a wall, seeing only shadows cast by objects behind them. They mistake these shadows for reality, never comprehending the three-dimensional world that produces them. The shadow man is such a prisoner: confined to 2D, he perceives silhouettes but never their source.

This mirrors the perceptual limits of AI. An LLM cannot see. It can only read descriptions of sight. It processes language about the world without directly accessing the world itself. The shadow man embodies this gap. Both are confined to representations.

The piece is a meditation on AI as cave-dweller - processing patterns without grasping the higher-dimensional reality that casts them.

Software

The interface is written in JavaScript and is web-based. A camera captures the projected shadow and the camera feed is thresholded by color such that the shadow is separated from the background. The figure's shadow is cancelled out.

The world view is converted into ASCII and sent to an LLM, where the shadows, the figure, the ground, and the light switch are represented by different symbols (such as "#", "P", "G", "S"). The LLM is prompted with the scene information and the available movements the figure can make (such as walk, jump, climb, and kick). The goal is for the figure to turn off the light switch. However, the figure is bound by the game-world physics (done by Matter.js), and cannot move through shadows and can stand on top of shadows. For example, the figure is able to kick away obstacles.

The obstacles in the real world are controlled via serial port connection with a microcontroller; details are described in the Hardware section.

Hardware

The light switch is connected to a solenoid and can sense when it is turned on or off. This is how the piece turns on—it invites people to interact with it.

The obstacles have magnets embedded inside and are lightweight. When the figure decides to kick an object, a command is sent to a connected ESP32 XIAO which controls an electromagnet. There is a Darlington pair, flyback diode, and bleed resistor to protect the GPIO of the microcontroller. When the electromagnet turns on, it creates a magnetic field which interacts with a magnet embedded in the blocks, causing the blocks to move.

The different movements of the blocks are controlled by the relative weight of each of the blocks and the placement of the magnets.