Creative Computation played a part in this year's SXSW with Tweetlantis, a 30-foot projected digital aquarium that used Processing sea creatures to represent top Twitter-trending topics at the event. Photo courtesy of Lisa Kaselak.
Creative computation – the blending of art and code – is Meadows’ newest form of artistic expression.
Creating Coding Artists
The student stood at the front of the darkened classroom and cleared his throat. “This,” he said, “is the result of a sleepless night.”
His classmates laughed in recognition as a stick-like windmill filled the wall behind him, its orange sails slowly rotating on a pale green background.
They, too, had spent many late-night hours in front of their computers, using a coding language called Processing to build a Rube Goldberg sequence of images involving a windmill, a mouse (the rodent variety) and a triangle of cheese. The assignment was an exercise in two-dimensional creative computation: images and sequences created using code, displayed on a screen or wall.
But producing creative computation in 2D is only one way to express the craft. In the hands of experienced practitioners, it can bloom into 3D interactive art. Companies, artists, designers, architects and researchers are increasingly using it to create an incredibly diverse range of projects: television commercials for Nike and Budweiser; music videos for Radiohead and Modest Mouse; immersive museum exhibits; interactive floors and walls; and more.
Combining technology and art, creative computation is the newest generation of human expression.
Coding : The next required language?
According to Ira Greenberg, director of the SMU Center of Creative Computation, the day is coming when learning coding will be a requirement for college students and programming literacy will no longer be just for the “engineer” types. “In the UK, they are having a serious debate about national programming literacy requirements in schools, expanding the three Rs (reading, writing and arithmetic) to include coding,” says Greenberg.
Despite ever-increasing technology influencing every aspect of our world, Greenberg recognizes that not everyone is comfortable with math and logic. But he wants to change that. “Coding should become more innate, natural, not this big separation,” he says. “I want the Center to develop new programming languages that are radical departures from the coding barriers currently in place for non-technical people.
“In short, we want to bring code to the masses.”
To do that, Greenberg designed the Center to be different from typical university computer science programs, which traditionally focus on engineering practices, mathematics and operations research. Instead, the SMU Center takes a cross-disciplinary approach, reaching students across the humanities and arts as well as engineering and computer science. Since its establishment in 2010, the Center has crosspollinated with dance, sculpture, theatre, music and digital humanities to create projects that are equal parts art, computer science, creativity and ingenuity.
The Center of Creative Computation is a place of possibilities, where future applications for technology are emerging.
Kinecting the Dots
Christopher Dolder, assistant professor of dance and former soloist with the Martha Graham Dance Company, wants to create a system that shows which muscles are being used when a dancer moves. An expert in kinesiology, he is researching ways to create the first-ever physical data capture lab, with capabilities that go beyond current motion-capture technology. To get there he is collaborating with faculty and students associated with the Center of Creative Computation.
In a traditional motion-capture setup, an individual is outfitted with white dots placed strategically along the body: shoulders, head, torso. The person moves while a camera registers reference points of where the white dots move in space, the results of which are often used for applications such as motion picture animation. Dolder says the motion-capture model of following the dots isn’t enough.
“I want the data readings to tell me more about the architecture of movement – not just what arm is moving where, but exactly which muscles are being used at any given moment in time. With traditional motion-capture, if a person is standing on the forcemeasuring platform, the data may or may not give enough information to decide whether the individual is leaning slightly forward from the lower, middle or upper back,” says Dolder. “I’m creating something that will tell me what is happening when someone’s hips are moving from point A to point B, at what angles and speed; we’ll be able to dissect the force values and see from a probability standpoint which muscles are doing what. We’ll be able to analyze what happens kinesiologically; this can be tremendously helpful to a dancer who wants to correct or maximize his or her technique.
“What we’re approaching is a scientific readout of a dancer’s aesthetic. It’s scientific patterns of art.”
Collaborating with faculty, researchers and students from both Meadows School of the Arts and Lyle School of Engineering, Dolder is creating a force-measuring, sensor-embedded platform on which an individual can stand and move. The prototype platform will be a 10’ x 10’ stage that will “feel” and measure where the individual, in this case a dancer, moves. He’ll then add three Xbox Kinects, which map the individual using infrared scanning, to create a three-dimensional image. Then, he’ll connect the Kinects to powerful software that will render the data surrounding the detailed captured images.
“My thought is to record not just points in space but an entire map of the body that yields data of where the individual is at any given moment. What I’m after is an actual pressure mapping of what exactly is on the force-measuring platform. If I know, for example, that there is more pressure on this metatarsal as opposed to that one, then I can know which muscles are being used – and I can correct or improve the technique of the dancer.”
Dolder says it’ll take research funding to provide the level of processing power and development needed to see the project all the way through.
Fusing Traditional Art with Technology
Greenberg, author of three books on creative coding and the Processing programming language, says “creative computation” can be described in three simple words: It’s a medium. Like a painter uses paint, the creative computation practitioner uses code to create art, interactive pieces and a whole new way of looking at things.
Trained as a traditional artist, Greenberg appreciates the tactile nature of working with paint and the unpredictable characteristics it can present, such as dripping or smearing. But he also hears the siren call of things logical and mathematical; he has a drive to understand the mechanics of the universe.
“As a painter, I used to stare at nature and study how things were put together and then try to reproduce those kinds of things,” he says. “But as a software artist I ask, ‘what are the generative algorithms in nature?’ And so, using code, I can construct a tree in a kind of similar way, except I build a model of a tree that can be iterated – I can build families of trees.
“People tend to think of ‘art’ as a synthetic model for reflecting our lives. With ‘computation,’ people tend to think of it as a tool to do math, build bridges and so forth. But computation is really a synthetic model of cognition. It’s something much more dimensional than paint or clay, or even words.
“I’ve come to understand that creative computation can create practically anything. Any information can be turned into digital information. Even the human genome is data now. Computation is a multi-dimensional, universal creative medium.
“In some ways, it’s the purest art form.”
Bringing Innovative Minds Together
A smorgasbord of professors from SMU Meadows, Lyle and Dedman convey the many aspects of creative computation, covering opensource Processing, computer science, algorithmic studies, programming, interactive applications and computer animation.
New media specialist Brittany Ransom joined the faculty in Fall 2012. Ransom is one of a growing number of new media artists interested in the boundary between human and animal societies. Her projects aim to heighten awareness of the world from the point of view of other species. Her work is interactive and often relies on direct contact between organisms and technology; past projects have included work with termites and beetles, and seedlings designed to grow and display Twitter hashtags.
Students in her Art and Code classes will spend time with the Arduino board, a small, programmable micro-controller that has the ability to affect its surroundings by controlling lights, motors and other actuators.
“They’ll learn how to use code – open-source Processing software – to make it work within their projects, from idea to execution,” says Ransom.
All together, the creative computation curriculum currently offers approximately 50 different classes for students to choose from while they pursue a B.A. or a minor in creative computing.
Hard Work Required
“Creative computation is not a ‘lazy major,’” according to junior Richard Braxton, who is double-majoring in creative computation and geology. Braxton says the curriculum quickly weeds out students who think they can come to class and relax with a game of FarmVille™.
“With Professor Greenberg’s accelerated teaching style, you have to stay on top of things,” he says. “Even for experienced programmers, at the end of each class there is always something that students have to read up on, chase down or compare notes on with other students in the class. Even though you may walk into the class knowing a lot about computer science and art, you quickly figure out there’s a lot more to learn.”
Braxton recognizes that creative computation combines both math and art and says students will probably emphasize one or the other. He leans more toward the computation side, and remembers fondly the capstone project he did for an Art and Code class.
“I synced a Mac with a Nintendo Wiimote and set it to play two audio files at the same time. One file played ambient noise, like people talking, chairs scraping across the floor, while the other played a compilation of Samuel L. Jackson quotes. Depending on which direction you moved the Wiimote, the volume on one or the other audio track would change. It gave the person interacting with the piece a sense of control over the ambient environment.”
When the Center opened in 2010 it started out in a cramped office space just off the Bob Hope Theatre lobby. But Greenberg’s vision for the Center was much bigger.
“I see the Center as an interdisciplinary research and teaching center,” he says. “It’s the whole package: exploration of computation as a universal generative medium, integration of creative development, quantitative analysis and interdisciplinary synthesis.” Examples of Center projects include hardware and software development, digital media/arts production, visualization, interactive performance, intermedia practice, digital humanities and pedagogical development, among others.
In 2011, the Center moved from its original small office to its own dedicated lab and classroom next to the Hamon Arts Library in the Owen Arts Center, complete with state-of-the-art, ceiling-to-floor projections, 7.1 surround sound, a physical computing lab and a 10-screen, highresolution video wall. Greenberg says they’re just getting started.
“It’s a very, very exciting time to be at SMU,” he says. “In addition to the Center of Creative Computation, the University is expanding its high-performance computing capabilities, developing new interdisciplinary programs and centers, and in general strategically raising the bar. SMU is the most entrepreneurial and forward-thinking institution I’ve worked at, and I’m really excited to continue to grow the Center of Creative Computation at such a world-class university.”