AI for creativity

For many centuries, graphic representation in science has been based on two dimensional sketches, like Leonardo da Vinci’s famous drawings. “Such visual representations are mandatory to understand and create. They increase the intuition about the observed phenomena and help structuring them into series of simpler stages” explains Marie-Paule Cani, professor at École Polytechnique and president of the Computer Science Department (DIX*). She is renowned in the field of computer graphics, the goal of which is to help generate three dimensional, animated virtual worlds.

Usually, designing animated 3D contents requires an iterative process between the scientist and a digital artist trained on dedicated tools. Furthermore, the scientist cannot modify and “play” with the final result. “Could digital images be turned into a creative media, more expressive but as simple to use as a pen, enabling scientists to directly express these visions in the form of interactive 3D environments, and then manipulate them, to get inspiration and refine their thoughts?” asks Marie-Paule Cani. This is the goal of her project with HI! PARIS.

Creating computer-generated images involves three stages: geometric modelling, animation and rendering, i.e. the calculation of the final image. The first two stages are at the heart of Marie-Paule Cani’s research, who compares 3D creators to “little gods who create their virtual worlds and choose their laws”. During her career, she has developed a series of new methodologies for computer graphics, such as expressive modelling that combines gesture-based design and interactive models embedding knowledge. Creative industries, from video games to cinema, are the primary users of her work, but her approach is also relevant for scientists from other disciplines, with whom she collaborates a lot: biologists for the modelling of trees, geomorphologists for the formation of mountain ranges, vulcanologists for the visual simulation of volcanic plumes, prehistorians for the reconstruction of the ecosystem of the Tautavel valley 450,000 years ago… and the list goes on.

Obviously, artificial intelligence has its place in the toolbox of computer graphics.  “But certainly not to create for us!” warns the researcher. “AI should come in as an assistant that provides us in real time with the most suitable tool and allows the models to react better to our creative gestures. For example, we may want to make an intelligent cut and paste, such as taking the clothes from one character and putting them on another, making sure that they automatically adapt to the new morphology.” The AI can also learn from examples to place distributions of elements in a virtual world, for example by providing a “vegetation” brush that allows to create landscapes in a plausible way, considering the climate and nature of the soil, the local slope, and the interactions between plant species.

This project requires a new convergence between human-computer interfaces, 3D graphics and AI. Hopefully, it will grant scientists access to a testbed environment where they will be able to experiment with their ideas: draw a 2D sketch of their mental model, such as a cell for a biologist, and immediately see a 3D world that matches, animate it and then inject their knowledge on the fly, for example, the fact the cell deforms at constant volume. AI will be used to make the models react in an expected way to their creative gestures. This should help with design thinking in science, as well as in engineering and education.