Creativity and Innovation at the Intersection of AI, Computer Graphics, and Design
At SIGGRAPH 2024, multi-institutional research teams to showcase their novel work bringing digital characters and environments to life.
At SIGGRAPH 2024, which will be held 28 July–1 August in
"The impact of generative AI on 3D graphics and interactive techniques is profound, poised to democratize content creation and lower barriers to entry in creative industries. Generative AI can facilitate real-time, personalized experiences in gaming, architecture, and industrial design, revolutionizing fields through procedural content generation and rapid prototyping," Qixuan (QX) Zhang, project lead of research that will debut at SIGGRAPH 2024, says. QX and his collaborators will present CLAY, a large-scale 3D native generative model that accurately captures complex geometric features and generates realistic textures in a variety of objects.
"We anticipate a future where generative AI enhances productivity and creativity across professional domains, making advanced 3D content creation and interaction a part of everyday life for a global audience," QX adds.
Following SIGGRAPH's 50th conference milestone in 2023, the conference this year will showcase exciting new research fueling the future — advances developed from a core place of imagination and creativity.
As a preview of the Technical Papers program, here is a sampling of three novel methods and their unique approaches incorporating generative AI to the vast field.
Dreaming Up High-Quality, Realistic Images via Text Prompts
Creating high-quality appearances of objects is a critical task in computer graphics because they can significantly improve the realism of rendering in a variety of applications — movies, games, and AR/VR. An international team of researchers from Zhejiang University,
The researchers set out to make this specific artistic task easier and faster with a tool for efficiently creating object appearances such that even novice users can do so with just simple text prompts. In this work, they fine tuned a new light-aware 2D diffusion model to condition on a given lighting environment and generate the shading results on this specific lighting condition. Then, by applying the same environment lighting in the material distillation, DreamMat can generate high-quality PBR (physically based rendering) materials that are not only consistent with the given geometry but also free from any unwanted and unrealistic shading effects.
"DreamMat has significantly lowered the barrier to creating PBR materials for untextured meshes (only using text prompts), enabling the generation of high-quality appearance that can be seamlessly integrated into games and films — a feat previously unattainable with earlier generative techniques,"
DreamMat's automatic creation of PBR materials can be imported directly into modern graphics engines such as Blender, GameMaker, or Unity for efficient rendering. In addition to Jin, the team comprises
Transforming Realistic 3D Assets, Quickly and Efficiently
In just recent years, the world has witnessed rapid advancements in generative AI across various modalities — text, images, audio, and video. With the public release of ChatGPT and the like, a much wider audience now has a direct connection and firsthand experience with how AI works. But despite significant research achievements in the 3D computer graphics field, industrial adoption has lagged.
In a collaboration between ShanghaiTech University and Deemos Tech-China, researchers set out to better understand the limitations game developers and CG teams are facing with existing 3D generation approaches. What they learned is that the traditional "reconstruction" methods fall short of industry needs. True 3D generative AI "must go beyond mere reconstruction; it must comprehend user requirements, support multitasking, and offer customization options akin to the vibrant 2D generative AI community," QX, a member of the ShanghaiTech-Deemos team that will be presenting their research at SIGGRAPH 2024, says.
The researchers have developed CLAY, a native 3D generative model with 1.5 billion parameters that accurately captures complex geometric features and generative realistic textures. Trained on a comprehensive 3D dataset, CLAY's uniqueness, say the researchers, lies in its alignment with the specific needs of the 3D industry — usability being its primary focus and design ability to cater directly to practical applications.
Within a minute, CLAY can generate detailed 3D assets with PBR materials adaptable to various conditions, from text, image to 3D. Their approach denoises 3D data, and the researchers say that when compared to previous similar methods, CLAY generates detailed geometries in seconds while preserving essential geometric features, such as flat surfaces and structural integrity, of multiple objects.
"Looking ahead, we envision generative AI technologies in the 3D field liberating artists from repetitive tasks, allowing them to focus on creativity and innovation," QX adds. "We aim to extend 3D generation technologies to everyday applications such as virtual try-ons for online shopping, enhanced creative in the metaverse, and novel gameplay experiences in gaming."
Working with QX on the CLAY development are collaborators Longwen Zhang,
Effortless Character Control and Real-Time Diverse Movement
Real-time character control is the holy grail in computer animation, playing a pivotal role in creating immersive and interactive experiences. Despite significant advances in this area, real-time character control continues to present several key challenges. These include generating visually realistic motions of high quality and diversity, ensuring controllability of the generation, and attaining a delicate balance between computational efficiency and visual realism.
In gaming, for example, consider the non-player characters (NPCs) and how they have been limited to exhibiting monotonous movements due to their pre-programmed reliance on a limited set of animations. These characters often repeat the same motions, lacking individuality and realism.
Indeed, to achieve more dynamic characters carrying out realistic motions in the virtual world, a global team of computer scientists has created a novel computational framework that overcomes these key obstacles. The team, comprising researchers from the Hong Kong University of Science and Technology, the University of Hong Kong, and
At the core of their method, say the researchers, is a transformer-based Conditional Autoregressive Motion Diffusion Model (CAMDM), which takes as input the character's historical motion and can generate a range of diverse potential future motions, in real time. Their method can generate seamless transitions between different styles, even in cases where the transition data is absent from the dataset.
"With this new technique, we're on the verge of witnessing a more realistic digital world. This world will be populated with thousands of digital humans empowered by our technique, each exhibiting unique features and diverse movements,"
The team is currently working with industry to incorporate their technique into a video game product, with an expected launch date in late 2024. In future work, they intend to expand their method to be applied to a few key areas, including animation and film and to enhance virtual reality user experiences. At SIGGRAPH this year, Chen will be joined with co-authors and collaborators
This research is a small preview of the vast Technical Papers research to be shared at SIGGRAPH 2024. Visit the Technical Papers listing on the full program to discover more generative AI content and beyond.
About ACM, ACM SIGGRAPH, and SIGGRAPH 2024
ACM, the Association for Computing Machinery, is the world's largest educational and scientific computing society, uniting educators, researchers, and professionals to inspire dialogue, share resources, and address the field's challenges. ACM SIGGRAPH is a special interest group within ACM that serves as an interdisciplinary community for members in research, technology, and applications in computer graphics and interactive techniques. The SIGGRAPH conference is the world's leading annual interdisciplinary educational experience showcasing the latest in computer graphics and interactive techniques. SIGGRAPH 2024, the 51st annual conference hosted by ACM SIGGRAPH, will take place live 28 July–1 August at the
View original content to download multimedia:https://www.prnewswire.com/news-releases/creativity-and-innovation-at-the-intersection-of-ai-computer-graphics-and-design-302185164.html
SOURCE SIGGRAPH
Serious News for Serious Traders! Try StreetInsider.com Premium Free!
You May Also Be Interested In
- The TADROS brothers, founders of Liberty City, from Montreal, Canada receive the support of a globally recognized billionaire from Silicon Valley for a technology project in Quebec
- Maximizing Financial Freedom: A Guide to Applying for Credit Cards Online
- From Concept to Launch: How Trikon Booster Transforms Game Development with $1 Million Grant
Create E-mail Alert Related Categories
PRNewswire, Press ReleasesSign up for StreetInsider Free!
Receive full access to all new and archived articles, unlimited portfolio tracking, e-mail alerts, custom newswires and RSS feeds - and more!