New AI-Powered Software Developed to Edit 3D Models
Researchers have developed a new software that allows three-dimensional designs generated by artificial intelligence to be repaired and prepared for printing using natural language and visual prompts.
Developed through a collaboration among researchers from MIT CSAIL, Google, and Northeastern University, the new software called InstructMesh aims to solve the practical problems of three-dimensional models generated by artificial intelligence.
Shortcomings of AI Models
AI models know what objects should look like, but they cannot fully understand how they should function. This leads to the emergence of impractical designs that remain non-functional in daily use.
Function of InstructMesh Software
The new software, named InstructMesh, makes designing and printing objects that will function in the real world—such as household items, accessories, and robots—much easier. Users can make modifications to the created designs.
Visual and Text-Based Power
The software combines Microsoft's TRELLIS system, which generates 3D models from text and visual prompts, with GPT-4, the large language model that powers ChatGPT. In this way, visual and textual information meet in a common space.
Testing Process and Success Rate
Researchers conducted tests to see whether novice users could detect design flaws prior to production. Users were able to correct these errors, which were reviewed by experts, at a rate of 90 percent.