| |
CARNEGIE MELLON RESEARCHERS TEACH COMPUTERS TO PERCEIVE THREE DIMENSIONS IN 2-D IMAGES
03 August 2006 - Carnegie Mellon Universtity
| We live in a three-dimensional world but, for the most part, we see it in two dimensions. Discerning how objects and surfaces are juxtaposed in an image is second nature for people, but it's something that has long flummoxed computer vision systems. |
Now, however, researchers in Carnegie Mellon University's School of Computer Science have found a way to help computers understand the geometric context of outdoor scenes and thus better comprehend what they see. The discovery promises to revive an area of computer vision research all but abandoned two decades ago because it seemed insoluble. It may ultimately find application in vision systems used to guide robotic vehicles, monitor security cameras and archive photos. Using machine learning techniques, Robotics Institute researchers Alexei Efros and Martial Hebert, along with graduate student Derek Hoiem, have taught computers how to spot the visual cues that differentiate between vertical surfaces and horizontal surfaces in photographs of outdoor scenes. They've even developed a program that allows the computer to automatically generate 3-D reconstructions of scenes based on a single image. "The technique provides an approximate sense of the scene, a qualitative grasp of the structure of a scene," said Efros, assistant professor of computer science and robotics. In their latest work, to be presented at the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, June 17-22 in New York City, the Carnegie Mellon researchers will show that having a sense of 3-D geometry helps computers identify objects, such as cars and pedestrians, in street scenes. Identifying vertical and horizontal surfaces and the orientation of those surfaces provides much of the information necessary for understanding the geometric context of an entire scene. Only about three percent of surfaces in a typical photo are at an angle, they have found. Using 300 images gleaned from a Google search, Hoiem showed the computer numerous examples of vertical and horizontal surfaces, allowing a machine learning program to develop statistical associations between certain shapes, shadings and other characteristics typical of each orientation. The program also takes advantage of the constraints of the real world, skies are blue, horizons are horizontal and most objects sit on the ground. "In our world," noted Hebert, a professor of robotics, "things don't just float." To demonstrate the utility of this technique, the researchers have designed a graphics program to automatically generate 3-D reconstructions by "cutting and folding" along vertical and horizontal lines in an image. "It's like a children's pop-up book," Efros said. "The amazing thing they did was show that it was actually possible," said computer vision pioneer Takeo Kanade, the U.A. and Helen Whitaker University Professor of computer science and robotics at Carnegie Mellon. "I would say it's a breakthrough." Inability to understand the geometric context of a scene has limited the ability of computers to recognize objects. Though researchers have had some success at identifying objects, such as faces or cars, the lack of context results in preposterous mistakes, such as faces seen in clouds, or cars perched in treetops. Scientists have struggled since early times to understand how people visually perceive three dimensions. Ancient Greeks reasoned that the eyes must emit rays that bounce off objects, measuring distances much like today's laser rangefinders. By the 19th century, scientists realized that a pair of eyes gives humans binocular vision, allowing them to perceive depth. But stereoscopic vision is useful at distances of no more than 50 meters. Even then, the mind often overrides binocular vision, such as when watching a football game on television. Vision was an early problem that artificial intelligence researchers tried to tackle and "context-based" outdoor scene analysis was a favorite subject during the 1970s. Researchers found they could describe the geometry of an object, such as a chair, but matching the description with actual pixels proved a herculean task. Statistical learning tools were limited then and research computers were about 100 times less powerful than a typical laptop today. By 1980, most had concluded that the feat was either impossible or, if possible, computationally impractical. Even when Efros and Hebert assigned Hoiem to use machine learning techniques to teach visual context to a computer two years ago, they regarded it primarily as a learning exercise for their student. "We didn't believe it would work," Efros said. To their surprise, Hoiem found the computer often discerned which surfaces were vertical or horizontal, and whether a vertical surface faced left, right or toward the viewer. Based on the examples it was shown, the computer identified each feature in an image and assigned to it a probability that it had a horizontal or vertical orientation. In their latest work, the researchers have used the geometric context information to improve the ability of computer programs to recognize objects within the scene. And improved object recognition, they note, should ultimately provide feedback to further improve understanding of the geometric context.
http://www.cmu.edu
About: Carnegie Mellon Universtity
The Carnegie Institution of Washington (www.carnegieinstitution.org) has been a pioneering force in basic scientific research since 1902. It is a private, nonprofit organization with six research departments throughout the U.S. Carnegie scientists are leaders in plant biology, developmental biology, astronomy, materials science, global ecology, and Earth and planetary science. Since its founding in 1900 by industrialist and philanthropist Andrew Carnegie, Carnegie Mellon University has been a pragmatic institution, adapting rapidly to change. In fewer than 100 years it has changed its name three times--each transition marking a milestone in the institution's 20th century evolution. Whether it was Carnegie Technical Schools, as it was in its first 12 years, Carnegie Institute of Technology, its name from 1912 to 1967, or Carnegie Mellon University, three primary purposes formed its foundation. Throughout this century, Carnegie Mellon has focused on delivering distinctive and first-quality education, fostering research, creativity and discovery, and using the new knowledge created on campus to serve our larger society. When Arthur A. Hamerschlag served as the school's first president, Carnegie Technical Schools' 12 professors and six administrators sought to educate the sons and daughters of Pittsburgh workers for employment in the region's growing industries. These educators served the vision of Carnegie by organizing into four faculties: the School of Science and Technology, the School of Fine and Applied Arts, the School of Apprentices and Journeymen, and the Margaret Morrison Carnegie School for Women. In its earliest years, the institution served primarily part-time and undergraduate students. The faculty, many of whom did not have doctor's degrees, focused on teaching and curriculum development. But research efforts began as early as 1916 when the Division of Applied Psychology of the Carnegie Institute of Technology developed rating scales for job placement. This rating system was used to classify two million men for placement in the armed forces during World War I. Research bureaus were organized in coal mining, nuclear physics, applied chemistry and metallurgy. And by granting the nation's first undergraduate degree in drama in 1917, the institution began a tradition of leadership in the arts that spanned the century. Through research and the education of its students during the administration of President Thomas S. Baker in the 1920s and '30s, the institution began its strong tradition of transferring knowledge and skills to industry and government. Building on this firm foundation, the administration of President Robert E. Doherty introduced a new approach to education that would be used as a model by similar institutions around the nation. The Carnegie Plan for Professional Education, initiated in 1939-40, required engineering and science students to take a quarter of their courses in a new Humanistic and Social Relations sequence. In addition, its curriculum focused on teaching students problem-solving techniques, a hallmark of the Carnegie Mellon educational experience today. While the Doherty administration has been credited with this educational innovation, it also oversaw growth in the institution's research capability. Between 1936 and 1950, the number of graduate students grew from 36 to more than 260. The research budget ballooned from $156,000 to $1 million. In the 1950s, the newly formed Graduate School of Industrial Administration, endowed by William Larimer Mellon, emerged as one of the three or four best business schools in the nation. (In 2004 the school was renamed the David A. Tepper School of Business after benefactor and alumnus David Tepper (MBA '82).) Today, the school is recognized as a pioneer in the field of management science and one of the top business schools in the world. The Warner administration oversaw the institution's burgeoning research enterprise. This period of research growth was aided by the work of the institution's Computation Center, founded in 1956 to provide computing services to the campus. A major grant from benefactor Richard K. Mellon in 1965 aided the establishment of a Computer Science Department, a department which would be the genesis of Carnegie Mellon's worldwide reputation in computer science. By the end of the Warner administration and the start of the administration of President H. Guyford Stever in 1966, Carnegie Tech had most elements of a university. Its merger in 1967 with the Mellon Institute created Carnegie Mellon University and brought a $60 million endowment, extensive research facilities and renowned research personnel to the institution. Five years later, President Richard M. Cyert (1972-90) began a tenure that was characterized by unparalleled growth and development. The university's research budget soared from about $12 million annually in the early 1970s to more than $110 million in the late 1980s. The work of researchers in new fields such as robotics and software engineering helped the university build on its reputation for innovative ideas and pragmatic solutions to the problems of industry and society. Carnegie Mellon began to be recognized as a truly national research university able to attract students from across the nation and around the world. The Cyert administration stressed strategic planning and comparative advantage, pursuing opportunities in areas in which Carnegie Mellon could outdistance its competitors. An archetypal example of this approach was the introduction of the university's "Andrew" computing network in the mid-1980s. This pioneering network, which linked all computers and workstations on campus, set the standard for educational computing and firmly established the university as a leader in the uses of technology in education and research. Education and teaching also benefited in this period with the establishment of a University Teaching Center to improve faculty teaching and the renovation of many of the university's classrooms. Cognizant of the university's heritage, President Robert Mehrabian (1990-97) invited alumni from the era of the institution's first president, Arthur A. Hamerschlag, to attend his inauguration in 1990. President Mehrabian emphasized Carnegie Mellon's traditional strengths in education, research and service to society while focusing on initiatives for leadership in the 21st century. With the appointment of the university's first Vice Provost for Education, President Mehrabian placed renewed emphasis early in his administration on the quality of undergraduate education. He also moved aggressively to complete the most ambitious campus building plan since the Warner era. The University Center, which opened in August 1996, and the Purnell Center for the Arts, to be completed by the fall of 1999, are keys to enhancing the quality of life on campus, another priority of the Mehrabian administration. Confronted by shrinking governmental support of university research, President Mehrabian diversified the university's research agenda. He stressed the need to build strong relationships with the business world, matching industry's needs with the university's areas of research strength. He also put new emphasis on productivity, improvement of administrative services and strategic management of university resources. President Mehrabian established strong, new partnerships with the greater Pittsburgh community. He led a community-wide economic development initiative, spurred collaboration with primary and secondary schools, and worked closely with local community groups. On April 15, 1997, Jared L. Cohon, former dean of Yale University's School of Forestry and Environmental Studies, was elected by the university's Board of Trustees to succeed President Mehrabian, who resigned to spend more time with his family in California. "Since I was chosen, since this wonderful event has occurred, it has made me reflect on why you are choosing me," President Cohon said in his first speech to the university community. "And I've said to people since this was announced that the more I think about it, the more I realize how well I think this institution and I fit together. We'll see if that's true. I think it is. "When I was at Johns Hopkins we used to always hold up Carnegie Mellon as an example," Cohon said. "So, for many years I've ... been jealous of what has been accomplished here across departmental lines. I celebrate that. I think it is so valuable in every aspect of this university and it will position Carnegie Mellon to be even better...." During Cohon's presidency, Carnegie Mellon has continued its trajectory of innovation and growth. Today, President Cohon is leading implementation of a comprehensive strategic plan that aims to leverage the university's existing strengths to benefit society in the areas of biotechnology and the life sciences, information and security technology, environmental science and practices, the fine arts and humanities. The university is also committed to broadening and enhancing undergraduate education to allow students to explore various disciplines while maintaining a core focus in their primary area of study. Realizing that today's graduates must understand international issues, Carnegie Mellon is committed to providing a global education for its students and is striving to expand its international offerings and to increase its presence on a global scale. Increasing diversity, in all aspects, and fostering the economic development of southwestern Pennsylvania, are also top priorities. Over the years Carnegie Mellon's leaders have reflected Andrew Carnegie's original dedication and commitment to this institution. In his 1900 letter to the mayor of Pittsburgh establishing Carnegie Technical Schools, Andrew Carnegie wrote, "My heart is in the work." These words have been echoed by students, faculty and administrators throughout this century and they live on the Carnegie Mellon campus today. |
More News:
• For August 2006
• From Carnegie Mellon Universtity
• For University
|
