Try It Now

(Registration required.)

What is Feature Extraction Service for IBM Multimedia Analysis and Retrieval System?

Feature Extraction Service for IBM® Multimedia Analysis and Retrieval System (hereafter referred to as "Feature Extraction Service") automatically classifies the visual contents of images. With the growing deluge of image and video data, manual annotation is too labor-intensive to be practical. An automated solution is needed. Feature Extraction Service automates the annotation process using powerful, computer-based technologies that analyze images visually. The service uses the semantic annotation capabilities of IBM Multimedia Analysis and Retrieval System (also here on alphaWorks) and exposes its core image classification functionality as an online, real-time service interface.

IBM's technology is unique in its approach of applying machine learning techniques across multiple visual features to automatically index visual content. Because there are many important dimensions of visual content, such as color, texture, shape, and spatial layout, this system uses many visual feature extraction algorithms that extract descriptors across a wide array of visual dimensions. These descriptors are then processed with machine learning algorithms in order to identify relevant semantic categories for each image.

How does it work?

Feature Extraction Service is an online image annotation service that consists of a sophisticated framework for analyzing images in real time. Machine learning techniques are used to model different semantic categories based on the visual content of training data. The models are then used to automatically classify new image content. Any application or Web service can send images to this service through the HTTP protocol. The system will analyze the visual content for each mage in real time and return a list of relevant semantic categories and their confidence scores in XML format. Applications can use these semantic annotations and confidence scores for search, filtering, classification, or content management purposes.

The creators of this technology all work at the IBM T. J. Watson Research Center. John R. Smith, Ph.D., is senior manager of the Intelligent Information Management Department. He is the leader of the project. Dr. Smith's research interests include multimedia databases, content analysis, indexing, and retrieval. Matthew Hill is a software engineer in the Multimedia Research group. Mr. Hill has been working with the project since 2004. He has worked in several areas at IBM, including multimedia databases, intelligent oil fields, and watermarking. Apostol (Paul) Natsev, Ph.D., is a research staff member and manager of the Multimedia Research Group. He is a co-founding member of the project. Dr. Natsev's research interests include multimedia analysis, indexing, and retrieval. Quoc-Bao (Bao) Nguyen is a software engineer in the Multimedia Research Group. Mr. Nguyen joined the project in 2006. He has worked in several areas at IBM, including rule-based systems, parallel processing, e-commerce auction, and business performance monitoring. Jelena Tesic, Ph.D., is a research staff member in the Multimedia Research Group. Her research interests include scalable multimedia management and content analysis. Lexing Xie, Ph.D., is a research staff member in the Multimedia Research Group. Her research interests include multimedia content analysis and mining. Rong Yan, Ph.D., is a research staff member in the Multimedia Research Group. His research interests include multimedia information retrieval and video content analysis. IBM is a trademark of IBM Corporation in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of others.