Mobile visual search and applications

Abstract

The increasing popularity of smartphones has significantly changed people’s lives. All kinds of convenient services are provided by the powerful mobile system. People use wide variety of available mobile applications to navigate, to shop, and to communicate with others. To further improve the user experience, mobile visual search applications are developed to provide searching service by using images instead of text. Mobile visual search applications have been applied to various areas such as landmark recognition, books, clothes and so on. This project designs a mobile visual search application for tourism. The application is designed for trip planning with a specific tour book. A database is built for the tour book. When users scan an image on the tour book, related information including address, opening hours, telephone number, introduction, website and so on will be displayed. Users could add the place to his/her trip plan. Visiting date for each place could be set in the trip plan. Map of single object and multiple objects are both available. Recommend list based on the places users added is provided. The target customers including tour book publishers and readers. The application could be sold to tour book publishers as a companion to tour books and publishers may provide database for the application. The users may download and use the application when reading the tour book. Mobile visual search applications are developed base on image recognition technology. The main image recognition technology involved in this project is Scale Invariant Feature Transform. Client side captures the image and sends the image to server side. Server side extracts the features of database images and a vocabulary tree is formed employing the Bag of Words method. By comparing the feature of query image with the vocabulary tree, the matched image is selected. Geometric verification using RANdom Sample Consensus (RANSAC) method is applied to selected image. Then the query result is sent to client side. The application is integrated with Augmented reality in this project. Augmented reality technique enables the application to dynamically display different query object’s recognition result on the screen without leaving this camera page.