From social media platforms like Facebook, Instagram, Snapchat etc. to digital photo frames, to instant sharing of images via internet messages or Direct Messages (DMs), millions of digital images are viewed, processed, indexed, searched and shared every day. Thus, handling such a large database using usual text based search methods like Full-Text Searching or metadata based searching technique is not a conclusive or complete in any way and fails on many levels when it comes to an image database. This gave rise to the concept of content based image retrieval (CBIR). CBIR also known as Query based Image Retrieval (QBIC) refers to image content that is retrieved directly, by which the images with certain features or containing certain content will be searched in an image database. Initially, CBIR mostly included Query by Visual Example (QVE) or Query by subjective description (QBD). In QVE, (Hirata, Kyoji, and Toshikazu Kato) the user inputs a rough sketch or an image similar to the desired image to retrieve the original image and the similar images, while QBD is more of a cognitive model or semantic representation. In 1992, Toshikazu Kato came up with visual interaction mechanisms for image database system which combined both QBD and QVE. With the current advancements in technology, CBIR is no longer restricted to QVE or QBD however uses a large number of both supervised and unsupervised mechanisms to perform fast and conclusive retrieval operations.Content Based Image Retrieval
Image databases are huge in nature, they may contain tens of millions of images. In the most common cases, the search is performed using indexes based on keywords, but such search techniques are not fulfilling, thus a need arises to search for images based on their content like color or color distribution, shape or even texture. Content Based Image Retrieval (CBIR) or Query based Image Retrieval (QBIC) is the field of research which deals with such scenarios. CBIR refers to the method of direct retrieval of the image content from images stored in a large image database on the basis of certain contents or features. The basic concept of CBIR is to evaluate image information by low level features of an image, like texture, shape, color and space relationship of objects etc., and to set up feature vectors of an image as its index. In a typical CBIR model, the users provide queries to retrieve images. The queries can be in the form of example image or even a sketch or a description of the image in text form, depending on the type of CBIR technique in use. The CBIR system then converts the query into a set of feature vectors. In the image database, the visual contents from the images are extracted to form the feature vectors.
Now depending on the similarity measure used, both the feature vectors of the query and the image database are compared and leads to the retrieval of images from the image database similar to the query. There are a large number of image databases available on the World Wide Web on which different CBIR techniques have been performed. Image processing contains a large number of image database examples for the purposes of CBIR. There are two basic ways in which CBIR is usually performed, Query by Visual Example (QVE) and Query by subjective description (QSD).
There is also a large number of CBIR systems available for further study. Typical examples of the CBIR retrieval systems include QBIC, Virage, Photobook, VisualSEEk, SaFe and SIMPLIcity etc.
Query by Visual Example (QVE)
QVE is the most common and easiest of techniques in which an example image is provided as input to the system, and the search is performed using certain low-level features like those reflecting color, shape, texture, and prominent points in an image to find the images closest to the input image. Because of the robustness, effectiveness, simplicity of implementation and tremendously low storage requirements advantages, color has been the most effective feature and most CBIR systems employ colors.
Apart from having low storage requirements, another reason for color being most desired feature is color being invariant to image size and orientation or even basic geometric transformations like translation, rotation, scaling or shearing etc. For representing color, say in RGB scheme, in terms of intensity values, a color space is defined as a model. A color component is one of the dimensions and a combination of Red, Green and Blue forms the other colors. When images are compared on the basis of their color similarity, the most common means used is a statistical method called color histogram (CH).
A color histogram (CH) is a statistical method most commonly used to isolate color features. Color histogram is based on a co-occurrence matrix used to form the feature vectors. Based on this co-occurrence matrix a number of images can be compared.
CH based Image Retrival (IR) techniques make use of the color intensity values of each pixel. CH based IR is done to only RGB or HSV systems. Based on the number of occurances of pixel values, a histogram is created using the formula given by Sangoh in 200. Where is the number of pixels of type or which represents the three color channels Red, Green and Blue and is the probability. Then a color space is defined, which is nothing but a model for representing the colors in the IR system. A Color space defines multi dimensional space where each dimension is a color channel or component like Red, Green or Blue. It is common knowledge that the combination of Red, Green and Blue gives the other colors of the visible range of the electromagnetic spectrum. Each of the color components or color spaces are related to one another using some mathematical formulae. Thus an image is characterised on the basis of its color distribution
CH provides a global color distribution for images, which is very helpful in spatial images where segmentation or object recognition is difficult. Each of the color space is divided into several small intervals to calculate the color histogram. Each interval is called a bin. The color histogram can be calculated by counting the pixels of colors belonging to each interval or bin. Now, if each of the intensity value creates a bin then it would lead to almost 256 bins for each color space. Such great number of bins lead to increased computational cost and certain bins might have have redundant or irrelevant contents and will be in appropriate for building efficient indexes for image database. Hence to overcome this overhead, these levels need to be quantized using a process commonly known as histogram equalization.
Histogram equalization quantizes the number of bins by reducing the number of bins by taking intensity values that are very similar to one another and placing them in the same bin. Color features include global color histogram and block color histogram for a small region. Jeong Sangoh (2001) in his paper performs a different adaptation of the CH method, where instead of computing histograms separately for each of the red, green or blue components of a pixel, the he attempts to create bins that contain information from all three. Thus, the number of bins is much larger.
Now for images to be characterised as similar to the query image, similarity measures have to be calculated. The most common is Euclidean distance calculation which computes the differences between the number of a certain set of pixels found in one image versus another for each bin in the histogram. Some of the other techniques include a histogram intersection method in which colors not present in either one of the images were not used to compare the images. The histogram values need to be normalized first . This is done by dividing the number of pixels in each histogram bin by the number of pixel values used in the comparison.However, CH being a statistical method, on its own has its limitations. Thus to make color histogram more effective for image indexing Rao, Aibing, Rohini K. Srihari, and Zhongfei Zhang came up with a way of incorporating spatial information as well. They proposed a new techniques to extend color histogram in an additional dimension. In addition to the statistics in the dimensions of a color space, the distribution state of each single color in the spatial dimension is also taken into account. The combination of the CH and spatial layout was coined as spatial color histogram. They made use of density distribution.