Image Retrieval Using CIFAR-10 Dataset

Pattern Recognition and Machine Learning

Spring 2024 IIT Jodhpur

Github |
Abstract |
Dataset |
Challenges |
Models |
Results |
Contribution |
Takeaways |
References

Abstract

Image retrieval is important in various applications that allows users to search and retrieve relevant images with ease. In our project, we focused on image retrieval using traditional machine learning techniques on the CIFAR-10 dataset which consist of 60,000 colored images from ten classes. We have tried classical ML methods like classification and clustering based techniques for effective retrieval of the best pictures. At initial stages after data visualization and feature extraction we used Single-layer perceptrons, K-nearest neighbors, Naive Bayes, Logistic Regression, Random forests which gave us a good insights along-with with accuracies ranging between 33% and 55%. In the later stages of this project we implemented Support vector machines and Multi-layer perceptrons(MLP) to improve model performance. Although the scratch version of SVM was working good on other data sets, it failed badly on CIFAR-10 so we did a failure-case analysis for that and went ahead with sklearn SVM. To manage dataset complexity issues we used techniques such as PCA and K-Means were but finally proceeded with PCA. This project helped us in understanding image retrieval techniques and handling complex & large datasets, aiming to identify effective strategies for real-world applications in a concise yet comprehensive manner.

Keywords: Image retrieval, CIFAR-10, Clustering, Classification, Model Performance

Dataset

The dataset used in this project is the CIFAR-10 dataset. The CIFAR-10 dataset consists of 60000 32x32 colour images in 10 classes, with 6000 images per class. There are 50000 training images and 10000 test images. It was created by researchers at the Canadian Institute for Advanced Research (CIFAR) as a benchmark for evaluating machine learning algorithms and models for image classification tasks. It was released in 2009 and has since become one of the most popular datasets for image classification research. The following are the classes in the CIFAR-10 dataset:

Airplane
Automobile
Bird
Cat
Deer
Dog
Frog
Horse
Ship
Truck

The images in the CIFAR-10 dataset were collected from a variety of sources, including web searches, image repositories, and image-sharing platforms. Each image was manually labelled with one of the 10 class labels by human annotators.

CIFAR-10 Images — Figure 1: Sample Images from the CIFAR-10 Dataset

Challenges

Despite its popularity, the CIFAR-10 dataset presents several challenges for machine learning algorithms, including:

Low-Resolution Images: The small size (32x32 pixels) of the images makes it more challenging to extract meaningful features.
Noisy Images: The images in the CIFAR-10 dataset are often noisy, which can make it difficult for models to learn the underlying patterns.
Similar Classes: Some classes in CIFAR-10 (e.g., cat and dog) share similar visual features, leading to higher classification difficulty.
Limited Diversity: The dataset contains a limited variety of objects compared to real-world scenarios, which may limit the generalization ability of models trained on CIFAR-10.

Models Implemented

We implemented almost all the machine learning models encompassing all the topics taught during the course. This covers techniques involving both supervised and unsupervised learning. For dimensionality reduction, we implemented PCA. We also implemented K-means clustering for unsupervised learning. For supervised learning, we implemented Decision Trees, Random Forest, SVM, Naive Bayes, KNN, ANN, and Perceptron. Logistic Regression was also implemented. Emphasis was laid on implementing the models from scratch to get a better understanding of the mathematics involved.

Results

The majority of models produced accuracy scores in the range of 48-60 %. SVM with 'RBF' kernel produced an accuracy score of 61.04 %.The results showed that deep learning models would definitely outperform traditional machine learning models on the CIFAR-10 dataset, achieving higher accuracy and better generalization. There were some models like Naive Bayes which performed poorly on the dataset due to its inherent assumptions about the data distribution given that the images have a property of having pixels not entirely independent.

Lets try out the model

Actual Label: Deer

Predicted Label: Deer

Correct

Contribution of Members

Anushk Gupta

Percpetron, ANN

Yashraj Chaturvedi

Decision Tree, SVM

Aatif Ahmad

Naive Bayes, SVM

Khushi Agrawal

KNN, ANN

Ishita Sancheti

PCA, KMeans

Aditi Rawat

Logistic Regression

Key Takeaways

The project provided valuable insights into the practical aspects of machine learning, including data preprocessing, model selection, training, evaluation, and hyperparameter tuning. It also helped us understand the challenges involved in working with real-world datasets and the importance of feature engineering and model interpretation. Overall, the project enhanced our understanding of machine learning concepts and techniques and provided us with hands-on experience in implementing and evaluating machine learning models for image classification tasks.