Hi I was curious if anyone has tried to store ORB or SIFT descriptors to use for an image similarity search?
At a high level i’m using opencv ORB and SIFT implementations to get feature vectors ie descriptors which results in a Mx32 for ORB and and Mx128 for SIFT matrix where M is based on the number of keypoints found in an image.
I was looking to see if there was a way to store those feature vectors in such a way that I could get feature vectors from a query image and find similar matches
The trouble with storing traditional image descriptors (such as ORB, SIFT, SURF, etc) is that those algorithms give you an array of unordered local features. Content-based image retrieval works best when you have a single representation for the entire image – a global descriptor.
There are ways to turn a collection of local ORB/SURF/SIFT descriptors into a single global descriptor – “Bag of Features” or “Bag of Visual Words” is one common technique, but there are others.
In general, it feels like once neural networks came onto the scene, most attention has focused on using a neural network to use the latent network features as a single global descriptor for each image. For more info on this, check out something like Keras Applications, which makes it straightforward to extract global feature descriptors from a pre-trained model. You will have less visibility into what those features mean / represent than you will with something like SIFT / ORB, but you will be working with a single global descriptor rather than an array of local descriptors.
I hope some of that makes sense?
Thanks Lex! It does. I ended up going experimenting with a CNN, I’m my case I decided to try ResNet50. I ended up storing the image descriptors and using that as a second pass as part of a re-ranking scheme from a knn search that came from using the feature descriptors that came from ResNet50.
Thanks again for the response.
Perfect! Sounds like you’re on the right track – well done! 