Hybrid AI-powered computer vision combines physics and big data
SOURCE: HTTPS://WWW.SCIENCEDAILY.COM/
AUG 14, 2023
What is image registration and how does it work?
SOURCE: ANALYTICSINDIAMAG.COM
MAY 07, 2022
The image registration process is an automated or manual operation that attempts to discover matching spots between two photos and spatially align them to minimise the desired error, i.e. a uniform proximity measurement between two images. Medical sciences, remote sensing, and computer vision all use image registration. This article discusses the technicality of image registration with implementation using OpenCV. Following are the topics to be covered.
Let’s start with a high-level overview of image registration.
Once a correspondence is established between the pictures, the study of the link between two or more photos can typically be simply regulated or dealt with. The process of establishing this correlation is known as image registration.
It could be said that Image registration is the process of calculating spatial transforms which align a set of images to a common observational frame of reference, often one of the images in the set. Registration is a key step in any image analysis or understanding task where different sources of data must be combined. During the registration process, two situations become evident:
Are you looking for a complete repository of Python libraries used in data science, check out here.
Image registration could be used in two ways:
It is frequently used to align pictures from diverse camera sources in medical and satellite photography. Image registration is a technique used by digital cameras to align and link nearby pictures into a single panoramic image.
There are major four steps that every method of image registration has to go through for image alignment. These could be listed as follows:
Image registration methods are majorly classified into two types: area-based approaches and feature-based methods. When significant features are lacking in photos and distinguishing information is given by grey levels/colours rather than local forms and structure, area-based approaches are preferred.
When picture intensities provide more local structural information, feature-based matching algorithms are used. Image characteristics produced from the feature extraction technique are used in these procedures. But these two classifications could be further classified into various methods. Let’s have a look at those classifications.
For registration, a cross-correlation statistical methodology is employed in this procedure. It is frequently used for template matching or pattern recognition, which involves finding the location and orientation of a template or pattern in an image. Cross-correlation is a measure of similarity or a match metric.
For example, the two-dimensional normalised cross-correlation function assesses the similarity for each translation for template (an image for reference) and image, where the template is tiny in comparison to the image.
If the template fits the image, the cross-correlation will be at its peak. Because the measure might be influenced by local picture intensity, the cross-correlation should be adjusted.
The key disadvantages of correlation approaches are the flatness of the similarity measure maximum (owing to the self-similarity of the pictures) and the high processing complexity. The maximum can be successfully sharpened by pre-processing or by applying edge or vector correlation.
Grey values are not used to describe matched entities in point-based matching algorithms. It employs picture features produced from a feature extraction algorithm/process.
The fundamental goal of feature extraction is to abstract a huge quantity of information from the original data input and filter out the redundant information, i.e. to filter out the information that is no longer needed.
Features that are detected only in both photos and are more tolerant of local distortions are chosen. These characteristics influence the computation of suitable transformation. As a result, a significant number of characteristics must be discovered for computations to be performed.
After detecting characteristics in each image, they must be matched. This is the most common method for registering two photos with unknown misalignment.
Control points for point matching are crucial in this strategy. Corners, lines of intersection, points of locally greatest curvature on contour lines, centres of windows with locally maximum curvature, and centres of gravity of closed-boundary areas are all examples of control points.
Point mapping methods are divided into three stages, which are as follows:
The downside of the Feature-based technique is the exceptionality of the window content. Because of its non-saliency, a window containing a smooth area devoid of critical information is likely to be mistakenly matched with other smooth areas in the reference picture.
The registration characteristics should preferably be recognised in discrete areas of the picture. Windows may lack this feature, and their selection is frequently not based on their content evaluation.
In this strategy, strong statistical characteristics are employed to match picture feature points. Colour picture segmentation is used to extract regions of interest from images.
To produce the contour of an image, the mean for a given collection of colours is computed. Then each RGB pixel in an image is categorised as having a colour in a specific range or not during the segmentation process. In addition, the Euclidean distance is necessary to determine similarity.
The locus of points is a sphere with a radius equal to the threshold value, and every point situated inside or on the surface of the sphere meets the stated colour requirement. By coding these two sets of points in the image with black and white, a binary, segmented image is produced.
A Gaussian filter is used to eliminate noise following the segmentation process. Thresholds blurred the image, and then the contour of an image was obtained.
The accuracy of the Contour Based Image Registration method is but the limitation is that it is manual and slow.
Registration of multispectral/multisensory pictures is a difficult task. In general, such pictures have varying grey level properties, and basic approaches based on area correlation cannot be easily used.
It is discovered that the image’s entropy remains constant even though the histogram changes. Even after randomly shuffling the image’s pixels, the entropy of the image stays constant. Natural photographs also have less ambiguity.
In a realistic image, the pixel’s value is likely to be quite near to that of some of its neighbours. As a result, this reliance decreases overall entropy.
When pictures have a poor resolution or the overlap region is narrow, mutual information causes misregistration.
The Correlation theorem is important in this procedure because the Fourier function contains two values of the image’s real and imaginary parts at each frequency of the frequency spectrum.
By calculating the inverse of the function, we get an impulse function that is essentially zero everywhere except at displacement, which is required to optimally register two pictures. The approach described above is used to register pictures that just have the translation.
It should be emphasised that some type of interpolation must be utilised in the frequency domain.
Genetic Algorithms (GA) are founded on the natural notion of evolution, which suggests that variety aids in the survival of a population in the face of changing environmental conditions. GAs are iterative methods with inherent parallelism.
They keep a population of candidate solutions recorded in the form of chromosomes, which are generally the binary symbols “0” and “1.” The beginning population might be chosen at random.
Fit candidates, defined as those with fitness values greater than a given threshold, will be chosen for reproduction in the following generation. The chosen candidates are merged using a genetic reproduction procedure known as “crossover.” The crossover operator swaps out chromosomal segments to create superior candidates with greater fitness in the following generation.
In the next generation, the entire population is reevaluated. The operation is repeated until a termination requirement is met. The termination criterion might include discovering an acceptable approximation solution, attaining a certain number of generations, or solution convergence.
For satellite imageries based on digital vector maps, automated registration using a genetic algorithm has shown to be particularly efficient and trustworthy.
In the field of Medical Science
In the axial view of brain imaging, the top-yellow dots indicate anatomical markers or fiducial points (anatomical information). Bottom: The pink dots show the positions of the MEG sensors, while the green dots represent the sites of the scalp-EEG sensors. The functional information is contained in these MEG and EEG data, and the bottom image depicts the coregistered brain image (sagittal view)
In the field of Photography
Let’s implement one of the mentioned methods using OpenCV
Import necessary libraries:
import numpy as np import imutils import cv2
If you are using google colab notebook then also need to import an extra library as mentioned below for displaying images because “cv2.imshow” has been disabled as it crashes the terminal.
from google.colab.patches import cv2_imshow
Read the images and convert them into grey values
img_align = cv2.imread("IMG_2523.jpg") img_temp = cv2.imread("IMG_2524.JPG") img1 = cv2.cvtColor(img_align, cv2.COLOR_BGR2GRAY) img2 = cv2.cvtColor(img_temp, cv2.COLOR_BGR2GRAY) height, width = img2.shape
Setting up some key points and descriptors that will be used for matching the pixels from the template file.
orb_detector = cv2.ORB_create(5000) kp1, d1 = orb_detector.detectAndCompute(img1, None) kp2, d2 = orb_detector.detectAndCompute(img2, None) matcher = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck = True) matches = matcher.match(d1, d2)
Create a homography for the validation of the resultant image
homography, mask = cv2.findHomography(p1, p2, cv2.RANSAC)
Registering Image:
transformed_img = cv2.warpPerspective(img_align, homography, (width, height)) matchedVis =cv2.drawMatches(img1, kp1, img2, kp2, matches, None) matchedVis = imutils.resize(matchedVis, width=1000) cv2_imshow(matchedVis) cv2.waitKey(0)
Comparing the resultant image:
Output image
Input image
Validating the result:
A total of 5000 pixels have been selected from the input image which is to be matched from the template image based on which the output image has been generated. By observing the output image we can confidently say that the image is properly aligned to the reference image.
Image registration is a necessary step in integrating, fusing, and evaluating data from numerous sensors (sources). It has a wide range of applications in medical sciences, computer vision, and remote sensing.
Image registrations with complicated nonlinear distortions, multi-modal registration, and registrations of occluded images, among other things, contribute to the robustness of the techniques belonging to the hardest tasks in the current environment.
In this article, we have understood the concept of image registration, different methods of registering images and implementation in Python using OpenCV.
Sourabh has worked as a full-time data scientist for an ISP organisation, experienced in analysing patterns and their implementation in product development. He has a keen interest in developing solutions for real-time problems with the help of data both in this universe and metaverse.
LATEST NEWS
Augmented Reality
Hi-tech smart glasses connecting rural and remote aged care residents to clinicians
NOV 20, 2023
WHAT'S TRENDING
Data Science
5 Imaginative Data Science Projects That Can Make Your Portfolio Stand Out
OCT 05, 2022
SOURCE: HTTPS://WWW.SCIENCEDAILY.COM/
AUG 14, 2023
SOURCE: INDIANEXPRESS.COM
OCT 24, 2022