|
 
 |
| ORIGINAL ARTICLE |
|
| Year : 2017 | Volume
: 3
| Issue : 4 | Page : 178-185 |
|
Nonrigid registration of multimodal medical images based on hybrid model
Nuo Tong1, Shuiping Gou1, Teng Xu1, Ke Sheng2, Shuyuan Yang1
1 Key Lab of Intelligent Perception and Image Understanding of Ministry of Education, School of Electronic Engineering, Xidian University, Xi'an, Shaanxi Province, China
2 Department of Radiation Oncology, University of California, Los Angeles, California, USA
| Date of Web Publication | 26-Mar-2018 |
Correspondence Address:
Shuiping Gou
No. 2 South Taibai Road, Xi'an 710071, Shaanxi Province
China
 Source of Support: None, Conflict of Interest: None  | Check |
DOI: 10.4103/digm.digm_39_17
Background and Objectives: Multimodal image registration is a crucial step in prostate cancer radiation therapy scheme. However, it can be challenging due to the obvious appearance difference between computed tomography (CT) and magnetic resonance imaging (MRI) and unavoidable organ motion. Accordingly, a nonrigid registration framework for precisely registering multimodal prostate images is proposed in this paper. Materials and Methods: In this work, multimodal prostate image registration between CT and MRI is achieved using a hybrid model that integrates multiresolution strategy and Demons algorithm. Furthermore, to precisely describe the deformation of prostate, B-spline-based registration is utilized to refine the initial registration result of multiresolution Demons algorithm. Results: To evaluate our method, experiments on clinical prostate data sets of nine participants and comparison with the conventional Demons algorithm are conducted. Experimental results demonstrate that the proposed registration method outperforms the Demons algorithm by a large margin in terms of mutual information and correlation coefficient. Conclusions: These results show that our method outperforms the Demons algorithm and can achieve excellent performance on multimodal prostate images even the appearances of prostate change significantly. In addition, the results demonstrate that the proposed method can help to localize the prostate accurately, which is feasible in clinical.
Keywords: B-spline, demons algorithm, multimodal prostate images, nonrigid registration
How to cite this article:
Tong N, Gou S, Xu T, Sheng K, Yang S. Nonrigid registration of multimodal medical images based on hybrid model. Digit Med 2017;3:178-85 |
| Introduction | |  |
Prostate cancer affects the male reproductive system and is one of the most commonly diagnosed cancers in men. Clinically, radiation therapy is considered as the most effective method of treating prostate cancer. Currently, computed tomography (CT) is frequently used in clinical prostate cancer radiation therapy for dose planning. However, due to the low-tissue contrast, fuzzy boundary, and complex background in prostate CT, only using CT during the treatment of prostate cancer cannot meet the clinical requirements. To compensate for these limitations of CT, magnetic resonance imaging (MRI) is often employed to provide abundant boundary information of soft tissues to improve the efficiency of prostate cancer radiation therapy. Therefore, nonrigid multimodal registration between prostate CT and MRI images becomes the prerequisite of precise radiotherapy for prostate cancer. However, the huge imaging differences between CT and MR and large deformation of prostate make the registration of prostate CT and MR challenging. Up to now, numerous nonrigid registration methods have been proposed to register MRI to CT. They generally fall into two categories: spatial transformation based and physical model based.
Spatial transformation-based nonrigid registration method uses the spatial transformation to fit the image deformation, such as polynomial method, basis function method, and spline function method.[1] The other category is based on the physical model. It considers the difference between imaging modalities comes from physical deformation such as elastic model, viscous fluid model, and optical flow model.[1] Broit [2] proposed considering the registration process as a physical stretch process of elastic materials such as rubber. This physical process was supported by the internal forces generated by the deformation of the elastomer and the external forces exerted on the elastomer. Wu et al.[3] proposed a viscous fluid model to simulate the deformation of organs, while fluid particle was used to model the motion of each pixel. Thirion [4] proposed the optical flow model-based Demons algorithm. The registration process was regarded as the process from the source image to the target image and carried out by the optical flow field. The change trend of the intensity of each pixel was considered as the instantaneous velocity field produced by the motion of the pixel with grayscale on the image plane. However, as only depends on the intensity gradients of the reference image to drive the floating image transforms, the effect of Demons algorithm may be limited when gradients are small. Clinically, there are a number of deformable registration methods have been proposed for multimodal prostate image registration. Hu et al.[5] proposed to employ a novel “model-to-image” registration approach in which a deformable model of the gland surface, derived from an MR image, is registered automatically to a TRUS volume by maximizing the likelihood of a particular model shape given a voxel-intensity-based feature that represents an estimate of surface normal vectors at the boundary of the gland. Mitra et al.[6] proposed to perform nonrigid registration of transrectal ultrasound and magnetic resonance prostate images based on a nonlinear regularized framework of point correspondences obtained from a statistical measure of shape-contexts. Alterovitz et al.[7] developed a 2D model to register prostate diagnostic probe-in MRI to therapeutic probe-out MR images for treatment planning, which is based on biomechanical modeling of soft tissues and estimation of uncertain tissue parameters using nonlinear optimization. [Table 1] shows part of the existed multimodal prostate image registration studies, modalities, and properties. | Table 1: Summarization of part of the existed prostate multimodal images registration technique
Click here to view |
To meet the requirements of prostate cancer radiation therapy, we propose to integrate intensity gradient information and local deformation correction in this work. In this paper, a nonrigid registration framework for registering multimodal prostate images in the treatment process of prostate cancer is proposed, which is based on the Demons algorithm and is improved by B-spline-based registration, to achieve high accuracy registration on multimodal prostate images with large deformation. It is notable that the registration procedure is applied on the prostate area that segmented from the whole image by level set to eliminate the interference of other irrelevant areas.
The primary contributions of this paper can be summarized as follows: (1) with the combination of the symmetric Demons algorithm and the B-splines registration, the deformation of prostate can be described accurately with low computation load; (2) by leveraging multiresolution Demons algorithm, the computation time can be saved greatly which is advantageous in clinical.
| Materials and Methods | | |
Ethics statement
The doctors obtained signed informed consent forms from all selected patients before the routine clinical course of treatment. All relevant ethical safeguards have been met in relation to patient or individual protection or animal experimentation.
Demons algorithm
Demons algorithm is an intensity-based automatic nonrigid algorithm by relying exclusively the image intensity gradients of the reference image to drive the floating image transform. It calculates the offset of each pixel in the floating image by the intensity gradients of the reference image to obtain the registration result. The basic idea of Demons algorithm is as follows:
Assuming f is the floating image, r is the reference image, the offset from f to r at position x is calculated as:
where f(x) and r(x) are the gray values of the floating image and the reference image, respectively. ∇r(x)) represents the intensity gradient of the image and α is the normalization constant.
Many improved variants have been introduced on the basic Demons algorithm concept after being proposed for nonrigid registration of medical images, especially the Active Demons algorithm proposed by Wang et al.[9] and the Symmetric Demons algorithm proposed by Rogelj et al.[10] show superior performance.
The active demons algorithm drives the deformation with the gradient information of the floating image and the reference image at the same time. The offset of each pixel can be calculated as follows:
Compared to the conventional Demons algorithm, the active Demons algorithm can achieve higher registration performance as the intensity gradient of the reference image and the floating image are both considered during the registration process.
Different from the active Demons algorithm, the intensity gradients of the reference image and the floating image will be averaged in the symmetric Demons algorithm,[11] then the offset of each pixel can be written as:
After each iteration, the offset will be smoothed by Gaussian filtering to normalize the transformation.
However, compared with the active Demons algorithm, symmetric intensity gradients are utilized in the symmetric Demons algorithm, which increases the amount of information and lower the registration error greatly. Hence, the symmetric Demons algorithm is employed to perform the registration initialization between the prostate CT and MR images in this paper.
Multiresolution strategy
In each iteration during the nonrigid registration process, the floating image will be transformed according to the estimated offset of each pixel,[12] which is time-consuming. Consequently, multiresolution strategy is employed in this work to further improve the efficiency and accuracy of multimodal prostate image registration, which is detailed below.
To register multimodal prostate images precisely, multiresolution strategy is adopted which downsamples the original image into a set of images in different resolution levels by a factor of 0.5 to form a multiresolution pyramid model. Considering the resolution of the prostate CT and MR images, five-level multiresolution pyramid model is employed here. In specific, the original image which has the highest resolution is placed at the bottom of the pyramid, while the lowest resolution image is placed at the top. When dealing with the complex image, the operation starts from the lowest resolution image, the upsampled result then will be set as the initial value for the next layer image of the pyramid until the image can reflect the details of the highest resolution image in each iteration.
B-spline registration
Demons algorithm is good at capturing large deformation between multimodal medical images. However, it does not have performance in small deformation. To achieve the best compromise between the degree of nonrigid image registration and the associated computational cost, B-spline registration is then employed to refine the demons registration result.
B-spline is a special form of spline curve, which has strong local control ability and superior smoothness.[13],[14] For this, it is commonly employed to model the local deformation of the target organs in nonrigid multimodal image registration. [Figure 1] illustrates the B-spline curve.
Assume p(t) is the position vector on the curve, the expression of B-spline curve takes the following form:
Here Bi denotes the position vector of the control node, Ni, k denotes the base function of B-spline, and k represents the order. In addition, the total number of control points is n+1.
The i-th B-spline basis function takes the following form:
where xi represents the value of control node, and xi≤xi+1x0,x1,...,xn forms a control node vector of B-splines. [Figure 2] illustrates the influence of control point movement upon the curve. Uniform cubic B-spline curve [15] is the most commonly used, which is the weighted sum of the uniform cubic B-spline basis functions and can be written as: | Figure 2: Influence of the control points movements on the curve. Bi,i = 1,...11 indicates the position vectors of the control nodes
Click here to view |
Its basis function can be defined as:
Consequently, uniform cubic B-spline curve takes the following form:
In practice, the density of the control point is an important factor that affects the registration accuracy and speed. The denser the control points are, the higher registration accuracy can be obtained and the longer computation time it will take. Therefore, to balance the registration accuracy and speed, control point mesh is utilized to represent the prostate CT and MR images, which is appropriate to this work.
Hybrid model-based nonrigid registration
To eliminate the interference of other irrelevant areas, the proposed registration framework is applied on the prostate CT and MR images with background removed.[16] The proposed hybrid model based nonrigid registration framework is detailed illustrated in [Figure 3].
The initial registration is carried out by Demons algorithm combined with multiresolution strategy. Then, prostate CT and MRI are divided into the control point grids to model the local deformation of prostate by the movements of control points. The sum of the squared differences of prostate CT and MRI is used as the similarity function, which is optimized by steepest descent method.
Details of algorithm implementation are given below:
The proposed hybrid model based nonrigid registration procedure is performed in a coarse-to-fine resolution manner to improve the registration accuracy and speed up the algorithm. Therefore, a 5-level multiresolution Gaussian pyramid is established with original prostate image in different resolution levels. In every resolution level, symmetric Demons algorithm is performed iteratively to obtain the deformation map. Then, the optimized deformation field is upsampled and linearly interpolated to serve as the initial estimate for the next high-resolution level. These procedures are iterated until the highest resolution level is reached. Afterward, the final optimized deformation field is applied on the prostate MRI to obtain the initial registration result.
To capture the local motion of prostate, B-spline-based nonrigid registration is employed to further refine the initial registration result due to its strong power in modeling deformable objects. Prostate CT and MRI are represented as control point meshes with each junction point represents a control point. Let  denotes the vector of the whole control point mesh with spacing δx in horizontal direction X and δy in vertical direction Y, where represents single control points and indices nx and ny correspond to the number of control points in horizontal direction X and vertical direction Y, where φi,j respectively. For every control point in the initial registration result, we randomly move it in the range of its surrounding four control points and subsequently calculate the displacement (Δx;Δy)T of its surrounding control points.
where △x and △y are the horizontal and vertical displacement of the control point Bl represents the l-th spline basis function of cubic B-spline l = 0,1....3, Bm represents the Bm-th spline basis function of cubic B-spline, l = 0,1....3.
The basis function of cubic B-spline takes the following form:
Afterward, estimated deformation is applied on the initial registration result according to the acquired displacement of each control point in the initial registration result. This procedure is iteratively performed to refine the registration result until the similarity function reaching a stable maximum. The properties of each steps are illustrated in [Table 2].
Materials
Nine prostate patient datasets were included in this work under an institutional review board protocol. The prostate CT images used for this work were acquired using a SIEMENS sensation open with voltage of 120kv, electric current of 250 mAs, and slick thickness of 3 mm. The CT images have the same size of 256 × 256 pixels with a pixel dimension being 0.976 × 0.976 mm 2. MRI images from the same patients with repetition time of 2200 ms, echo time of 201 ms, and slice thickness of 1.5 mm were acquired on SIEMENS Skyral. The size of the MR images used in this study is 256 × 256 pixels with pixel size of 0.664 × 0.664 mm 2. Note that the correspondences between CT slices and MRI slices are determined by an expert.
The proposed registration method is implemented using matlab. The experiments are conducted on a computer with an Intel Core i3 3.3GHz processor with 4GB RAM.
| Results | | |
[Figure 4] and [Figure 5] demonstrate the registration results between prostate CT and MR images by the proposed registration method and the symmetric Demons algorithm, respectively. Compared the registration results by our method with the results by the symmetric Demons algorithm in [Figure 4] and [Figure 5], it can be observed that our registration, which combines B-spline and Demons algorithm, can better preserve the prostate contour during registration. Furthermore, to quantitatively evaluate the registration result, mutual information (MI) and correlation coefficient (CC) between CT and the transformed MR images are calculated as shown in [Table 3] and [Table 4]. The mutual information (MI) of two images expresses how much the uncertainty on one of the image decreases when the other one is known. It is assumed to be maximum when the images are registered. The MI of two images takes the following form:[17] | Figure 4: Registration result by Demons algorithm. First, second, and third rows correspond to 3 patients, respectively. First and second columns are magnetic resonance and computed tomography images, respectively. Third and final columns are the registration result by Demons algorithm and the deformation grid, respectively
Click here to view |
 | Figure 5: Registration result by the proposed method. First, second, and third rows correspond to three patients, respectively. First and second columns are magnetic resonance and computed tomography images, respectively. Third and final columns are the registration result by the proposed method and the deformation grid, respectively
Click here to view |
 | Table 3: Comparison of mutual information values of 9 patients after nonrigid registration based on the symmetric demons algorithm and our proposed hybrid model-based registration framework
Click here to view |
 | Table 4: Comparison of correlation coefficient values of nine patients after nonrigid registration based on the symmetric demons algorithm and our proposed hybrid model-based registration framework
Click here to view |
where A and B denote the fixed CT and the transformed MRI, respectively, H(·) represents the corresponding Shannon entropy, which is computed on the probability distribution of the gray values. H (B\A) denotes the conditional entropy, which is based on the conditional probability p (b\a), the chance of gray valuein image given that the corresponding voxel in has gray value a. In general, a higher MI value represents a better alignment between the prostate regions after registration.
The CC measures the similarity between the prostate contours of the fixed CT image and the moving MRI image after registration, which is defined as:[18]
where the covariance is,
and where E [A] and E [B] represent the means,  and  and denote the variances of the random variables.
As the MI and CC values shown in [Table 3] and [Table 4], our hybrid model-based registration method outperforms the symmetric Demons algorithm by a large margin.
In addition, the average processing time of one scan is about 24s.
| Discussion | | |
In this work, the registration performance was evaluated by calculating MI and CC between CT and the registered MRI. Experimental results demonstrate that the proposed registration method outperforms the symmetric Demons algorithm by a large margin and can achieve satisfactory registration performance even the appearances of prostate change significantly.
A potential limitation of our study is that the estimated deformation field might not be accurate enough because the proposed method only takes the information of prostate region into account for removing the interference of other organs. However, this operation will lead to the lack of contextual information and result in inaccurate registration results. In the future work, we expect to introduce the contextual information of prostate into our registration method for better registration performance.
The evaluation of registration method still remains a hot topic. The improvement of MI and CC can roughly indicate the improvement of the performance of the registration method. However, MI is a good global similarity measurement, which has limited power in capturing local anatomical details, and thus has limited capacity in tackling local deformations. In addition, MI is sensitive to the amount of overlap between images that is part of the reason why the MI value of Patient 1 and Patient 3 increased greatly. Although CC is a natural choice when registering two images from the same modality, it may be limited for multimodality image registration in terms of statistical efficiency and computational efficiency. Better evaluation strategy for multimodal prostate image registration is still of high demand.
In addition, the verification and validation of our method on phantom data and the evaluation against clinical prostate radiation therapy are among our future work plans.
| Conclusions | | |
In this paper, a hybrid model-based nonrigid registration framework for registering multimodal prostate images is proposed to facilitate prostate cancer radiation therapy. Compared with the conventional symmetric Demons algorithm, which only utilize intensity information and cannot capture the large deformation of prostate accurately, we adopt the B-spline-based registration method combined with the symmetric Demons algorithm to improve the performance of multimodal prostate image registration. Furthermore, a coarse-to-fine resolution multilevel strategy is employed to reduce the computation cost and speed up the runtime. Experiments on clinical prostate multimodal images demonstrate that the proposed registration method achieves accurate registration even the appearances of prostate change significantly. In addition, the results demonstrate that the proposed method can help to localize the prostate accurately, which is feasible in clinical.
Financial support and sponsorship
This work was supported in part by the National Natural Science Foundation of China (Nos. 61472306, U1401255), the National Basic Research Program (973 Program) of China (No. 2013CB329402).
Conflicts of interest
There are no conflicts of interest.
| References | | |
| 1. | Zitová B, Flusser J. Image registration methods: A survey. Image Vis Comput 2003;21:977-1000.  |
| 2. | Broit C. Opitmal registration of deformed images. Univ Pa 1981;39:556-62. |
| 3. | Wu F, Qian ZC, Hang QS, Shi MG, Ma D. Principal axes algorithm based on image contour applied to medical image registration[J]. J Fourth Mil Med Univ 2001;22:567-9. |
| 4. | Thirion JP. Image matching as a diffusion process: An analogy with Maxwell's demons. Med Image Anal 1998;2:243-60. |
| 5. | Hu Y, Ahmed HU, Taylor Z, Allen C, Emberton M, Hawkes D, et al. MR to ultrasound registration for image-guided prostate interventions. Med Image Anal 2012;16:687-703. |
| 6. | Mitra J, Kato Z, Martí R, Oliver A, Lladó X, Sidibé D, et al. Aspline-based non-linear diffeomorphism for multimodal prostate registration. Med Image Anal 2012;16:1259-79. |
| 7. | Alterovitz R, Goldberg K, Pouliot J, Hsu IC, Kim Y, Noworolski SM, et al. Registration of MR prostate images with biomechanical modeling and nonlinear parameter estimation. Med Phys 2006;33:446-54. |
| 8. | Ou YM, Shen DG, Feldman M, Tomaszewski J, Davatzikos C. Non-rigid registration between histological and MR images of the prostate: A joint segmentation and registration framework. CVPR Workshops 2009;125-32. |
| 9. | Wang H, Dong L, O'Daniel J, Mohan R, Garden AS, Ang KK, et al. Validation of an accelerated 'demons' algorithm for deformable image registration in radiation therapy. Phys Med Biol 2005;50:2887-905. |
| 10. | Rogelj P, Kovacic S. Symmetric image registration. Med Image Anal 2006;10:484-93. |
| 11. | Vercauteren T, Pennec X, Perchant A, Ayache N. Diffeomorphic demons: Efficient non-parametric image registration. Neuroimage 2009;45:S61-72. |
| 12. | Vercauteren T, Pennec X, Malis E, Perchant A, Ayache N. Insight into efficient image registration techniques and the Demons algorithm. Inf Process Med Imaging 2007;20:495-506. |
| 13. | Lester H, Arridge SR. A survey of hierarchical non-linear medical image registration. Pattern Recogn 1999;32:129-49. |
| 14. | Rueckert D, Sonoda LI, Hayes C, Hill DL, Leach MO, Hawkes DJ, et al. Nonrigid registration using free-form deformations: Application to breast MR images. IEEE Trans Med Imaging 1999;18:712-21. |
| 15. | Holden M. A review of geometric transformations for nonrigid body registration. IEEE Trans Med Imaging 2008;27:111-28. |
| 16. | Abramoff MD, Magelhaes PJ, Ram SJ. Image processing with imageJ. Biophotonics Int 2004;11:36-42. |
| 17. | Pluim JP, Maintz JB, Viergever MA. Mutual-information-based registration of medical images: A survey. IEEE Trans Med Imaging 2003;22:986-1004. |
| 18. | Kim J, Fessler JA. Intensity-based image registration using robust correlation coefficients. IEEE Trans Med Imaging 2004;23:1430-44. |
[Figure 1], [Figure 2], [Figure 3], [Figure 4], [Figure 5]
[Table 1], [Table 2], [Table 3], [Table 4]
|
Search Pubmed for