Recently there has been a great deal of interest in developing computer aided diagnosis systems for identifying prostate cancer presence in vivo on MRI and ultrasound separately1-3, no work we are aware of has attempted to address the issue of fusing computer derived features from MRI and ultrasound to create the best possible predictor of cancer in vivo. In this work we attempt a systematic and quantitative evaluation of the discriminability of computer extracted MRI and ultrasound features in terms of cancer detection in patients undergoing radical prostatectomy.Our study design comprised 12 2D planar images obtained from the MRI and US scans of 3 patients, all of whom had biopsy confirmed prostate cancer and scheduled for a radical prostatectomy. A 3D B-mode ultrasound scan was performed followed by a 3 Tesla MRI prior to surgery. Following surgery and histologic sectioning of the gland via a microtome, the H&E stained whole mount histologic (WMH) sections were digitized via a whole slide scanner and the regions of cancer annotated by an expert pathologist. Deformable co-registration methods were used to spatially align the in vivo MRI, TRUS, and ex vivo histology. In particular, we used fully automatic Multiattribute probabilistic prostate elastic registration (MAPPER) approach to fusion of ultrasound and MRI4. We also manually delineated corresponding landmarks between MRI and WMH for deformable co-registration of WMH to MRI. A total of 129 computer extracted image features including Haralick, Gabor, Law, LBP, Laplacian features were extracted from both the prostate MRI and TRUS. Each of the computer extracted MRI and ultrasound features were then ranked via the Fisher criteria to identify the features that best identified the region of cancer. Figure 1 illustrates the MRI-TRUS-WMH registration and mapping of the cancer extent on MRI and TRUS.The top 3 features for each modality and corresponding Fisher criteria values are shown in Table.1. The classification is per region of interest (ROI), i.e. the texture features for the cancerous part of the prostate is compared to the texture features of the noncancerous confounding regions. Top three texture features, contrast variance, contrast entropy, and contrast inverse moment were selected by the theoretical linear discriminant analysis (LDA) classifier for MRI and yielded an area under the receiver operating characteristic curve (AUC) of 0.83, 0.77 and 0.70 for identifying cancerous ROIs in MRI. By comparison, top three most predictive features identified for TRUS were contrast inverse moment, contrast variance, and contrast entropy. These features yielded an AUC of 0.75, 0.69, and 0.66, respectively. By combining the top two texture features on MRI and the most informative texture feature on TRUS, the LDA based predictor yielded an AUC of 0.88 in predicting presence of prostate cancer. Figure 2 illustrates the scatter plot of the prostate cancer versus the non-cancer cases in three dimensional most informative texture feature space.We presented a framework to rank the performance of computer extracted MRI and ultrasound features in terms of their ability to identify prostate cancer. Our results in a small cohort suggests that we may be able to combine the MRI and ultrasound radiomic features to create a better classifier for prostate cancer detection compared to MRI or ultrasound alone.