This talk focuses on automatically extracting disease specific biomarkers from medical images to aid diagnosis and labelling of diseases. Medical researchers commonly extract biomarkers from scans via hypothesis driven testing based on volumetric measurements. We instead propose to analyze the mapping of medical scans to a particular population. In this talk, we specifically focus on two applications:We first analyze the spatial distribution of brain gliomas across 125 patients by automatically segmenting the MR scans of that population and registering the scans to an atlas generated from a healthy subjects. Our algorithm jointly perform these two tasks by artificially seeding the tumor inside the atlas and then simulating its growth to model the mass effect and diffusivity of the pathology on the healthy tissue. Our approach compares favorable to state of the art in this domain. In addition, we show that the distribution of the tumor across the 125 cases is a non-uniform function which reaches a maximum in the left temporal lobe within our patient population.In the second application, we use manifold learning to detect diffuse cardiac diseases as well as Alzheimer from MR scans. Learning the manifold of deformations across MR scans allows us to extract a low dimensional encoding for these high dimensional mappings. We then compare the accuracy of different classifiers based on our proposed low dimensional encoding versus other popular measurements.