Normal aging is associated with changes in volumetric indices of brain atrophy. A quantitative understanding of age-related brain changes can shed light on successful aging. To investigate the effect of age on global and regional brain volumes and cortical thickness, 3514 magnetic resonance imaging scans were analyzed using automated brain segmentation and parcellation methods in elderly healthy individuals (69–88 years of age). The machine learning algorithm extreme gradient boosting (XGBoost) achieved a mean absolute error of 2 years in predicting the age of new subjects. Feature importance analysis showed that the brain-to-intracranial-volume ratio is the most important feature in predicting age, followed by the hippocampi volumes. The cortical thickness in temporal and parietal lobes showed a superior predictive value than frontal and occipital lobes. Insights from this approach that integrate model prediction and interpretation may help to shorten the current explanatory gap between chronological age and biological brain age.