Osteoporosis is a condition causing significant morbidity and mortality in the elderly population worldwide. Age-related testosterone deficiency is the most important factor of bone loss in elderly men. Androgen can influence bone health by binding to androgen receptors directly or to estrogen receptors (ERs) indirectly via aromatization to estrogen. This review summarized the direct and indirect effects of androgens on bone derived from in vitro, in vivo, and human studies. Cellular studies showed that androgen stimulated the proliferation of preosteoblasts and differentiation of osteoblasts. The converted estrogen suppressed osteoclast formation and resorption activity by blocking the receptor activator of nuclear factor k-B ligand pathway. In animal studies, activation of androgen and ERα, but not ERβ, was shown to be important in acquisition and maintenance of bone mass. Human epidemiological studies demonstrated a significant relationship between estrogen and testosterone in bone mineral density and fracture risk, but the relative significance between the two remained debatable. Human experimental studies showed that estrogen was needed in suppressing bone resorption, but both androgen and estrogen were indispensable for bone formation. As a conclusion, maintaining optimal level of androgen is essential in preventing osteoporosis and its complications in elderly men.