Sexual practices among heterosexual men may differ between female sex workers (FSWs) and casual partners. We surveyed 203 heterosexual men and investigated the attributes associated with inconsistent condom use among them. Lower educational attainment was positively associated with inconsistent condom use with FSWs (adjusted prevalence ratio (aPR) 2.63; P = 0.018) and casual partners (aPR 1.55; P = 0.022), whereas early age of sexual debut (aPR 3.00; P = 0.012) and alcohol use during sex (aPR 7.95; P < 0.001) were positively associated with inconsistent condom use with FSWs. Socioecological factors may explain such differences.