The National Institute for Environmental Studies (NIES) of Japan established a strain of Japanese quail (Coturnix japonica) known as NIES-L by rotation breeding in a closed colony for over 35years; accordingly, the strain has highly inbred-like characteristics. Another strain called NIES-Brn has been maintained by randomized breeding in a closed colony to produce outbred-like characteristics. The current study aimed to characterize intermale aggressive behaviors in both strains and to identify possible factors regulating higher aggression in the hypothalamus, such as sex hormone and neuropeptide expression. Both strains displayed a common set of intermale aggressive behaviors that included pecking, grabbing, mounting, and cloacal contact behavior, although NIES-Brn quail showed significantly more grabbing, mounting, and cloacal contact behavior than did NIES-L quail. We examined sex hormone levels in the blood and diencephalon in both strains. Testosterone concentrations were significantly higher in the blood and diencephalon of NIES-Brn quail compared to NIES-L quail. We next examined gene expression in the hypothalamus of both strains using an Agilent gene expression microarray and real-time RT-PCR and found that gene expression of mesotocin (an oxytocin homologue) was significantly higher in the hypothalamus of NIES-Brn quail compared to NIES-L quail. Immunohistochemistry of the hypothalamus revealed that numbers of large cells (cell area>500μm2) expressing mesotocin were significantly higher in the NIES-Brn strain compared to the NIES-L strain. Taken together, our findings suggest that higher testosterone and mesotocin levels in the hypothalamus may be responsible for higher aggression in the NIES-Brn quail strain.