This paper deals with the effects of three low-carbon steel filler metals consisting of ferritic and austenitic phases on the weld joints of the tungsten inert gas (TIG) welding of Hardox 500 steel. The correlation between the microstructure and mechanical properties of the weld joints was investigated. For this purpose, macro and microstructure were examined, and then microhardness, tensile, impact, and fracture toughness tests were carried out to analyze the mechanical properties of joints. The results of optical microscopy (OM) images showed that the weld zones (WZ) of all three welds were composed of different ferritic morphologies, including allotriomorphic ferrite, Widmanstätten ferrite, and acicular ferrite, whereas the morphology of the heat-affected zone (HAZ) showed the various microstructures containing mostly ferrite and pearlite phases. Further, based on mechanical tests, the second filler with ferritic microstructure represented better elongation, yield strength, ultimate tensile strength, impact toughness, and fracture toughness due to having a higher amount of acicular ferrite phase compared to the weld joints concerning the other fillers consisting of austenitic and ferritic-austenitic. However, scanning electron microscopy (SEM) images on the fracture surfaces of the tensile test showed a ductile-type fracture with a large number of deep and shallow voids while on the fracture surfaces resulting from the Charpy impact tests and both ductile and cleavage modes of fracture took place, indicating the initiation and propagation of cracks, respectively. The presence of acicular ferrite as a soft phase that impedes the dislocation pile-up brings about the ductile mode of fracture while inclusions may cause stress concentration, thus producing cleavage surfaces.