Regression models are developed in various field of applications to help researchers to predict certain variables based on other predictor variables. The dependent variables in the regression model are estimated by a number of independent variables. Model utility test is a hypothesis testing procedure in regression to verify if there is a useful relationship between the dependent variable and the independent variable. The hypothesis testing procedure that involves p-value is commonly used in model utility test. A new technique that involves coefficient of determination R2 in model utility test is developed in this paper. The effectiveness of the model utility test in testing the significance of regression model is evaluated using simple linear regression model with the significance level α = 0.01, 0.025 and 0.05. The study in this paper shows that a regression model that is declared to be a significant model by using model utility test, however it fails to guarantee a strong linear relationship between the independent variable and dependent variable. Based on the evaluation presented in this paper, it is shown that the p-value approach in model utility test is not a good technique in evaluating the significance of a regression model. The results of this study could serve as a reference for other researchers applying regression analysis in their studies.