This paper examines the intellectual structure of evaluation by means of citation analysis. By using various article attributes and citation counts in Google Scholar and (Social) Science Citation Index Web of Science, we analyze all articles published in Evaluation and Program Planning from 2000 until 2012. We identify and discuss the characteristics and development of the field as reflected in the history of those citations.
Theory-based evaluation (TBE) is an effectiveness assessment technique that critically analyses the theory underlying an intervention. Whilst its use has been widely reported in the area of social programmes, it is less applied in the field of energy and climate change policy evaluations. This paper reports a recent study that has evaluated the effectiveness of the national biofuel policy (NBP) for the transport sector in Malaysia by adapting a TBE approach. Three evaluation criteria were derived from the official goals of the NBP, those are (i) improve sustainability and environmental friendliness, (ii) reduce fossil fuel dependency, and (iii) enhance stakeholders' welfare. The policy theory underlying the NBP has been reconstructed through critical examination of the policy and regulatory documents followed by a rigorous appraisal of the causal link within the policy theory through the application of scientific knowledge. This study has identified several weaknesses in the policy framework that may engender the policy to be ineffective. Experiences with the use of a TBE approach for policy evaluations are also shared in this report.
This study evaluates the impact of public export promotion programs (EPPs) among small and medium-sized enterprises (SMEs) in Malaysia. Three indicators, level of awareness, frequency of use, and perception of usefulness, were examined according to a firm's export status. The global evaluation suggests that exporters are more frequent users of EPPs and perceive them to be more useful than non-exporters. Nonetheless, both groups demonstrate higher levels of awareness, are frequent users, and perceive the programs relating to export info/knowledge are more usefulness than programs relating to financial assistance. Further analysis also reveals that the frequency of use and the perception of usefulness for most programs are positively related to export experience, but not to export turnover. This study offers insights into the effectiveness of export programs for encouraging export initiation and expansion in an emerging economy.
Enterprise Architecture (EA) implementation evaluation provides a set of methods and practices for evaluating the EA implementation artefacts within an EA implementation project. There are insufficient practices in existing EA evaluation models in terms of considering all EA functions and processes, using structured methods in developing EA implementation, employing matured practices, and using appropriate metrics to achieve proper evaluation. The aim of this research is to develop a hybrid evaluation method that supports achieving the objectives of EA implementation. To attain this aim, the first step is to identify EA implementation evaluation practices. To this end, a Systematic Literature Review (SLR) was conducted. Second, the proposed hybrid method was developed based on the foundation and information extracted from the SLR, semi-structured interviews with EA practitioners, program theory evaluation and Information Systems (ISs) evaluation. Finally, the proposed method was validated by means of a case study and expert reviews. This research provides a suitable foundation for researchers who wish to extend and continue this research topic with further analysis and exploration, and for practitioners who would like to employ an effective and lightweight evaluation method for EA projects.
The study described the perceived importance of, and proficiency in core agricultural extension competencies among extension workers in Peninsular Malaysia; and evaluating the resultant deficits in the competencies. The Borich's Needs Assessment Model was used to achieve the objectives of the study. A sample of 298 respondents was randomly selected and interviewed using a pre-tested structured questionnaire. Thirty-three core competency items were assessed. Instrument validity and reliability were ensured. The cross-sectional data obtained was analysed using SPSS for descriptive statistics including mean weighted discrepancy score (MWDS). Results of the study showed that on a scale of 5, the most important core extension competency items according to respondents' perception were: "Making good use of information and communication technologies/access and use of web-based resources" (M=4.86, SD=0.23); "Conducting needs assessments" (M=4.84, SD=0.16); "organizing extension campaigns" (M=4.82, SD=0.47) and "Managing groups and teamwork" (M=4.81, SD=0.76). In terms of proficiency, the highest competency identified by the respondents was "Conducting farm and home visits (M=3.62, SD=0.82) followed by 'conducting meetings effectively' (M=3.19, SD=0.72); "Conducting focus group discussions" (M=3.16, SD=0.32) and "conducting community forums" (M=3.13, SD=0.64). The discrepancies implying competency deficits were widest in "Acquiring and allocating resources" (MWDS=12.67); use of information and communication technologies (ICTs) and web-based resources in agricultural extension (MWDS=12.59); and report writing and sharing the results and impacts (MWDS=11.92). It is recommended that any intervention aimed at developing the capacity of extension workers in Peninsular Malaysia should prioritize these core competency items in accordance with the deficits established in this study.
From the rise of restorative justice to evidence-based approaches to reducing recidivism, the intellectual landscape of criminal justice has seen considerable change in recent decades. The result is that an increasing number of countries have tackled the task of shifting preexisting political institutions to confirm with these new understandings. This is, of course, no easy feat. A great number of challenges confront willing policymakers, a reality that often puts philosophy at loggerheads with practicality. Moreover, the political process of change is subject to the influence of cultural and institutional norms. In this paper, we look at one particular case study - that of Malaysia's juvenile justice system - to understand the challenges faced in changing criminal justice policy. We identify three primary categories of challenges and elucidate their shape and impact through the Malaysian example. We also briefly analyze potential opportunities to mitigate and overcome these challenges. Furthermore, we also conclude with several implications for future research that we deem are important to be taken place. In sum, we argue that criminal justice reform must be undertaken with an eye toward important societal and institutional norms, each requiring thoughtful analysis of complex local cases.
The primary goal in project portfolio management is to select and manage the optimal set of projects that contribute the maximum in business value. However, selecting Information Technology (IT) projects is a difficult task due to the complexities and uncertainties inherent in the strategic-operational nature of the process, and the existence of both quantitative and qualitative criteria. We propose a two-stage process to select an optimal project portfolio with the aim of maximizing project benefits and minimizing project risks. We construct a two-stage hybrid mathematical programming model by integrating Fuzzy Analytic Hierarchy Process (FAHP) with Fuzzy Inference System (FIS). This hybrid framework provides the ability to consider both the quantitative and qualitative criteria while considering budget constraints and project risks. We also present a real-world case study in the cybersecurity industry to exhibit the applicability and demonstrate the efficacy of our proposed method.