Development of computer-based medical inference systems is always confronted with some difficulties. In this paper, difficulties of designing an inference system for the diagnosis of arthritic diseases are described, including variations of disease manifestations under various situations and conditions. Furthermore, the need for a huge knowledge base would result in low efficiency of the inference system. We proposed a hierarchical model of the fuzzy inference system as a possible solution. With such a model, the diagnostic process is divided into two levels. The first level of the diagnosis reduces the scope of diagnosis to be processed by the second level. This will reduce the amount of input and mapping for the whole diagnostic process. Fuzzy relational theory is the core of this system and it is used in both levels to improve the accuracy.