Sistemlerin, sentetik ortamların, insanların performansının değerlendirilmesi genellikle karmaşık olup çok zaman gerektirmektedir. Mevcut değerlendirme sistemleri belli bir alana yönelik olarak geliştirilmişlerdir ve sistemin değerlendirme sonuçlarına nasıl ulaştığını açıklamazlar. Elde edilen yeni değerlendirme bilgilerinin, değerlendirme sisteminde güncellenmesi kolay değildir. Değerlendirme süreci, uzmanlık gerektirmektedir. Fakat uzmanlar az sayıda olup, bilgilerinin bilgisayar ortamına aktarılarak daha fazla istifade edilmeleri gerekir. Bu çalışmada, değerlendirme sürecini kolaylaştıran, hızlandıran ve farklı alanlarda kullanılabilen “Genel Değerlendirme Modeli” ve “Zeki Değerlendirme Sistemi” (ZeDeS) geliştirildi. Bu kapsamda, farklı alanlardaki uzmanlardan elde edilen sezgisel bilgilerin ve farklı kaynaklardan elde edilen bilgilerin bilgisayarla değerlendirme amaçlı kullanılabilmeleri için bir yöntem geliştirildi. Bu yöntemde, değerlendirme bilgileri, değerlendirme amaçlarının, değerlendirme kurallarının, ölçümlerinin, metotlarının ve parametrelerinin referans modeli olarak ifade edildi. Melez uzman sistem ve bulanık mantıktan meydana gelen “Zeki Değerlendirme Sistemi”, öğrencileri, eğitmenleri, işe başvuranları, bilgisayar tarafından meydana getirilmiş kuvvetler gibi sentetik kuvvetleri değerlendirdiği gibi gerçek sistemleri de değerlendirebilmekte olup “Genel Değerlendirme Modeli”ne ve değerlendirme ihtiyaçlarına göre geliştirildi. Değerlendirme bazı açılardan belirsizlik içerdiğinden, değerlendirmede genel çıkarım için bulanık mantıkla uzman sistemler beraber kullanıldı. ZeDeS, Hava Savunma Sistemi, öğretici performansı, pilot performansı değerlendirmesi ve eleman seçimi gibi çeşitli alanlarda ilk defa olarak kullanıldı. Makalede bir Hava Savunma Sistemi değerlendirmesinin ZeDeS kullanılarak nasıl yapıldığı ayrıntılı şekilde verilmiştir.

Evaluation of systems, synthetic environments and human performance are generally complicated and time-consuming tasks. Evaluation is needed nearly for all engineering tasks and the obstacles related with evaluation are increased proportional with complexity. Existing evaluation systems are domain dependent and do not provide explanation on how the system reaches the evaluation results. Expertise is needed for the evaluation process. Elicited new evaluation information cannot be updated to the system easily. Forming an evaluation definition is a complicated and time-consuming task. Finding out and formulating the required knowledge from the domain for which the evaluation is to be performed, is generally difficult due to lack of structured approach. It is not only important to formulate the knowledge, but finding out the right source of knowledge is also essential. Structured knowledge architecture is especially important in order to utilize evaluation knowledge automatically, especially in distributed environments. In this study, Common Evaluation Model (CEM) and INtelligent Evaluation System (INES), which simplify, speed up the evaluation process and decrease the evaluation cost, were developed. The study indicates that it is possible to put knowledge related to evaluation into a structured format. In this scope, a methodology was developed to handle the heuristic knowledge of experts from different domains and information from different sources for evaluation purposes. In this method, evaluation knowledge was represented as a reference model of evaluation objectives, production rules, measures, methods and parameters. Evaluation Objectives indicate what is going to be evaluated. Evaluation rules are criteria used to assess the collected parameters or calculated evaluation measures. Evaluation parameters are variables needed for applying rules or calculating the result of methods. The results of methods are defined as measures in order to simplify the evaluation rules and provide reusability. Evaluation methods are the algorithms for analyzing the collected parameters or / and calculating measures used in the rules. CEM shows the relation between evaluation objectives, rules, measures, methods and parameters. Using Reference Model of Evaluation Knowledge and CEM decreases the number of evaluation rules that are necessary to perform an evaluation to the related application. CEM also simplifies the representation of evaluation knowledge. INES is a hybrid expert-fuzzy system and was developed based on CEM and evaluation needs. Before development of INES, AI techniques including expert systems, fuzzy logic, neural networks, genetic algorithms, intelligent agents and conventional programming were investigated and compared with respect to achieving high level requirements of Evaluation Systems. INES’s Knowledge Base (KB) and KB Editor were developed for forming, editing and updating evaluation knowledge. INES’s Inference Engine was developed for executing the evaluation definition, which includes evaluation objectives, production rules, measures, methods and parameters. Backward chaining technique was used for INES’s inferencing. Some benefits of INES, which are mostly AI related, are speeding up the evaluation process, decreasing the evaluation cost, explaining the reason of evaluation results, modelling the uncertainty on an overall evaluation, providing reasoning on linguistic variables, providing a flexible structure, allow updating evaluation knowledge base without changing the source code, reducing the complexity associated with the evaluation and providing an objective and a reliable evaluation. INES was successful and was tested in the following conditions: • Knowledge of experts from the related domain and knowledge (or information) from the related sources for evaluation purposes are existed. • Identifying evaluation criteria from the expert knowledge and information from different sources is possible. INES was implemented for the first time in various areas from different domains such as evaluation of Air Defence System, instructor performance, personnel selection, and pilot performance. Evaluation of an Air Defence System using INES is given in the paper. As the evaluation includes uncertainty in some aspects, Fuzzy Logic was used for reasoning. But it was realized that Fuzzy Logic could be used to perform overall performance or assessment instead of the evaluation itself for complex tasks. In other words, fuzzy logic can be more beneficial and more easily used for overall evaluation of main objective instead of all aspects of evaluation. A lot of parameters for evaluation are required and writing a lot of rules for these parameters in fuzzy logic is not an efficient way. As more rules are needed for complex systems, it becomes increasingly difficult to relate these rules to the system. Therefore, fuzzy system was used at an abstract level of evaluation.