Evaluation of Research Quality (VQR): a case study based on DINDSCAL for compositions

The eValuation of Research Quality (VQR) is one the most important assessment process achieved by the National Agency for the Evaluation of Universities and Research Institutes (ANVUR). Its main task is to provide information on the status of the Italian research system assessing the performance of universities in various scientific areas. The entities measured are made up of researchers, assistants, first and second band professors, fixed-term professors and researchers, technology and research executives. For the purposes, ”research products” as journal contributions, volume contributions, and other types of scientific products are considered. The basic evaluation criteria were defined by groups of experts (GEV) according to the specific characteristics of each subject area and through a synthetic statement on the products. In this framework differences between GEV groups on a differential set of quality judgment should be explained in terms of compositional dissimilarity matrices. In literature the INDSCAL (Individual Differences Scaling) model is used to study the individual differences in three-way data by doubly centered a set of matrices of squared dissimilarity measures between a range of stimuli. A direct approach is here preferred, defined DINDSCAL (Direct INDividual Differences SCALing), in order to directly analyze simultaneous slices of dissimilarity matrices organized as compositional data. The compositional aspect of data allow to understand, at a first glance, which is the research product with the highest assessment compared to the remaining ones, irrespective of the role and the type of institutions to which researchers belong. Additionally, the DINDSCAL algorithm underlines the main divergencies made by each GEV group in terms of research output classification.