Much more lately 850140-72-6a study have revealed that the inter-rater agreement of peer reviewers for journal content articles is as lower as .23. Panel A guidance advised that outcomes should be informed by quotation metrics but REF evaluators have been provided no standardised advice as to how it was to be related to the ranking of 3 or four.To take a look at whether or not the described doubling of globe-leading research top quality in the REF was supported by bibliometric quality, we acquired details of all Panel A outputs submitted to the RAE 2008 and REF 2014, and in comparison them to different bibliometric indices, together with worldwide functionality on the identical indicators in that time body.We attained all the outputs submitted for Panel A and B for both RAE and REF from the community internet site. The outputs submitted for REF have been categorised into journal articles or blog posts, chapters, publications and other kinds of analysis communications. Considering that citation metrics are most appropriate for journal articles or blog posts, we restricted even more evaluation to these analysis outputs which shaped ninety seven%/ninety nine% of the total outputs in Panel A for RAE/REF. The RAE and REF altered the nomenclature of panels and subpanels and to make sure comparability, we reworked all the RAE panel data into REF panel terminology, thus enabling a immediate comparison.The RAE/REF content articles have been matched from the Centre for Science and Technologies Studies, Leiden variation of the Internet of Science database which includes data on more than forty two million articles or blog posts from in excess of 18 thousand journals and tracks a lot more than 555 million citations. To compare like-with-like, the citation to a certain report was compared to all other content articles in the same discipline and from the same yr of publication allowing the perseverance of the globally percentile of that write-up. Some content articles are in journals that relevant to a lot more than one group, in which circumstance the overall standing of the report is the sum of its fractional standing in the fields to which it is assigned. We excluded self-citations in our analysis as we needed to evaluate externally gained quotation impact, therefore excluding up-front any source of distortion. Finally, to validate our use of bibliometrics we examined how typically the journal articles or blog posts in our RAE/REF set referred to other posts which were also in this dataset, a evaluate referred to as internal coverage, figures larger than 70% supply legitimate and steady actions of bibliometric indicators. Given that the fields of Medical and Existence Sciences in this databases are mainly coterminous with Panel A in RAE/REF, we received bibliometric info relating to Uk vs. With this info available for all RAE and REF journal articles we questioned the adhering to inquiries: a) how a lot of RAE/REF content articles ended up available for bibliometric examination b) how did the top quality of posts submitted in the RAE vs. REF evaluate in terms of international percentile standing c) how did high quality of all British isles Healthcare and Daily life Sciences posts adjust for the duration of that interval d) has there been a decreasing of the percentile thresholds at which papers are labelled three/four and e) is this adjust of threshold noticed only in Panel A or also in Panel B .The REF Panel A received 50,044 posts, reflecting a drop in general variety from fifty six,639 received in RAE 2008. Of these, 96% of the RAE and 98% of the REF articles or blog posts have been linked to the CWTS Internet of Science database and thus amenable to bibliometric analyses.