Accurate evaluation and consistent results: the case of the optimized version of the preselected items evaluation method

Resumen

Replicable/reliable translation evaluation methods are the most rudimentary thrusts in translation testing. The present research paper was an attempt to contribute to the omnipresence of product quality evaluation. In doing so, this sketch compared two translation evaluation methods, namely the PIE (Preselected items evaluation) and the OPIE (Optimized version of preselected items evaluation). The PIE method as a perturbative testing technique is used to evaluate the product quality. Despite its ubiquity, it could not accurately measure product quality through its parameters: p-value and d-index. To address this critical issue, this paper introduced an optimized version of this method (OPIE) to precisely evaluate the product quality through the application of Feldt's pG-value and the corrected item-total correlation (rit-value). One-hundred translation participants and seven evaluators partook in this research. The evaluators were asked to score translation drafts through the PIE and OPIE methods. To measure up the degree of reliability, Pearson product-moment correlation and regression variable plots were applied. The results demonstrated that the OPIE method was more consistent/reliable based on docimologically acceptable items. Research limitations and implications were discussed.