Gender Bias in Machine Translation: A Statistical Evaluation of Google Translate and DeepL for English, Italian and German

Despite the significant advancements made in the field of Machine Translation (MT) technology, there are still some challenges that need to be addressed. One such challenge is represented by the issue of gender bias in machine translation systems. The main objective of this study is to examine and investigate the presence of gender bias in MT systems and identify any potential issues related to the use of sexist language. The research evaluates the performance of Google Translate and DeepL in terms of natural gender translation, particularly the frequency of male and female forms used in translating sentences that refer to professions without any other gender-specific words. The evaluation is carried out using the MT-GenEval corpus [2] contextual subset, for English-Italian and English-German language pairs. The paper presents the statistical findings obtained from the evaluation.