Bilingual Evaluation by Phrase Structure
DOI:
https://doi.org/10.5281/zenodo.23222254Palavras-chave:
Machine Translation, BEPhS, Translation Evaluation, Bilingual Translation, Evaluation by Phrase StructureResumo
This article presents a new metric for evaluating translation between two languages. Bilingual evaluation by phrase structure (BEPhS) uses corresponding sentence structure across two languages in order to calculate a translation score for text candidates. By using an electronic dictionary, the metric searches for correlated terms and words, defining the weight of all parts of the sentence. The tests were carried out using samples of texts translated in English-Portuguese and Portuguese-English pairs. Experiments on these two language pairs show the ability for more accurate scoring compared to other classical translation evaluation methods. The results of the experiments were validated by professional human translators.
Referências
AGRAWAL, R.; JOLLY, S. K. Big data and machine learning for evaluating machine translation. Cham: Springer International Publishing, 2020. DOI: https://doi.org/10.1007/978-3-030-39119-5_2.
BANERJEE, S.; LAVIE, A. Meteor: an automatic metric for MT evaluation with improved correlation with human judgments. In: PROCEEDINGS OF THE ACL WORKSHOP ON INTRINSIC AND EXTRINSIC EVALUATION MEASURES FOR MACHINE TRANSLATION AND/OR SUMMARIZATION, jun. 2005, Ann Arbor, Michigan. Proceedings [...]. Ann Arbor, Michigan: Association for Computational Linguistics, 2005. p. 65-72. Disponível em: https://aclanthology.org/W05-0909.
BABYCH, B. Automated MT evaluation metrics and their limitations. Revista Tradumàtica: tecnologies de la traducció, n. 12, p. 464-470, 2014. DOI: https://doi.org/10.5565/rev/tradumatica.70.
BHATTACHARYYA, P. Machine translation. 1. ed. Boca Raton: CRC Press, 2015.
DAVE, S.; PARIKH, J.; BHATTACHARYYA, P. Interlingua-based English-Hindi machine translation and language divergence. Machine Translation, v. 16, p. 251-304, 2001. DOI: https://doi.org/10.1023/A:1021902704523.
DOUGLAS, M. Trading in the zone: master the market with confidence, discipline, and a winning attitude. Upper Saddle River: Prentice Hall Press, 2001.
ESPLÀ-GOMIS, M.; SÁNCHEZ-MARTÍNEZ, F.; FORCADA, M. L. Predicting insertion positions in word-level machine translation quality estimation. Applied Soft Computing, v. 76, p. 174-192, 2019. DOI: https://doi.org/10.1016/j.asoc.2018.11.036.
HATIM, B.; MUNDAY, J. Translation: an advanced resource book for students. 2. ed. London: Routledge, 2019.
HUTCHINS, W. J. Machine translation: past, present, future. Chichester: Ellis Horwood Ltd., 1986. Disponível em: https://dl.acm.org/doi/10.5555/6607.
HUTCHINS, W. J. Machine translation. Chichester: John Wiley and Sons Ltd., 2003.
HUTCHINS, W. J.; SOMERS, H. L. An introduction to machine translation. London: Academic Press, 1992. v. 362.
ISLAM, M. A.; MUKTA, M. S. H. A comprehensive understanding of popular machine translation evaluation metrics. International Journal of Computational Science and Engineering, v. 25, n. 5, p. 467-478, 2022. DOI: https://doi.org/10.1504/IJCSE.2022.126258.
MALMKJAER, K.; WINDLE, K. The Oxford handbook of translation studies. Oxford: Oxford University Press, 2011. DOI: https://doi.org/10.1093/oxfordhb/9780199239306.001.0001.
MATHUR, N.; BALDWIN, T.; COHN, T. Tangled up in bleu: reevaluating the evaluation of automatic machine translation evaluation metrics. arXiv preprint, arXiv:2006.06264, 2020. DOI: https://doi.org/10.48550/arXiv.2006.06264.
PAPINENI, K.; ROUKOS, S.; WARD, T.; ZHU, W. J. BLEU: a method for automatic evaluation of machine translation. In: ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, 40., jul. 2002, Philadelphia, Pennsylvania. Proceedings [...]. Philadelphia, Pennsylvania: Association for Computational Linguistics, 2002. p. 311-318. Disponível em: https://aclanthology.org/P02-1040.
SHI, C.; XIANG, Y.; YU, J.; SOOD, K.; GAO, L. Machine translation-based fine-grained comments generation for Solidity smart contracts. Information and Software Technology, v. 153, p. 107065, 2023. DOI: https://doi.org/10.1016/j.infsof.2022.107065. Disponível em: https://www.sciencedirect.com/science/article/pii/S0950584922001744.
SHUTTLEWORTH, M. Dictionary of translation studies. 4. ed. London: Routledge, 2014.
SOANES, C.; STEVENSON, A. Concise Oxford English dictionary. 11. ed. Oxford: Oxford University Press, 2004.
SU, C.; HUANG, H.; SHI, S.; JIAN, P.; SHI, X. Neural machine translation with Gumbel Tree-LSTM based encoder. Journal of Visual Communication and Image Representation, v. 71, p. 102811, 2020.
WAGNER, R. A.; FISCHER, M. J. The string-to-string correction problem. Journal of the ACM, v. 21, n. 1, p. 168-173, 1974. DOI: https://doi.org/10.1145/321796.321811.
WILKS, Y. Machine translation: its scope and limits. New York: Springer Science & Business Media, 2009.
ZHAO, Y.; KOMACHI, M.; KAJIWARA, T.; CHU, C. Region-attentive multimodal neural machine translation. Neurocomputing, v. 476, p. 1-13, 2022. DOI: https://doi.org/10.1016/j.neucom.2021.12.076. Disponível em: https://www.sciencedirect.com/science/article/pii/S0925231221019317.
