Framework for MT Evaluation (FEMTI)


The following announcement is from FEMTI coordinators Eduard Hovy, Margaret King and Andrei Popescu-Belis.

FEMTI, the Framework for Machine Translation Evaluation in ISLE, is available for consultation at the following mirrored URLs:

http://www.isi.edu/natural-language/mteval/
http://www.issco.unige.ch/projects/isle/femti/

FEMTI is a structured repertoire of methods used to evaluate MT systems. The first part of FEMTI enables evaluators to specify a context of use for the MT system to be evaluated, while the second part defines relevant quality characteristics and metrics.

FEMTI is a result of the ISLE Project, funded by the US National Science Foundation and the Swiss and Danish Governments. A series of workshops provided valuable input to the framework (see http://www.issco.unige.ch/projects/isle/ewg.html).

The framework is discussed in the following article: Eduard Hovy, Margaret King and Andrei Popescu-Belis (2002) - 93 Principles of Context-Based Machine Translation Evaluation 94, Machine Translation, 17:1, p.43-75, 2002.

Users are welcome to browse the framework at the above URLs and to use the feedback buttons to provide comments.