BenchIEFL: A Manually Re-Annotated Fact-Based Open Information Extraction Benchmark

BenchIEFL: A Manually Re-Annotated Fact-Based Open Information Extraction Benchmark

Speaker: Fabrice

Topics: Open Information Extraction

Abstract

Open Information Extraction (OIE) is a field of natural language processing that aims to present textual information in a format that allows it to be organized, analyzed and reflected upon. Numerous OIE systems are developed, claiming ever-increasing performance, marking the need for objective benchmarks. BenchIE is the latest reference we know of. Despite being very well thought, we noticed a number of issues we believe are limitative. Therefore, we propose BenchIEFL, a new OIE benchmark which fully enforces principles of BenchIE while containing less errors, omissions and shortcomings when candidate facts are matched towards reference ones. BenchIEFL allows to draw some insightful conclusions on the actual performance of OIE extractors.