TAC KBP 2016 : Multi-lingual Event Detection and Coreference Tasks

Overview

The Event Detection and Coreference tasks at NIST TAC KBP aims to identify the explicit mentioning of Events in English, Chinese, and Spanish documents. Every instance of a mention of the relevant Event types must be identified. The Event Detection task is required to detect the selected Event Types and Subtypes taken from the Rich ERE Annotation Guidelines: Events (current version is v2.9.). Also, the task is to identify three REALIS {ACTUAL, GENERIC, OTHER}, which are described in the Rich ERE guidelines.

Source data will be provided by LDC. For English, about 200 annotated documents will be provided prior to the evaluation as training set. For Chinese, a training set containing 200,000 words will be provided. For Spanish, a training set containing 120,000 words will be available by January 2016.

For the formal evaluation, the minimum of 60 documents will be given to the participants for each language. In the evaluation corpora, the minimum of 30 mentions of event types and subtypes will be included. We plan to evaluate both newswire articles and discussion forum text. The Event Detection Task requires participants to identify all relevant Events Mention instances within each sentence. If the same Event is mentioned in several places in the document, the participants will list them all. The Event Coreference Task requires participants to identify the coreference links between Event Mention instances within a document (but not across documents).

Updates

  • Aug 24 2016: Instructions provided for the evaluation format converter.
  • Aug 24 2016: The task definitation is updated to correct evaluation dates.
  • Aug 11 2016: The task definitation is updated.
  • July 28 2016: Scorer and the scoring documentation is online.
  • Feb 16 2016: The task definitation is online.