TAC KBP 2015 : Event Detection and Coreference Tasks for English

Overview

The Event Detection and Coreference tasks at NIST TAC KBP focus on the explicit mentioning of Events in text. One long-term goal of Event Detection and Coreference is to detect subevent structures in a future task.

In the Event Detection task participants must:

  • detect each mention of an event (including multiple mentions within a sentence),
  • identify for each mention its Event Type and Subtype, as listed in the Rich ERE Annotation Guidelines Overview.
  • identify for each event mention one of three REALIS states (ACTUAL, GENERIC, OTHER), which are described in the Rich ERE guidelines. If the same Event is mentioned in several places in the document, the participant must detect them all.

In the Event Coreference task participants must:

  • identify all coreference links between Events Mention instances within a document.
The data sources are provided by LDC: (TO BE UPDATED). About 150 or more annotated texts will be provided prior to the evaluation as a training set. For the formal evaluation, about 200 texts will be provided. The texts will include newswire articles and discussion forums.

Overview Paper

The Notebook version of the TAC KBP 2015 overview paper can be found here. Stay tuned for the the proceeding version of the KBP workshop official publications.

Updates

  • Sep 11: Validator now also check whether the mention IDs in the submissio are unique.
  • Aug 28: Scorer will now canonicalize the mention types to help reduce some inconsistency in annotation guidelines. See scoring page for details.
  • Aug 28: Schedule change of the evaluation window, see schedule page for details.
  • Aug 20: Slight updates on documentation about the token file formats, the scoring guildine is of version 2.7 now.
  • Aug 12: Update scorer and converters to be align with LDC's new annotation, see the Scorer page for more details. It is highly suggested to get the most recent version now.
  • May 6: The task page is online.