chapter27-Question Answering

Speech and language Processing, chapter27:Question Answering

Question Answering System

什么是 Question Answering?

主要分为两类:

  1. 多文档的智能问答系统
  • 针对一系列文档提出的问题
  • 答案可能出现多次,也可能没有出现
  • 主要应用在于互联网搜索引擎,文本资料库的搜索,比如新闻档案、医学文献、科学文章等
  1. 单个文档的智能问答

Question Type

  • Simple (factoid) questions (most commercial systems): 简单的问题,可以用简单的事实回答,答案简短通常是一个 named entity
    • Who wrote the Declaration of Independence?
    • What is the average age of the onset of autism?
    • Where is Apple Computer based?
  • Complex (narrative) questions: 稍微复杂的叙述问题,答案略长
    • What do scholars think about Jefferson’s position on dealing with pirates?
    • What is a Hajj?
    • In children with an acute febrile illness, what is the efficacy of single medication therapy with acetaminophen or ibuprofen in reducing fever?
  • Complex (opinion) questions: 复杂的问题,通常是关于观点/意见
    • Was the Gore/Bush election fair?

问答系统分类

现代智能问答系统的主要有两个范式(two major modern paradigms of question answering)以及混合方法, 都是关注于 事实性的回答(factoid question)。:

  • IR-based question answering 基于信息检索的智能问答
  • knowledge-based question answering 基于知识库的智能问答
  • Hybrid approaches (IBM Watson)

对比下三者,具体了解之后可以再会过头来看看:

IR-based Factoid question answering

IR-based factoid AQ 的流程图,包括三个阶段:问题处理(question processing), 篇章检索(passage retrieval and ranking), 和 答案处理(answer processing).

  • Question Processing
    Answer Type Detection: 分析 question,决定 answer type
    Query Formulation: 形成合适的查询语句进行检索
  • Passagge Retrieval
    通过检索得到 top N documents
    把 documents 拆分称合适的单位(unit/passage)
  • Answer Processing
    得到候选的 answer
    进行排序,选出最佳 answer

Question Processing

两个任务:

  • answer type detection
    • what kind of entity (person, place) is the answer?
  • 形成合适的 query
    • what is the query to the IR system

Answer type: the kind of entity the answer consists of (person, location, time, etc.)
Query: the keywords that should be used for the IR system to use in searching for documents
Focus: the string of words in the question that are likely to be replaced by the answer in any answer string found

在这个阶段需要做的事:

举个栗子:

Answering Type Detection (Question classification)

通常而言,我们可以把它当作一个机器学习的分类问题。

  • 定义类别
  • 注释训练数据,给数据打上分类标签
  • 训练分类器,所用特征可以包括 hand-written rules.
定义分类类别

前人已经提供了一些 answer type 的层次型分类结构,如 answer type Taxonomy(from Li & Roth).

  • Two-layered Taxonomy
  • 6 coarse classes:(coarse-grained tag)
    ABBEVIATION, ENTITY, DESCRIPTION, HUMAN, LOCATION, NUMERIC_VALUE
  • 50 fine classes:(fine-grained tags)
    HUMAN: group, individual, title, description
    ENTITY: animal, body, color, currency...
    LOCATION: city, country, mountain...
提取特征

将 answer type 看作一个监督学习任务。question中可以用来分类的特征:

  • words in the questions,
  • the part-of-speech of each word
  • named entities in the questions
  • answer type word or question headword: 通常 wh-word 之后的第一个 NP 可以用来作为特征
  • semantic information about the words,WordNet synset ID
分类方法

总结下可以用来分类的方法:

  • hand-written rules
  • machine learning
  • hybrids

其中 rules 包括:

  • regular expression-baesd rules
    • \(\text{who {is | was | are | were} PERSON}\)
    • PERSON(YEAR-YEAR)
  • Other rules use the question headword
    • headword of first noun phrase after wh‐word

当然也可以把上述某些分类方法当作特征一起进行训练。就分类效果而言,PERSON, LOCATION, TIME 这类的问题类型有更高的准确率,REASON,DESCRIPTION 这类的问题更难识别。

Query Formulation

根据 question 产生一个 keyword list,作为 IR 系统的输入 query。可能的流程是去除 stopwords,丢掉 question word(where, when, etc.),找 noun phrases,根据 tfidf 判断 keywords 的去留等等。
如果 keywords 太少,还可以通过 query expansion 来增加 query terms。

keyword selection algorithm:
- select all non-stop words in quotations - select all NNP words in recognized named entities - select all complex nominals with theor adjectival modifiers - select all other complex nominals - select all nouns with their adjetival modifiers - select all other nouns - select all verbs - select all adverbs - select the QFW word(skipped in all previous steps) - select all other words

最终得到的 query:

Passage Retrieval

有了 query,我们进行检索,会得到 top N 的文档,然而文档并不是得到 answer 的最好的单位,下一步我们需要从文档抽取 potential answer passages,来方便后面的 answer processing。passage 可以是 sections, paragraphs, sentence,具体情况具体分析。

1.rank the documents by relevance
2.extract a set of potential answer passages from the retrieved set of documents. the passage could be sections, paragraphs, sentences.
3.passage retrieval:
- run a named entity or answer type classification on the retrieved passages.
- 然后对剩下来的 passages 进行排序,通过监督学习进行分类,有以下一些列特征:

对于基于 web 的 QA 系统,我们可以依靠网页搜索来做 passage extraction, 简单的说,可以把网页搜索产生的 snippets 作为 passages。

总结下 passage retrieval:

Answer Processing

现在,我们已经有了 answer type,也选出了相关的 passages,就可以进一步缩小 candidate answer 的范围。

answer-extraction task
主要有两种方法

answer-type pattern extraction

基于正则化匹配的机制。如果 answer types 是 HUMAN 或者 DISTANCE-QUANTITY. 可以直接通过模式匹配得到:

有时候光用 pattern-extraction 方法是不够的,一方面我们不能创造规则,另一方面 passage 里也可能有多个 potential answer。另外,对于没有特定定命名实体类型的答案,我们可以使用正则表达式(人工编写或自动学习)。

真“人工”智能!!! 前两天北理工交流会,有个自动化所的博士讲的论文就是用神经网络来生成很自然的回答,赞!

N-gram tiling/redundancy-based approach

N-gram tiling 又被称为 redundancy-based approach(Brill et al. 2002, Lin 2007),基于网页搜索产生的 snippet,进行 ngram 的挖掘,具体步骤如下:

  1. N-gram mining
    提取每个片段中的 unigram, bigram, and trigram, 并赋予权重
  2. N-gram filtering
    根据 ngram 和预测的 answer type 间的匹配程度给 ngram 计算分数
  3. N-gram tiling 将重叠的 ngram 连接成更长的答案

    standard greedy method

    1. start with the highest-scoring candidate and try to tile each other candidate with this candidate
    1. add the best-scoring concatenation to the set of candidates
    2. remove the lower-scoring candidate
    3. continue until a single answer is built

其中怎么求这个 weigth, 也就是 ngram 和 answer type 的匹配度:

Ranking Candidate Answers

In other cases we use machine learning to combine many rich features about which phrase is the answer

可能用到的 feature:

总结 IR-based question answering

  • factoid question answering
    • answer type detection
    • query formulation
    • passage retrieval
    • passage ranking
    • answer extration
  • web-based factoid question answering

Knowledge-based Question Answering

当大量的信息以结构化的形式存储时, 通过语义分析(semantic parsers) 将 query 映射成一个 logical form.

未完待续。。

reference: