著者
Steve Whittaker, Loren Terveen, Bonnie A. Nardi
タイトル
Let's stop pushing the envelope and start addressing it: a reference task agenda for HCI
ページ
75-106
日時
2000
概要
We identify a problem with the process of research in the HCI community - an overemphasis on "radical invention" at the price of achieving a common research focus. Without such a focus, it is difficult to build on previous work, to compare different interaction techniques objectively, and to make progress in developing theory. These problems at the research level have implications for practice, too; as researchers we often are unable to give principled design advice to builders of new systems. We propose that the HCI community try to achieve a common focus around the notion of reference tasks. We offer arguments for the advantages of this approach, as well as considering potential difficulties. We explain how reference tasks have been highly effective in focussing research into information retrieval and speech recognition. We discuss what factors have to be considered in selecting HCI reference tasks and present an example reference task (for searching speech archives). We conclude with recommendations about necessary steps to execute the reference task research agenda, including both required technical research, as well as changes in HCI research community practice. The technical research involves: identification of important user tasks by systematic requirements gathering; definition and operationalisation of reference tasks and evaluation metrics; execution of task-based evaluation along with judicious use of field trials. Perhaps more important, we also suggest changes in HCI community practice. We must create forums for discussion of common tasks and methods by which people can compare systems and techniques. Only through this can the notion of reference tasks be integrated into the process of research and development, enabling the field to achieve the focus it desperately needs.
コメント
CHIPlaceでの Henry Liebermanの議論|http://web.media.mit.edu/~lieber/Misc/Tyranny-Evaluation.html あたりで参照されていた論文。 使えるかどうかよくわからない「新発明」ばっかりにこだわらず Reference Taskをきめてじっくり研究すべきだと 言っているらしい。
概要
「評価中心主義: 逝ってよし」の方が説得力あると 思うんだけどなぁ。(2003/8/29 増井)
カテゴリ
UI, Evaluation
Category: UI Evaluation
Journal: Human Computer Interaction
Comment: CHIPlaceでの
        <a href="/~masui/PIM/programs/search.cgi?title=Henry+Lieberman%1B%24B%24N5DO%40%1B%28B%7Chttp%3A%2F%2Fweb.media.mit.edu%2F%7Elieber%2FMisc%2FTyranny-Evaluation.html">Henry Liebermanの議論|http://web.media.mit.edu/~lieber/Misc/Tyranny-Evaluation.html</a>
        あたりで参照されていた論文。
        使えるかどうかよくわからない「新発明」ばっかりにこだわらず
        Reference Taskをきめてじっくり研究すべきだと
        言っているらしい。
Abstract: We identify a problem with the process of research in the
        HCI community - an overemphasis on "radical invention" at
        the price of achieving a common research focus. Without
        such a focus, it is difficult to build on previous work,
        to compare different interaction techniques objectively,
        and to make progress in developing theory. These problems
        at the research level have implications for practice,
        too; as researchers we often are unable to give
        principled design advice to builders of new systems. We
        propose that the HCI community try to achieve a common
        focus around the notion of reference tasks. We offer
        arguments for the advantages of this approach, as well as
        considering potential difficulties.  We explain how
        reference tasks have been highly effective in focussing
        research into information retrieval and speech
        recognition. We discuss what factors have to be
        considered in selecting HCI reference tasks and present
        an example reference task (for searching speech
        archives). We conclude with recommendations about
        necessary steps to execute the reference task research
        agenda, including both required technical research, as
        well as changes in HCI research community practice. The
        technical research involves: identification of important
        user tasks by systematic requirements gathering;
        definition and operationalisation of reference tasks and
        evaluation metrics; execution of task-based evaluation
        along with judicious use of field trials. Perhaps more
        important, we also suggest changes in HCI community
        practice. We must create forums for discussion of common
        tasks and methods by which people can compare systems and
        techniques.  Only through this can the notion of
        reference tasks be integrated into the process of
        research and development, enabling the field to achieve
        the focus it desperately needs.
Bibtype: Article
URL: http://citeseer.nj.nec.com/whittaker00lets.html
Pages: 75-106
Author: Steve Whittaker
        Loren Terveen
        Bonnie A. Nardi
Title: Let's stop pushing the envelope and start addressing it:
        a reference task agenda for HCI
Comment1: 「評価中心主義: 逝ってよし」の方が説得力あると
        思うんだけどなぁ。(2003/8/29 増井)
        
Year: 2000
Date: 2003/08/29 00:31:10
Volume: 15