Sakai Tetsuya 酒井哲也

Researcher Number	80723519
Other IDs
Affiliation (Current)	2025: 早稲田大学, 理工学術院, 教授
Affiliation (based on the past Project Information) *help	2016 – 2021: 早稲田大学, 理工学術院, 教授
Review Section/Research Field	Principal Investigator Library and information science/Humanistic social informatics Except Principal Investigator Library and information science/Humanistic social informatics
Keywords	Principal Investigator 人工知能 / 情報検索 / 情報アクセス / 自然言語 / 評価 / 評価指標 / データセット / ordinal quantification / ナゲット / ヘルプデスク / 対話 … More Except Principal Investigator … More 検索実験の妥当性 / 多段階要約 / マルチファセット検索 / クエリ推薦 / 検索結果の多様化 / タスク / クエリマイニング / デジタル空間探索と実空間探索 / 評価実験用データセットの品質 / パーソナライズド検索 / Ostensive Search / 博物館鑑賞体験 / 気づき / 提示型検索 / 対話 / ユーザビヘイビア / ファセット検索 / ユーザの状況 / 探索的検索 Less

Nugget-based Automatic Evaluation of Task-oriented dialoguesPrincipal Investigator
- Principal Investigator
  
  Sakai Tetsuya
- Project Period (FY)
  2017 – 2020
- Research Category
  
  Grant-in-Aid for Scientific Research (B)
- Research Field
  
  Library and information science/Humanistic social informatics
- Research Institution
  Waseda University
Exploratory Search Considering the User's Situation
- Principal Investigator
  
  Kando Noriko
- Project Period (FY)
  2016 – 2019
- Research Category
  
  Grant-in-Aid for Scientific Research (A)
- Research Field
  
  Library and information science/Humanistic social informatics
- Research Institution
  National Institute of Informatics

All 2022 2021 2020 2019 2018 2017 2016

All Journal Article Presentation

[Journal Article] Relevance Assessments for Web Search Evaluation: Should We Randomise or Prioritise the Pooled Documents?2022
- Author(s)
  Tetsuya Sakai, Sijie Tao, and Zhaohao Zeng
- Journal Title
  
  ACM Transactions on Information Systems (TOIS)
  
  Volume: 40(4) Issue: 4 Pages: 1-35
- DOI
  10.1145/3494833
- Peer Reviewed / Open Access
- Data Source
  KAKENHI-PROJECT-16H01756
[Journal Article] On the Two-Sample Randomisation Test for IR Evaluation2021
- Author(s)
  Tetsuya Sakai
- Journal Title
  
  Proceedings of SIGIR '21: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2021)
  
  Volume: - Pages: 1980-1984
- DOI
  10.1145/3404835.3463002
- Peer Reviewed
- Data Source
  KAKENHI-PROJECT-16H01756
[Journal Article] WWW3E8: 259,000 Relevance Labels for Studying the Effect of Document Presentation Order for Relevance Assessors2021
- Author(s)
  Tetsuya Sakai, Sijie Tao and Zhaohao Zeng
- Journal Title
  
  Proceedings of SIGIR '21: The 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2021)
  
  Volume: - Pages: 2376-2382
- DOI
  10.1145/3404835.3463236
- Peer Reviewed
- Data Source
  KAKENHI-PROJECT-16H01756
[Journal Article] A Closer Look at Evaluation Measures for Ordinal Quantification2021
- Author(s)
  Tetsuya Sakai
- Journal Title
  
  Proceedings of the CIKM 2021 Workshops co-located with 30th ACM International Conference on Information and Knowledge Management (CIKM 2021)
  
  Volume: -
- Peer Reviewed / Open Access
- Data Source
  KAKENHI-PROJECT-16H01756
[Journal Article] On the Instability of Diminishing Return IR Measures2021
- Author(s)
  Tetsuya Sakai
- Journal Title
  
  Proceedings of 43rd European Conference on IR Research, ECIR 2021 Part I (LNCS 12656)
  
  Volume: - Pages: 572-586
- DOI
  10.1007/978-3-030-72113-8_38
- ISBN
  9783030721121, 9783030721138
- Peer Reviewed
- Data Source
  KAKENHI-PROJECT-16H01756
[Journal Article] Retrieval Evaluation Measures that Agree with Users’ SERP Preferences: Traditional, Preference-based, and Diversity Measures2020
- Author(s)
  Tetsuya Sakai and Zhaohao Zeng
- Journal Title
  
  ACM Transactions on Information Systems (ACM TOIS)
  
  Volume: 39(2) Issue: 2 Pages: 1-35
- DOI
  10.1145/3431813
- Peer Reviewed
- Data Source
  KAKENHI-PROJECT-16H01756
[Journal Article] Good Evaluation Measures based on Document Preferences2020
- Author(s)
  Tetsuya Sakai and Zhaohao Zeng
- Journal Title
  
  Proceedings of SIGIR '20: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2020)
  
  Volume: - Pages: 359-368
- DOI
  10.1145/3397271.3401115
- Peer Reviewed
- Data Source
  KAKENHI-PROJECT-16H01756
[Journal Article] Randomised vs. Prioritised Pools for Relevance Assessments: Sample Size Considerations2020
- Author(s)
  Tetsuya Sakai and Peng Xiao
- Journal Title
  
  Proceedings of Asia Information Retrieval Symposium2019, AIRS2019 (LNCS 12004)
  
  Volume: - Pages: 94-105
- DOI
  10.1007/978-3-030-42835-8_9
- ISBN
  9783030428341, 9783030428358
- Peer Reviewed
- Data Source
  KAKENHI-PROJECT-16H01756
[Journal Article] ユーザの感覚に近い多様化検索評価指標2020
- Author(s)
  酒井哲也, Zhaohao Zeng
- Journal Title
  
  FIT2020講演論文集
  
  Volume: CD-002
- Peer Reviewed / Open Access
- Data Source
  KAKENHI-PROJECT-16H01756
[Journal Article] Which Diversity Evaluation Measures Are “Good”?2019
- Author(s)
  Tetsuya Sakai and Zhaohao Zeng
- Journal Title
  
  Proceedings of SIGIR'19: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2019)
  
  Volume: - Pages: 595-604
- DOI
  10.1145/3331184.3331215
- Peer Reviewed
- Data Source
  KAKENHI-PROJECT-16H01756
[Journal Article] Comparing Two Binned Probability Distributions for Information Access Evaluation2018
- Author(s)
  Sakai Tetsuya
- Journal Title
  
  Proceedings of the 41st International ACM SIGIR Conference on Research & Development in Information Retrieval (SIGIR 2018)
  
  Volume: 41 Pages: 1073-1076
- DOI
  10.1145/3209978.3210073
- Peer Reviewed
- Data Source
  KAKENHI-PROJECT-16H01756
[Journal Article] Topic Set Size Design for Paired and Unpaired Data2018
- Author(s)
  Sakai Tetsuya
- Journal Title
  
  Proceedings of the 2018 ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR 2018)
  
  Volume: 8 Pages: 199-202
- DOI
  10.1145/3234944.3234971
- Peer Reviewed
- Data Source
  KAKENHI-PROJECT-16H01756
[Journal Article] Towards Automatic Evaluation of Customer-Helpdesk Dialogues2018
- Author(s)
  Zhaohao Zeng, Cheng Luo, Lifeng Shang, Hang Li, Tetsuya Sakai
- Journal Title
  
  Journal of Information Processing
  
  Volume: 26 Issue: 0 Pages: 768-778
- DOI
  10.2197/ipsjjip.26.768
- NAID
  130007511649
- ISSN
  1882-6652
- Language
  English
- Peer Reviewed / Open Access / Int'l Joint Research
- Data Source
  KAKENHI-PROJECT-17H01830
[Journal Article] Towards Automatic Evaluation of Multi-Turn Dialogues: A Task Design that Leverages Inherently Subjective Annotations2017
- Author(s)
  Tetsuya Sakai
- Journal Title
  
  Proceedings of the 8th International Workshop on Evaluating Information Access (EVIA 2017)
  
  Volume: 8 Pages: 24-30
- Peer Reviewed / Open Access / Int'l Joint Research
- Data Source
  KAKENHI-PROJECT-16H01756
[Journal Article] LSTM vs. BM25 for Open-domain QA: A Hands-on Comparison of Effectiveness and Efficiency2017
- Author(s)
  Sosuke Kato, Riku Togashi, Hideyuki Maeda, Sumio Fujita, and Tetsuya Sakai
- Journal Title
  
  Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '17)
  
  Volume: 40 Pages: 1309-1312
- DOI
  10.1145/3077136.3084147
- Peer Reviewed / Int'l Joint Research
- Data Source
  KAKENHI-PROJECT-16H01756
[Journal Article] Investigating Users' Time Perception during Web Search2017
- Author(s)
  Luo Cheng、Li Xue、Liu Yiqun、Sakai Tetsuya、Zhang Fan、Zhang Min、Ma Shaoping
- Journal Title
  
  Proceedings of the 2017 Conference on Conference Human Information Interaction and Retrieval (CHIIR '17)
  
  Volume: 2 Pages: 127-136
- DOI
  10.1145/3020165.3020184
- Peer Reviewed / Int'l Joint Research
- Data Source
  KAKENHI-PROJECT-16H01756
[Journal Article] Ranking Rich Mobile Verticals based on Clicks and Abandonment2017
- Author(s)
  Mami Kawasaki, Inho Kang, and Tetsuya Sakai.
- Journal Title
  
  Proceedings of the 2017 ACM on Conference on Information and Knowledge Management (CIKM '17)
  
  Volume: 2017 Pages: 2127-2130
- DOI
  10.1145/3132847.3133059
- Peer Reviewed / Int'l Joint Research
- Data Source
  KAKENHI-PROJECT-16H01756
[Journal Article] Mobile Vertical Ranking based on Preference Graphs2017
- Author(s)
  Yuta Kadotami, Yasuaki Yoshida, Sumio Fujita, and Tetsuya Sakai
- Journal Title
  
  Proceedings of the ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR '17).
  
  Volume: 2017 Pages: 225-228
- DOI
  10.1145/3121050.3121082
- Peer Reviewed / Int'l Joint Research
- Data Source
  KAKENHI-PROJECT-16H01756
[Journal Article] Evaluating Mobile Search with Height-Biased Gain2017
- Author(s)
  Cheng Luo, Yiqun Liu, Tetsuya Sakai, Fan Zhang, Min Zhang, and Shaoping Ma
- Journal Title
  
  Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '17)
  
  Volume: 40 Pages: 435-444
- DOI
  10.1145/3077136.3080795
- Peer Reviewed / Int'l Joint Research
- Data Source
  KAKENHI-PROJECT-16H01756
[Journal Article] The Probability that Your Hypothesis Is Correct, Credible Intervals, and Effect Sizes for IR Evaluation2017
- Author(s)
  Tetsuya Sakai
- Journal Title
  
  Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '17)
  
  Volume: 40 Pages: 25-34
- DOI
  10.1145/3077136.3080766
- Peer Reviewed / Int'l Joint Research
- Data Source
  KAKENHI-PROJECT-16H01756
[Journal Article] he Effect of Inter-Assessor Disagreement on IR System Evaluation: A Case Study with Lancers and Students2017
- Author(s)
  Tetsuya Sakai
- Journal Title
  
  Proceedings of the 8th International Workshop on Evaluating Information Access (EVIA 2017)
  
  Volume: 8 Pages: 31-38
- Peer Reviewed / Open Access / Int'l Joint Research
- Data Source
  KAKENHI-PROJECT-16H01756
[Journal Article] Does Document Relevance Affect the Searcher's Perception of Time?2017
- Author(s)
  Luo Cheng、Liu Yiqun、Sakai Tetsuya、Zhou Ke、Zhang Fan、Li Xue、Ma Shaoping
- Journal Title
  
  Proceedings of the Tenth ACM International Conference on Web Search and Data Mining (WSDM '17)
  
  Volume: 10 Pages: 141-150
- DOI
  10.1145/3018661.3018694
- Peer Reviewed / Int'l Joint Research
- Data Source
  KAKENHI-PROJECT-16H01756
[Journal Article] Two Sample T-tests for IR Evaluation2016
- Author(s)
  Sakai Tetsuya
- Journal Title
  
  Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval (SIGIR '16)
  
  Volume: 39 Pages: 1045-1048
- DOI
  10.1145/2911451.2914684
- Peer Reviewed
- Data Source
  KAKENHI-PROJECT-16H01756
[Journal Article] Statistical Significance, Power, and Sample Sizes2016
- Author(s)
  Sakai Tetsuya
- Journal Title
  
  Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval (SIGIR '16)
  
  Volume: 39 Pages: 5-14
- Peer Reviewed
- Data Source
  KAKENHI-PROJECT-16H01756
[Journal Article] Evaluating Search Result Diversity using Intent Hierarchies2016
- Author(s)
  Wang Xiaojie、Dou Zhicheng、Sakai Tetsuya、Wen Ji-Rong
- Journal Title
  
  Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval (SIGIR '16)
  
  Volume: 39 Pages: 415-424
- DOI
  10.1145/2911451.2911497
- Peer Reviewed / Int'l Joint Research
- Data Source
  KAKENHI-PROJECT-16H01756
[Journal Article] A Simple and Effective Approach to Score Standardisation2016
- Author(s)
  Sakai Tetsuya
- Journal Title
  
  Proceedings of the 2016 ACM International Conference on the Theory of Information Retrieval (ICTIR '16)
  
  Volume: 2 Pages: 95-104
- DOI
  10.1145/2970398.2970399
- Peer Reviewed
- Data Source
  KAKENHI-PROJECT-16H01756
[Journal Article] Topic set size design2016
- Author(s)
  Sakai Tetsuya
- Journal Title
  
  Information Retrieval Journal (June 2016)
  
  Volume: 19 Issue: 3 Pages: 256-283
- DOI
  10.1007/s10791-015-9273-z
- Peer Reviewed
- Data Source
  KAKENHI-PROJECT-16H01756
[Journal Article] The Effect of Score Standardisation on Topic Set Size Design2016
- Author(s)
  Sakai Tetsuya
- Journal Title
  
  Ma S. et al. (eds) Information Retrieval Technology. AIRS 2016. (Lecture Notes in Computer Science, vol 9994)
  
  Volume: 12 Pages: 16-28
- DOI
  10.1007/978-3-319-48051-0_2
- ISBN
  9783319480503, 9783319480510
- Peer Reviewed
- Data Source
  KAKENHI-PROJECT-16H01756
[Presentation] RSLNV at the NTCIR-15 Dialogue Evaluation (DialEval-1) Task2020
- Author(s)
  Ting Cao, Fan Zhang, Haoxiang Shi, Zhaohao Zeng, Sosuke Kato, Tetsuya Sakai, Injae Lee, Kyungduk Kim, and Inho Kang
- Organizer
  NTCIR-15
- Int'l Joint Research
- Data Source
  KAKENHI-PROJECT-17H01830
[Presentation] Overview of the NTCIR-15 Dialogue Evaluation (DialEval-1) Task2020
- Author(s)
  Zhaohao Zeng, Sosuke Kato, Tetsuya Sakai, and Inho Kang
- Organizer
  NTCIR-15
- Int'l Joint Research
- Data Source
  KAKENHI-PROJECT-17H01830
[Presentation] SKYMN at the NTCIR-15 DialEval-1 Task2020
- Author(s)
  Junjie Wang, Yuxiang Zhang, Tetsuya Sakai, and Hayato Yamana
- Organizer
  NTCIR-15
- Int'l Joint Research
- Data Source
  KAKENHI-PROJECT-17H01830
[Presentation] Overview of the NTCIR-14 Short Text Conversation Task: Dialogue Quality and Nugget Detection Subtasks2019
- Author(s)
  Zhaohao Zeng and Sosuke Kato and Tetsuya Sakai
- Organizer
  NTCIR-15
- Int'l Joint Research
- Data Source
  KAKENHI-PROJECT-17H01830
[Presentation] SLSTC at the NTCIR-14 STC-3 Dialogue Quality and Nugget Detection Subtasks2019
- Author(s)
  Sosuke Kato and Rikiya Suzuki and Zhaohao Zeng and Tetsuya Sakai
- Organizer
  NTCIR-15
- Int'l Joint Research
- Data Source
  KAKENHI-PROJECT-17H01830
[Presentation] Comparing Two BinnedProbability Distributions for Information Access Evaluation2018
- Author(s)
  Tetsuya Sakai
- Organizer
  ACM SIGIR 2018
- Int'l Joint Research
- Data Source
  KAKENHI-PROJECT-17H01830
[Presentation] Towards Automatic Evaluation of Multi-Turn Dialogues: A Task Design that Leverages Inherently Subjective Annotations2017
- Author(s)
  Sakai, T.
- Organizer
  The 8th International Workshop on Evaluating Information Access (EVIA 2017)
- Int'l Joint Research
- Data Source
  KAKENHI-PROJECT-17H01830
[Presentation] Test Collections and Measures for Evaluating Customer-Helpdesk Dialogues2017
- Author(s)
  Zeng, Z., Luo, C., Shang, L., Li, H., and Sakai, T.
- Organizer
  The 8th International Workshop on Evaluating Information Access (EVIA 2017)
- Int'l Joint Research
- Data Source
  KAKENHI-PROJECT-17H01830
[Presentation] On Estimating Variances for Topic Set Size Design2016
- Author(s)
  Tetsuya Sakai*, Lifeng Shang
- Organizer
  Proceedings of the Seventh International Workshop on Evaluating Information Access (EVIA 2016), June 7, 2016 Tokyo Japan　pp.9-12 【査読あり】
- Int'l Joint Research
- Data Source
  KAKENHI-PROJECT-16H01756
[Presentation] Two-layered Summaries for Mobile Search: Does the Evaluation Measure Reflect User Preferences?2016
- Author(s)
  Makoto P. Kato*, Virgil Pavlu, Tetsuya Sakai, Takehiro Yamamoto, Hajime Morita
- Organizer
  Proceedings of the Seventh International Workshop on Evaluating Information Access (EVIA 2016), June 7, 2016 Tokyo Japan　pp.29-32 【査読あり】
- Int'l Joint Research
- Data Source
  KAKENHI-PROJECT-16H01756

1. Kando Noriko (80270445)

# of Collaborated Projects: 1 results

# of Collaborated Products: 0 results
2. 吉岡真治 (40290879)

# of Collaborated Projects: 1 results

# of Collaborated Products: 0 results
3. 山本岳洋 (70717636)

# of Collaborated Projects: 1 results

# of Collaborated Products: 1 results
4. 相澤彰子 (90222447)

# of Collaborated Projects: 1 results

# of Collaborated Products: 0 results
5. 大島裕明 (90452317)

# of Collaborated Projects: 1 results

# of Collaborated Products: 0 results

Sakai Tetsuya 酒井 哲也

Research Projects

Research Products

Co-Researchers

Nugget-based Automatic Evaluation of Task-oriented dialoguesPrincipal Investigator

Principal Investigator

Project Period (FY)

Research Category

Research Field

Research Institution

Exploratory Search Considering the User's Situation

Principal Investigator

Project Period (FY)

Research Category

Research Field

Research Institution

[Journal Article] Relevance Assessments for Web Search Evaluation: Should We Randomise or Prioritise the Pooled Documents?2022

Author(s)

Journal Title

DOI

Data Source

[Journal Article] On the Two-Sample Randomisation Test for IR Evaluation2021

Author(s)

Journal Title

DOI

Data Source

[Journal Article] WWW3E8: 259,000 Relevance Labels for Studying the Effect of Document Presentation Order for Relevance Assessors2021

Author(s)

Journal Title

DOI

Data Source

[Journal Article] A Closer Look at Evaluation Measures for Ordinal Quantification2021

Author(s)

Journal Title

Data Source

[Journal Article] On the Instability of Diminishing Return IR Measures2021

Author(s)

Journal Title

DOI

ISBN

Data Source

[Journal Article] Retrieval Evaluation Measures that Agree with Users’ SERP Preferences: Traditional, Preference-based, and Diversity Measures2020

Author(s)

Journal Title

DOI

Data Source

[Journal Article] Good Evaluation Measures based on Document Preferences2020

Author(s)

Journal Title

DOI

Data Source

[Journal Article] Randomised vs. Prioritised Pools for Relevance Assessments: Sample Size Considerations2020

Author(s)

Journal Title

DOI

ISBN

Data Source

[Journal Article] ユーザの感覚に近い多様化検索評価指標2020

Author(s)

Journal Title

Data Source

[Journal Article] Which Diversity Evaluation Measures Are “Good”?2019

Author(s)

Journal Title

DOI

Data Source

[Journal Article] Comparing Two Binned Probability Distributions for Information Access Evaluation2018

Author(s)

Journal Title

DOI

Data Source

[Journal Article] Topic Set Size Design for Paired and Unpaired Data2018

Author(s)

Journal Title

DOI

Data Source

[Journal Article] Towards Automatic Evaluation of Customer-Helpdesk Dialogues2018

Author(s)

Journal Title

DOI

Sakai Tetsuya 酒井哲也