UMich
2026
Improving How HRI Experimental Studies Are Evaluated
Contributing to a large-scale review of experimental human–robot interaction research to identify methodological practices, reporting gaps, and opportunities for more rigorous study design.
Overview
Evaluating how experiments are designed and reported in HRI
I contributed to a large-scale review of experimental Human-Robot Interaction (HRI) research, screening studies and coding methodological characteristics such as study design, participants, measures, statistical analysis, and reporting practices. I also helped refine the review framework when recurring edge cases made existing criteria difficult to apply consistently.
What I Did
Evaluated research methodology in human-robot interaction.
Reviewed studies and identified the information needed to determine whether they met project criteria.
Noticed recurring areas of ambiguity and inconsistency across papers. Analyzed papers for patterns so we can resolve the ambiguities.
Improved the analysis framework by turning edge cases into clearer rules that could be applied more consistently.
Challenge
Finding patterns & defining our selection criteria
HRI papers varied widely in how they described manipulations, comparison conditions, measures, and analyses. Some studies clearly met our experimental criteria, while others fell into a gray area.
The challenge was to make sense of ambiguity and make consistent methodological decisions across very different studies, rather than interpreting each paper differently.
A number of papers are double screened to increase reliability. This also increased opportunities for discussion and collaboration during team meetings.
My Approach
Clarified the goals of the review and researched definitions of experiments and literature-review methodology.
Identify edge cases that exposed gaps or ambiguity in our existing criteria.
Look for patterns Compared difficult cases instead of treating each one as an isolated decision.
Refine requirements by updating inclusion, exclusion, and classification criteria.
Lessons
Good research depends on asking the right questions
Treat ambiguity as information.
If the same question keeps coming up, it may be a sign that the process or requirements need to change. Tracking edge cases helped us improve the framework instead of repeatedly solving the same problem. These habits helped me become more independent and made the work more efficient as I gained experience.
Ask questions, but also bring recommendation.
There were many cases where there was not an obvious right answer. Ilearned that it was more useful to identify the problem, explain my reasoning, and propose how I thought we should handle it. This opens opportunity for discussion and feedback from the team/supervisor.
Understand the "why" before jumping into the task.
At the beginning, I focused on learning the definitions and goals behind the literature review rather than immediately trying to screen papers as quickly as possible. Having that foundation made it easier to make decisions independently when I later encountered ambiguous cases.
