Overview

AI-powered personal assistants such as Siri, Alexa, and Google Assistant are becoming increasingly embedded in our daily lives. While they offer convenience, their effectiveness varies based on user needs and expectations. This study explores real-world interactions with AI assistants through a diary study and follow-up interviews to uncover user experiences, frustrations, and suggestions for improvement.

Research Objective

Question: "How do users engage with AI-powered personal assistants, and how do these interactions shape their experiences?"

To answer this, participants documented their AI assistant usage over three days, responding to these prompts:

• Which AI assistant(s) did you use today?

• What tasks did you perform with the assistant?

• How did the AI assistant’s performance align with your expectations?

• Would you rely on this AI assistant for the same task in the future?

Following the diary study, we conducted semi-structured interviews to delve deeper into users’ experiences and concerns

Methods

Participant Selection & Study Design

We recruited individuals who frequently use AI assistants to participate in a three-day diary study. Each day, they submitted responses via a digital form to capture real-time experiences. The diary entries helped document recurring themes and variances in usage.

Following the diary study, we conducted one-on-one interviews with participants to:

• Expand on diary responses

• Identify common pain points and areas of satisfaction

• Explore user concerns related to privacy and reliability

This mixed-method approach allowed for a holistic understanding of how AI assistants fit into daily routines.

Key findings and insights 

1. AI Assistants Are Convenient but Often Fall Short

Most participants found AI assistants helpful for simple, routine tasks like setting reminders, playing music, and retrieving basic information. However, they frequently encountered issues with misinterpretations and inconsistent performance.

2. Privacy Concerns and Data Security Hesitations

A major theme that emerged was uncertainty about data privacy. Many participants expressed concerns over what AI assistants track and store, influencing how much personal information they were willing to share.

3. AI Assistants Are Primarily Used for Low-Stakes Tasks

Participants overwhelmingly used AI assistants for routine, non-critical tasks rather than complex decision-making or high-stakes needs. Their reluctance to rely on AI for anything urgent stemmed from inconsistent reliability.

4. User Recommendations for Improvement

Participants provided several suggestions for making AI assistants more reliable and effective:

• Better Context Awareness: AI should remember previous interactions and respond accordingly rather than treating each request in isolation.

• More Accurate Responses: Reducing vague or repetitive answers would improve trust and usability.

• Enhanced Privacy Transparency: Users want clearer explanations of how their data is stored and used.

• Improved Integration with Other Tools: AI assistants should work more seamlessly with third-party apps and smart home devices.

Challenges and Areas for improvement 

Future Considerations

To improve research depth and credibility, future studies should:

• Implement structured reminders to encourage real-time diary logging.

• Expand the participant pool to capture diverse usage patterns.

• Extend the study duration to analyze evolving AI interactions over time.

Acknowledgments: This project was a collaborative effort as part of my INFO 690 UX Research Methods class at Drexel University. I had the privilege of working with an amazing team: Marian Gasinu, Katherine Cassandra Stolaki, Katherine Wilhelm. Their insights, skills, and dedication were instrumental in making this research successful.