Simon LutterSenior UX Designer
← All work

Voice assistants that work where you work

Bachelor thesis · HfG Schwäbisch Gmünd · 2018/19

A concept for multimodal voice assistants that work inside your computer, not your living room. Its three stages of system access describe what MCP-connected AI agents do today.

Thesis exhibition stand with printed booklet
Role
Research, concept and interaction design (team of three).
Team
Anita Kerbel, Esther Bullig, Simon Lutter · Prof. David Oswald, Prof. Dr. Ulrich Barnhoefer
Output
Parameters, a 3-stage access model, scenarios, exhibition
Context
Bachelor thesis, 7th semester, winter 2018/19

Challenge

In 2018, Alexa, Google Assistant and Siri couldn't solve complex tasks; they stayed toys in the living room. We asked: how must a voice assistant work to be useful in everyday computer work?

A good assistant needs integration, intelligence and context, not just a microphone.

The model

We defined four parameters: deep integration into the system, real AI (NLP and personalisation), one assistant across all devices, and context knowledge. Then we described three stages of access:

  1. Stage I · External accessThe assistant can open, send and manage files and applications.
  2. Stage II · Internal accessIt understands the content of files and apps, not just that they exist.
  3. Stage III · Full accessIt processes and interprets content and structure on its own and actively supports the user.
Parameters: integration and AI
Parameters: integration and AI
Parameters: one assistant and context
Parameters: one assistant and context
The three stages of a voice assistant
The three stages of a voice assistant

Why it matters now

Six years later, the Model Context Protocol gives AI agents exactly this kind of access to files, tools and content. My daily workflow uses stage II–III: with MCP servers configured, Claude reads and writes in Jira, Confluence and GitHub, and I approve what gets published.

See how the thesis became my working practice. Thesis → practice →