BETWEEN LINES:
CRITICALLY SPECULATING FUTURES WITH LOCATION-BASED RECOMMENDATION SYSTEMS
An overview of Between Lines, a critical speculative design project, completed as a portion of requirement for my Masters Degree in Interaction Design at the University of Washington
Between Lines is a critical, speculative design project that envisions uses of location data through four, near-future, design fiction films.
ROLE
INTERACTION DESIGNER
DESIGN RESEARCHER
FILMMAKER
GRAPHIC DESIGNER
Duration
30 WEEKS
Project type
DESIGN RESEARCH
DESIGN FILMMAKING
PUBLICATION DESIGN
WEB DESIGN
GOALS
Personal motivations and a desire to explore intangible systems around us drove the development of this project. This project aims to understand how we perceive and interpret these intangible systems. I have always been keenly interested in design ethics surrounding privacy, security, and surveillance. With this project, I combined my prior design research interest regarding intangible (vibration) data with a new angle on speculative futures. I wanted to grow as both a filmmaker and designer leveraging my filmmaking and interaction design skillset.
PROCESS
This was a one-year project (roughly 30 weeks). I spent the first 10 weeks defining and scoping it. Additionally, I developed the conceptual framework to create a wide composition of critiques, which would be the basis for the four films. I spent the next 10 weeks writing, developing, and making the four films in a rolling production cycle. At the end of week 20, I began a five-week-long qualitative research study. I then analyzed the data and compiled the findings in a bespoke publication and public installation. See the case study below for further details.
REFLECTION
This project was grueling and transformative, pushing my limits as a researcher, designer, and filmmaker. Working with my thesis committee, mentors, and collaborators profoundly reshaped my approach to design, particularly in ethical considerations and the role of the designer. 'Between Lines' instilled in me the power to be ambitious: to create four short films, conduct a qualitative study, design a publication, and develop a public installation in under 30 weeks. It allowed me to refine my perspective and use design to probe designers' ethical responsibilities.
Interested in learning more about the project? CONTACT ME.
CASE STUDY ↓
Onset filming Evidence of Insurability.
Post Mortem: A Method and an Approach; Design Filmmaking in Practice
This case study details the process of creating the four speculative design films and the films usage in qualitative research—as a tool to co-speculate with professionals in the field about the future of location-based recommendation systems.
Want to watch the FIlms first?
00_Project Definitions—
There are a few important terms to define to situate this work in the larger context of human-computer interaction (HCI) research and industry futuring practices:
Speculative Design is an artistic approach that emphasizes new ideas and solutions for the present that involve thinking through the future. This practice was termed by Anthony Dunne and Fiona Raby in their book Speculative Everything.
Critical Design artifacts challenge an audience's preconceptions, provoking new ways of thinking about the object, its use, and the surrounding culture. The artifact challenges the viewers' notions and assumptions.
Design Fiction: is a design practice that aims to explore and criticize possible futures by creating speculative and often provocative scenarios narrated through designed artifacts.
Co-Speculation: is a form of collaborative speculating that allows communities to co-create potential futures rather than being passive observers in top-down assessment processes. It draws on participants' lived experience and professional knowledge.
01_How do designers think about the future?
The future is inherently unstable. Many disciplines prioritize spending resources to best orient themselves to capitalize on the most likely outcome. Designers, on the other hand, are often tasked with realizing a specific vision of the future (their vision or, and quite often, their organization leaders' vision). Designers introduce objects, products, and services into the wild, often without thinking through the ethical ramifications. These artifacts often permanently change society (see the AIGA's writing on ontological design).
Small design teams often make these decisions. While many firms have adopted participatory approaches, methods for collaborative futuring remain few and far between. With this project, I simultaneously wanted to expand and iterate on existing methods (leveraging by background in film production) to question an increasingly relevant and pressing topic: the usage of location data in recommender systems.
Many critical scholars argue that location privacy is a critical component of autonomy from state-sponsored or commercial systems. Yet, these systems are often hidden away, along with the data they collect. Going into this thesis project, I ask myself, as designers, how can we provide tools, artifacts, and speculations to question always-on location surveillance practices through critical design approaches? How can designers engage people in the near future using compelling design forms like concept or design fiction videos?
The cone of possible futures refined by Anthony Dunne and Fiona Raby for situating speculative designs along axis of distance and preference.
"Although most would agree that design is everywhere and serves particular purposes, we often fail to consider how profoundly design structures the possibilities for existence in the world."
Ontological Designing, Anne-Marie Willis
02_How We Think About Location Data
Location-based recommendation systems are recommender systems that incorporate location information, such as that from a mobile device, into algorithms to provide users with more relevant recommendations.
These are increasingly complex recommendation systems behind the stable interfaces of the Google and Apple Maps, Uber and Lyft apps. These systems are are dynamic, often morphing systems, taking into account current and past locations, browsing, and purchasing activity across the web to inform increasingly contextual recommendations — and navigating our bodies through space to get there.
These systems run on location data. If you read the headlines, you know this data is being used for some controversial, if not downright dystopian, stuff:
ICE to pursue privacy approvals related to controversial location data
Automakers Are Sharing Consumers’ Driving Behavior With Insurance Companies
FCC fines AT&T, Sprint, T-Mobile, and Verizon nearly $200 million for sharing access to users’ location data.
Many critical scholars argue that location privacy is a critical component of autonomy from state-sponsored and commercial surveillance systems. Yet, these systems are often hidden away, along with the data they collect. Going into this thesis project, I asked myself:
Many critical scholars argue that location privacy is a critical component of autonomy from state-sponsored or commercial systems. Yet, these systems are often hidden away, along with the data they collect. Going into this thesis project, I ask myself, as designers, how can we provide tools, artifacts, and speculations to question always-on location surveillance practices through critical design approaches? How can designers engage people in the near future using compelling design forms like concept or design fiction videos?
A still from Routine Repetitions.
How can we use design to anticipate concerning and contentious future scenarios of location-based recommendations?
02_Creating a composition of critiques
To explore how the act of design might be used to anticipate futures. I developed the conceptual framework across two phases. Instead of committing to an extensive literature review of the specific topic, I chose to draw research-based inspiration for the scenarios. Due to the films intended use (as probes for elicitations), this literature survey enabled me to discern specific trends and technology that would be readable by professionals in the field. The first phase consisted of a literature search, drawing inspiration from grounded research and timely news media articles. As part of this effort, I conducted a comprehensive search cataloging local and emerging national stories in the news media, capturing just-breaking news stories that existing HCI papers had yet to catch up to. Additionally, I surveyed computer science journals and publications on recommendation systems, audited existing forms of recommendation in apps, and surveyed the ACM Digital Library for publications in the domains of privacy, surveillance, location, and recommendation systems. With this portion of the survey complete, I created a list of areas where these emerging technologies could have future impacts (in both positive and negative ways).
The second phase of the framework development then drew on existing theory and work regarding critiques of Surveillance Capitalism, Critical Race Theory, and historical precedents regarding the technology industry and gentrification. Based on these existing works, I created a catalog of critiques and technologies that embodied them, considering areas of impact and existing trends. This comprehensive survey of location-based recommendation systems and emerging technologies facilitated the development of my future scenarios. Due to the framework's application, the scenarios present realistic, near-future speculations of products, services, and technologies rooted in theoretical critiques.
Many critical scholars argue that location privacy is a critical component of autonomy from state-sponsored or commercial systems. Yet, these systems are often hidden away, along with the data they collect. Going into this thesis project, I ask myself, as designers, how can we provide tools, artifacts, and speculations to question always-on location surveillance practices through critical design approaches? How can designers engage people in the near future using compelling design forms like concept or design fiction videos?
A simplified project diagram. Reduced to its core components, this project is about envisioning likely futures at high fidelity levels so we can then understand ramifications of design decisions before they happen.
03_Scenario Development
With the conceptual framework developed, I then began using the framework to develop possible scenarios. I developed twenty initial scenarios covering a broad range of futures and technologies. The scenarios included a logline (one-to-two sentence description of the scenario, setting, and characters), which critiques, and which trends they were responding to. With each scenario, I included a three-frame storyboard, using the digital service ShotDeck to pull stills from films, visually and thematically linked to the logline, creating a combined mood and storyboards.
These scenarios were then critiqued by other designers, fellow researchers, and one subject matter expert. This feedback was pivotal in the process of downselecting the final four scenarios that would later be adapted into short films. When downselecting, I balanced the subject matter of the video, the way in which location-based recommenders manifest, and the critique being leveraged. Over a period of rapid iteration, I took each scenario and diagramed out possible alternatives, encompassing a range of positive and negative implications. The final four scenarios consisted of 1) Avoidable Inconveniences, 2) December 26th, 3) Evidence of Insurability, and 4) Routine Repetitions. With these scenarios locked, I moved forward with pre-production.
Many critical scholars argue that location privacy is a critical component of autonomy from state-sponsored or commercial systems. Yet, these systems are often hidden away, along with the data they collect. Going into this thesis project, I ask myself, as designers, how can we provide tools, artifacts, and speculations to question always-on location surveillance practices through critical design approaches? How can designers engage people in the near future using compelling design forms like concept or design fiction videos?
The original 20 scenarios. Each scenario contains three images conveying plot and tone through careful image selection.
04_Pre-Production-Production-Post-Production
Once the scenarios were locked in, I moved on to the pre-production phase. Due to the project's condensed timeline and the number of films I aimed to create, I began a rolling production schedule. For each of the four films, I would write a script on Monday, create a storyboard on Tuesday, and send it to my talent on Wednesday. I would assemble a skeleton crew, including a gaffer, onset sound, and myself (camera operating and directing). Each week (often Thursday and Friday), I would assemble a rough cut of what had been shot the week before, doing a rapid color grade and making other minor adjustments. Occasionally, if I were not able to find help, I would record motion without sound (MOS) adding all audio in post-production. While I sacrificed greatly on lighting quality, camera stabilization, and a layer of refinement that might otherwise have been there, this approach allowed me to be incredibly flexible (shooting alternate versions, coming up with shots on the fly, etc.) in a more documentary style.
Once the films were edited (often a fast one to two-day process), I began to approach screen replacements. Artistically, I was opposed to text overlays on screen. While they provide greater flexibility, I prefer the naturalism and realism of showing UI on-device. This was debated heavily throughout due to the time on-screen text and UI graphics would save. However, I decided that the extra effort would pay off, not breaking the illusion of an alternate reality. To this end, each screen was replaced using Davinci Resolve’s built-in Fusion compositing tools. While I was initially able to replace all of the phone screens, due to some of them looking poorly (often due to difficult tracks and complex rotoscoping work), I pivoted to taking the interface I’d created for the screen replacements and using them in a series of reshoots. I reshot nearly every scene from Avoidable Inconveniences and Evidence of Insurability. In the end, 50% of the interfaces were replaced in post production as originally intended, and 50% were reshot with stand-in actors.
Many critical scholars argue that location privacy is a critical component of autonomy from state-sponsored or commercial systems. Yet, these systems are often hidden away, along with the data they collect. Going into this thesis project, I ask myself, as designers, how can we provide tools, artifacts, and speculations to question always-on location surveillance practices through critical design approaches? How can designers engage people in the near future using compelling design forms like concept or design fiction videos?
Behind the scenes filming Avoidable Inconveniences.
What can we learnby showing these scenarios to various domain experts?
05_ Pre-Pilot & Pilot
After completing the initial versions of the films, I conducted a preliminary screening with two colleagues who specialize in data and privacy. This was less a traditional pilot and more akin to a test screening aimed at evaluating the clarity of the narrative, the critique being applied, and the depiction of algorithms within each film. The feedback was critical: the narratives lacked clarity, leading to misinterpretations of the plots; the role of recommendation systems was obscured; and the interfaces failed to effectively convey their intended messages.
In response, I revised each film, focusing on strengthening the narratives through strategic edits—including shortening the duration—to enhance narrative clarity. I also reshot and updated the user interfaces, ensuring each was of higher fidelity and more effectively communicated within the story's context.
Upon completing these revisions, I conducted another pilot with a subject matter expert in location data, privacy, and security. The revised films elicited fewer questions about the narratives and the roles of recommendation systems in the story. Instead, they sparked visceral reactions, indicating a high level of engagement with the themes presented. This feedback was encouraging as I proceeded to engage with my main study participants.
Many critical scholars argue that location privacy is a critical component of autonomy from state-sponsored or commercial systems. Yet, these systems are often hidden away, along with the data they collect. Going into this thesis project, I ask myself, as designers, how can we provide tools, artifacts, and speculations to question always-on location surveillance practices through critical design approaches? How can designers engage people in the near future using compelling design forms like concept or design fiction videos?
Behind the scenes filming December 26th, 4:36pm.
06_Participants + Recruiting + Study Design
This project poses a critical inquiry about our roles within the field of HCI and technology, questioning how we might drive change and envision alternative futures. Reflecting this, a diverse group was recruited for participation. I conducted five 90-minute ethnographic interviews with professionals from the tech industry who are either directly involved with or adjacent to projects concerning location data or recommendation systems. Additionally, three interviews were carried out with HCI researchers specializing in recommendation and surveillance systems. Employing a grounded theory approach, I further expanded the participant pool to include one professional artist with a focus on surveillance themes, one community activist engaged in philanthropy and nonprofits, and one assistant city attorney dealing with public data, culminating in a total of eleven interviews.
The primary aim was to gauge the reactions of tech industry professionals to the films, especially considering their implications on individual agency. However, to deepen the discourse on key issues like policy, legislation, and public awareness, I intentionally recruited a broader array of voices. These participants were likely to provide additional insights into the research questions and helped evaluate how well the films communicated complex ideas to non-experts.
Each participant engaged in a detailed viewing and speculation process. During our meetings, we watched the four films together. Following each video, I conducted semi-structured interviews to discuss their thoughts and gather feedback, repeating this process for each video.
Many critical scholars argue that location privacy is a critical component of autonomy from state-sponsored or commercial systems. Yet, these systems are often hidden away, along with the data they collect. Going into this thesis project, I ask myself, as designers, how can we provide tools, artifacts, and speculations to question always-on location surveillance practices through critical design approaches? How can designers engage people in the near future using compelling design forms like concept or design fiction videos?
A spread from the final publication highlighting a selection of participant quotes grouped thematically.
07_Analysis
Once the interviews had been completed, I began analyzing the data. Approaching this process from a constructivist perspective, I began the process of open coding. I worked through the eleven interviews tagging and developing tag groups. Once this process had been completed, I began to draw connections between the codes I had developed—placing them into larger thematic groups. From this point I began grouping similarly tagged quotes and developing formal, written observations, which were then translated into findings.
To see the findings and final publication in their full form, read
Between Lines: On Location and Recommendation on Issu ↗
Many critical scholars argue that location privacy is a critical component of autonomy from state-sponsored or commercial systems. Yet, these systems are often hidden away, along with the data they collect. Going into this thesis project, I ask myself, as designers, how can we provide tools, artifacts, and speculations to question always-on location surveillance practices through critical design approaches? How can designers engage people in the near future using compelling design forms like concept or design fiction videos?
Findings and insights developed from thematic analysis. Read the thesis publication for more developed analysis of the most impactful findings.
08_Dissemination
With the project largely completed at this point, I was interested in how this work might live on. I wondered how the findings and insights developed during my research might find find relevance for designers and other researchers working in industry or academia. These were two of the driving questions behind the project site and the thesis publication.
In a short 1 week sprint, I developed the Between Lines project site on Webflow. The design and interactions went beyond what was possible with vanilla Webflow, so I turned to ChatGPT 3.5 and 4 to help generate custom code. While not perfect (there are still a few annoying bugs remaining), I am happy with what I was able to accomplish with only a basic understanding of HTML and CSS.
The thesis publication, which is covered on the Between Lines: On Location & Recommendation project page grew out of a similar mindset. I aimed to create a standalone publication that could exist without the short films. Additionally, I wanted to challenge myself—I had no formal visual communication training, nor publication design experience prior to my time at UW. It'd always been illusive. I taught myself the fundamentals of publication design to create a bespoke 212 page coptic bound book. The publication and fabrication process was completed in 2 weeks.
Additionally, I have continued to work, formalizing my findings and documentation for publication at CHI 2025 and DIS 2025 conferences. Even if rejected, 10 peer reviewers will have throughly engaged with the work—a new form of guerrilla dissemination.
These ideal were carried into the design of the installation at the Jacob Lawrence Gallery, in which the publication and videos were presented to a wide ranging public audience comprising industry professionals and everyday people.
Many critical scholars argue that location privacy is a critical component of autonomy from state-sponsored or commercial systems. Yet, these systems are often hidden away, along with the data they collect. Going into this thesis project, I ask myself, as designers, how can we provide tools, artifacts, and speculations to question always-on location surveillance practices through critical design approaches? How can designers engage people in the near future using compelling design forms like concept or design fiction videos?
The project microsite was developed in 1 week with Webflow and Chat GPT for custom code. Visit the site at critical-futures.com
The final thesis publication. Designed and fabricated within 2 weeks.
Between Lines installed at the Jacob Lawrence Gallery.
Interaction Design
Wyatt Olson
Visual Design
Wyatt Olson
Burke Smithers
Filmmaking
Wyatt Olson
Laura Le
Rachael Winkler
Anne Winkler
Mary Olson
Nelly Quezada
Teresa McDade
Kenneth Nguyen
AK McDade
Mark Olson
Publication Design
Wyatt Olson
Installation Design
Wyatt Olson
Fabrication
Wyatt Olson
Laura Le
Mark Olson