rewind_icon copy.png
 

REWIND: DIGITAL MEMORY REPLAY

Automatically Generated Location-based Videos for Reconstructing Personal Memories

 

Born out of the Brown University Human-Computer Interaction Lab, Rewind is a web application featuring a video compilation of up to 700 Google Maps street view images automatically generated from its location tracking mechanism.

Time of day and weather conditions are replicated using computer vision techniques, according to local precipitation and weather data. All locations the user visited on any chosen day are gathered and sorted, after which a route is mapped between the locations using the Google Maps API.

I have spent a lot of love, time, and care on this project, and I'm lucky to work with such an amazing team (Han Sha, Kelly Wang, Jeff Huang). I'm responsible for designing our formal user study, conducting our user research, and revamping our site's UX/UI design.

Our work has been submitted to CHI 2018 and is currently under review.

 
r-sea.png
 
Rewind videos are adjusted for time of day (above), and season (left). Along with temporal cues, Rewind incorporates local weather data. If a precipitation was high on the selected day, there is an animated filter overlaid onto the video, indicating that it was raining.

Rewind videos are adjusted for time of day (above), and season (left). Along with temporal cues, Rewind incorporates local weather data. If a precipitation was high on the selected day, there is an animated filter overlaid onto the video, indicating that it was raining.


designing our user Study


Our user study was led by the guiding questions: 

* How accurately did our computer vision techniques capture season, weather, and temporal cues?
* How helpful were Rewind videos in reconstructing of memories their past?
* How did Rewind compare to their baseline memory of that day?
* Would you use Rewind again?

In designing our user study, I used inspiration from methods in cognitive psychology research. After observing how participants used our program, I formed my user study questions based on standard memory tasks, and adjusted these accordingly for our Rewind system.

 

Timeline of our formal study, lasting three weeks (recording participants' data for two weeks, and an additional week to act as a memory buffer]

Timeline of our formal study, lasting three weeks (recording participants' data for two weeks, and an additional week to act as a memory buffer]

The formal study 


We conducted 22 interviews over the course of one and a half months, asking participants to record their location tracking data and compare the accuracy of Rewind videos to personal photos taken on the same day.

We advertised our study throughout the local community, and online local chat forums. Each interview lasted 45 minutes to an hour long and was audio recorded with the participants' permission. 
 

Our study methods are as follows: 

* Participants enabled geolocation tracking on their cellular device.

* Their location is then recorded for two weeks (Recording Phase), followed by a one-week buffer [Intervention Period] to avoid any recency effects during the interview process.

* Participants were interviewed [Recall Session], reviewing a Rewind from a randomly selected weekday and comparing their memory with any pictures taken on the same day. Participants were judged on accuracy, asked to report their emotions during each event they recalled, determining if the overall event was memorable.  


web application ui/ux

I am currently in the process of redesigning our application's interface. The working application is seen below, in which a user can select a date using a calendar and review all Rewind videos taken on that day.

Over the past summer user experience aspects were reimagined: the time slider, in which users can scroll between Rewind videos, and the look around feature, which incorporates a 360 degree view of the selected frame.

r1-min (1).png