Collection No 5
Without context, sensing data tells an incomplete story.
As humans, we are driven to seek ever-deeper understandings of both the world around us and the world within us. A growing number of us do just that by tracking the hours we sleep, the calories we eat, the miles we run, and many other types of inputs, states, and measures of performance.
This metrics minded pursuit of self-knowledge has in recent years spawned a movement, known simply as the quantified self. The process of self-tracking begins by collecting quantitative data, which is then aggregated, processed, and distilled into meaningful insight. Until recently, self-tracking required keeping notebooks or spreadsheets filled with numbers and observations. To extract meaning from this static record required many potentially daunting additional steps, such as input and analysis. Comparing metrics with peers was possible, but usually difficult.
Wearable devices are changing all that. They have made self-quantification highly accessible and easily comparable, and are thus remaking measures of self-awareness. Typically, they pair with services that store, analyze, and even share the data for you—often via sophisticated visualizations.
Today’s wearables face other constraints, such as limited modes of sensing and the sometimes isolated nature of the services that process the data. Viewing the data without the context of other data, or choosing the wrong data to assess, can lead to invalid conclusions. Likewise, choosing too many data points can lead to overwhelming noise or muted results.
These shortcomings are a reminder that context humanizes the numbers and places them back into our lives in meaningful ways. For example, a fitness tracker can tell us that our physical activity is down from the previous month. But it cannot tell us that the inactivity is due to a sprained ankle.
Given that context, those declining numbers might tell a different story: that we are recovering steadily rather than slacking off. Even in that simple scenario, it is clear that a small bit of context can frame data in a much more insightful way.
With richer context, we can better understand the quality of these quantities, and thereby better understand our being. As this capacity advances, the emphasis shifts to more metaphysical ways of describing ourselves.
This is what we describe as the qualified self. Where the quantified self gives us raw numbers, the qualified self completes our understanding of those numbers. The second half completes the first half.
To give a more concrete example, we have taken the sleep data from one of our Fitbit wearable sensors. Using this data, we will show in charts how context, story, and additional data sets can turn a quantified aspect of our lives into a qualified story.
The first chart shows the Fitbit sleep data as it is exported and presented from their service. It is a classic example of a data visualization of the quantified self. The vertical white lines show time slept and the gray lines are the actual time spent in bed each night as measured by the Fitbit. The yellow horizontal line shows the average sleep time per night, roughly 7 hours and 7 minutes as calculated by the app. An important additional data point, not recorded by the Fitbit has been added: On October 1st, a second child was added to the family.
The total sleep time per night only deviates by a minute or two from the average when evaluated monthly. The weekly average is more volatile, though, deviating by up to 10 minutes. When examined daily, a pattern emerges showing that a spike to catch up almost always follows a night with less sleep.
We can clearly see the quantitative side of these sleep behaviors. But without any extra information, there isn’t much visible meaning. We have run into a wall using solely the quantitative measures. We have reached the point where greater context and story can help us push through into more meaningful, more qualitative results.
The first, and most obvious bit of context we can add is time of day. The same data as the previous chart is shown aligned with bedtimes and wake up times. The gray bars show the time in bed. The yellow line is a rolling average over the course of each week. With this context we can quickly see that there is regular bedtime between midnight and 12:30 a.m. followed by a regular wakeup time between 7:00 and 7:30 a.m.
Here the white bars representing time asleep are overlaid on the gray bars. We start to see three clear segments over the course of the seven-month study. The first segment, spanning the first two months, shows a pattern of earlier bedtimes and earlier wake up times, by an average of about 20 minutes per night. The second segment appears after the addition of another family member. The bedtimes then become more volatile with later mornings, as well as some later nights, and some very early mornings. The average bedtime is later by 27 minutes. Past and present parents of newborns will recognize this as a common change in sleep patterns. The third segment is the span from August to October. Here bedtimes get later but wake up times are not quite as late as post newborn.
We will now add additional data to further illuminate what is happening in the three main segments. Here we are using data from the Strava app, which is used to track cycling activities. We can see, in yellow, which day’s commute was done via bicycle. During the initial two-month period, bicycle commuting happened on 61% of the days. In that time, total sleep duration didn’t shrink or grow, but the sleep period moved: with earlier bedtimes and earlier wake times. Outside of those two months cycling was very sporadic.
So the first few months seem to be influenced by exercise and the last few were altered by a newborn. What then explains the downward trend in August and September?
To answer that question, we crosschecked viewing data from Netflix with our sleep patterns. The yellow lines now show days when there was Netflix activity. There is a downward trend—with later bedtimes and less total sleep—in the period preceding the finale of “Breaking Bad”. The culprit: a binge of back episode viewing before the finale.
In that period, bedtimes were on average of 29 minutes later than the two months prior. Each night, there was an average of 14 minutes less total sleep. Also, there is a trend of catching up on sleep during the second half of the binge, once the viewing activity slowed.
This case study shows the power of context to elevate the isolated meaning of quantified data into a more qualified context. This sleep data, at first, appeared to be mostly random and revealed relatively little. The additional data, context, and some humanization of the information aided in making sense and uncovering meaning.
To be sure, correlation doesn’t always mean causation, but there is plainly some linkage between these layers of data. Through this we were able to begin to qualify the data and find evidence of influences that can be changed and controlled.
Just as stories yield data, data yield stories. And just as it is difficult to quantify our lives without data, we cannot qualify them without context or narrative. When we bring the two sides together, we achieve deeper self-knowledge.
Eric is an avid quantifier of my life, data-viz enthusiast, and a music zealot, pursuing how they intersect. He frequently thinks about data of all sizes and the future.
Jarrett is a Principal Design Technologist at frog.