Donating my body, er… being!… to (citizen) science.
Over the past few years I’ve been sharing my self-tracking/Quantified Self and biohacking adventures here at QuantifiedBob.com. And over this time I have accumulated a lot of personal data** — biometric, health/wellness/fitness, medical, behavioral, financial, activity/location, meals, books I’ve read, and even driving.
Born out of my own needs to aggregate and share my data, today I am happy to announce BobAPI, my personal API.
Why build a personal API?
- My data was fragmented and siloed. My data was split across a number of different proprietary platforms (apps, devices, 3rd-party APIs), manually recorded data points (spreadsheets, notebooks, Post-Its, voice memos, etc.), and pdfs (medical records, lab test results).
- I wanted a safe, permanent backup of all of my personal data. I wasn’t keen to lock myself into a proprietary solution where the underlying data is stored or accessible in ways that aren’t fully usable to me, not to mention risk those services deciding to change or restrict access, or even go out of business (RIP Basis?).
- I wanted the ability to look back at my data at any point in time. “You can’t manage what you don’t measure.”
- I wanted to contribute to citizen science. I frequently get requests from researchers and other self-quantifiers asking for some of my data, so I wanted an easy way to share and provide access.
Requirements for a personal API
My requirements were relatively straightforward:
- Non-proprietary. Should not be tied into anyone’s “data vault.”
- Integration. Make it easy to connect other devices, services, apps, and providers, and allow for manually collected/self-reported data. My data consists of a combination of passively collected data (through apps, devices, 3rd-party APIs) along with data I have been recording manually (spreadsheets, notebooks, etc.)
- Data normalization. Standardize data fields and timestamps regardless of data source/provider to best suit my needs. The biggest challenge with aggregating so much data is coming up with the best way to normalize it. And don’t get me started on timestamps and time zones!
- Data portability. In addition to running locally on a laptop, should be able to deploy easily to any server.
- Security. Allow/restrict 3rd-party access to different data sets (i.e., genetic data or sexual activity vs. heart rate). And of course, require SSL.
Although I had been kicking around the idea of building a personal API for some time, it wasn’t until earlier this year that I started cobbling together an early version of what exists today. While the code has undergone several rounds of refactoring (and several starts and stops in between), there are admittedly still a few pieces being held together with rubber bands and duct tape. But hey, it seems to be working!
I’ve integrated and normalized data from a number of sources such as wearables and connected devices from Basis and Withings, apps like Moves and HRV4Training, manually entered data from non-connected tools like glucose meters, pH test strips, and pulse oximeters, as well as from my own spreadsheets and tools.
Initially, I am sharing some of my wellness, biometric, and activity data, and will continue to bring additional data sets online in the coming weeks.
So, what can a personal API be used for?
The beauty of having a personal data warehouse is that it’s simply…data. But aside from sharing my raw (but normalized) data with others, I now have an easy way to integrate with other best-in-class tools and projects:
- Data Visualizations: Make your data look sexy — Gyroscope, Reporter, roll-your-own graph/chart widgets using tools like D3.
- Data Analysis: Find correlations in your data — Zenobase, Statwing (now Qualtrics), R.
- Contribute to Citizen Science: Share your data with researchers and for public good — Open Humans, university researchers, Personal Genome Project. And transposing data to be Open mHealth-compatible is trivial.
- Build New Apps and Services: Having a solid data backend will allow me to build new apps and tools on top of it such as voice- and bot-based interfaces, or create IFTTT recipes to do things like, say, determine and maintain the optimal temperatures and airflow in my bedroom throughout the night for best sleep quality.
Additionally, I can implement social experiments to explore new ways to leverage my data:
- Data Monetization?: Fitness companies are already selling collected data to third parties with no compensation to users. However, employers and insurance companies are starting to reward people for sharing their data. And someone even sold their “digital soul” to the highest bidder (for $480 USD :). What if in the future we were able to “opt in” and selectively share our data in exchange for monetary compensation? Is this the evolution of the Nielsen family?
- Blockchain: I can even publish my data onto a blockchain database.
- Ephemeral Data: Wouldn’t it be cool if services could only gain temporary/ephemeral access to your data and not store it permanently on their servers?
Under the Hood
The code is mostly Node.js/Express and MongoDB, along with a handful of Python scripts to handle data fetching and import. There is also a separate front-end web app that allows me to enter data manually. Everything is wrapped in Docker, for easy portability and deployment, and is currently hosted for free using Carina by Rackspace (thanks, Rob, for the Docker help!).
Thanks, and let’s connect!
So, go ahead and feel free to play around with “v1” of my personal API. I’ve also placed data samples on GitHub. While looking at my data will probably bore you to death, I’ve love to get your feedback and ideas for how to improve BobAPI. And hopefully it will inspire others to work on similar projects.
** Although I am an avid self-quantifier, I make no claims about being the “most quantified” person in the world (nor do I desire to be), nor would I ever claim to be the first person to share my data publicly, as folks like Beau, Buster, and Naveen have been doing so for some time.