Pebbl
Wearable Opportunistic Crowdsensing for Open-Vocabulary Activity Data
Collection Through User-Scheduled Trigger–Action Routines
Pebbl originated from my desire to address the complexity of data annotation
in machine learning while simultaneously tackling existing issues
in habit-forming apps, achieving two goals with one solution.
Full video on
YouTube
Timeframe
‍Jun-Sep2025
Role
‍Developer
Experimenter
Designer
Author rank
‍Co-Author
Academic recognition
ACM CHI '26 Revise
(RR, RR, RR, RR)
*This paper is submitted to the CHI Conference on Human Factors in Computing Systems (CHI ’26). It is currently under review and temporarily open for application purpose. People who see this page are kindly requested not to spread it.
 Motivation‍
In the last century, Google cleverly harnessed humanity's cognitive surplus through reCAPTCHA. Hundreds of millions of users, in their effort to prove "I'm not a robot," spent seconds voluntarily labeling vast amounts of street numbers and ancient texts for Google. This formed a perfect symbiotic relationship: users received security services, while AI obtained high-quality data.
Today, as large models sweep through the fields of vision and NLP, wearable AI and embodied intelligence are facing a severe "data famine."
· If you have users act according to a script in a lab setting, the data will not be authentic (lacking ecological validity).
· If you allow users to record freely in their daily lives, no one can accurately recall what they did at specific times afterward, and the data will be full of noise.
 Introduction‍
Is there a human activity that requires users to be extremely focused in the present moment while also being highly repetitive, like entering a CAPTCHA? Pebbl is an opportunistic crowdsensing system that wraps up data collection as a tiny habit tracking app.
‍
This figure demonstrates the process of using Pebbl’s habit-tracking function and how data collection + annotation is done during the interactions.
(1) Initiation (Create a recipe like doing three push-ups after flushing the toilet, and the AI agent will assist in completing it.)
(2) Onboarding (Users can choose to try it once after creation.)
(3) Execution (Once created, each time the watch detects a trigger, it will remind you to perform the target behavior and record the data.）
 Theoretical Basis (Fogg Behavior)‍
Existing habit-tracking and cultivation approaches, particularly those inspired by Fogg's Behavior Model (Presented by Prof. Fogg in Stanford), emphasize that target behaviors (B) occur with sufficient motivation (M), ability (A), and a timely trigger (T). Fogg's Tiny Habits further refine this by advocating for small, anchor-based actions linked to existing routines.
‍
Pebbl transforms data collection into a habit-forming activity by linking it to users’ everyday routines. It lets users set personalized “if-this-then-that” rules where natural cues (like sounds or time of day) trigger simple motion-based actions. By embedding data annotation into meaningful personal habits, Pebbl enhances motivation and sustained participation, effectively collecting high-quality, context-rich activity data.
 Tech Stack‍
Pebbl was developed as a native mobile application primarily for the Apple ecosystem, specifically targeting iOS (iPhone) and watchOS (Apple Watch). The implementation prioritized on-device processing, privacy preservation, and battery efficiency to ensure real-time responsiveness for capturing in-the-wild behaviors. System architecture and deployment workflow are as follows.
Data Storage Flow
Pebbl stores recipes and execution metadata in a local structured database, while raw audio and motion sensor data are stored as files, all linked to provide a comprehensive, time-bounded, and user-annotated record of activities. The data storage flow is as shown in the diagram on the right.
 User Study‍
Study1: Evaluation of Pebbl
‍
This is a within-subjects lab study with 21 participants (10 male, 11 female, aged 21-31). It aimed to compare Pebbl against standard habit-tracking apps and baseline annotation tools, assessing both habit-tracking outcomes and data labeling reliability. The experiment lasted approximately 120 minutes in a moderately noisy meeting room environment. The process is illustrated in the figure below.
Left Figure: As a habit-tracking app, users found Pebbl significantly preferable to baselines, citing its substantially lower mental and physical burden and higher ratings for willingness to use, recommendability, and effectiveness.
‍Right Figure: From an annotator's perspective, Pebbl was the most preferred data collection method, imposing the lowest physical and mental demand while yielding the highest labeling confidence at the lowest perceived production cost.
Annotation Quality Analysis is as follows. We show results of both description-level and segment-level evaluation metrics per participant.Orders were rearranged by annotation IoU, which is the primary evaluation metric. “Length Ratio” represents the average intervallength of the participant’s annotations over Pebbl’s recordings. We draw the vertical line of Length Ratio = 1 in the last columninstead of the average value. Electronic Image Stabilization (EIS)represents two camera types used in our experiment.
Study2: ExpertWorkshop
‍
To gauge the perception of Human Activity Recognition (HAR) experts—the potential end-users of the collected dataset—on Pebbl's viability, this study aimed to evaluate its strengths, weaknesses, and potential compared to existing data collection methods. Six experienced HAR experts from four research institutions were recruited and co-created with us.
‍
‍Procedure:
1. Experts firstly received an introduction and demo of Pebbl.
2. Brainstormed and categorized existing data collection methods.
3. Established a shared set of evaluation criteria.
4. Independently scored each method, including Pebbl, based on these criteria.
5. Concluded with a group discussion to analyze the scores and provide qualitative feedback on Pebbl's pros, cons, and future directions.
Experts identified Pebbl's core advantage as its minimal user burden, achieved by cleverly reframing data collection as a personally valuable habit-tracking app. This innovative approach promotes high scalability and ecological validity, enabling the collection of diverse, open-vocabulary activities from motivated users in real-world settings, all while strongly preserving their privacy. The results of each step are as follows:
 Supplement:
To contribute to the development of the open-source community, the project will still be
open-sourced on my GitHub after the paper submission is completed. Your feedback is welcome.
 Acknowledgments:
Sincerely thank the user study participants for their positive teamwork and the reviewers in CHI2026 for their valuable and supportive feedback.
