Carepatron synthetic onboarding test
Current onboarding vs three setup-checklist prototypes
Bias controls used: anonymous randomized target labels in-agent, screenshot-only interaction, same rubric and goals for every target, no prototype hypotheses in prompts, and prototype-aware scoring that excludes backend completeness while still judging visible goal support.
Reading the ranking: two scores per target. Flow quality (clarity/trust/motivation/ease) measures the visible UX independent of persona scope. Goal fit (value/concept-fit/likelihood-to-subscribe) measures how well the prototype serves THIS persona's specific job. A prototype with high flow quality but low goal fit is well-designed for one job and poorly-aligned for another — that's information, not a bug. In-scope-only columns recompute both scores using only the reviewers for whom this prototype's scope was relevant.
Ranking Table
| Rank | Target | Runs | Flow quality (all) | Goal fit (all) | Flow quality (in-scope) | Goal fit (in-scope) | In-scope runs | Subscribe | Goal completed | Attempt allowed | Median friction |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | v2 Personalized: profession-first wizard | 5 | 5.7 | 5.9 | 6.3 | 7.0 | 4 | 4.4 | 60.0% | 80.0% | 3.0 |
| 2 | v1 Refined: closest to current | 8 | 5.1 | 5.1 | 5.4 | 5.5 | 7 | 3.5 | 25.0% | 87.5% | 3.0 |
| 3 | v3 Focused: first booking | 9 | 4.6 | 3.6 | 6.6 | 6.1 | 4 | 2.8 | 33.3% | 44.4% | 3.0 |
| 4 | Control: current production onboarding | 0 | 0.0 | 0.0 | 0.0 | 0.0 | 0 | 0.0 | 0.0% | 0.0% | 0.0 |
Friction Heatmap
| Step / friction zone | Control: current production onboarding | v1 Refined: closest to current | v2 Personalized: profession-first wizard | v3 Focused: first booking |
|---|---|---|---|---|
| All turns: No import or migration entry point | 5 | |||
| Below completion panel — Explore top features | 2 | |||
| Completion screen — CTA hierarchy | 3 | |||
| Completion screen — page header | 4 | |||
| Completion screen: no visible booking link URL | 2 | |||
| Early onboarding steps (pre-T11) | 2 | |||
| Entire onboarding flow | 4 | |||
| Entire onboarding flow (T1–T24) | 2 | |||
| Location setup — Step 3 of 4 (T20) | 3 | |||
| Onboarding Step 3 - Where you see clients | 3 | |||
| Onboarding Step 4 - Try a test booking | 5 | |||
| Onboarding completion — 100% milestone visual tr | 2 | |||
| Onboarding completion — Explore top features sec | 3 | |||
| Onboarding completion — Open calendar CTA | 4 | |||
| Onboarding completion — tooltip overlay | 2 | |||
| Open calendar CTA — 100% completion screen (T18, | 4 | |||
| Page header subtitle at completion state | 2 | |||
| Post-onboarding (never reached) | 4 | |||
| Post-setup sidebar (T7+) | 4 | |||
| Practice type selection (T4–T15) | 5 | |||
| Set your hours (step 2 of 4) | 5 | |||
| Settings nav item (T7–T25) | 5 | |||
| Setup complete screen (T7–T25) | 5 | |||
| Share booking link CTA — 100% completion screen | 5 | |||
| Sidebar navigation — all items dimmed | 3 | |||
| Sidebar navigation — throughout onboarding | 2 | |||
| T1 - Onboarding welcome screen | 5 | |||
| T1 — Get Started onboarding (landing state) | 10 | |||
| T1 — Onboarding welcome screen | 5 | |||
| T1 — Welcome screen | 4 | |||
| T1 — onboarding wizard | 4 | |||
| T1-T20 — All turns on onboarding dashboard | 5 | |||
| T1-T20 — Entire prototype scope | 5 | |||
| T1-T20 — Sidebar navigation | 4 | |||
| T10: Secondary feature grid CTA labels | 2 | |||
| T10–T25 — Booking link live confirmation (repeat | 5 | |||
| T11 onward - Clients section locked | 5 | |||
| T11-T24: 'Open my calendar' CTA on booking link | 5 | |||
| T11–T23 — Booking ready completion screen | 4 | |||
| T15 - Onboarding complete | 5 | |||
| T1: Get Started onboarding dashboard | 4 | |||
| T1: Initial onboarding screen | 3 | |||
| T1: Left nav sidebar | 4 | |||
| T1–T19: Onboarding checklist sequencing | 3 | |||
| T1–T25: Get Started onboarding dashboard | 5 | |||
| T1–T5: Onboarding wizard (Step 1 of 4) | 5 | |||
| T2 - Pick a service | 4 | |||
| T2 — Services setup | 2 | |||
| T20–T25: Tapping step 3 directly | 3 | |||
| T22–T23 — Headline typography | 1 | |||
| T25: Session end | 5 | |||
| T3 - Onboarding Step 1 service types | 5 | |||
| T3: Get Started onboarding dashboard | 4 | |||
| T3–T5: Feature teaser at bottom of wizard | 4 | |||
| T4 onwards: Onboarding headline | 4 | |||
| T4 — Set a location | 4 | |||
| T4-T24: Onboarding dashboard / Settings nav item | 5 | |||
| T4-T5 — Location model | 3 | |||
| T4–T19: Onboarding step 1 'Add what you offer' | 5 | |||
| T4–T22: Onboarding / Get Started screen | 5 | |||
| T5-T8 - Set a location | 4 | |||
| T6 - Step 4 test booking skip | 3 | |||
| T7 — Explore top features cards | 3 | |||
| T7-T21 - Completion screen loop | 5 | |||
| T7–T24 — Explore top features / post-completion | 5 | |||
| T8 — Celebration illustration container | 2 | |||
| T8–T23: Open calendar CTA post-onboarding | 5 | |||
| T9-T10 - Test your booking link | 3 | |||
| T9: Bottom banner | 4 | |||
| T9–T24: Open my calendar CTA loop | 4 |
Persona x Prototype Matrix
| Persona | Control: current production onboarding | v1 Refined: closest to current | v2 Personalized: profession-first wizard | v3 Focused: first booking |
|---|---|---|---|---|
| Dr. Maya Chen, LCSW | Not run | Blocked | Not run | Not run |
| Sam Rivera, LMHC | Not run | Not run | Completed | Not run |
| Priya Shah | Not run | Not run | Not run | Not run |
| Dr. Tom Walsh | Not run | Attempted | Not run | Not run |
| Lara Okeke, PT | Not run | Attempted | Completed | Completed |
| Anna Bauer, OT | Not run | Not run | Not run | Not run |
| Chris Park | Not run | Not run | Not run | Blocked |
| David Knight, DC | Not run | Not run | Not run | Not run |
| Nancy Hanisch | Not run | Not run | Not run | Blocked |
| Jordan Lee | Not run | Not run | Not run | Not run |
| Dr. Elena Vargas | Not run | Not run | Not run | Attempted |
| Mary McKinnon | Not run | Attempted | Attempted | Not run |
| Becky Tran | Not run | Attempted | Not run | Blocked |
| Marc Robinson | Not run | Not run | Not run | Blocked |
| Liz Chen, SLP | Not run | Not run | Blocked | Blocked |
Quote Wall
Marc Robinson · v3 Focused: first bookingI spent 22 turns trying to click into Settings and never got there — if I can't even find your compliance controls, I can't put 20 clinicians' data on this platform.
Nancy Hanisch · v3 Focused: first bookingI spent twenty turns looking for a GDPR notice or a German-language option and found nothing — just Australian dollars and a booking setup checklist, so I cannot trust this platform with my patients' data.
Liz Chen, SLP · v3 Focused: first bookingI clicked Continue setup over and over and never left the availability screen — nothing here tells me this is built for speech therapy or that a parent can even log in.
Liz Chen, SLP · v2 Personalized: profession-first wizardI selected Allied health because it literally said speech on the card, went through four setup steps, and ended up at a generic booking link with no templates, no parent portal, and a Clients section I couldn't open — this told me nothing about whether it can actually support my pediatric practice.
Dr. Maya Chen, LCSW · v1 Refined: closest to currentI came here to check if my clients and insurance billing could transfer, but I spent five turns stuck in a booking wizard and never got a single answer to either question.
Becky Tran · v1 Refined: closest to currentI clicked Billing over and over for 25 turns and just kept getting blocked — I have absolutely nothing to bring back to the owner about pricing or where to go for training.
Becky Tran · v3 Focused: first bookingI spent the whole session trying to check billing and it was locked behind setup — I have nothing to report back to the owner on pricing or training.
Chris Park · v3 Focused: first bookingI spent 25 turns trying to find a single settings screen and never got there — if I can't even see the customization options, I have no reason to trust this can replace my Airtable setup.
Dr. Elena Vargas · v3 Focused: first bookingI've spent 15 turns watching a booking link get set up — I came here to find out if this handles psychiatric practice, not to send myself a calendar link.
Lara Okeke, PT · v1 Refined: closest to currentI can see step 3 says 'In-person, virtual, or both' which is exactly what I need, but I've been tapping step 1 for what feels like forever and nothing is happening.
Dr. Tom Walsh · v1 Refined: closest to currentI spent nearly the entire session clicking Skip on the same onboarding screen and never once saw what a telehealth video session actually looks like.
Mary McKinnon · v2 Personalized: profession-first wizardI got through the setup fine — the language made sense — but I never found anyone to call or chat with if I got stuck, and that's the one thing I really needed to see.
Mary McKinnon · v1 Refined: closest to currentI got through all the setup steps fine but then I spent the whole rest of the time clicking around looking for someone to call or chat with and never found a single thing.
Lara Okeke, PT · v2 Personalized: profession-first wizardHome visit being right there as a built-in service type felt like this was designed for someone like me, but I spent what felt like forever just trying to get past the practice type screen.
Lara Okeke, PT · v3 Focused: first bookingThe setup flow felt genuinely mobile-friendly right up until I tried to open my calendar — and then nothing happened, over and over, which is exactly the moment I needed to see whether this actually works between client homes.
Conversion expert · v1 Refined: closest to currentThe checklist and the $1,000 unlock are genuinely strong activation mechanics — but the 'Share booking link' CTA is a dead end, and that's the only action that actually matters for the signup-to-paid conversion I came here to evaluate.
Brand expert · v1 Refined: closest to currentThe completion screen has genuine polish — the purple, the reward banner, the dual CTAs all land well — but when 'Open calendar' goes nowhere after fifteen clicks, the activation moment becomes a beautiful dead end.
Sam Rivera, LMHC · v2 Personalized: profession-first wizardThis is exactly the finish line I was hoping for — my booking link is live, I'm set up to take my first booking, and there's even A$1,000 in free online payments unlocked.
Behavioural psychology expert · v3 Focused: first bookingThe activation arc builds to a genuinely well-designed peak, but clicking 'Open my calendar' fourteen times without advancing exposed a dead end at the exact moment the behavioral commitment loop needed to close.
Behavioural psychology expert · v1 Refined: closest to currentThe peak is exceptionally well-crafted — $1,000 unlocked at the exact moment of completion — but a header still reading 'You're 4 steps from your first booking' at 100% done is the kind of copy slip that quietly erodes the trust you just spent five steps earning.
Brand expert · v2 Personalized: profession-first wizardThe completion screen lands with genuine brand confidence — the purple value banner and bold headline earn their moment — but the contradictory 'Available once setup is complete' bottom bar is a single UX artifact that actively undermines what should be Carepatron's cleanest activation payoff.
UX heuristics expert · v3 Focused: first bookingThe activation milestone screen is one of the strongest I've seen for a healthcare SaaS prototype — clear artifact, clear reward, clear next step — but clicking the primary CTA sixteen times without advancing is a trust-destroying prototype gap that must be closed before any real user testing.
Top 3 Fix List
1. Unlock the Billing sidebar item (or add a dedicated 'I'm switching from another practice' onboarding path) so migration-intent users can immediately verify insurance billing support without being forced through a booking setup wizard first.
Seen in 1 run(s): v1 Refined: closest to current
Seen in 1 run(s): v1 Refined: closest to current
2. Make the Skip for now button functional so reviewers exit onboarding and land on the Calendar, then add a prominent Start telehealth session entry point on the Calendar or dashboard so the video session experience can be discovered and previewed without requiring a completed booking.
Seen in 1 run(s): v1 Refined: closest to current
Seen in 1 run(s): v1 Refined: closest to current
3. Surface a persistent 'Plans & Pricing' link in the sidebar that is always accessible regardless of setup completion status, and add a 'Help & Training' or 'Resources' nav item — these two items directly unblock both stop conditions and are the minimum required for an evaluation-mode visitor to have a productive session.
Seen in 1 run(s): v1 Refined: closest to current
Seen in 1 run(s): v1 Refined: closest to current
Production Calibration Check
| Expected persona signal | Matched? | Evidence |
|---|---|---|
| tom_walsh | No / not run | |
| maya_chen | No / not run | |
| priya_shah | No / not run | |
| chris_park | No / not run | |
| nancy_hanisch | No / not run | |
| david_knight | No / not run | |
| becky_tran | No / not run |
Visual Evidence Index
| Target | Reviewer | Status | Screenshots | Video | Run JSON |
|---|---|---|---|---|---|
| v1 Refined: closest to current | Becky Tran | Scored | 26 final | video | result.json |
| v1 Refined: closest to current | Behavioural psychology expert | Scored | 26 final | video | result.json |
| v1 Refined: closest to current | Brand expert | Scored | 25 final | video | result.json |
| v1 Refined: closest to current | Conversion expert | Scored | 26 final | video | result.json |
| v1 Refined: closest to current | Dr. Maya Chen, LCSW | Scored | 6 final | video | result.json |
| v1 Refined: closest to current | Dr. Tom Walsh | Scored | 26 final | video | result.json |
| v1 Refined: closest to current | Lara Okeke, PT | Scored | 26 final | video | result.json |
| v1 Refined: closest to current | Mary McKinnon | Scored | 25 final | video | result.json |
| v2 Personalized: profession-first wizard | Brand expert | Scored | 24 final | video | result.json |
| v2 Personalized: profession-first wizard | Lara Okeke, PT | Scored | 26 final | video | result.json |
| v2 Personalized: profession-first wizard | Liz Chen, SLP | Scored | 22 final | video | result.json |
| v2 Personalized: profession-first wizard | Mary McKinnon | Scored | 26 final | video | result.json |
| v2 Personalized: profession-first wizard | Sam Rivera, LMHC | Scored | 25 final | video | result.json |
| v3 Focused: first booking | Becky Tran | Scored | 26 final | video | result.json |
| v3 Focused: first booking | Behavioural psychology expert | Scored | 26 final | video | result.json |
| v3 Focused: first booking | Chris Park | Scored | 26 final | video | result.json |
| v3 Focused: first booking | Dr. Elena Vargas | Scored | 16 final | video | result.json |
| v3 Focused: first booking | Lara Okeke, PT | Scored | 26 final | video | result.json |
| v3 Focused: first booking | Liz Chen, SLP | Scored | 24 final | video | result.json |
| v3 Focused: first booking | Marc Robinson | Scored | 23 final | video | result.json |
| v3 Focused: first booking | Nancy Hanisch | Scored | 21 final | video | result.json |
| v3 Focused: first booking | UX heuristics expert | Scored | 25 final | video | result.json |
