“I’m not going to budget just to budget. Budgeting has to be a means to an end. Give me a reason to feel good about budgeting. It’s not something I would naturally enjoy doing… It’s kind of like eating healthy or working out.” (Andy W.)
I have been an avid budgeter for a long time, and it’s helped my wife and I to do some fairly “radical” things with money over the years, like save a 60%+ down payment for our yet-to-be-purchased home and avoid any form of debt. I’ve experienced firsthand what can happen when you get a grip of your finances, stop letting your money be the boss of you, and start putting it to work on your dreams.
That’s why I want to work for YNAB. They are bringing budgeting to the masses with some awesome products that I fully believe have the potential to change peoples’ lives. What’s more, they understand that ultimately, it’s not about the software. Nor is it about the budget. Nor is it even about the money. It’s about the people—their dreams, their ambitions.
Not everyone has a budget, but everyone I’ve ever met has dreams and ambitions. Many of them don’t see any correlation between those things and a budget, but I do!
I believe that YNAB helps people connect with their dreams, not just stay on top of their bills.
“In 2018, we’ll focus intensely on making YNAB effortless and helping new budgeters get started effectively, from their very first experience.” (YNAB job ad)
Over the course of 2 weeks, I conducted a research study focused on identifying areas of friction encountered by newly-minted YNABers in their first experience with the mobile app.
The study simulated a situation where the customer’s very first exposure to YNAB is via the mobile app. I primarily focused on their experience from signing up through creating their first budget.
Here are the top 5 things that surfaced (I’ll elaborate more on these findings later):
- All four participants wanted to personalize the categories and make it their budget, with two of them insisting on doing this right away, and two who were content to explore the app before making any changes.
- All four participants had some level of confusion or hesitancy around setting up an account (e.g. checking), and two of them never managed to set one up, which meant they missed out on the whole experience of adding transactions.
- Three participants dismissed the Intercom message almost immediately, without reading it, believing it to be an “ad”, “sales pitch”, or “wizard”.
- All four participants had some kind of struggle or confusion dealing with income, especially in terms of how it related to their account balance and “To Be Budgeted” amount.
- All four participants approached the app with the belief that budgeting is primarily about handling expenses, and were unconvinced that the app could help them with their dreams or ambitions.
None of these findings are black and white. They don’t provide me with a concrete set of problems to solve, i.e. where I could simply say “If you go with X design, you’ll most likely get Y result.” They do, however, provide an excellent springboard for additional research and experimentation.
I have some ideas on what my next steps would be, and I’ve described those near the end.
I’m responsible for the entire study, everything from recruiting participants to synthesizing the findings. I love this stuff.
As I mentioned above, this study focused primarily on new users’ experience from signing up through creating their first budget.
I presented participants with common tasks that any new budgeter would be likely to attempt, such as budgeting for their normal expenses, handling a grocery purchase, and handling a monthly utility bill paid by check.
I also asked them to find ways to use the app to help them with their bigger financial goals, which is not necessarily something a new budgeter would think to try, but that’s probably because “[they’ve] never budgeted like this.”
Let’s talk about constraints. I had 2 weeks to get everything done, but gave myself only 7 days to recruit participants and conduct all of the sessions. Because people are doing anything and everything on their phones these days, I decided to focus my research exclusively on the YNAB mobile app.
While recruiting participants I used a screener to find people who had never used YNAB before. I ended up with six people who met my criteria, but was only able to schedule sessions with four of them.
Only one of the group had ever heard of YNAB. All four of them had created a personal budget at some point, but it had been months or years since three of them had last kept a budget.
The participants ranged in age from their early 30s through late 70s.
The study followed a pretty standard usability test format—moderated, largely task-based, and, in this case, conducted in person.
After a brief introduction where I assured them “I’m not testing you, I’m testing this app to see how well it works for you,” and getting their consent to record, we began with a pre-test interview covering their past experience with budgeting and any big financial goals they have.
Next, I presented them with a series of tasks. No explanations. No help. Just some things a new budgeter would probably be inclined to try. I gave them plenty of latitude to explore the app on their own.
At the end of the test, I asked them two questions that Jared Spool likes to use in these types of studies.
- What are two things about the design that you really liked?
- What are two things about the design that you didn’t like?
Finally, we took a couple of minutes to debrief and discuss specific areas they struggled and why.
“I never felt comfortable with how I enter my income. Expenses? I think they’ve got this down on lock. It’s very obvious that I’m entering a monthly expense for rent… The income is fuzzy. And to me that’s half of a budgeting app—entering income.” (Jon O.)
Steps I Took
1. Establish Why the Study Should Be Done
Research is a wonderful yet paradoxical thing. It often provokes more questions than it answers. It can confirm things we already know, and uncover things we’ve never thought of. It relies on us asking good questions, and often reveals an even better question that we could be asking.
Any research effort must be grounded in something relatively concrete, something that will keep the curiosity of the researcher accountable. In this case, I had a clear business objective to ground me: “making YNAB effortless and helping new budgeters get started effectively, from their very first experience.”
That business objective was the why behind this study.
(Normally, I would collaborate with business stakeholders on this to make sure I’m aligned with their priorities.)
2. Decide How To Do the Study
I had a lot of research methods to choose from: user interviews, usability testing, a heuristic review, etc.
I chose usability testing because it excels at revealing points of friction in working software, and helps me to connect a person’s behavior with the thought process behind it.
Please note that my goal was not to find specific usability issues (though I did find some) that could be fixed with some straightforward tweaks to the interface.
My primary goal was to find any systemic problems or areas of friction for brand new users. Remember, this is their very first time in the app. Their decision to either open the app again or delete it from their phone may be based entirely on this experience.
3. Recruit Participants
I reached out to people I knew, and decided to stick with folks in my local area since I had so little time to execute the study. This step involved creating a screener in Typeform, and then some email exchanges to figure out when and where to meet each participant.
4. Decide What to Test
Technically, a new YNABer’s first experience could involve dozens of things. I had to determine what they were most likely to encounter in their first go-around.
To do that, I created a new account and went through the onboarding process for both the web and mobile apps. I unpacked the mobile onboarding process and accounted for those steps in my test plan.
I wanted to get into their mental model about money and budgeting, so I decided to ask them to “find some ways to use this app to help you accomplish your financial goals.”
Would they see their goals and their budget as related?
I also decided to touch on a few of the more common actions a new user would be likely to explore, such as handling transactions.
I carefully worded each task so as not to create bias or provide more information than a new user would have on their own.
5. Do a Dry Run
It’s crucial to do a dry run before sitting down with the first participant. My wife helped me with this (thanks, sweetheart!) and it helped me to refine my test plan and script.
6. Execute the Sessions
This part was super fun! I cruised around to meet people at coffee shops, their homes—wherever was most convenient for them. My testing setup worked perfectly, and was even more efficient once I figured out that I could record a mobile device’s screen directly onto my laptop using QuickTime.
7. Capture the Data
Even though I took notes during each session, I’m no shorthand expert, and my notes were not as thorough as I like them to be.
I spent a number of hours reviewing the video from each session and capturing detailed notes about what the participants did and said.
Because this study was so focused on people’s mental models regarding money and budgeting, I felt that it was worthwhile to spend the time capturing exact quotes. There is always a great deal of nuance and meaning in how people choose to express their thoughts!
8. Analyze the Data
The gold was still encased in ore. I had lots of data but data alone does not equal information.
My process involved going through the notes from each session and layering information by means of tags and themes. (Researchers often refer to this as “coding” your data). This step isn’t as much about interpreting the data, as it is about adding dimensionality to prepare it for interpretation.
9. Synthesize the Data and Produce Findings
This is one of my favorite parts. Finally, all the hard work of planning, preparing, executing, and analyzing has been completed, and we’re ready to refine some shiny nuggets!
I came away with 5 major findings. These were things I observed across all four sessions and that formed a clear pattern.
1. They Wanted It to Be Their Budget
All four participants wanted to personalize the categories and make it their budget.
“It’s me telling my needs up front. If that’s the philosophy—your budget is your own—how do you give [me] intelligent options? It doesn’t seem like much of a partnership [unless] I tell you what my goals are and you help me to get there.” (Andy W.)
For some, this was a highly emotional issue.
“That’s why working on paper is like ‘I know what’s going on here.’ They don’t know what’s going on here. Yeah, they think I’ve got some basic budget that I can just use. No!” (Shirley S.)
I consider this to be the most important finding from the study. YNAB isn’t trying to tell people what to do with their money, but is simply trying to be helpful by illustrating how their budget could look and also offer some subtle cues about the 4 rules (e.g. the “Immediate Obligations” category group).
This approach really backfired in the study, though. Instead of making the budget more relatable, the example categories made the budget seem impersonal, like someone had already decided what their priorities should be. Three participants figured out how to edit the categories, and two of them made some customizations.
Even so, the first impression for all four people was that it didn’t feel like their budget, and it would clearly require some effort to make it fit their life.
“…When I look at something like this it’s like somebody is trying to tell me what to do, and I already know what I’m doing. I don’t want to be told and then have to fit into somebody else’s idea of what my budget should be.” (Shirley S.)
This kind of friction is unnecessary. I propose a few ways we could overcome it below.
2. They Struggled With Setting Up An Account
All four participants had some level of confusion or hesitancy around setting up a checking account. Two of them never managed to do so, which also meant they missed out on the whole experience of adding transactions.
The hesitancy stemmed from a lack of trust.
“I didn’t necessarily want to give it my bank information!” (Andy W.)
“I’m hoping it’s not going to ask to connect to my bank yet. We don’t really have much of a relationship yet, me and YNAB, and I definitely don’t trust it yet to connect it to my bank.” (Jon O.)
The confusion was mostly around not realizing that you could create a non-linked account now, and then come back later and link it to your bank.
“I guess I clicked ‘skip’ just to see what ‘skip’ did, not that I knew I could manually enter anything.” (Calvin S.)
Obviously, there a host of benefits to linking an account to your bank—things most new budgeters care about, like saving time entering transactions—but they are left with the impression that this is an either/or decision. It’s not. I provide an idea for how to smooth out the wrinkles in this experience below.
3. They Dismissed the Intercom Message Without Reading It
Three participants dismissed the Intercom message almost immediately, without reading it, believing it to be an “ad”, “sales pitch”, or “wizard”.
“This guy’s in the way, how do I get him out of the way?” (Shirley S.)
“I just assumed it was like a sales pitch. I didn’t even read it.” (Andy W.)
“There’s Todd again. I don’t know Todd or like Todd so I’m going to swipe. Get the heck out of here.” (Jon O.)
The problem here is that Todd’s message is a new user’s “lifeline.” They literally don’t know what they’re doing or how their budget works. Below, I explore a few ways we could experiment with this message to make it more effective.
4. They Struggled When Dealing With Income
All four participants had some kind of struggle or confusion dealing with income, especially in terms of how it related to their account balance and “To Be Budgeted” amount.
“There’s no place here… that gives me a space to write down ‘this is how much money I have coming in.’ That seems a little strange to me for a budget, because you’ve got to know how much is coming in before you can figure out how much can go out.” (Shirley S.)
“[I was ] left with the unsettling feeling that, without Googling, I don’t know how people are handling income with this app.” (Jon O.)
Two of them were clearly thinking of budgeting as a forecasting tool to help them plan for their future money.
“I could have nothing in my account, but I’m getting paid here, and by the end of the month I know I’ll have $5,000 in my checking account but it might not necessarily be there up front.” (Calvin S.)
There are two issues here. The first is their mental model and the notion that budgeting = forecasting. The second is the actual way YNAB supports entering income. I think an introduction to the 4 rules would help to clear this up, but I’m getting ahead of myself—see below.
5. They Were Unconvinced That Their Budget Would Help Them With Their Dreams
All four participants approached the app with the belief that budgeting is primarily about handling expenses, and were unconvinced that the app could help them with their dreams or ambitions.
“What this tells me is ‘you have $800 budgeted to spend’, as opposed to ‘you saved $800, it counts toward your $40,000 goal and you’re $800 closer.’ Woohoo! Confetti, blah blah blah.” (Andy W.)
“My experience with budgeting software is it takes all the fun out of my life.” (Jon O.)
What I Would Do Next
- Discuss my findings with business stakeholders.
- Look for ways to correlate these findings with any important business metrics (e.g. churn, conversions), which would help me to determine which behaviors have a direct bearing on business success.
- Talk with the support team and workshop facilitators about the questions and problems they hear from brand new users to see if my findings from this small sample of four people may be representative of the broader population of new users.
Experiments We Could Run
Here are some ideas for experiments we could run to address findings #2 and #3. These are “tweaky” things that we could probably evaluate by watching the right metrics and using quantitative data.
Time the Intercom Message Differently
Time the first Intercom message so that it appears after the user has made their first move (e.g. budgeting money to a category, viewing the “Accounts” screen for the first time, or trying to add a transaction).
I realize that this would involve switching from an “in-app” to a “push” message along with the requisite notification. If someone disallows all notifications from YNAB, well… it’s tricky.
Try a Different Message Type
A/B test the first Intercom message with the message type set to “Note” and sent as “Show the full message”. The Intercom team recommends, “When something is very important and you want your customers to take action right away, deliver your note in ‘full’.”
I would also suggest trying a more concise message that emphasizes the importance of creating an account with a clearer call to action.
Provide a “Dummy” Account
Instead of requiring new users to set up their own account right away, provide them with a “dummy” account with a starting balance. This reduces the friction of them having to decide whether they trust YNAB with their bank account credentials and allows them to experience all of the basic features with zero administrative work.
Explain That They Can Link the Account Later
Whether they create an account themselves or are provided with a dummy account, emphasize that they may link it to their bank at any time.
The current “New Account” workflow makes it feel like a mutually exclusive choice: you can either create a linked account or “Skip” to create an unlinked account. The reality is that you can link certain types of accounts (e.g. checking) to your bank later. No pressure. You can do that when you’re ready.
You’re not really “skipping” anything permanently. Perhaps a different button label like “Not Now” would perform better than “Skip.”
Things We Could Prototype
Here are a few ideas that could help with some of the deeper mental model issues I discovered (findings #1, #4, and #5). These are all things that would require some deeper discussion, planning, iteration, and testing before anything goes live.
Let’s provide a better onboarding experience:
Introduce the 4 Rules
Introduce new YNABers to the 4 rules. As Jesse is fond of saying, “You’ve never budgeted like this,” which is great… But that’s precisely the problem.
People aren’t used to thinking this way. They aren’t used to budgeting all of their money. They aren’t used to budgeting only money they have right now. They aren’t used to feeling so close to their money, so in tune with what’s happening.
“I’ve done this in Excel in the past… I can make a future-cast. If I’m consulting I can say, well, I have three clients right now… in May there’s going to be this large payment and I need to be able to live until then. I expect to be able to enter that such that I see a chart that’s like money over time, going down down down, then a big $20,000 check comes in and it shoots all the way up. To me, that’s a big motivator for using a budgeting application.” (Jon O.)
Their mental model for what a budget is for and how it works runs against some of the things they’re experiencing in the YNAB app. This creates friction and uneasiness. We may be able to reduce this by prefacing what they’re about to do in the app with the reasons why it works that way.
Let Them Choose Their Categories
Allow them to tell YNAB what’s important to them, opting in to the categories of their choosing rather than getting “everything on the menu”. I get the reasons for having a bunch of predefined categories. There are arguments for and against it.
My thought is we could get the best of both worlds by at least offering the option to opt in to specific categories during the onboarding process. If they choose to do so, great! If not, just give them the standard list. People should feel as though YNAB is a good partner, consulting them about their priorities, not dictating what they should be.
“It already added all these things and I didn’t tell it to… This is like working with a very uncooperative spouse. Stop!” (Shirley S.)
Prompt Users to Name Their Budget
Allow users to give their budget a name. This is subtle. It may not have the effect I intend, but my theory is that people feel a kind of ownership when they name something. Susie will probably feel more invested in “Sue’s Personal Budget” than she would in, “My First Budget.”
What I Learned
I’ve been immersed in budgeting, YNAB, and the 4 rules for so long that it was a reality check to hear people say things like:
“Budgeting sucks… I give it to my wife because it’s not fun. It’s so business-like.” (Calvin S.)
As I mentioned in the beginning, I believe that YNAB helps people connect with their dreams, not just stay on top of their bills. I truly believe that.
This study reminded me that other people aren’t like me. They think differently than I do. They do things differently than I would. They will choose whether to budget or not for their own reasons, not mine. And that’s the whole point! It’s called personal finance, after all.
It’s their choice. It’s their money. It’s their life.
There are things I want for people, but at the end of the day, they have to want those things for themselves. And I think their first experience with YNAB could very well be the inaugural step toward figuring out what they really want, and then making it happen.