Closings gone wrong are painful

If you've purchased a home you know how important it is to close on time, otherwise you risk losing that low interest rate and may not get the keys to your brand new house. However, you may not know that behind the scenes there's a frantic dance that happens between your lender, settlement agent, and notary to make sure all of the proper documents are accurately drawn up, printed, signed, and returned on time. Until your lender has a stack of accurately signed documents in their hand, your loan will not be funded.

Snapdocs aims to power the perfect closing

Snapdocs exists to leverage technology and digital solutions to create stress-free closing experiences for real estate transactions nationwide and currently processes over 15% of the nations real estate transactions.

At the time I'm writing this, our big company objective was to get our customers to put 100% of their signings through Snapdocs. If we could guarantee that every Snapdocs closing is perfect (i.e. accurate, on-time or faster) we felt we could increase wallet share with existing customers and also attract new ones.

Narrowing in on a core problem

Being able to guarantee that every closing is perfect is a monumental task. I began by breaking it down and compartmentalizing the parts of the closing my team had direct impact over. Turning to the data, I analyzed metrics around how documents flow through our product and noticed when scanbacks* are requested they're only returned 66% of the time! Additionally, when referencing market data, I noticed lenders are requesting scanbacks more frequently month over month making this initiative even more important.

*Note: A scanback is an industry term that essentially means the notary must scan a copy of the signed paper documents back to the settlement agent and lender. This allows the lender to fund the loan faster, which usually means the borrower can get the keys to their house the same day instead of waiting for the documents to get shipped (or potentially even lost) in the mail.

Scanbacks are requested more often and returned 66 percent of the time

Aligning on a strategy and target to hit

My product manager and I agreed our core success metric would be to increase our scanback return rate from 66% to 85% by the end of the quarter. If we could reliably get signed documents back into our system, we could leverage Snapdocs' document intelligence tools to verify the accuracy of these documents and catch errors faster. However, we realized we must walk before we can run, so my team chose to first focus on improving our scanback return rate.

Our primary goal was to increase our scanback return rate from 66% to 85% by the end of the quarter.

In order to do my job, I needed to become a scanback expert

I put together a research guide and set out to learn from users in the field. My big question was to understand why our scanback return rate was so low. To start answering this question, I interviewed three different personas – Settlement Agents, Notary Schedulers, and Notaries. The below photo is a zoom interview with one of our rockstar notaries!

Some of the key questions I was curious about were...

What are the top reasons scanbacks don’t go well?
How soon after the signing do scanbacks need to be uploaded?
What is happening with orders that originally required scanbacks but scans were never uploaded?
What are the top reasons why a notary will not upload scanbacks to an order?
After a notary accepts a signing where scanbacks are required, do they take extra steps to make sure scanbacks can be uploaded on time?

Redefining what a successful scanback means

Based on our interview learnings, I was able to create a more objective definition of what it meant for a scanback to be successful.

It's important to note that scanbacks can become “unsuccessful” at any layer of this pyramid. For example, a scanback could be received but not within the desired turnaround time of “within 3 hours of the appointment time” – that could leave a borrower sitting around unable to close the transaction and obtain their keys. Likewise, a scanback could be received on-time, but the file may have pages where the text isn’t legible.

Our first major roadblock

We now knew what it took for a successful scanback, however, we didn't have a reliable way of measuring most of the criteria mentioned in the above pyramid. Our data was missing or incomplete!

Wait, what happened to the 33% of scanbacks that aren’t returned?

We didn’t have the data yet to know for sure, but our user interviews pointed us in the right direction:

Notaries were returning scanbacks via channels we didn't yet have visibility into such as email or fax (e.g. 18% of scanback instructions mentioned an email or fax number)
Sometimes scanbacks were no longer needed and the requirement wasn't updated on the order (e.g. the notary was close enough to drop off the documents at the office)
Some scanbacks simply didn't get returned at all (e.g. notary forgot or wasn't able to accommodate the request)

You can’t improve what you can’t measure

Our lack of reliable scanback data made it difficult for us to understand, size, and prioritize between various scanback challenges and accurately measure our progress toward scanback goals.

We created several new data fields and added them into the product:

Scanback deadline – "When do you need scanbacks returned by?"
Scanback return method - "How do you want scanbacks returned to you?"
Additional return methods – "If absolutely necessary, what other ways can the notary return scanbacks?"

Notaries benefitted greatly from this data too

Before we added these new data fields, notaries would often get instructions communicated to them in an unorganized fashion causing important information to fall through the cracks.

I mapped out a scanback product flow from end-to-end and identified opportunities where we could present these instructions in a more standardized way. For example, now when a notary goes to accept an offer for a new signing they can clearly see that scanbacks are needed as well as when and how they should be returned.

Customer adoption of new fields was high

I leveraged Appcues to design onboarding flows which helped educate our users and get them to engage with the new data fields. Customer adoption of our new scanback fields rose to 79% usage in a little over a month!

First time user onboarding for new scanback fields

My product manager and I worked with our business intelligence team to spin up a dashboard to track things like scanback return rate, usability rate, and on time rate.

After some weeks passed we revisited our metrics and were quite surprised to see our scanback return rate shot up from 66% to 88%!

Refocusing on the biggest near-term opportunity

Using our new metrics, we noticed in the last 12 months 32% of scanback files were larger than 20MB. Going back to our interview learnings, I knew that files of that size start to cause issues for all of our users and can slow down or disrupt the funding process.

Notaries lack the time or technical know-how to easily downsize files or find their file is too large to upload to Snapdocs. Schedulers & Settlement Agents have to recombine split files before sharing with others and may run into size limitations when trying to upload files to their system of record.

Getting to a solution to reduce large files

I led a brainstorming meeting with our team, members from customer success, as well as other core stakeholders to identify potential solutions. Our prompt was "How might we eliminate scanback file size as an issue?"

I put together a lightweight comic strip to summarize our interview findings and help others in the brainstorming session connect with our learnings. As not everyone in the session was as close to the problem, I found the comic strips to be a valuable way to catch people up to speed in an engaging way.

Along with my core team, I took the outputs from our brainstorming sessions and affinity mapped everyone's ideas into core themes which we would vote on and prioritize against our project goals and KPIs.

Testing concepts early

My product manager and I identified all the potential lightweight concepts we could test out with users. I created super low-fi mockups in Figma and got them in front of users, asking questions such as:

“Tell me what you’re seeing?”
“What do you think you can do on this screen?”
“On a scale of 0-10, how useful would this be to you? Why?”

Concept testing scanback file size ideas

Concept testing main takeaways

Auto-resize - Most users saw value in the system being able to auto-resize their scanback files, however had concerns about the quality of the scanbacks after the file size had been reduced.
Send to lender - Having a quick button to send the scanbacks to the lender seemed like a good idea, but many users had concerns if this would work with different processes across lender and title companies.
Low-hanging fruit - Having better error handling and instructions when a large file is uploaded felt like a safe win.

What we actually built & released

Based on our findings, I proposed we take a multi-angled approach and also tackle some low-hanging fruit along the way. We designed an experience where the system automatically compresses any scanback file larger than 5 MB.

Additionally, for any file larger than 20 MB, I added a soft stop to educate the user on the dangers of large files and encourage them to downsize their file. I also wrote a support article to give notaries instructions on how to safely decrease their file size.

Lastly, our document upload widget was inconsistent across our client, scheduler, and notary UIs so I took this as an opportunity to standardize the component and improve the overall UX to be more clear.

Results

18%

Usable file size

Scanback usable file size increased from 68% to 86% in a little over 3 months.

26%

Scanbacks returned

Scanback return rate went from 66% to 94% (with "Upload to Snapdocs" as the return method)

93%

Scanbacks returned via Snapdocs

"Upload to Snapdocs" is the primary return method and increased from 88% to 93%.

85%

Usage of new scanback fields

User adoption of our new fields is steadily climbing and increased from 78% to 85%.

What I learned & what's next

Through this effort I learned you can’t take product metrics at face value. You need to dig in and understand where they’re coming from and how they’re being calculated in order to determine if they’re an accurate representation of how you users are using the product.

Since standardizing our instructions and data, we noticed our scanback on time rate also increased from 67% to 75% but we still have room to improve here. I’m going to run another round of brainstorming sessions to identify more concepts to test, measure, and launch.

Summary Defining the problem Early research Refocusing efforts Design approach What was shipped Final results

HOW MIGHT WE...

Guarantee that every home closing is perfect?