ivan's blog: posts tagged process

New feed entry point · Tinkoff

Ivan Zviahin — Mon, 26 Aug 2024 22:36:06 +0000

How we made the personal feed in the bank app more readable by increasing 1-week feed retention rate several times over.

Tinkoff is an online ecosystem or super app based on financial and lifestyle services. The best mobile privetely-held bank in the world according to Global Finance. The third largest bank in the CIS with 30,000,000 customers.

A personal feed is a product. With the help of which we are building a communication platform where people learn all about finance, save money and have fun. Businesses need a personal feed to sell financial products through it. Users need to have fun, learn something new, be part of an expert community. In simple words, it’s just a feed of events and news.

The personal feed is having a positive impact. We test disabling the feed on new users and compare metrics across our products. There has been a drop in metrics, including cashback, bonuses and airline tickets. The product is working well, but we need to make it more efficient.

The feed is in a difficult-to-reach place on the interface. To get to it, you have to open the “City” tab, which doesn’t tell you what’s on there. You have to look at a complicated screen and open the bottom sheet with the feed.

People say the feed is hard to find. If people can’t access the feed, they won’t come back. If they don’t come back, they won’t use the product.

By comparison, the stories on the home screen get visited several times more often and people come back more often, even though both are feeds with content, just in different formats.

Problem for users

We want to make the feed accessible. To make you more likely to open the feed on the mobile app. It’s heavily hidden at the moment.

Problem for business

Increase retention rate in the feed. If we increase the retention rate in the feed, customers will be more engaged with our products. If customers use more products, the bank will make more money.

Audience

The audience is very wide and diverse. But we aimed at all our bank users with an active lifestyle. I can’t share the audience of the personal feed, but it makes up the majority of the total number of customers. Considering that we have 30,000,000 customers, the scale of the product is confirmed. Wow!

Success criteria

↑ 1-week feed retention rate
↑ 1-week app retention rate
We also need to keep people engaged in the feed, which will be difficult.

Discovery

We’re looking for a solution in other products, but we haven’t found any yet. So, we’ll have to come up with a solution ourselves. We looked at what our users had to say. We’ve put together a job spec for a Data Analyst. But...

But I find interesting is this. It’s not just about moving the feed. How do you get people to go there? The analysis shows us this:

We already have a big audience that goes to the bank every day. We just need to put the feed on the home screen and get people interested.
We already have content on the home page in the form of Stories, which is a great product.

So, what we have

This scheme helped us make some guesses.

Hypothesis formulation and low-fi prototypes

These are the hypotheses.

Main screen. Adding the feed to the home screen will make it easier to access and more likely to be used.

We had many more options than I’m going to show you. I’ll show you the best ideas and how they look. I thought the best idea was to put content in a separate tab.

We stopped because we realised it was unrealistic to expect people to read content on secondary screens in the bank. We tried adding stories to secondary screens. It doesn’t work. Right, let’s move on.

The second idea was to just put the feed under all the accounts, as there aren’t that many of them on average.

We saw that people don’t scroll past the second screen. How can we open it from the first screen? We added a right swipe option.

It’s cool, but it looks complicated. It’s not natural. Turn the story feed into a smart widget.

Interesting idea, but we don’t have the guts to do it. There’s a big risk we’d mess up the stories and it’d all fall apart. But on day five, we had a brainwave...

It’s much more straightforward to swipe down than to swipe right. Pull to refresh is no longer necessary as all the data is updated in real time. It seems to be the best idea, so we’ve decided to go with it. And opening quick tools works the same way in iOS, which gave us the green light to go ahead with it. We showed this idea to everyone who was interested: the product managers, the designers and the stakeholders of the home screen. We got a few doubts in response, but everyone was generally on board with this solution. We gave this to user testing, and we were pleased with the results. Let’s consider some other possibilities.

Interest. Make an entry point interesting to increase its open and retention rate.

The simplest way to get people interested is to create an information trigger on the outside. I think stories are probably the best trigger, but it’s tricky to make stories and a feed one platform. It’ll probably be a separate story soon.
But for now, we just need something simple. And here’s the thing: if you’ve been keeping up with our content, you’ll have noticed that we recently added a news card in the feed. It would be great to take the most interesting words from the introductory text and put them outside.

Let’s move on to the next item on the agenda.

Clear. Make the opening of the feed simple and clear to increase accessibility.

Somehow we have to teach people how to use the new feed. At first we tried to come up with some kind of detailed training, but then we realised that the easiest way to do was to just pull the feed a few times. And stop when the person has done it.

Habit. If we don’t change user habits, we’ll keep old stuff.

Some people use the current feed the old way. We shouldn’t break their habit. Let’s add a history button. This is about the new way to access the feed. Don’t show it for a long time. If the user opens the feed, we’ll remove the button.

Additional entry points. Finding more entry points will increase availability.

When we spoke with users we found something interesting. When people contact the support team (chat) they go to watch stories on the home screen because they’re afraid to close the app (in general!!!). We must find a way to deal with these situations. This could affect how accessible the feed is.

We knew this idea wouldn’t make it into the first version, so we didn’t make a prototype. But you can imagine it in your mind.

Priority

I met with the product manager and engineers to estimate the hypotheses according to two parameters: business value and technical complexity. We made a list of priorities.

Main screen. Adding the feed to the home screen will make it easier to access and more likely to be used.

∞ ⁄ 5 = ∞

Interest. Make an entry point interesting to increase its open and retention rate.

3 ⁄1 = 3

Clear. Make the opening of the feed simple and clear to increase accessibility.

3/1 = 3

Habit. If we don’t change user habits, we’ll keep old stuff.

3 ⁄ 2 = 1.5

Additional entry points. Finding more entry points will increase availability.

4 ⁄ 4 = 1

Scoping

We break down our ideas into iterations to get the product to users quickly and make sure it’s perfect.

Iteration 1

Main screen. If we make the feed on the home screen, it will increase its accessibility and retention rate. This is our main idea, we will make it.
Interest. Make an entry point interesting to increase its open and retention rate. I really want to do it at once, so that it will be the cherry on the cake. But if we don’t have time, we’ll put it in a separate iteration. Of course, it will be an ab-test to understand the impact of this particular idea.

Next iterations

Clear. Make the opening of the feed simple and clear to increase accessibility. Let’s test it separately, in case this problem doesn’t exist and it is clear and simple without some additional activities.
Habit. If we don’t change user habits, we’ll keep old stuff. Important stuff. But we’ll run the first iteration only for new users and if everything is good there, we’ll make this idea and publish an update for everyone.
Additional entry points. Finding more entry points will increase availability. Oh, it’s definitely a contender to make it out of the first round!

— Let’s go!

Iteration 1 · hi-fi

I just wanted to point out that our lo-fi prototypes already sounded pretty hi-fi. Having said that, we didn’t come across any major technical issues. So, all the ideas from the concepts made it through to the hi-fi stage, which is pretty unusual.

But the final solution was this. The feed on the main screen looks like a top sheet that you can open by swiping from the top to the bottom. Plus, we made it so you can just click on the dragger and text to open it. We figured this out during the UX tests, but we assumed it in advance.

Above the dragger is a shortened descriptor from the feed, which changes every day along with the full descriptor inside the feed. You can read about this in the case study about the news card.

Just then, Sergei Egorov came along. The second designer to join us. I needed to move on to other things. Sergei took the reins as the second designer on the project. The second half of the work on the final layouts is creating animations of opening and closing the feed. Based on my prototype, Sergei put together a much more detailed one with a description of all the details.

To close the feed, just hit the close button or swipe down. When we were testing it out, we realised that we needed to make closing by swiping up, so that it’s the reverse of opening. This meant there were three ways to close the feed in total. This way, everyone would know how it worked.

We had to get the most complex interactive feature in the Tinkoff app up and running at the time, which was a big challenge for the whole team. It was pretty straightforward to develop the animation and the feed itself on Android, but we hit a few snags on iOS. But we managed to get around all those limitations in the end. Sometimes we had to switch up our strategy and come up with quick fixes, so Sergei spent a lot of evenings with the engineers. Sergei says,

“We often sat with the engineers late into the night tweaking details and catching endless bugs together. I even picked up a few lines of code in Xcode.”

At some point, we got everything to a state that could be called a release state. And it was so lovely to see that happen!

First launch and results

We released it as an AB test for a new audience. At the same time, we ran a separate AB test on the description. Here’s what we found out.

We saw a big jump in the number of people who stuck around and checked out the feed on a weekly basis (1-week feed retention rate).
The percentage of users accessing the feed on the app shot up several times. This indicator saw a nice increase, even if it couldn’t help but grow.
The engagement rate of the feed saw a slight decrease. This is because more people are now entering the feed, but the quality of these users has dropped. They’re not as active as they could be. We’ll be addressing this in the next episodes.
There was no change to the weekly retention rate of the app, statistically speaking. We were disappointed, but not overly so. It’s really difficult to make changes.
The descriptor did the job. Without it, the retention and open rates in the feed were lower.

I’m happy to say that we met our goals, and we’re all proud of that.

Role and platforms

I participated in the formulation and confirmation of the problem together with the product manager.
I did the primary research.
I formulated hypotheses and participated in prioritization together with the product manager and engineers.
Together with the manager, we divided all the hypotheses into iterations.
I designed lo-fi and and first part of hi-fi mockups and described the requirements together with the technologists and engineers.

Half of the layouts and animations were designed by Sergei Egorov. He joined us in the middle of the project as the second designer. Sergei helped QA engineers to run all the flow and described bugs.

Basically, all solutions work on android and iOS.

Reflection

I think the first version of the personal feed opening could have been made much simpler, but it’s great that we overestimated the requirement at the start and made the transition above the level of the traditional design approach in the company. I’m really proud of this little project. See you in the next episodes, thank you!

Story about the same, but from Sergei, only in Russian

News card in personal feed · Tinkoff

Ivan Zviahin — Tue, 21 Mar 2023 20:31:00 +0000

How we made a news card in the feed and increased CTR of first card by 32%, 1-week retention rate of the feed by 8%, open rate by 4% and engagement rate by 35%.

Problem for users

Analyzing different types of cards and collections, we noticed an increase in interest in news cards, especially when we combine them into collections. Based on this insight, we decided that we need to develop in this direction.

Hence the problem — people want to be able to see an up-to-date selection of news based on their interests. They don’t want to collect this news in different places, they want to see it all at once. Most importantly, this was confirmed in a series of user interviews that were aimed at finding the answer “Why do users use the feed?”

We decided that it needed to be checked somehow cheaply. And we collected a selection of news based on the type of card already available.

After such an experiment, we noticed a significant increase (+24%) in the CTR of this card relative to the most popular card in this place. So we realized that we need to improve the user experience of this card and come up with maximum ways to engage a person.

Problem for business

We are always looking for ways to increase CTR with the feed cards. Because with the growth of CTR → engagement grows. With the growth of engagement, the loyalty of users grows, and with the growth of loyalty, the revenue from sales of our internal services grows.

Audience

Success criteria

We will consider the success criterion to be CTR of this card, which will be higher than CTR of the news card from the experiment above (which means it will be higher than any other card in the first place of the feed).

Additionally, we will look at the open and 1-week retention rate of the feed, because the first card will be visible in a hidden state.

Discovery · Interview

Fortunately, quite by chance, we conducted a series of interviews with our users in parallel. We were trying to find an answer to the question of why users read the feed. We had a small product market fit, but we didn’t understand exactly why it happened.

Total 19 in-depth interviews. The latest news got into the top 3 insights. This confirms our initial insight. In addition, we realized that we need to recommend news in a personalized format and give only a selection of headlines.

Discovery · Analysis of similar products

Next, we went to see how different products aggregate different information into a card. Here we found some interesting ideas.

First, almost all cards with no more than four lines of content. If more, then do a horizontal scroll. Secondly, it is important to understand the date of publication, how new the material is. Third, currency exchange rates and investment stickers look cool together with a selection of news as a single widget. Fourth, the introductory description, as an engagement tool, creates the effect of personalization. Additionally a personal address by name attracts a lot of attention. (everyone already knows that)

Wow! As a result, we had a lot of ideas. We have described them in the form of hypotheses below

Hypothesis formulation and low-fi prototypes

Now the card is colored and we know that this is important. When the feed is hidden, a piece of the card is visible and it attracts attention. We decided not to experiment with this and further considered it only in color.

Dates. If we display the publication dates of articles, it may affect card CTR.

Pictures. If we add pictures to the news, it may affect card CTR.

Personalisation. If we display news based on user interests, it may affect CTR and retention rate.

Name. If we contact to you personally, it may affect feed open rate.

Introductory text. If we write a humorous introductory text, it may affect retention rate.

Currency rates. If we display currency rates and share price, it may affect retention rate.

Priority

I set up a meeting with the product manager and engineers to estimate all hypotheses according to two parameters: technical complexity of the implementation (from 1 to 5) and business value (from 1 to 5). We divided one into the other and got an hi-level prioritisation coefficient. So...

Card CTR

Dates. If we display the publication dates of articles, it may affect card CTR.
3 ⁄ 1 = 3
Pictures. If we add pictures to the news, it may affect card CTR.
3 ⁄ 1 = 3
Personalisation. If we display news based on user interests, it may affect card CTR and retention rate.
5 ⁄ 5 = 1

Open rate

Name. If we contact to you personally, it may affect open rate.
3 ⁄ 1 = 3

Retention rate

Introductory text. If we write a humorous introductory text, it may affect retention rate.
3 ⁄ 2 = 1.5
Currency rates. If we display currency rates and share price, it may affect retention rate.
2 ⁄ 2 = 1

Scoping

We divide all into iterations in order to deliver a valuable product as quickly as possible. We focus on three metrics card CTR, retention rate and feed open rate. Therefore, we can take one hypothesis for each metric in each iteration. Of course, we understood that they could influence each other, but speed is more important.

Iteration 1
We definitely make a news card with dates right away. We had a dispute about the pictures, the product manager believed that it would work better without pictures. We agreed on an AB test. Additionally, we will take the appeal by name, since it affects another metric. We do not take hypotheses aimed at retention rate in the first iteration due to the fact that they are complex, and we want to deliver value as early as possible.

Dates. If we display the publication dates of articles, it may affect card CTR.
Pictures. If we add pictures to the news, it may affect card CTR.
Name. If we contact to you personally, it may affect open rate.

Iteration 2
The text-introduction will be launched separately. Potentially, this can affect the retention rate individually, it is interesting to check.

Introductory text. If we write a humorous introductory text, it may affect retention rate.

Iteration 3
After that, we will add a widget with currency rates and investment tickers. It is also interesting to test them separately because this hypothesis is also aimed at the retention rate.

Currency rates. If we display currency rates and share price, it may affect retention rate.

Iteration 4
Next, we try to personalize the news (a difficult thing), so in the end.

Personalisation. If we display news based on user interests, it may affect CTR and retention.

— Let’s go!

Iteration 1 · hi-fi

We added only 4 news, it seems enough to start and looks easy. We tried to make the interface as simple as possible, we have already made mistakes with this many times in the collections. And there was an insight from users that they want to see all the current news in a list at once. We borrowed the algorithm from a web magazine, it is already ready there, current news.

We had two options:

I chose the second option. It looks simpler and more aesthetically pleasing. Next...

Added dates. There are not too many options for how this can be shown, so we made it as simple as possible. We already have a date display style in other cards.

We enter the dates according to special rules that I have formed.

Have	Show
≤ 5 min	just now
< 1 hour	56 min ago
< 2 hours	1 hour ago
< 1 day	23 hours ago
< 2 days	yesterday
< 8 days	6 days ago
≥ 8 days	18 february
≥ 1 year	18 february 2020

Added an appeal by name. I made it as simple as possible, using the available styles from other cards. That’s where we stopped.

Added pictures. As I said before, we had disagreements on this, so it got into the first iteration, we will run them AB test. Option A without pictures, option B with pictures. We will show the card 50-50, we will look at the card CTR.

That is all.

We didn’t do any interface tests, because it seems to us that there is nothing to check here. I checked only the list of self-checking the interface. So, it was → will be.

First launch and results

AB test results. At the first visit, users tap better on the news card without pictures. The relative uplift in CTR is 9%. It does not affect taps when re-entering. It does not affect the return to the feed. The card without pictures won, I cried :-(

In general, first card CTR +8% (new format and dates) and feed open rate +4% (address by name). Cool! Let’s move on to the next iterations.

This is a product manager, his name is Nikita. Nikita is glad that he won the AB test. He was for the option without pictures

Moving on!

Iteration 2 · hi-fi

Hello again! It’s time to work with the retention rate as part of the news card and we have an introductory text on this. The main idea is that we want to make an introductory text that will change every day. It should have a welcoming meaning.

We strongly believe that the text should be humorous, this was confirmed in user interviews, people want something entertaining, a lot of sad things are happening in the world anyway. This can be a great reward for action.

We have not found an editor resource that will write an introductory text every day for several days ahead. So I came up with and wrote them myself. I really want to test this.

What might it look like? We didn’t want to come up with something new (so as not to complicate the idea of test) yet, so we took the text style from other cards as a basis and applied it in our news card. The main thing is that it should be readable, simple and understandable. We have reduced the number of articles to make it look easier visually. We removed the “current news” descriptor and moved it inside the introductory text to make it all look easier.

An important learn that we found. We are well aware that it is ideal to use the hook model for the retention rate boost. And we even figured out how to do it. The reason is the position of the feed in the app and the policy of contacts to send push messages. We promised ourselves to try it after moving the feed to a more accessible place in the app. But about this in another story.

While we design a text, another idea came up. What if we ask some interesting question in the introductory text, and somewhere in the middle of the feed we give an answer in the form of a story card. If we use the story card, we won’t need to develop anything additionally – perfect for the test. It seems that in addition to the retention rate, we can influence the engagement rate of the feed.

feed engagement rate = (user tap + like + bookmark + share + swipe) / all users × 100%

The idea should work because people will look for an answer, react to it and get involved in other content. Sounds interesting?

Sure! Sounds interesting. So, it was and will be. Of course, guys, we started with an AB test to understand the impact of the second iteration on the result.

Second launch and results

In general, feed 1-week retention rate +8% (introductory text with a question and answer) and feed engagement rate +35% (introductory text with a question and answer). It’s hard to understand why it affected engagement so much, but I like it. The test did not affect the CTR of the first card and the open rate in any way, which is also important.

Iteration 3 and 4 · not done

An interesting thing happened, we decided to redo the concept of the feed as a whole and separate the news selection from the text and the address by name. The introductory text and the appeal should become part of the feed itself and be independent, but that’s another story.

So, we have postponed working with currency exchange rates and personalisation. But I want to show you the layouts and developments on these tasks, they should not be lost. We will return to them later, but in another story.

Role and platforms

I participated in the formulation and confirmation of the problem together with the product manager. I communicated with analysts to find out insides. I participated in user interviews and analysis of similar products. The research team helped me a lot, thanks to them.

I formulated hypotheses and participated in prioritization together with the product manager and engineers. Together with the manager, we divided all the hypotheses into iterations. I designed lo-fi and hi-fi mockups and described the requirements together with the technologists and engineers. I helped QA engineers to run all the flow and described bugs. I wrote dozens of introductory texts for the news card. Basically, all solutions work on android and iOS.

Reflection and final results

Although the results cannot be called super-high, but they can be called real.
We achieved:

first card CTR without development +24%
first card CTR with development +8%
feed open rate +4%
feed 1-week retention rate +8%
feed engagement rate +35%

I would certainly like more. Unfortunately, we couldn’t do all the planned iterations, but it was influenced by external factors. Plus, in the second iteration, it was hard for me to throw out the subscription model. But we understood that it was too early to give such functionality, the card does not take into account interests.

The story looks real and confirms the theory of my design process, which works in practice. Next time we’ll try to do more iterations. See you in the next episodes, thank you!

Here I’m telling the same story, only live:

Don’t forget to turn on the subtitles, this presentation is in English and I’m speaking in my native language

Design process

Ivan Zviahin — Mon, 18 Apr 2022 19:19:46 +0000

на русском · in english

Globally, each and every designer faces three types of tasks in their daily life. The first type is just about fixing something. The second group are likely to resolve some small problem. And lastly, they happen to do a big project. Depending on the type of task, there may come a different approach.

“Fix something” is by far the simplest one. There is no rocket science in here. What we do is go handle it. It might be difficult from the point of view of the process to do something wrong here.

“Solve a small problem” in a large project. For instance, we may desire to add up the ability to like publications in the news feed or attach the ability to build on QR codes to your profile. To read process for a small task go straight to point five.

“Do a large project”. This is something way bigger, which consists of dozens, hundreds, thousands of tasks from the second point. And most importantly, in this approach we are to break a large project into smaller tasks. So, when it comes to the third type, then first you need to ↓

1. Understanding goals

As a rule, this is a meeting with all stakeholders. Together we describe goal and mission and define the audience for which we the product will be made. We write out the main hypotheses that have a direct connection with the goal and mission. For each hypothesis, the success criteria are described and outlined — a set of high-level metrics and indicators.

More about understanding Naer and Zviahin so far only in russian

2. Taking a birdview of product

In here we have to understand the scope of the product and what other systems the product can potentially hook. There are a few simple ways that I tap into to sort it out. It must be noted that most often is it better to make use of everything to broaden your horizons.

System diagrams. The temptation for many designers is to go straight to interface development. Yet designing interactions at such an early stage can interfere with the development of the basics of your product. As part of a workshop with all stakeholders, you need to describe the project through system diagrams.

Applying systems thinking in product design Shekman Tang
Creating systems not destinations Paul Adams

User journey map. It helps tremendously to look at the product from another perspective. We go beyond the edges of the product and try to understand how the user will find the product, how he will understand the specifics of the work, etc. What are the roles, what are the stages, what are the goals and actions at these stages. As part of a workshop with all stakeholders, you need to describe the product through a user journey map. It is important to note that this is just our representation and the representation of stakeholders. In reality, everything may be different and you need to verify this card with potential users at the discovery stage.

Miroverse Step – 2 – CJM Kate Syuma

It is a good practice at this meeting to outline a potential discovery plan.

3. Make sure we are on the right track

It is time to validate the concept and search for growth points. I have a few simple ways to accomplish this.

In-depth interviews. A great way to find new insights, point-by-point. However, you should not immediately take them to work, it is best to validate them with points 2 and 3.
Surveys. A quantitative study that can confirm the insights from point 1 and give reasons for reflection in isolation from qualitative research.
Data analysis. A quantitative study that can confirm the insights from point 1 and give reasons for reflection in isolation from qualitative research.
Competitor analysis. Top-level view of the set of functions of your direct and indirect competitors. This will help find something new and check out what you already have.
Analysis of the metrics used. Desk research. As a rule, we are not the first in the world to do this, and quite a lot of articles with research and best solutions have already been written about many practices. We cannot say that they will suit us, but they are worth studying.
Analysis of user feedback. Work with reviews in stores and with what arrives at the support service. There are often diamonds there that can confirm your idea or hypothesis.

All these ways can change your model from point two.

4. Prioritizing it and splitting into versions

Next, we need to divide our entire concept into small user stories, prioritize them and divide them into versions.

So, a map of user stories. This is such a visualization of the solution scope, which helps us determine the minimum valuable product and its evolution. Below is a good video that explains how it works in three minutes.

To prioritize stories by importance, you can use moscow framework, and t-shirt size framework for technical complexity.

As a result, we have dozens, hundreds, thousands of user stories that are prioritized and divided into versions.

5. Approaching each story individually

In a while, we work with each story (a small task) by a similar process, but still a little different.

Understanding the task
Discovery
Formulation of hypotheses and low-fi prototypes
Scouping and high-fi prototypes
Review the result and launch!
Result analysis and new iterations

Learn more about the design process of a small task →
Poster, PDF 2.3 MB

Corridor tests

Ivan Zviahin — Tue, 18 Jan 2022 08:47:11 +0000

на русском · in english

What are these tests for

So, at this stage you already have prototypes and formulated hypotheses. Now it remains to shake this bag and make sure that everything works as intended. Or rather, check the interface, icons, signatures, accents, and so on.

It’s simple

We have already formulated hypotheses, so we just write them out. We formulate tasks for users.

We are preparing a clickable prototype, if it is needed. We fix the “gates”, in other words, the conversion marks of the interface. For example: opening a form, successful completion, sending, and so on.

We are looking for participants for testing. We conduct testing and mark all the most important things. We draw conclusions and analyze them. Bingo, 100% percent you will change your attitude to your ideal prototypes.

1 step. Define hypotheses

Hypotheses are assumptions that we are going to test in the course of the study. It is better to formulate it according to the template: “If we do (the idea), then he will be able to positively influence (the criterion of success), since (why is this idea good)”.

2 step. Choose a testing method

All testing methodologies are divided into unmoderated and moderated according to the degree of independence of the task by respondents, as well as by the location of the respondent and the interviewer at the time of testing.

First click. The research participant follows the link and gets a very simple task. For example: “Where would you click to find information about leasing programs?”, “Choose leasing to purchase the selected car”. Only the first click is recorded. If the majority of respondents click “wrong way”, it signals that there are usability problems on the screen.

Side by side. The essence of the method is to compare two options and choose one based on the principle of better visibility or recognition of the target object. Respondents are asked to compare two images of the interface or select a specific element on two versions of the image. For example: “Find and select the A1 payment method”, “Which of the banner options is more recognizable for the material ‘Cars for leasing on ave’”.

For patency. The method is closest to moderated testing. It assumes the presence of the interviewer next to the respondent at the time of the test. We select conversion points and give the respondent a task that should lead him along the path you need. As a result, the facts of passing the points are recorded and notes are made about the difficulties encountered by the user.

Moderated testing. With such a check, the user performs the assigned tasks in relation to the product or service, while the researcher or moderator watches him in real time. Each of the above methods can be moderated. The reliability of the results and the depth of the insights obtained during such testing is considered better than when using methods without moderation.

3 step. Respondents’ choice

For testing, we need representatives of the target audience who are faced in real life with the tasks that are described in the testing tasks. However, the corridor test makes allowances in this matter, and absolutely all direct or indirect Internet users can become respondents.

Five to eight respondents are enough to identify interface problems. If all respondents face the same difficulty— this is a reason to stop testing, make edits to the prototype and start testing a new version.

If the resources and time to work on the site are very limited, we are guided by the principle: “less is better than nothing.” Three respondents are better than none.

4 step. Task formation

The quality of the results and their objectivity directly depends on the formulation of questions and tasks in testing. Checklist for the formulation of the task:

Described accurately.
Announced in full.
Relevant to the respondent’s experience.
It is possible to perform without prompts.

5 step. Preparation of the prototype

A test prototype can be:

Interactive clickable prototype.
A test version or a working website.
Just a picture.

Checklist for testing the prototype:

Scripts are executed.
There are different options for executing the script.
Texts, numbers and visual content are similar to the real ones.
There are no errors and typos in texts and figures.
There are no hints.
It is possible to quickly return to the beginning of the script.

6 step. Testing process

Properly conducted testing will be less stressful for the respondent, will be easier to moderate and, therefore, will give more relevant results. Advice – shut up, listen to the respondent and do not prevent him from not understanding anything.

The respondent speaks most of the time during testing. The moderator speaks only if absolutely necessary. Any intervention by the moderator affects the course of the experiment and distorts the data.

If the respondent has stopped “thinking out loud” – remind him about it: “So… So, you think that... What do you see here? Tell me what happened.”

Questions often force the respondent to take a defensive position. You can replace questions with motivational phrases like: “Tell me a little bit about it... Describe it in more detail... Share your feelings... Let’s talk about it... Help me understand...”.

In what cases to help:

The respondent should feel that you are encouraging him, not the product.
If the respondent has tried several methods of action and asks for help.
The respondent thinks that the task is completed, but it is not.

How to do it:

Focus on the task, remind the purpose of the task.
Make a general hint: “Remember how you started doing the task”, “You have already seen it”.
To say directly what to do next.

It is important to record the moments where the respondent encountered difficulties. After testing, we ask the respondent a few questions. You don’t have to follow the direct recommendations for changing the site that the respondent will express during the conversation. Conclusions and recommendations are usually made from observing behavior, not from a conversation.

7 step. Recording of results and conclusions

Even at the stage of hypothesis formulation, we recommend creating a table containing decomposed hypotheses and tasks for testing with the content of user steps or beaten conversion points.

After each meeting, record the test result in the table: whether the hypothesis was confirmed, whether the respondent coped with the task without errors. We also add insights and other notes describing the user’s behavior at a particular step of the task.

Based on this document, we draw conclusions about the confirmation or non-confirmation of the hypothesis, as well as formulate new ones, if necessary.

Table for recording results

Table template in Google Docs

In-depth interview

Ivan Zviahin — Tue, 11 Jan 2022 08:46:26 +0000

на русском · in english

In-depth interview is a communication in the form of questions and answers, the purpose of which is to obtain the information needed by the researcher. The key task is not just to ask, but to see the world through the eyes of others and feel what they feel. Below I will tell you about the advantages and disadvantages of open and closed questions, about popular techniques, about the stages of interviews and solving difficult situations.

Closed questions

Almost always asking closed-ended questions is bad. They assume a binary answer: yes or no.

Advantages:

allow you to get specific information
confirm or refute hypotheses
nice change of subject

Disadvantages:

we get little information
there are no details and details
we can impose our opinion

Open questions

These usually begin with the words: “why, why, how, describe, tell us what you think and so on.”

Advantages:

allow the respondent to answer without restrictions
give the respondent the opportunity to freely talk about their feelings and comment on events
provoke a person to reflect, analyze their actions and thoughts that may not have occurred to them before

Disadvantages:

may provoke a long confused response
the respondent may go “wrong way”

A dirty trick, how to turn a closed question into an open one. Closed question + “Tell me...” = Open question.

The “Five Whys” technique

In order to find the cause of behavior, problems, inconsistencies, it is necessary to consistently ask the same question — “Why?”, and look for answers to this question. Each new question is added to the answers to the previous question.

Often the repetition of the same question causes discomfort. How to make it better:

Reformulate, for example, “What is the reason?” or “Because of what... is it happening?” or “Why is that?”.
Say that you will ask “Why?” because you want to get to the root cause.

Technique “What? Who? · Where? When? · How? Why?”

The technique works well if you need to understand who the people who will be the users of your product are. Find out what they are doing, what problems they are facing.

What happened? What was the hardest part? What does a person do? Who else is involved? What’s wrong with the current solution?

Find out the context. Where it happens and when.

Where did this happen? When was the last time this happened?

How they do it. How to solve problems already. Why is it so.

How exactly did this happen? Why was it difficult? How did you manage? How often is it repeated? Has the solution to the problem changed over time and why?

Stage: Preparation

Preparation consists of six steps:

Understand the purpose of the interview.
Do desk research on the topic.
Make a plan: questions, situations, features.
Formulate hypotheses.
Think about what else might be useful for an interview.
Have 3 main questions.

Stage: Interview

Start

Tell us who you are and why you are conducting this interview.
Indicate the duration.
There are no wrong answers. You are here to find out the real experience of a person.
The first questions should be made introductory to relax the respondent.

The main part

Talk about the respondent’s life, not about your idea.
Ask about specific things that happened in the past, and not about views or opinions on the future.
Talk less, listen more.
Use the techniques of “Five Whys” and “What? Who! · Where? When? · How? Why?” to dig deeper.
Show friendliness and interest. Be simpler.
Speak in your native language
Keep the question neutral. Don’t give your opinion. Don’t rate.
Ask to compare.

Ending
Thank the respondent. Find out if the person wants to add something or ask you. Ask for the contacts of other people you can talk to. Agree on a potential sequel.

Difficult situations: the respondent is silent

Sometimes the respondent comes across silent. You should tell him about the value of the conversation. Tell me who you are and why this is all happening now. So to speak, to interest the respondent. Push the goal aside and establish personal contact.

Difficult situations: indecisive respondent

Sometimes the respondent does not give clear answers to each question. And yes and no, and so and so. Perhaps this is even very good. The respondent may have different experiences and it is worth digging deeper here. Ask about each option separately, why yes, why not. Ask for examples of all situations.

Difficult situations: talkative respondent

The respondent walks away after the question and gives completely useless information. Sometimes they can’t even be stopped, and the interview time is ticking. Here it is important to skillfully stop the respondent. For example, offer to discuss it later. It is also necessary to interrupt carefully, try to do it “on the inhale”.

Competitor analysis

Ivan Zviahin — Mon, 03 Jan 2022 08:37:03 +0000

на русском · in english

I consider the analysis of competitors within a specific feature as one of the ways to analyze and search for ideas. This is not about creating something bigger and not about the overall assessment of the market.

It is important to note that we already know the problem and the goal. We have a success criterion and an audience is defined. What’s next?

We describe the evaluation criteria

The criteria will be different for each feature. We describe briefly, but as concretely as possible:

How well the problem or task is solved. There may be several solutions. For example, adding a photo on the submission form or something else.
The first interaction with interface.
Keyboard shortcuts and interactivity.
Anything else that is important in the context of a particular feature...

Looking for competitors

As a rule, 3-4 direct competitors are already known. We include it in the list and divide it into platforms, if necessary. Each platform is a separate analysis.

Do not forget about indirect competitors. Newspaper, radio, apartment sale service, visa application form. They help to find the “new”, but they should be treated skeptically. A feature that works cool for them may not be needed by our audience at all.

Write to the table

We write down the criteria and add the “score” and “comment” field next to each one.

Template in Google Docs

The “score” is from 0 to 5. The better the competitor’s criterion is implemented, the higher the score.

“Comment” – What influenced the assessment? Why not 5? Optionally, you can add more screenshots for clarity.

Making conclusions

In the process of filling out the table, ideas will appear that solves the problem well, and what is not very good and how it can be improved. Based on the information received, we make hypotheses.

Audience

Ivan Zviahin — Mon, 27 Dec 2021 07:55:58 +0000

на русском · in English

Why do I need to understand the audience? To catch the right mental model when designing the interface, to show empathy, to make the most understandable product. And often it’s not very easy to do.

There is, for example, such a tool — persons or portraits of audiences. I don’t really believe in these ways. Why? First, most often, they are collected on the basis of superficial information. Secondly, the audience is dynamic and often very diverse. Yesterday’s personas may not be the reality of today.

And if everything is so bad, then what to do? I have a couple of thoughts on this.

1. Use the data

We go to analytics and try to identify our audience there based on banal characteristics:

gender,
age,
place of residence,
income.

If the data doesn’t work, take a survey or look at the statistics of your social networks.

You may notice that based on the data alone, it is already possible to split the audience into several portraits. Now we have a very superficial idea of who these people are and into which segments they are roughly divided. Let’s take the statistical middle of each segment and try to find out more about them.

2. Communicate with users weekly

In theory it sounds very scary and difficult, in practice it is very interesting and useful. And yes, I’m talking about weekly interviews. We already have an idea of the average audience. It remains to find them and talk.

What is there to talk about so often? It turns out that there is always a lot there: to check prototypes of current tasks, to talk about life, about the future, about problems, to discuss functions from the backlog. In a couple of months, you will have a good understanding of who your audience is. What segments it has and what segments it have:

problems,
motivations,
context,
expectations.

Where can I get users? Good question. Unloading the database and sending emails with an invitation to talk. Here is already your fantasy how to position it. Perhaps this is some kind of closed club of users close to the product or just an opportunity to get a T-shirt with the company logo.

An alternative to an interview or an additional source of information can be any kind of community or commentators in stores and social networks.

3. Complete portraits in the process of communication

Perhaps you need to fix the new information somehow. Therefore, it seems logical to take the portraits from the first paragraph and supplement them in the process.

⌘⌘⌘

Example of portraits of the car sales ad board av.by after years of work.

Goal and mission

Ivan Zviahin — Tue, 21 Dec 2021 08:23:21 +0000

на русском · in english

Or how I formulate the goal and mission when working with a task.

Goal

The goal is the final point in the business world that we want to achieve by doing something. For example, we want to increase revenue by 33% for the third quarter by increasing the number of service payments.

The goal should be specific. What exactly do we want to increase? Can I segment it in more detail?
Measurable. You need to specify a number. Numerical definition, quantity in absolute or percentage form.
With a deadline. How much time do we need to achieve success? When should the planned result be received?

Mission

The mission is what we will help the user with if we do something. For example, we want to help the user quickly find out about the appearance of new ads for interesting searches. If 25% percent of users use it in a month, it will be considered a success.

Can they be together?

Yes. The coolest thing is when you manage to make decisions in the field of intersections of mission and goal, but sometimes it happens that the solution is only in the field of mission or goal. And that’s fine.

How to formulate it?

Discuss the problem with your product manager. Discuss how this affects our end users and our business. Discuss what success looks like by defining success criteria. Write down your open questions and think about how you will answer them. Look at past research. Challenge the wording — is this the right problem, is it formulated correctly?