The Quadri-Golden tools: Continuous UX Research habit for your product team

How to create a sustainable and continuous research mindset thanks to the experimental continuous UX Research Framework

Who this approach is for

This iterative research methodology is indicated for all teams, whether strategic, tactical, or working on the entire perimeter of the product lifecycle.

More specifically, it is indicated to Product Owners, Designers et researchers working on the same user experience and value proposition.

Each will need to inform and act on one or more parts of the proposed tools to help prioritize, choose, and conduct research in an iterative and, above all, continuous manner.

Which problem do we solve ?

First of all, why do we need to apply this approach? Here is a list of problems we may have in everyday life

1) Difficulty prioritizing UX studies and aligning resources to launch UX research.

The problem of difficulty in prioritizing research and aligning resources to launch UX research refers to the lack of clear research direction and ineffective distribution of resources. Often, teams do not know what research to conduct to improve the user experience, or lack the resources to conduct the research. This can lead to a lack of understanding of user needs, and thus a lack of clarity about research priorities and goals. In addition, this can lead to inefficient allocation of resources, as the team may not have the resources needed to conduct the research effectively.

2) Difficulty in understanding what and why we need to search and understand from UX research

The difficulty of understanding what and why we need to look for and understand from UX research can lead to a lack of direction in research and ineffective allocation of resources. Often, teams do not know what research to conduct to improve the user experience, or do not have the resources to conduct the research. It is important to understand what the user wants and needs so that we can create a user experience that is satisfying and functional. In this way, UX research can help guide product development and improve the user experience.

3) Difficulty in understanding the impact our research will have.

The difficulty of understanding the impact our research will have can lead to a lack of clarity about the expected results and the usefulness of the research for the team and the organization. It is important to define from the outset the objectives of the research and the success criteria that will be used to evaluate the results. In this way, it will be possible to understand whether the research achieved its objectives and whether it had a positive impact on the product or the organization as a whole. In addition, it will be possible to identify any gaps in the research and make corrections in a timely manner.

4)The erroneous habit of choosing methodology before we know what to study and investigate (our research questions)

Many times, people make the mistake of choosing research methodology before they know what to study or investigate in depth. This can lead to ineffective research or a waste of time and resources. It is important to have a clear understanding of the research subject and its needs and problems before choosing the appropriate research methodology. Once you have a clear understanding of the research subject, you can choose the research methodology that works best to gather the necessary information. In this way, more effective and focused research can be created, leading to more useful and meaningful results.

5) Incorrect and inconsistent ways of collecting user information from UX research

The problem of incorrect and inconsistent ways of collecting user information from UX research refers to the collection of inconsistent or incomplete data that can lead to incorrect or unreliable conclusions. This can occur when research methodologies are not well defined or when researchers are not properly trained or do not follow proper procedures. In addition, the lack of adequate and easily accessible documentation can further contribute to this problem, as it can be difficult for team members to access relevant information. To avoid this problem, it is important to precisely define research methodologies and ensure that researchers are properly trained to conduct research accurately and consistently. In addition, it is important to fully and accurately document all information gathered during research so that it is easily accessible to all team members and can be used to make informed decisions and guide product development.

6) Insufficient documentation, not accessible and scattered in unused reports

This problem refers to the lack of adequate and easily accessible documentation regarding UX research results. Often, research reports are created but then remain unused or scattered, making it difficult for team members to access relevant information. This can lead to a lack of clarity regarding research findings and necessary actions to be taken. To avoid this problem, it is important to ensure that research reports are easily accessible and available to all team members so that they can be used to make informed decisions and guide product development. In addition, it is important to keep documentation up-to-date and organized so that it is easily searchable.

The 4 indispensable tools of the framework: The Quadri-Golden tools

Here we are at the heart of our methodology. I will first present the 4 essential tools to conceive and understand before we delve into the how step by step we apply them one after the other in a simple and coherent path.

• The library of research papers (categorized for each experiment or research we would be conducting)
• The backlog of the hypothesis to be tested (what and why we should look for)
• The list of experiments to conduct (how we would go about looking for information)
• The atomic research repository (what we discovered from the experiments conducted)

Let’s start analyzing these tools one by one

1) The Documents Library

Nothing innovative or new. This tool will essentially allow us to save UX research raw data preparation and analysis documents.

Essentially we would then need a space to save these documents within a folder (you can use any solution such as Google Drive, Notion, Microsoft Onedrive or even better Microsoft Teams by opening a group and saving your documents in the “files” section)

Within the main folder related to our product and research field we would go and create a folder for each research project or experiment chosen naming it like this

• Month/year + title of research related to hypothesis to be tested + type of methodology
• Example: 03/2023 — Send mortgage request journey — Moderated Interviews
• Example: 04/2024 — Compare mortgage offers — Unmoderated Usability Study (Think out Loud)

For each folder created we would go in and insert metadata to classify even better what we are talking about and share even more of the information.

For each folder created, in addition to the name as already indicated, we would add in the information

• The status of the research: To do / Ongoing / Done
• The link to the hypothesis being tested (URL)

Once the folder created we should create 5 more sub-folders

• Research script: where to save the guide with the questionnaire or questions
• Legal Documents: various legal documents for your participants to sign.
• Report of the research: Where to save the final report in power point of PDF.
• Analysis and raw data: Where to save the raw data from the research to be analyzed
• Design and Artifacts: Where to save designs or mock-ups to testera if need be

Finally, here is what your library of research documents should look like

📁 Domain Folder: My home

(Project Folder 1) 03/2023 — Send mortgage request journey — Moderated Interviews

• (Sub Folder 1) Script
• (Sub Folder 1) Legal documents
• (Sub Folder 1) Analysis & Raw Data
• (Sub Folder 1) Design & Artifacts
• (Sub Folder 1) Report

(Project Folder 2) — 04/2024 — Compare mortgage offers — Unmoderated Usability Study (Think out Loud)

• (Sub Folder 2) Script
• (Sub Folder 2) Legal documents
• (Sub Folder 2) Analysis & Raw Data
• (Sub Folder 2) Design & Artifacts
• (Sub Folder 2) Report

This way everyone will know where to find the UX research preparation, analysis and return documents made available in a centralized folder shared to all.

As said nothing new or old…just simple, good documentation habits.

2) The Hypothesis Backlog

And here we come to the heart and essence of the framework

Let us analyze the structure of a hypothesis

First we need to define what a hypothesis is:

Determination or condition relating to the possible or eventual occurrence or configuration of a fact, on the level of practical or speculative experimentation.

As we can see in the scientific method, the hypothesis is the central element that allows us, after a primary search and the identification of a research question, to identify what specifically we want to test.

Going more specific, in UX research we will use a well-defined type of hypothesis called “Empirical”

Empirical Hypothesis An empirical hypothesis is a hypothesis that is currently tested through scientific investigation and is based on hard data. It is also known as a “working hypothesis.”

The variables: Once the research question and the type of hypothesis have been determined, it is necessary to identify the independent and dependent variables.

In the example given there are two variables

• Independent: the amount of corn
• Dependent: the life span of the cow

How do we write a hypothesis in our working domain ?

To make the creation and understanding of hypotheses accessible and consistent, there is a well-defined structure and taxonomy for writing hypotheses that varies depending on what we want to look for and test, but globally a hypothesis is written like this

The are 2 different types of hypothesis

1) Based on the solution to be tested (Solution centered)

• We believe that… (Introductory form to announce that the statement we are making is merely an assumption that has not yet been verified)
• if (the if is important because it tells us the independent variable)
• our targets (here we indicate the subject, who is taken into account in our hypothesis)
• will use (we indicate the functionality and new behavior proposed by our solution)
• to/when (we define what goal and objective he will be able to achieve as a result of our idea — his job to be done)
• they will be able to (indicate what unmet need we are going to fill; our dependent variables)
• We will know this to be true when we see (success criteria to measure the impact and the validity of our hypotheses, it all depends on your market and benchmark in your domain)

📌 Example We believe that if first home buyers in Paris will receive a report about their financial capacity when Looking for a new home to buy they will be able to maximize chances to choose the right home based on their financial capabilities and minimize time to select the right one

We will know this to be true when we see more than 80% of customers come back to us during the next month to ask a mortgage

2) Based on our user’s behavior, attitude or lived experience (Solution agnostic)

• We believe that… (Introductory form to announce that the statement we are making is only an assumption not yet verified)
• our targets (here we indicate the subject, who is taken into account in our assumption)
• Needed to / is frustrated by (need / pain)
• When (job step or activity to get done)

📌 Example We believe that first home buyers need to maximize chances to choose the right home based on their financial capabilities and is difficult for them to select the right home without the financial capacity vision when they start to looking for new a new home to buy

We will know this to be true when we see & listen more than 60% of customers share these pains & needs with us

How to create the hypothesis backlog.

First, we need to see what features and metadata a hypothesis must contain in order to be properly defined and manipulated by all members of the working team.

Here is the list of information to have for each hypothesis:

• ID: is used to identify the hypothesis directly during discussions or presentations without reading the entire description
• Title: a title that summarizes the description of your hypothesis. You can take the title you gave your document folder initially.
• Description: as seen before, there are two major categories that affect how you should write the hypothesis
• Priority: there are 4 priorities you have to choose for each hypothesis: low, medium, high, really high. This way you can distinguish what to do first and what to do next. You can change the priorities time after time of course but an important rule is that there should be only 1 high priority hypothesis for each available researcher on your team. Avoid working on 3 researches at the same time.
• Status: each hypothesis has 6 statuses that follow the progress of the work : To be verified / choice of methodology / in preparation / under verification / analysis / verified. Don’t forget to update these items slowly.
• Conclusion : true / false / to be re-verified / new hypothesis. At the end you will need to see whether your hypothesis is true or false with respect to the data you have derived. In some cases the data are insufficient or lead to the creation of a new question is connected new hypothesis.
• Hypothesis category: Strategic (Discovery and concept design) or Tactical (product design or optimization)
• Hypothesis Subcategory: linked to the main category we can then better identify what we are really testing. Strategic: The behavior and goal to be accomplished , the unmet needs nowadays and problems; the concept and value proposition. Tactical: The functionality, the interactions, the journey/process or the content
• Research methodology: The type ti methodologies chosen to test the hypothesis
• The link to the LIST OF EXPERIMENTS (one or more)
• We found that: The link tied to the Atomic Research Repository to eventually tie the hypothesis to the research insights
• So we act on: the action to take as the next step on the product or its strategy once we know the answer to the hypothesis.
• Topic: add subjects related to your hypothesis so you can more easily find the hypothesis through keywords
• Linked JTBD: create a list regarding your domain of what activity or goal to accomplish and the Agate your hypothesis (you can extract it directly from the hypothesis description)
• Artifact / Design link: link to the tested prototype and design or artifacts you have verified
• Design backlog link: the link to the design backlog that the UX designer follows to complete the designs before sending them to the UX testing phase.
• Business reasons: Add the why do you need to deep dive into this hypothesis and why you add this priority to the hypothesis
• Research questions: What do you want to learn ? (these questions will help us to define the script of the research protocol)
• Name of the creator
• Date of the last modification

3) The list of experiments

Experiments are the “HOW” you want to test your hypotheses. In this part we will use the name “Experiments” as a synonym for “Research Methodologies”

One of the rules to take into account is that you can.

• Test more than one hypothesis with one experiment
• Test one hypothesis with multiple experiments

Depending on what you need to test there are experiments more specific to the strategic phase (Discovery & Concept) or the Tactical phase (Evaluative & System Optimisation). In addition, we can plan to launch an initial generative research (discovery) and then clarify some needs with a survey to test one hypothesis and then mix 2 types of research to test one hypothesis.

Naturally we have to remember that:

The more experiments we do to test a hypothesis the more exact the decision we will make to test the hypothesis will be.

So try to prioritize each time for which experiment to choose and also how many to launch to be sure of your final decisions.

Another factor to keep in mind is that included in the research methods we also find the facilitation of workshops to look for ideas to test keeping in mind that these ideas will then turn into hypotheses to be tested later…but they still remain experiments.

How to create the List of Experiments.

First of all, we need to see what features and metadata an experiment should contain in order to choose them carefully to test our hypotheses.

Here is the list of information to have for each experiment:

• Category: Research or test, or, Generative ideation
• Experiment title: Resume document folder title (example: 04/2024 — Compare mortgage offers — Unmoderated usability study)
• Type of experiment: choose from survey, interview, observation, desk-research, workshop
• Status: choose from waiting, preparing, in progress, done
• Data collected: choose from Quantitative, qualitative, both
• Conduct: choose from Moderate, Unmoderated
• Method: choose from usability, think aloud, split test, 1-on-1 discussion, market survey, idea generation
• Total participants: add the total number of participants in the experiment.
• Key Target Roles: add the people you need to talk to (your Key Person/Job Performer).
• Country: select the country where you want to launch the experiment.
• Link to HYPOTHESIS BACKLOG: link the related hypothesis you want to test with this experiment
• Trials of the experiment: add the link for each trial from the RESEARCH REPOSITORY
• Link to DOCUMENT LIBRARY: add the link to the library of documents related to this experiment.
• Tool: select the tool you are using to run the experiment.
• Time: add the time needed to run the experiment with 1 participant (example: 1h interview)
• Expected outcomes: add the list of expected outcomes from the research and their outcomes achieved
• Launch and end date: when you want to launch and end the experiment (you can create a roadmap visualization with this data).
• Attachments: add some files, if necessary

When you have created your repository in one of the tools you like (Microsoft List, airtable, Notion, Trello) you can create two types of views of your items

• View by status: so you can divide the experiments depending on the status
• Roadmap: display the experiments in a roadmap

4) The Atomic Research Repository

By speaking of atomic, we are identifying all the primary elements of a research project, whatever it may be. They are bits of information from our studies that we tag and categorize in a precise sequence and then find and manipulate them for other purposes.

The concept of the atom will help us use raw data in different ways and for different purposes like a token can be used for different goals.

Like a detective searching for the truth, UX Researchers have to do the same thing: understand the purpose of the research, research for the information, encode it and finally find the culprit (for us the insights) that tell us what is going on and the truth. Finally, we must then find a solution to the problem based on the evidence and facts found.This is only possible for us if we go into detail about the evidence from a research.

We are not talking at this point about research projects and how to create a library of projects, but we are talking about the raw data we are going to get when we do some observations or interviews with our users, whether qualitative or quantitative.

Why do we need a repository? (The key value: Share knowledge)

I do not want to make a big deal about why we need to have a library not only for research projects but also for raw data:- Make a strategic decision based on real evidence and insights- Share knowledge throughout the company- Be reliable when talking about our users or customers- Measure the evolution of a topic, behavior or attitude over time- Create a library to store wisdom within the company- Introduce new members quickly in every area of the company

How to create the atomic research repository

First of all, we need to understand the meta-structure of a UX research repository.

The first element concerns the central point on which our entire repository is based: The evidence.

Evidence is data derived from research that helps us understand a real situation that has happened or will happen.

The second element concerns the type of evidence we are talking about, and there are three types with three different levels:

• First Level — Fact / Verbatim: The basis of our knowledge, also called ‘Snippets’. These are all raw data from interviews, observations, and surveys taken directly from the voice, thoughts or actions of your users. This is data that is pure and has not yet been analyzed or interpreted by anyone. And a good method is to put these types of # evidence in inverted commas “…” .
• Second Level — Insight / Learning: By combining different Facts/Verbatim from one topic or category we derive what we call an Insight. Like an investigator, looking for connections from real facts we deduce a conclusion. This conclusion will be our discovery to share with the team. The same fact/verbatim can be used to create other insights based on other evaluation criteria or analysis. An insight will therefore be created from at least 2 or more Facts/Verbatim based on the researcher’s interpretation.
• Third Level — Action/Next Steps: To make our research actionable and concrete, we must measure its impact on business or product design. Each insight will then be associated with a description of what needs to be done to solve a problem or respond to a need of our user. By doing so, we ensure that our research has an impact on the strategic decisions of your product or business.

For each #evidence we have to associate a long list of data to categorize them, this activity is called: Data Coding.

To make a list of all the items in the repository, we can take as a basis the fact that a First Level “Verbatim X” (Fact/Verbatim) from research we are conducting enters our repository.

For our “Verbatim X” we then have to choose from various categories to assign it to in order to classify it in a specific category that will help us find it to create insight or share it with someone.

• ID: The identification number of the evidence in order to refer to it (Number)
• Description of the evidence: the pure data describing the evidence (Open Field)
• Type of data: Qualitative or Quantitative
• Participant info: add what role or particular characteristic the participant owner of the evidence has (Open Field)
• Evidence type: Fact/Verbatim (first level) | Insight/Learning (second level) | Action/Next steps (Third level)
• Valance: choose whether the evidence is positive, negative, or neutral (single choice)
• Content type: in which category the content of the evidence falls (pain, need, activity, emotion, thinking, desire, event, process, habit, idea, info) (multiple choice)
• Job Performer / Persona & Job to be done: which Person or Job performer is associated with the evidence and which activity is to be done (single choice from a predefined list)
• Link to the complete research report: Insert the hyperlink to the complete research document (hyperlink)
• Link to the tested design: Insert the link, if necessary, of the design you tested during the search with the page referring to the evidence (hyperlink)
• Link to the card of your Persona: if you have a Persona Repository or cards of your Persona then create a link (hyperlink)
• Research Question or Hypothesis: To which question or hypothesis is this evidence linked? What helps you to understand? Take this information from your research objectives (Open Field)
• Type of Hypothesis: Define if your hypothesis is related to 1) Discover the unmet needs (Discovery): 2) Concept Test: 3) New Design Test 4) Current Product
• Job Step Covered: To which step of the activity of our Persona does this evidence refer? Try to understand how your user performs his/her activity without your solution to define all the steps (Try to go into more detail with the Job to be done methodology) (Single choice from a predefined list)
• Task / Little job: which little action of our Person does this evidence refer to? (Single choice from a predefined list with the possibility to add entries)
• Desired Outcomes / Needs: what are the unmet needs to which the evidence refers? A need is expressed in the form of “Direction + metric + object + situation” (single choice from a predefined list with possibility of adding items)
• Emotional Jobs: Add some emotional feelings that you gather from the fact. An emotional job is expressed in the form of “feel + emotion + when (job)
• Section of the design (where): if you test a design, identify the section or page you tested (single choice from a predefined list with the possibility of adding entries)
• Feature / Element / Topic (What): if you test a design, enter which feature or element of the page you are testing or otherwise the topic is related to the evidence. (Single choice from a predefined list with the possibility to add entries)
• Product / Business Area: The area of your business or product according to your organisation (single choice from a predefined list)
• Customer/Clients: if you are in B2B choose which of your customers you have researched (Single choice from a predefined list with the possibility of adding entries)
• Testing Tool/Channel: Define with which tool you conducted the research and derived the evidence (Single Choice)
• Product Lifecycle stage: Define whether this evidence is related to #Strategy or #Tactics (Single choice)
• Research Method: define which research method you have used (single choice from a predefined list with the possibility to add entries)
• Data Reliability: Define how much we can trust the evidence based on your research channel and method: Very high if you gather information about what people DO and very low if you gather data about what other people tell you what your user do (Single choice from a list)
• Research Moderation: define whether moderated or unmoderated (single choice from a predefined list)
• Source age: define when you obtained this evidence (the date) (Calendar)
• Design system version: if you tested a design, define under which version you tested the design (single choice from a predefined list)
• Research Project Name: define under which research project you obtained the evidence (single choice from a predefined list with the possibility to add entries)
• Does this evidence confirm the hypothesis? Add a variable Yes or No to define whether or not your evidence supports your initial hypothesis
• Attachments: give the possibility to attach files or screenshots if needed
• Linked Evidence IDs: With a Look-up feature try to link some other evidence with the same topic or that could be connected. We mostly used this category when creating Insights to refer to all first-level evidence selected to create the insight

Once the data is saved, it will enter your database and you can from then on treat it as you wish to inform your business and user experience in a more robust and consistent manner.

Most important, you can now sort data by the 3 evidence levels and so have a full vision of Facts → Insights → Actions in a cascade visualization.

What to do after we have categorized the first-level data ?

Ok, perfect so what do we do once we have categorized our first-level data (facts/verbatim) we need to converge 2 or more facts to create insights or learnings.

Like a detective by combining multiple facts we uncover truths and we have to describe then what we infer. The simplest method is to categorize and join facts that talk about the same activity or topic. (Data clustering or Affinity Mapping). In this case a Evidence Affinity Mapping.

An #insight is therefore:

The collection of two or more facts (raw data) leads us to teach by interpreting facts that actually happened.

How is an Insight characterised?

• The name of the insight
• Its description: what we inferred from it possibly described in the form of a story :

📌 When (situation/event) I need to (underserved need) in order to (job to get done) but (pains) and I’m feel (Emotional Job)

• Whether or not it answers our initial hypothesis or research question.

These are these main features, but as we have seen before, an Insight and second-level evidence that follows the same coding features as the other first- and third-level evidence. Their characteristics and elements do not change.

In conclusion, once the insights are created and saved in our database we can filter them and then separate Facts from Insights.

At this point, once we have shared our findings with the team of designers, POs, and developers we could also enter what actions (third-level evidence) they are going to take to respond to the insight and ensure that our research will have an impact on the business and the product.

Once you have entered all the data you can share the link to your database with your colleagues or simply copy the first-level data into documents or add data from other research from other areas.

So…in a nutshell how to use these 4 tools ?

The research framework presented in the article consists of four tools that work together to help researchers conduct effective research projects. Here is a step-by-step guide on how to use them in a connected way:

1. Start by using the hypotheses backlog tool to create and manage research hypotheses. Categorize your hypotheses based on their strategic or tactical nature, as well as their related subcategories.
2. Once you have your hypotheses in place, use the experiments list tool to plan and execute experiments to test them. Categorize your experiments based on their type, status, and data gathered. Make sure to link each experiment to the corresponding hypothesis in your backlog.
3. As you conduct your experiments, gather and store your raw data in the document library tool. Make sure to categorize your data based on its type, participant information, and other relevant metadata.
4. Once you have gathered enough data, use the atomic research repository tool to categorize your evidence based on its level (fact/verbatim, insight/learning, or action/next steps) and other relevant metadata. Use the evidence affinity mapping feature to combine related evidence into insights.
5. Share your findings and insights with your team using the appropriate links from each tool. Use the research questions and business reasons fields to help your team understand the significance of your findings.

By using these four tools in a connected way, you can conduct effective research projects and share your findings and insights with your team in a clear and organised manner.

How to start applying this working methodology ? here’s a little story to finish it off

Once upon a time, a team of UX researchers were struggling to keep track of their research projects. They found themselves lost in a sea of data, unsure of how to organize it or make sense of it. They realized they needed a better way to manage their research, to gather and analyze raw data, and to develop insights.

So, they decided to implement a new research framework consisting of four tools: hypotheses backlog, experiments list, atomic research repository, and the documents library. They began by imposing and directing the work to ensure that every team member was involved and that there was effective organization of the work. They made it a habit to use these four tools for every project and to centralize discussion through comments. They also made sure that all stakeholders were directly involved, and communication was centralized and visible to everyone.

To keep everyone informed, they started sharing weekly updates on the progress of their hypotheses and research by sharing the appropriate links. They were also impeccable with the governance of their tools to ensure that the data was accurate and up-to-date. They started small, with a narrow perimeter, to test the methodology and evaluate if it worked for the team.

As a result, they were able to align with the backlog design to prioritize hypotheses based on design to test. They made stakeholders responsible for the tools, and all team members were trained on their use. With the new research framework in place, they were able to manage their research projects more efficiently, gather and analyze raw data, develop insights, and make strategic decisions based on real evidence and insights.

In the end, the team was able to overcome their struggles and achieve success with their research projects. They learned the importance of having an effective research framework and how it could help them to better manage their data, develop insights, and make informed decisions.

Ciao ciao,