Natalie Glance’s Post

One of our Duolingo operating principles is “test it first.” I love the sheer scale of our testing; we run hundreds of simultaneous A/B tests to find new ways to teach more effectively. We have tens of millions of learners (just under 30M daily active users last quarter) and we use innovative thinking and experimentation to improve how we teach. Whenever we make a change to the Duolingo app, small or big, we first test to make sure it doesn’t have a negative impact on our users or their ability to learn. We do the same for the course content, never making any change until it has been tested and shown to have positive effects on our learners’ progress. But today, I want to talk about a class of experiments that surprised me. A few years ago, we started a team called ✨ Delight ✨. Delight was not tasked with improving our metrics; its purpose was to make the app more delightful. This can be anything from: 💥Animating the explosions that appear when you answer a hard question.  🐝Satisfying haptic feedback throughout your lesson ✨Adding subtle sparkles across the app We didn’t expect these changes to have a positive effect on our numbers. But we tested them anyway. To our surprise, the animations do in fact keep people in the app longer and cause them to return to the app more often! ✨Delight facilitates learning✨, in ways we did not anticipate. Though we discovered that these animations *did* hurt the performance of one user segment: those who use Duolingo on lower-end devices. So we are able to turn off the more processing-intensive animations for these users to make sure their educational experience is not harmed 🥳 Another thing I love about our experiment-driven culture is how it facilitates collaboration. When a cross-functional team (made up of product, engineering, design and others) has to move a metric, they brainstorm together ➡️ come up with multiple ideas and hypotheses ➡️ test them to see what moves the needle ➡️ and iterate 🔁 If you are an engineer who wants to contribute meaningfully to education, with a strong sense of how products can be continuously improved, you’re one of us. Check out our open engineering roles here at Duolingo! https://lnkd.in/e9c8yXRz #duolingo #education #engineering #abtesting #hiring

One of our Duolingo operating principles is “test it first.” I love the sheer scale of our testing; we run hundreds of simultaneous A/B tests to find new ways to teach more effectively. We have tens of millions of learners (just under 30M daily active users last quarter) and we use innovative thinking and experimentation to improve how we teach. Whenever we make a change to the Duolingo app, small or big, we first test to make sure it doesn’t have a negative impact on our users or their ability to learn. We do the same for the course content, never making any change until it has been tested and shown to have positive effects on our learners’ progress. But today, I want to talk about a class of experiments that surprised me. A few years ago, we started a team called ✨ Delight ✨. Delight was not tasked with improving our metrics; its purpose was to make the app more delightful. This can be anything from: 💥Animating the explosions that appear when you answer a hard question.  🐝Satisfying haptic feedback throughout your lesson ✨Adding subtle sparkles across the app We didn’t expect these changes to have a positive effect on our numbers. But we tested them anyway. To our surprise, the animations do in fact keep people in the app longer and cause them to return to the app more often! ✨Delight facilitates learning✨, in ways we did not anticipate. Though we discovered that these animations *did* hurt the performance of one user segment: those who use Duolingo on lower-end devices. So we are able to turn off the more processing-intensive animations for these users to make sure their educational experience is not harmed 🥳 Another thing I love about our experiment-driven culture is how it facilitates collaboration. When a cross-functional team (made up of product, engineering, design and others) has to move a metric, they brainstorm together ➡️ come up with multiple ideas and hypotheses ➡️ test them to see what moves the needle ➡️ and iterate 🔁 If you are an engineer who wants to contribute meaningfully to education, with a strong sense of how products can be continuously improved, you’re one of us. Check out our open engineering roles here at Duolingo! https://lnkd.in/e9c8yXRz #duolingo #education #engineering #abtesting #hiring

One of our Duolingo operating principles is “test it first.” I love the sheer scale of our testing; we run hundreds of simultaneous A/B tests to find new ways to teach more effectively. We have tens of millions of learners (just under 30M daily active users last quarter) and we use innovative thinking and experimentation to improve how we teach. Whenever we make a change to the Duolingo app, small or big, we first test to make sure it doesn’t have a negative impact on our users or their ability to learn. We do the same for the course content, never making any change until it has been tested and shown to have positive effects on our learners’ progress. But today, I want to talk about a class of experiments that surprised me. A few years ago, we started a team called ✨ Delight ✨. Delight was not tasked with improving our metrics; its purpose was to make the app more delightful. This can be anything from: 💥Animating the explosions that appear when you answer a hard question.  🐝Satisfying haptic feedback throughout your lesson ✨Adding subtle sparkles across the app We didn’t expect these changes to have a positive effect on our numbers. But we tested them anyway. To our surprise, the animations do in fact keep people in the app longer and cause them to return to the app more often! ✨Delight facilitates learning✨, in ways we did not anticipate. Though we discovered that these animations *did* hurt the performance of one user segment: those who use Duolingo on lower-end devices. So we are able to turn off the more processing-intensive animations for these users to make sure their educational experience is not harmed 🥳 Another thing I love about our experiment-driven culture is how it facilitates collaboration. When a cross-functional team (made up of product, engineering, design and others) has to move a metric, they brainstorm together ➡️ come up with multiple ideas and hypotheses ➡️ test them to see what moves the needle ➡️ and iterate 🔁 If you are an engineer who wants to contribute meaningfully to education, with a strong sense of how products can be continuously improved, you’re one of us. Check out our open engineering roles here at Duolingo! https://lnkd.in/e9c8yXRz #duolingo #education #engineering #abtesting #hiring

One of our Duolingo operating principles is “test it first.” I love the sheer scale of our testing; we run hundreds of simultaneous A/B tests to find new ways to teach more effectively. We have tens of millions of learners (just under 30M daily active users last quarter) and we use innovative thinking and experimentation to improve how we teach. Whenever we make a change to the Duolingo app, small or big, we first test to make sure it doesn’t have a negative impact on our users or their ability to learn. We do the same for the course content, never making any change until it has been tested and shown to have positive effects on our learners’ progress. But today, I want to talk about a class of experiments that surprised me. A few years ago, we started a team called ✨ Delight ✨. Delight was not tasked with improving our metrics; its purpose was to make the app more delightful. This can be anything from: 💥Animating the explosions that appear when you answer a hard question.  🐝Satisfying haptic feedback throughout your lesson ✨Adding subtle sparkles across the app We didn’t expect these changes to have a positive effect on our numbers. But we tested them anyway. To our surprise, the animations do in fact keep people in the app longer and cause them to return to the app more often! ✨Delight facilitates learning✨, in ways we did not anticipate. Though we discovered that these animations *did* hurt the performance of one user segment: those who use Duolingo on lower-end devices. So we are able to turn off the more processing-intensive animations for these users to make sure their educational experience is not harmed 🥳 Another thing I love about our experiment-driven culture is how it facilitates collaboration. When a cross-functional team (made up of product, engineering, design and others) has to move a metric, they brainstorm together ➡️ come up with multiple ideas and hypotheses ➡️ test them to see what moves the needle ➡️ and iterate 🔁 If you are an engineer who wants to contribute meaningfully to education, with a strong sense of how products can be continuously improved, you’re one of us. Check out our open engineering roles here at Duolingo! https://lnkd.in/e9c8yXRz #duolingo #education #engineering #abtesting #hiring

One of our Duolingo operating principles is “test it first.” I love the sheer scale of our testing; we run hundreds of simultaneous A/B tests to find new ways to teach more effectively. We have tens of millions of learners (just under 30M daily active users last quarter) and we use innovative thinking and experimentation to improve how we teach. Whenever we make a change to the Duolingo app, small or big, we first test to make sure it doesn’t have a negative impact on our users or their ability to learn. We do the same for the course content, never making any change until it has been tested and shown to have positive effects on our learners’ progress. But today, I want to talk about a class of experiments that surprised me. A few years ago, we started a team called ✨ Delight ✨. Delight was not tasked with improving our metrics; its purpose was to make the app more delightful. This can be anything from: 💥Animating the explosions that appear when you answer a hard question.  🐝Satisfying haptic feedback throughout your lesson ✨Adding subtle sparkles across the app We didn’t expect these changes to have a positive effect on our numbers. But we tested them anyway. To our surprise, the animations do in fact keep people in the app longer and cause them to return to the app more often! ✨Delight facilitates learning✨, in ways we did not anticipate. Though we discovered that these animations *did* hurt the performance of one user segment: those who use Duolingo on lower-end devices. So we are able to turn off the more processing-intensive animations for these users to make sure their educational experience is not harmed 🥳 Another thing I love about our experiment-driven culture is how it facilitates collaboration. When a cross-functional team (made up of product, engineering, design and others) has to move a metric, they brainstorm together ➡️ come up with multiple ideas and hypotheses ➡️ test them to see what moves the needle ➡️ and iterate 🔁 If you are an engineer who wants to contribute meaningfully to education, with a strong sense of how products can be continuously improved, you’re one of us. Check out our open engineering roles here at Duolingo! https://lnkd.in/e9c8yXRz #duolingo #education #engineering #abtesting #hiring

One of our Duolingo operating principles is “test it first.” I love the sheer scale of our testing; we run hundreds of simultaneous A/B tests to find new ways to teach more effectively. We have tens of millions of learners (just under 30M daily active users last quarter) and we use innovative thinking and experimentation to improve how we teach. Whenever we make a change to the Duolingo app, small or big, we first test to make sure it doesn’t have a negative impact on our users or their ability to learn. We do the same for the course content, never making any change until it has been tested and shown to have positive effects on our learners’ progress. But today, I want to talk about a class of experiments that surprised me. A few years ago, we started a team called ✨ Delight ✨. Delight was not tasked with improving our metrics; its purpose was to make the app more delightful. This can be anything from: 💥Animating the explosions that appear when you answer a hard question.  🐝Satisfying haptic feedback throughout your lesson ✨Adding subtle sparkles across the app We didn’t expect these changes to have a positive effect on our numbers. But we tested them anyway. To our surprise, the animations do in fact keep people in the app longer and cause them to return to the app more often! ✨Delight facilitates learning✨, in ways we did not anticipate. Though we discovered that these animations *did* hurt the performance of one user segment: those who use Duolingo on lower-end devices. So we are able to turn off the more processing-intensive animations for these users to make sure their educational experience is not harmed 🥳 Another thing I love about our experiment-driven culture is how it facilitates collaboration. When a cross-functional team (made up of product, engineering, design and others) has to move a metric, they brainstorm together ➡️ come up with multiple ideas and hypotheses ➡️ test them to see what moves the needle ➡️ and iterate 🔁 If you are an engineer who wants to contribute meaningfully to education, with a strong sense of how products can be continuously improved, you’re one of us. Check out our open engineering roles here at Duolingo! https://lnkd.in/e9c8yXRz #duolingo #education #engineering #abtesting #hiring

One of our Duolingo operating principles is “test it first.” I love the sheer scale of our testing; we run hundreds of simultaneous A/B tests to find new ways to teach more effectively. We have tens of millions of learners (just under 30M daily active users last quarter) and we use innovative thinking and experimentation to improve how we teach. Whenever we make a change to the Duolingo app, small or big, we first test to make sure it doesn’t have a negative impact on our users or their ability to learn. We do the same for the course content, never making any change until it has been tested and shown to have positive effects on our learners’ progress. But today, I want to talk about a class of experiments that surprised me. A few years ago, we started a team called ✨ Delight ✨. Delight was not tasked with improving our metrics; its purpose was to make the app more delightful. This can be anything from: 💥Animating the explosions that appear when you answer a hard question.  🐝Satisfying haptic feedback throughout your lesson ✨Adding subtle sparkles across the app We didn’t expect these changes to have a positive effect on our numbers. But we tested them anyway. To our surprise, the animations do in fact keep people in the app longer and cause them to return to the app more often! ✨Delight facilitates learning✨, in ways we did not anticipate. Though we discovered that these animations *did* hurt the performance of one user segment: those who use Duolingo on lower-end devices. So we are able to turn off the more processing-intensive animations for these users to make sure their educational experience is not harmed 🥳 Another thing I love about our experiment-driven culture is how it facilitates collaboration. When a cross-functional team (made up of product, engineering, design and others) has to move a metric, they brainstorm together ➡️ come up with multiple ideas and hypotheses ➡️ test them to see what moves the needle ➡️ and iterate 🔁 If you are an engineer who wants to contribute meaningfully to education, with a strong sense of how products can be continuously improved, you’re one of us. Check out our open engineering roles here at Duolingo! https://lnkd.in/e9c8yXRz #duolingo #education #engineering #abtesting #hiring

One of our Duolingo operating principles is “test it first.” I love the sheer scale of our testing; we run hundreds of simultaneous A/B tests to find new ways to teach more effectively. We have tens of millions of learners (just under 30M daily active users last quarter) and we use innovative thinking and experimentation to improve how we teach. Whenever we make a change to the Duolingo app, small or big, we first test to make sure it doesn’t have a negative impact on our users or their ability to learn. We do the same for the course content, never making any change until it has been tested and shown to have positive effects on our learners’ progress. But today, I want to talk about a class of experiments that surprised me. A few years ago, we started a team called ✨ Delight ✨. Delight was not tasked with improving our metrics; its purpose was to make the app more delightful. This can be anything from: 💥Animating the explosions that appear when you answer a hard question.  🐝Satisfying haptic feedback throughout your lesson ✨Adding subtle sparkles across the app We didn’t expect these changes to have a positive effect on our numbers. But we tested them anyway. To our surprise, the animations do in fact keep people in the app longer and cause them to return to the app more often! ✨Delight facilitates learning✨, in ways we did not anticipate. Though we discovered that these animations *did* hurt the performance of one user segment: those who use Duolingo on lower-end devices. So we are able to turn off the more processing-intensive animations for these users to make sure their educational experience is not harmed 🥳 Another thing I love about our experiment-driven culture is how it facilitates collaboration. When a cross-functional team (made up of product, engineering, design and others) has to move a metric, they brainstorm together ➡️ come up with multiple ideas and hypotheses ➡️ test them to see what moves the needle ➡️ and iterate 🔁 If you are an engineer who wants to contribute meaningfully to education, with a strong sense of how products can be continuously improved, you’re one of us. Check out our open engineering roles here at Duolingo! https://lnkd.in/e9c8yXRz #duolingo #education #engineering #abtesting #hiring

One of our Duolingo operating principles is “test it first.” I love the sheer scale of our testing; we run hundreds of simultaneous A/B tests to find new ways to teach more effectively. We have tens of millions of learners (just under 30M daily active users last quarter) and we use innovative thinking and experimentation to improve how we teach. Whenever we make a change to the Duolingo app, small or big, we first test to make sure it doesn’t have a negative impact on our users or their ability to learn. We do the same for the course content, never making any change until it has been tested and shown to have positive effects on our learners’ progress. But today, I want to talk about a class of experiments that surprised me. A few years ago, we started a team called ✨ Delight ✨. Delight was not tasked with improving our metrics; its purpose was to make the app more delightful. This can be anything from: 💥Animating the explosions that appear when you answer a hard question.  🐝Satisfying haptic feedback throughout your lesson ✨Adding subtle sparkles across the app We didn’t expect these changes to have a positive effect on our numbers. But we tested them anyway. To our surprise, the animations do in fact keep people in the app longer and cause them to return to the app more often! ✨Delight facilitates learning✨, in ways we did not anticipate. Though we discovered that these animations *did* hurt the performance of one user segment: those who use Duolingo on lower-end devices. So we are able to turn off the more processing-intensive animations for these users to make sure their educational experience is not harmed 🥳 Another thing I love about our experiment-driven culture is how it facilitates collaboration. When a cross-functional team (made up of product, engineering, design and others) has to move a metric, they brainstorm together ➡️ come up with multiple ideas and hypotheses ➡️ test them to see what moves the needle ➡️ and iterate 🔁 If you are an engineer who wants to contribute meaningfully to education, with a strong sense of how products can be continuously improved, you’re one of us. Check out our open engineering roles here at Duolingo! https://lnkd.in/e9c8yXRz #duolingo #education #engineering #abtesting #hiring

One of our Duolingo operating principles is “test it first.” I love the sheer scale of our testing; we run hundreds of simultaneous A/B tests to find new ways to teach more effectively. We have tens of millions of learners (just under 30M daily active users last quarter) and we use innovative thinking and experimentation to improve how we teach. Whenever we make a change to the Duolingo app, small or big, we first test to make sure it doesn’t have a negative impact on our users or their ability to learn. We do the same for the course content, never making any change until it has been tested and shown to have positive effects on our learners’ progress. But today, I want to talk about a class of experiments that surprised me. A few years ago, we started a team called ✨ Delight ✨. Delight was not tasked with improving our metrics; its purpose was to make the app more delightful. This can be anything from: 💥Animating the explosions that appear when you answer a hard question.  🐝Satisfying haptic feedback throughout your lesson ✨Adding subtle sparkles across the app We didn’t expect these changes to have a positive effect on our numbers. But we tested them anyway. To our surprise, the animations do in fact keep people in the app longer and cause them to return to the app more often! ✨Delight facilitates learning✨, in ways we did not anticipate. Though we discovered that these animations *did* hurt the performance of one user segment: those who use Duolingo on lower-end devices. So we are able to turn off the more processing-intensive animations for these users to make sure their educational experience is not harmed 🥳 Another thing I love about our experiment-driven culture is how it facilitates collaboration. When a cross-functional team (made up of product, engineering, design and others) has to move a metric, they brainstorm together ➡️ come up with multiple ideas and hypotheses ➡️ test them to see what moves the needle ➡️ and iterate 🔁 If you are an engineer who wants to contribute meaningfully to education, with a strong sense of how products can be continuously improved, you’re one of us. Check out our open engineering roles here at Duolingo! https://lnkd.in/e9c8yXRz #duolingo #education #engineering #abtesting #hiring